Policy or Value ? Loss Function and Playing Strength in AlphaZero-like Self-play
Por um escritor misterioso
Last updated 29 março 2025

Results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Recently, AlphaZero has achieved outstanding performance in playing Go, Chess, and Shogi. Players in AlphaZero consist of a combination of Monte Carlo Tree Search and a Deep Q-network, that is trained using self-play. The unified Deep Q-network has a policy-head and a value-head. In AlphaZero, during training, the optimization minimizes the sum of the policy loss and the value loss. However, it is not clear if and under which circumstances other formulations of the objective function are better. Therefore, in this paper, we perform experiments with combinations of these two optimization targets. Self-play is a computationally intensive method. By using small games, we are able to perform multiple test cases. We use a light-weight open source reimplementation of AlphaZero on two different games. We investigate optimizing the two targets independently, and also try different combinations (sum and product). Our results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Moreover, we find that care must be taken in computing the playing strength. Tournament Elo ratings differ from training Elo ratings—training Elo ratings, though cheap to compute and frequently reported, can be misleading and may lead to bias. It is currently not clear how these results transfer to more complex games and if there is a phase transition between our setting and the AlphaZero application to Go where the sum is seemingly the better choice.

Reimagining Chess with AlphaZero, February 2022

Representation Matters: The Game of Chess Poses a Challenge to

Policy or Value ? Loss Function and Playing Strength in AlphaZero

Value targets in off-policy AlphaZero: a new greedy backup

AlphaGo/AlphaGoZero/AlphaZero/MuZero: Mastering games using

PDF) Expediting Self-Play Learning in AlphaZero-Style Game-Playing

Acquisition of chess knowledge in AlphaZero

Policy or Value ? Loss Function and Playing Strength in AlphaZero

Multiplayer AlphaZero – arXiv Vanity
Does the neural net of AlphaZero only evaluate the score of a
Simple Alpha Zero
Recomendado para você
-
Comparison of network architecture of AlphaZero and NoGoZero+ (529 março 2025
-
AlphaZero, Vladimir Kramnik and reinventing chess29 março 2025
-
AlphaZero - Wikipedia29 março 2025
-
DeepMind AlphaZero lernt übergreifend Spiele zu spielen29 março 2025
-
AlphaZero: DeepMind's AI Works Smarter, not Harder29 março 2025
-
AlphaZero: A General Reinforcement Learning Algorithm that Masters Chess, Shogi and Go through Self-Play29 março 2025
-
How AlphaZero Works – Augmented Lawyer29 março 2025
-
Creative' AlphaZero leads way for chess computers and, maybe, science, Chess29 março 2025
-
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play29 março 2025
-
Mastering TicTacToe with AlphaZero, by Noufal Samsudin, MLearning.ai29 março 2025
você pode gostar
-
Goosebumps: Jack Black & Slappy Official Movie Interview29 março 2025
-
DON'T LET HIM FIND YOU in Roblox Flee the Facility! on Vimeo29 março 2025
-
IGN - An eye-catching Steam sale has seen Rockstar's Red Dead Redemption 2 hit new heights, with more players on the platform than ever before. Link in comments for more.29 março 2025
-
Lorena Matos on X: De hoje #anagrama #rachacuca / X29 março 2025
-
Yuuna and the Haunted Hot Springs Vol. 24|Paperback29 março 2025
-
Review: Season two of 'Kamisama Kiss' beautifully blends humor & drama - Digital Journal29 março 2025
-
Why Wind Waker Could Be the Best Premise for The Legend of Zelda: Live-Action Movie29 março 2025
-
Santos encaminha empréstimo de Soteldo para o Grêmio, futebol29 março 2025
-
Top 10 Free Horror Games Worth Playing - Indie Game Bundles29 março 2025
-
Bola Basquete R500 Size 7 Resistente A Furo Tarmak Cd - Iguasport Ltda. - Bola de Basquete - Magazine Luiza29 março 2025