25,161
edits
Changes
no edit summary
==Network==
While [[AlphaGo]] used two disjoint networks for policy and value, [[AlphaZero]] as well as Leela Chess Zero, share a common "body" connected to disjoint policy and value "heads". The “body” consists of spatial 8x8 input planes, followed by convolutional layers with using B [[Neural Networks#Residual|residual]] blocks times 3x3xF with F filtersof kernel size 3x3, stride 1. So far, model sizes FxB specifies the model and size of the CNN (64x6, 128x10, 192x15, and 256x20 were used). Concerning [[Nodes per Second|nodes per second]] of the MCTS, smaller models are faster to calculate than larger models. They are faster to train and one may will earlier recognize progress, but they will also saturate earlier so that at some point more training will no longer improve the engine. Larger and deeper network models will improve the receptivity, the amount of knowledge and pattern to extract from the training samples, with potential for a [[Playing Strength|stronger]] engine.
As a further improvement, Leele Chess Zero applies the ''Squeeze and Excite'' (SE) extension to the residual block architecture <ref>[https://github.com/LeelaChessZero/lc0/wiki/Technical-Explanation-of-Leela-Chess-Zero Technical Explanation of Leela Chess Zero · LeelaChessZero/lc0 Wiki · GitHub]</ref> <ref>[https://towardsdatascience.com/squeeze-and-excitation-networks-9ef5e71eacd7 Squeeze-and-Excitation Networks – Towards Data Science] by [http://plpp.de/ Paul-Louis Pröve], October 17, 2017</ref>.
The body is fully connected to both the policy "head" for the move probability distribution, and value "head" for the evaluation score aka [[Pawn Advantage, Win Percentage, and Elo|winning probability]] of the the the current position and up to seven predecessor positions on the input planes.
