Difference between revisions of "Leela Chess Zero"

From Chessprogramming wiki
Jump to: navigation, search
Line 1: Line 1:
 
'''[[Main Page|Home]] * [[Engines]] * Leela Chess Zero'''
 
'''[[Main Page|Home]] * [[Engines]] * Leela Chess Zero'''
 +
 +
[[FILE:LC0-Logo.jpg|border|right|thumb|link=https://twitter.com/leelachesszero| Lc0 logo <ref>[https://twitter.com/leelachesszero Leela Chess Zero (@LeelaChessZero) | Twitter]</ref> ]]
  
 
'''Leela Chess Zero''', (LCZero, lc0)<br/>
 
'''Leela Chess Zero''', (LCZero, lc0)<br/>
an adaption of [[Gian-Carlo Pascutto|Gian-Carlo Pascutto's]] [[Leela Zero]] [[Go]] project <ref>[https://github.com/gcp/leela-zero GitHub - gcp/leela-zero: Go engine with no human-provided knowledge, modeled after the AlphaGo Zero paper]</ref> to [[Chess]], using [[Stockfish|Stockfish's]] [[Board Representation|board representation]] and [[Move Generation|move generation]]. No heuristics or prior [[Knowledge|knowledge]] are carried over from Stockfish. Leela Chess is [[:Category:Open Source|open source]], released under the terms of [[Free Software Foundation#GPL|GPL version 3]] or later, and supports [[UCI]]. The goal is to build a strong [[UCT]] chess AI following the same type of [[Deep Learning|deep learning]] techniques of [[AlphaZero]] as described in [[DeepMind|DeepMind's]] paper <ref>[[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2017'''). ''Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm''. [https://arxiv.org/abs/1712.01815 arXiv:1712.01815]</ref>, but using distributed training for the weights of the [[Neural Networks#Deep|deep]] [[Neural Networks#Residual|residual]] [[Neural Networks#Convolutional|convolutional neural network]]. The training process requires [https://en.wikipedia.org/wiki/CUDA CUDA] and a [[GPU]] accelerated version of [https://en.wikipedia.org/wiki/TensorFlow Tensorflow] installed <ref>[https://github.com/glinscott/leela-chess/blob/master/README.md leela-chess/README.md at master · glinscott/leela-chess · GitHub]</ref>.  
+
an adaption of [[Gian-Carlo Pascutto|Gian-Carlo Pascutto's]] [[Leela Zero]] [[Go]] project <ref>[https://github.com/gcp/leela-zero GitHub - gcp/leela-zero: Go engine with no human-provided knowledge, modeled after the AlphaGo Zero paper]</ref> to [[Chess]], initiated and announced by [[Stockfish]] co-author [[Gary Linscott]], who was already responsible for the Stockfish [[Stockfish#TestingFramework|Testing Framework]] called ''Fishtest''. Leela Chess is [[:Category:Open Source|open source]], released under the terms of [[Free Software Foundation#GPL|GPL version 3]] or later, and supports [[UCI]].  
 +
The goal is to build a strong chess playing entity following the same type of [[Deep Learning|deep learning]] along with [[Monte-Carlo Tree Search|Monte-Carlo tree search]] (MCTS) techniques of [[AlphaZero]] as described in [[DeepMind|DeepMind's]] 2017 and 2018 papers
 +
<ref>[[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2017'''). ''Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm''. [https://arxiv.org/abs/1712.01815 arXiv:1712.01815]</ref>
 +
<ref>[[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2018'''). ''[http://science.sciencemag.org/content/362/6419/1140 A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play]''. [https://en.wikipedia.org/wiki/Science_(journal) Science], Vol. 362, No. 6419</ref>
 +
<ref>[http://blog.lczero.org/2018/12/alphazero-paper-and-lc0-v0191.html AlphaZero paper, and Lc0 v0.19.1] by [[Alexander Lyashuk|crem]], [[Leela Chess Zero|LCZero blog]], December 07, 2018</ref>,  
 +
but using distributed training for the weights of the [[Neural Networks#Deep|deep]] [[Neural Networks#Convolutional|convolutional neural network]] (CNN, DNN, DCNN).
 +
 
 +
=Lc0=
 +
Leela Chess Zero consists of an executable to play or analyze [[Chess Game|games]], initially dubbed '''LCZero''', soon rewritten by a team around [[Alexander Lyashuk]] for better performance and then called '''Lc0''' <ref>[https://github.com/LeelaChessZero/lc0/wiki/lc0-transition lc0 transition · LeelaChessZero/lc0 Wiki · GitHub]</ref>. This executable, the actual chess engine, performs the [[Monte-Carlo Tree Search|MCTS]] and reads the self-taught [[Neural Networks#Convolutional|CNN]], which weights are persistent in a separate file.
 +
Lc0 is written in [[Cpp#14|C++14]] and may be compiled for various platforms and backends. Since deep CNN approaches are best suited to run massively in parallel on [[GPU|GPUs]] to perform all the [[Float|floating point]] [https://en.wikipedia.org/wiki/Dot_product dot products] for thousands of neurons,
 +
the preferred target platforms are [[Nvidia]] [[GPU|GPU’s]] supporting [https://en.wikipedia.org/wiki/CUDA CUDA] and cuDNN libraries <ref>[https://developer.nvidia.com/cudnn NVIDIA cuDNN | NVIDIA Developer]</ref>.
 +
None CUDA compliant GPUs ([[AMD]]) are supported through [https://en.wikipedia.org/wiki/OpenCL OpenCL], while much slower pure CPU binaries are possible using [https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms BLAS], target systems with or without a [https://en.wikipedia.org/wiki/Video_card graphics card] (GPU) are [[Linux]], [[Mac OS]] and [[Windows]] computers, or BLAS only the [[Raspberry Pi]].
 +
 
 +
=Description=
 +
Like [[AlphaZero]], Lc0's [[Evaluation|evaluates]] [[Chess Position|positions]] using non-linear function approximation based on a [[Neural Networks|deep neural network]], rather than the [[Evaluation#Linear|linear function approximation]] as used in classical chess programs.
 +
This neural network takes the board position as input and outputs position evaluation (QValue) and a vector of move probabilities (PValue, policy).
 +
Once trained, these network is combined with a [[Monte-Carlo Tree Search]] (MCTS) using the policy to narrow down the search to high­probability moves,
 +
and using the value in conjunction with a fast rollout policy to evaluate positions in the tree. The MCTS selection is done by a variation of [[Christopher D. Rosin|Rosin's]] [[UCT]] improvement dubbed [[Christopher D. Rosin#PUCT|PUCT]] (Predictor + UCT).
 +
 
 +
==[[Board Representation]]==
 +
Lc0's color agnostic board is represented by five [[Bitboards|bitboards]] (own pieces, opponent pieces, orthogonal sliding pieces, diagonal sliding pieces, and pawns including [[En passant|en passant]] target information coded as pawns on rank 1 and 8), two king squares, casting rights, and a flag whether the board is [[Color Flipping|color flipped]].
 +
While the structure is suitable as input for the neural network, getting individual pieces bitboards requires some [[General Setwise Operations|setwise operations]] such as [[General Setwise Operations#Intersection|intersection]] and [[General Setwise Operations#RelativeComplement|set theoretic difference]] <ref>[https://github.com/LeelaChessZero/lc0/blob/master/src/chess/board.h lc0/board.h at master · LeelaChessZero/lc0 · GitHub]</ref>.
 +
 
 +
==Network==
 +
While [[AlphaGo]] used two disjoint networks for policy and value, [[AlphaZero]] as well as Leela Chess Zero, share a common "body" connected to disjoint policy and value "heads". The “body” consists of spatial 8x8 input planes, followed by convolutional layers with B [[Neural Networks#Residual|residual]] blocks times 3x3xF filters. BxF specifies the model and size of the CNN (64x6, 128x10, 192x15, 256x20 were used).
 +
Concerning [[Nodes per Second|nodes per second]] of the MCTS, smaller models are faster to calculate than larger models. They are faster to train and one may earlier recognize progress, but they will also saturate earlier so that at some point more training will no longer improve the engine.
 +
Larger and deeper network models will improve the receptivity, the amount of knowledge and pattern to extract from the training samples, with potential for a [[Playing Strength|stronger]] engine.
 +
As a further improvement, Leele Chess Zero applies the ''Squeeze and Excite'' (SE) extension to the residual block architecture <ref>[https://github.com/LeelaChessZero/lc0/wiki/Technical-Explanation-of-Leela-Chess-Zero Technical Explanation of Leela Chess Zero · LeelaChessZero/lc0 Wiki · GitHub]</ref> <ref>[https://towardsdatascience.com/squeeze-and-excitation-networks-9ef5e71eacd7 Squeeze-and-Excitation Networks – Towards Data Science] by [http://plpp.de/ Paul-Louis Pröve], October 17, 2017</ref>.
 +
The body is fully connected to both the policy "head" for the move probability distribution, and value "head" for the evaluation score aka [[Pawn Advantage, Win Percentage, and Elo|winning probability]] of the the the current positions and up to seven predecessor positions on the input planes.
 +
 
 +
==Training==
 +
Like in [[AlphaZero]], the '''Zero''' suffix implies no other initial knowledge than the rules of the games, to build a superhuman player, starting with truly random self-play games to apply [[Reinforcement Learning|reinforcement learning]] based on the outcome of that games.
 +
However, there are derived approaches, such as [[Albert Silver|Albert Silver's]] [[Deus X]], trying to take a short-cut by initially using [[Supervised Learning|supervised learning]] techniques, such as feeding in high quality games played by other strong chess playing entities, or huge records of positions with a given preferred move.
 +
The unsupervised training of the NN is about to minimize the [https://en.wikipedia.org/wiki/Norm_(mathematics)#Euclidean_norm L2-norm] of the [https://en.wikipedia.org/wiki/Mean_squared_error mean squared error] loss of the value output and the policy loss. Further there are experiments to train the value head against not the game outcome, but against the accumulated value for a position after exploring some number of nodes with [[UCT]] <ref>[https://medium.com/oracledevs/lessons-from-alphazero-part-4-improving-the-training-target-6efba2e71628 Lessons From AlphaZero (part 4): Improving the Training Target] by [https://blogs.oracle.com/author/vish-abrams Vish Abrams], [https://blogs.oracle.com/ Oracle Blog], June 27, 2018</ref>.
  
=lc0=
+
The distributed training is realized with an sophisticated [https://en.wikipedia.org/wiki/Client%E2%80%93server_model client-server model].
Soon after the start of the project, some of the team led by [[Alexander Lyashuk]] started to rewrite the engine from scratch. The new engine dubbed '''lc0''' was able to search 4-8 times faster than lczero on [https://en.wikipedia.org/wiki/Nvidia Nvidia ]GPU’s <ref>[https://blog.lczero.org/2018/06/18/2-the-way-forward/ The Way Forward · Leela Chess Zero]</ref>. Training has changed to require lc0 instead of lczero <ref>[http://lczero.org/ LCZero]</ref>.
+
The client, written entirely in the [[Go (Programming Language)|Go programming language]], incorporates Lc0 to produce self-play games.
 +
Controlled by the server, the client may download the latest network, will start self-playing, and uploading games to the server, who on the other hand will regularly produce and distribute new neural network weights after a certain amount of games available from contributors.  
 +
The training software consists of [[Python]] code, the pipeline requires [https://en.wikipedia.org/wiki/NumPy NumPy] and [https://en.wikipedia.org/wiki/TensorFlow TensorFlow] running on [[Linux]] <ref>[https://github.com/LeelaChessZero/lczero-training GitHub - LeelaChessZero/lczero-training: For code etc relating to the network training process.]</ref>.  
 +
The server is written in [[Go (Programming Language)|Go]] along with [[Python]] and [https://en.wikipedia.org/wiki/Shell_script shell scripts].
  
 
=See also=
 
=See also=
 
* [[AlphaZero]]
 
* [[AlphaZero]]
 +
* [[Leela Zero]]
 
* [[Deep Learning]]
 
* [[Deep Learning]]
* [[UCT]]
+
* [[Monte-Carlo Tree Search]]
 +
: [[UCT]]
 
: [[Christopher D. Rosin#PUCT|PUCT]]
 
: [[Christopher D. Rosin#PUCT|PUCT]]
  
Line 17: Line 57:
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66280 Announcing lczero] by [[Gary Linscott|Gary]], [[CCC]], January 09, 2018
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66280 Announcing lczero] by [[Gary Linscott|Gary]], [[CCC]], January 09, 2018
 
: [http://www.talkchess.com/forum/viewtopic.php?t=66280&start=67 Re: Announcing lczero] by [[Daniel Shawul]], [[CCC]], January 21, 2018 » [[Bojun Huang#Rollout|Rollout Paradigm]]
 
: [http://www.talkchess.com/forum/viewtopic.php?t=66280&start=67 Re: Announcing lczero] by [[Daniel Shawul]], [[CCC]], January 21, 2018 » [[Bojun Huang#Rollout|Rollout Paradigm]]
 +
* [https://github.com/glinscott/leela-chess/issues/47 Policy and value heads are from AlphaGo Zero, not Alpha Zero Issue #47] by [[Gian-Carlo Pascutto]], [https://github.com/glinscott/leela-chess  glinscott/leela-chess · GitHub], January 24, 2018
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66452 LCZero is learning] by [[Gary Linscott|Gary]], [[CCC]], January 30, 2018
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66452 LCZero is learning] by [[Gary Linscott|Gary]], [[CCC]], January 30, 2018
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66824 LCZero update] by [[Gary Linscott|Gary]], [[CCC]], March 14, 2018
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66824 LCZero update] by [[Gary Linscott|Gary]], [[CCC]], March 14, 2018
Line 45: Line 86:
  
 
=Blog Posts=
 
=Blog Posts=
* [https://blog.lczero.org/ Leela Chess Zero - Blog]
+
* [https://medium.com/oracledevs/lessons-from-implementing-alphazero-7e36e9054191 Lessons From Implementing AlphaZero] by [https://medium.com/@akprasad Aditya Prasad], [https://blogs.oracle.com/ Oracle Blog], June 05, 2018
 +
: [https://medium.com/oracledevs/lessons-from-alphazero-connect-four-e4a0ae82af68 Lessons from AlphaZero: Connect Four] by [https://medium.com/@akprasad Aditya Prasad], [https://blogs.oracle.com/ Oracle Blog], June 13, 2018
 +
: [https://medium.com/oracledevs/lessons-from-alphazero-part-3-parameter-tweaking-4dceb78ed1e5 Lessons from AlphaZero (part 3): Parameter Tweaking] by [https://medium.com/@akprasad Aditya Prasad], [https://blogs.oracle.com/ Oracle Blog], June 20, 2018
 +
: [https://medium.com/oracledevs/lessons-from-alphazero-part-4-improving-the-training-target-6efba2e71628 Lessons From AlphaZero (part 4): Improving the Training Target] by [https://blogs.oracle.com/author/vish-abrams Vish Abrams], [https://blogs.oracle.com/ Oracle Blog], June 27, 2018
 +
: [https://medium.com/oracledevs/lessons-from-alpha-zero-part-5-performance-optimization-664b38dc509e Lessons From Alpha Zero (part 5): Performance Optimization] by [https://blogs.oracle.com/author/anthony-young Anthony Young], [https://blogs.oracle.com/ Oracle Blog], July 03, 2018
 +
: [https://medium.com/oracledevs/lessons-from-alpha-zero-part-6-hyperparameter-tuning-b1cfcbe4ca9a Lessons From Alpha Zero (part 6) — Hyperparameter Tuning] by  [https://blogs.oracle.com/author/anthony-young Anthony Young], [https://blogs.oracle.com/ Oracle Blog], July 11, 2018
 +
* [http://blog.lczero.org/2018/10/understanding-training-against-q-as.html Understanding Training against Q as Knowledge Distillation] by Cyanogenoid, [[Leela Chess Zero|LCZero blog]],
 
* [http://blog.lczero.org/2018/09/guide-setting-up-leela-on-chess-gui.html GUIDE: Setting up Leela on a Chess GUI] by Bob23, [[Leela Chess Zero|LCZero blog]], September 21, 2018
 
* [http://blog.lczero.org/2018/09/guide-setting-up-leela-on-chess-gui.html GUIDE: Setting up Leela on a Chess GUI] by Bob23, [[Leela Chess Zero|LCZero blog]], September 21, 2018
 
* [http://blog.lczero.org/2018/12/alphazero-paper-and-lc0-v0191.html AlphaZero paper, and Lc0 v0.19.1] by [[Alexander Lyashuk|crem]], [[Leela Chess Zero|LCZero blog]], December 07, 2018
 
* [http://blog.lczero.org/2018/12/alphazero-paper-and-lc0-v0191.html AlphaZero paper, and Lc0 v0.19.1] by [[Alexander Lyashuk|crem]], [[Leela Chess Zero|LCZero blog]], December 07, 2018
Line 51: Line 98:
 
=External Links=
 
=External Links=
 
==Chess Engine==
 
==Chess Engine==
 +
* [https://en.wikipedia.org/wiki/Leela_Chess_Zero Leela Chess Zero from Wikipedia]
 
* [http://lczero.org/ LCZero]
 
* [http://lczero.org/ LCZero]
 +
* [https://github.com/LeelaChessZero/lczero GitHub - LeelaChessZero/lczero: A chess adaption of GCP's Leela Zero]
 
* [https://github.com/LeelaChessZero/ LCZero · GitHub]
 
* [https://github.com/LeelaChessZero/ LCZero · GitHub]
 +
* [https://github.com/LeelaChessZero/lc0 GitHub - LeelaChessZero/lc0: The rewritten engine, originally for tensorflow. Now all other backends have been ported here]
 
* [https://github.com/LeelaChessZero/lc0/wiki Home · LeelaChessZero/lc0 Wiki · GitHub]
 
* [https://github.com/LeelaChessZero/lc0/wiki Home · LeelaChessZero/lc0 Wiki · GitHub]
 
* [https://github.com/LeelaChessZero/lc0/wiki/FAQ FAQ · LeelaChessZero/lc0 Wiki · GitHub]
 
* [https://github.com/LeelaChessZero/lc0/wiki/FAQ FAQ · LeelaChessZero/lc0 Wiki · GitHub]
* [https://github.com/LeelaChessZero/lc0 GitHub - LeelaChessZero/lc0: The rewritten engine, originally for tensorflow. Now all other backends have been ported here]
+
* [https://github.com/LeelaChessZero/lc0/wiki/Technical-Explanation-of-Leela-Chess-Zero Technical Explanation of Leela Chess Zero · LeelaChessZero/lc0 Wiki · GitHub]
 
* [https://github.com/mooskagh/lc0 GitHub - mooskagh/lc0: The rewritten engine, originally for cudnn. Now all other backends have been ported here]
 
* [https://github.com/mooskagh/lc0 GitHub - mooskagh/lc0: The rewritten engine, originally for cudnn. Now all other backends have been ported here]
* [https://github.com/LeelaChessZero/lczero GitHub - LeelaChessZero/lczero: A chess adaption of GCP's Leela Zero]
+
* [https://blog.lczero.org/ Leela Chess Zero - Blog]
 
* [https://groups.google.com/forum/#!forum/lczero LCZero – Google Groups]
 
* [https://groups.google.com/forum/#!forum/lczero LCZero – Google Groups]
 
* [https://www.facebook.com/LeelaChessZero/ Leela Chess Zero - Facebook]
 
* [https://www.facebook.com/LeelaChessZero/ Leela Chess Zero - Facebook]
* [https://en.wikipedia.org/wiki/Leela_Chess_Zero Leela Chess Zero from Wikipedia]
+
* [https://twitter.com/leelachesszero Leela Chess Zero (@LeelaChessZero) | Twitter]
 
* [https://en.chessbase.com/post/leela-chess-zero-alphazero-for-the-pc Leela Chess Zero: AlphaZero for the PC] by [[Albert Silver]], [[ChessBase|ChessBase News]], April 26, 2018
 
* [https://en.chessbase.com/post/leela-chess-zero-alphazero-for-the-pc Leela Chess Zero: AlphaZero for the PC] by [[Albert Silver]], [[ChessBase|ChessBase News]], April 26, 2018
 
==Misc==
 
==Misc==
Line 69: Line 119:
 
=References=  
 
=References=  
 
<references />
 
<references />
 
 
'''[[Engines|Up one level]]'''
 
'''[[Engines|Up one level]]'''
 
[[Category:UCI]]
 
[[Category:UCI]]

Revision as of 14:16, 6 January 2019

Home * Engines * Leela Chess Zero

Lc0 logo [1]

Leela Chess Zero, (LCZero, lc0)
an adaption of Gian-Carlo Pascutto's Leela Zero Go project [2] to Chess, initiated and announced by Stockfish co-author Gary Linscott, who was already responsible for the Stockfish Testing Framework called Fishtest. Leela Chess is open source, released under the terms of GPL version 3 or later, and supports UCI. The goal is to build a strong chess playing entity following the same type of deep learning along with Monte-Carlo tree search (MCTS) techniques of AlphaZero as described in DeepMind's 2017 and 2018 papers [3] [4] [5], but using distributed training for the weights of the deep convolutional neural network (CNN, DNN, DCNN).

Lc0

Leela Chess Zero consists of an executable to play or analyze games, initially dubbed LCZero, soon rewritten by a team around Alexander Lyashuk for better performance and then called Lc0 [6]. This executable, the actual chess engine, performs the MCTS and reads the self-taught CNN, which weights are persistent in a separate file. Lc0 is written in C++14 and may be compiled for various platforms and backends. Since deep CNN approaches are best suited to run massively in parallel on GPUs to perform all the floating point dot products for thousands of neurons, the preferred target platforms are Nvidia GPU’s supporting CUDA and cuDNN libraries [7]. None CUDA compliant GPUs (AMD) are supported through OpenCL, while much slower pure CPU binaries are possible using BLAS, target systems with or without a graphics card (GPU) are Linux, Mac OS and Windows computers, or BLAS only the Raspberry Pi.

Description

Like AlphaZero, Lc0's evaluates positions using non-linear function approximation based on a deep neural network, rather than the linear function approximation as used in classical chess programs. This neural network takes the board position as input and outputs position evaluation (QValue) and a vector of move probabilities (PValue, policy). Once trained, these network is combined with a Monte-Carlo Tree Search (MCTS) using the policy to narrow down the search to high­probability moves, and using the value in conjunction with a fast rollout policy to evaluate positions in the tree. The MCTS selection is done by a variation of Rosin's UCT improvement dubbed PUCT (Predictor + UCT).

Board Representation

Lc0's color agnostic board is represented by five bitboards (own pieces, opponent pieces, orthogonal sliding pieces, diagonal sliding pieces, and pawns including en passant target information coded as pawns on rank 1 and 8), two king squares, casting rights, and a flag whether the board is color flipped. While the structure is suitable as input for the neural network, getting individual pieces bitboards requires some setwise operations such as intersection and set theoretic difference [8].

Network

While AlphaGo used two disjoint networks for policy and value, AlphaZero as well as Leela Chess Zero, share a common "body" connected to disjoint policy and value "heads". The “body” consists of spatial 8x8 input planes, followed by convolutional layers with B residual blocks times 3x3xF filters. BxF specifies the model and size of the CNN (64x6, 128x10, 192x15, 256x20 were used). Concerning nodes per second of the MCTS, smaller models are faster to calculate than larger models. They are faster to train and one may earlier recognize progress, but they will also saturate earlier so that at some point more training will no longer improve the engine. Larger and deeper network models will improve the receptivity, the amount of knowledge and pattern to extract from the training samples, with potential for a stronger engine. As a further improvement, Leele Chess Zero applies the Squeeze and Excite (SE) extension to the residual block architecture [9] [10]. The body is fully connected to both the policy "head" for the move probability distribution, and value "head" for the evaluation score aka winning probability of the the the current positions and up to seven predecessor positions on the input planes.

Training

Like in AlphaZero, the Zero suffix implies no other initial knowledge than the rules of the games, to build a superhuman player, starting with truly random self-play games to apply reinforcement learning based on the outcome of that games. However, there are derived approaches, such as Albert Silver's Deus X, trying to take a short-cut by initially using supervised learning techniques, such as feeding in high quality games played by other strong chess playing entities, or huge records of positions with a given preferred move. The unsupervised training of the NN is about to minimize the L2-norm of the mean squared error loss of the value output and the policy loss. Further there are experiments to train the value head against not the game outcome, but against the accumulated value for a position after exploring some number of nodes with UCT [11].

The distributed training is realized with an sophisticated client-server model. The client, written entirely in the Go programming language, incorporates Lc0 to produce self-play games. Controlled by the server, the client may download the latest network, will start self-playing, and uploading games to the server, who on the other hand will regularly produce and distribute new neural network weights after a certain amount of games available from contributors. The training software consists of Python code, the pipeline requires NumPy and TensorFlow running on Linux [12]. The server is written in Go along with Python and shell scripts.

See also

UCT
PUCT

Forum Posts

2018

Re: Announcing lczero by Daniel Shawul, CCC, January 21, 2018 » Rollout Paradigm
LCZero update (2) by Rein Halbersma, CCC, March 25, 2018
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X by Gian-Carlo Pascutto, CCC, August 03, 2018
Re: Has Silver written any code for "his" ZeusX? by Alexander Lyashuk, LCZero Forum, August 02, 2018

2019

Blog Posts

Lessons from AlphaZero: Connect Four by Aditya Prasad, Oracle Blog, June 13, 2018
Lessons from AlphaZero (part 3): Parameter Tweaking by Aditya Prasad, Oracle Blog, June 20, 2018
Lessons From AlphaZero (part 4): Improving the Training Target by Vish Abrams, Oracle Blog, June 27, 2018
Lessons From Alpha Zero (part 5): Performance Optimization by Anthony Young, Oracle Blog, July 03, 2018
Lessons From Alpha Zero (part 6) — Hyperparameter Tuning by Anthony Young, Oracle Blog, July 11, 2018

External Links

Chess Engine

Misc

References

Up one level