Difference between revisions of "NNUE"
GerdIsenberg (talk | contribs) |
GerdIsenberg (talk | contribs) |
||
Line 113: | Line 113: | ||
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75724&start=60 Re: Pytorch NNUE training] by [[Gary Linscott]], [[CCC]], January 09, 2021 | * [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75724&start=60 Re: Pytorch NNUE training] by [[Gary Linscott]], [[CCC]], January 09, 2021 | ||
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76285 HalfKP Structure in NNUE] by Roger Stephenson, [[CCC]], January 12, 2021 | * [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76285 HalfKP Structure in NNUE] by Roger Stephenson, [[CCC]], January 12, 2021 | ||
+ | * [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76346 Andscacs nnue 0.1] by [[Daniel José Queraltó]], [[CCC]], January 17, 2021 » [[Andscacs]] | ||
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76356 NNUE and game phase] by [[Dann Corbit]], [[CCC]], January 18, 2021 » [[Game Phases]] | * [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76356 NNUE and game phase] by [[Dann Corbit]], [[CCC]], January 18, 2021 » [[Game Phases]] | ||
Revision as of 21:55, 19 January 2021
Home * Learning * Neural Networks * NNUE
NNUE, (ƎUИИ Efficiently Updatable Neural Networks)
a Neural Network architecture intended to replace the evaluation of Shogi, chess and other board game playing alpha-beta searchers running on a CPU.
NNUE was introduced in 2018 by Yu Nasu [3],
and was used in Shogi adaptations of Stockfish such as YaneuraOu [4],
and Kristallweizen [5], apparently with AlphaZero strength [6].
Contents
Stockfish NNUE
As reported by Henk Drost [7], Nodchip incorporated NNUE into the chess playing Stockfish 10 as a proof of concept. Stockfish NNUE was born, and in summer 2020 the computer chess community bursts out enthusiastically due to its rapidly raising playing strength with different networks trained using a mixture of supervised and reinforcement learning methods - despite the approximately halved search speed, becoming stronger than its original [8], finally responsible for the huge strength improvement of Stockfish 12.
NNUE Engines
see Category:NNUE
Being tempted by the success of Stockfish NNUE and attracted by how easy the method and small the code is, many engine developers have started testing and applying NNUE. For quick trials and evaluating before going into serious development, some of them borrowed and/or rewrote NNUE code and use networks from Stockfish NNUE. Most of them reported positive results, such as David Carteau with Orion [9], Ehsan Rashid with DON [10], various Stockfish derivatives by Michael Byrne [11], and Volodymyr Shcherbyna with Igel [12] using the Night Nurse NNUE net by Dietrich Kappe [13]. Daniel Shawul added NNUE support à la CFish into his egbbdll probing library of Scorpio [14] [15], making it even easier to use NNUE. The promising engines Halogen 7 and 8 by Kieren Pearson, and Seer by Connor McMonigle came with their own, distinct NNUE implementations, and on November 10, 2020, the commercial Dragon by Komodo Chess aka Komodo NNUE appeared [16], trying to close the gap to Stockfish NNUE.
NN Structure
The neural network of Stockfish NNUE consists of four layers, W1 through W4. The input layer W1 is heavily overparametrized, feeding in the board representation for various king configurations. The efficiency of the net is due to incremental update of W1 in make and unmake move, where only a fraction of its neurons need to be recalculated. The remaining three layers with 32x2x256, 32x32 and 32x1 weights are computational less expensive, best calculated using appropriate SIMD instructions like AVX2 on x86-64, or if available, AVX-512.
NNUE structure with incremental update [17]
See also
Publications
- Yu Nasu (2018). ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi. Ziosoft Computer Shogi Club, pdf, pdf (Japanese with English abstract) GitHub - asdfjkl/nnue translation [18]
Forum Posts
2020 ...
- The Stockfish of shogi by Larry Kaufman, CCC, January 07, 2020 » Stockfish, Shogi
- Re: The Stockfish of shogi by Gian-Carlo Pascutto, CCC, January 18, 2020
- Stockfish NNUE by Henk Drost, CCC, May 31, 2020 » Stockfish
- Stockfish NN release (NNUE) by Henk Drost, CCC, May 31, 2020
- NNUE shared library and tools by Adam Treat, CCC, June 10, 2020
July
- Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation... by Nancy M Pichardo, CCC, July 15, 2020
- Can the sardine! NNUE clobbers SF by Henk Drost, CCC, July 16, 2020
- NNUE accessible explanation by Martin Fierz, CCC, July 21, 2020
- Re: NNUE accessible explanation by Jonathan Rosenthal, CCC, July 23, 2020
- Re: NNUE accessible explanation by Jonathan Rosenthal, CCC, July 24, 2020
- Re: NNUE accessible explanation by Jonathan Rosenthal, CCC, August 03, 2020
- BrainLearn NNUE 1.0 by Andrea Manzo, FishCooking, July 25, 2020 » BrainLearn
- ShashChess NNUE 1.0 by Andrea Manzo, FishCooking, July 25, 2020 » ShashChess
- LC0 vs. NNUE - some tech details... by Srdja Matovic, CCC, July 29, 2020 » Lc0
- What does NNUE actually mean by Paloma, CCC, July 29, 2020
August
- What happens with my hyperthreading? by Kai Laskos, CCC, August 06, 2020 » Stockfish NNUE
- Re: Minic version 2 by Vivien Clauzon, CCC, August 08, 2020 » Minic
- Neural Networks weights type by Fabio Gobbato, CCC, August 13, 2020 » Stockfish NNUE
- Re: Introducing Igel chess engine - Igel and NNUE by Volodymyr Shcherbyna, CCC, August 19, 2020 » Igel
- Orion 0.7 : NNUE experiment by David Carteau, CCC, August 19, 2020 » Orion
- Night Nurse 0.2 by Dietrich Kappe, CCC, August 19, 2020 » A0lite, Igel
- NNUE by Bert Tuyt, World Draughts Forum, August 19, 2020 » Draughts
September
- Train a neural network evaluation by Fabio Gobbato, CCC, September 01, 2020 » Automated Tuning
- RubiChess NNUE player implemented by Andreas Matthies, CCC, September 06, 2020 » RubiChess
- Toga III 0.4 NNUE by Dietrich Kappe, CCC, September 07, 2020 » Toga
- Neural network quantization by Fabio Gobbato, CCC, September 08, 2020 » Neural Networks
- AVX-512 and NNUE by Gian-Carlo Pascutto, CCC, September 08, 2020 » AVX-512
- First success with neural nets by Jonathan Kreuzer, CCC, September 23, 2020 » Neural Networks
- Re: First success with neural nets by Jonathan Kreuzer, CCC, November 11, 2020 » Checkers
- Nemorino 6 (NNUE) by Florentino, CCC, September 28, 2020 » Nemorino
- A Crossroad in Computer Chess; Or Desperate Flailing for Relevance by Andrew Grant, CCC, September 29, 2020
- NNUE variation by Ed Schröder, CCC, September 29, 2020
October
- BONA_PIECE_ZERO by elcabesa, CCC, October 04, 2020
- Re: Final Release of Ethereal, V12.75 by Andrew Grant, CCC, October 09, 2020 » Ethereal
- Request for someone to train an NNUE for Ethereal by Andrew Grant, CCC, October 09, 2020
- Ethereal Tuning - Data Dump by Andrew Grant, CCC, October 10, 2020
- Dangerous turn by Dann Corbit, CCC, October 10, 2020
- Black crushing white, weird ? by Vivien Clauzon, CCC, October 14, 2020 » MinicNNUE
- Hacking around CFish NNUE by Maksim Korzh, CCC, October 15, 2020 » CFish
- Re: Hacking around CFish NNUE by Daniel Shawul, CCC, October 15, 2020 » Scorpio
- How to scale stockfish NNUE score? by Maksim Korzh, CCC, October 17, 2020
- Re: How to scale stockfish NNUE score? by Daniel Shawul, CCC, October 17, 2020
- Embedding Stockfish NNUE to ANY CHESS ENGINE: YouTube series by Maksim Korzh, CCC, October 17, 2020 » BBC
- Seer by Gerd Isenberg, CCC, October 18, 2020 » Seer
- BBC 1.3 + Stockfish NNUE released! by Maksim Korzh, CCC, October 21, 2020 » BBC
- Mayhem NNUE - New NN engine by JohnWoe, CCC, October 22, 2020 » Mayhem NNUE
- Centipawns vs Millipawns with NNUE by Madeleine Birchfield, CCC, October 23, 2020 » Centipawns, Millipawns
- NNUE Question - King Placements by Andrew Grant, CCC, October 23, 2020 » Stockfish NNUE - NNUE Structure
November
- Komodo 14.1 Release and Dragon Announcement by Larry Kaufman, CCC, November 02, 2020 » Komodo
- NNUE outer product vs tensor product by Madeleine Birchfield, CCC, November 02, 2020 [19] [20]
- Pytorch NNUE training by Gary Linscott, CCC, November 08, 2020 [21]
- TucaNNo: neural network research by Alcides Schulz, CCC, November 08, 2020
- Dragon by Komodo Chess by Larry Kaufman, CCC, November 10, 2020 » Dragon by Komodo Chess
- Tensorflow NNUE training by Daniel Shawul, CCC, November 10, 2020 [22]
- Speculations about NNUE development (was New engine releases 2020) by Madeleine Birchfield, CCC, November 11, 2020
- Re: Speculations about NNUE development (was New engine releases 2020) by Connor McMonigle, CCC, November 12, 2020
- Re: Speculations about NNUE development (was New engine releases 2020) by Connor McMonigle, CCC, November 12, 2020 » Dragon by Komodo Chess, Seer, Halogen
- Re: Final Release of Ethereal, V12.75 by Andrew Grant, CCC, November 12, 2020
- Maybe not the best diversity of strongest chess engines under development by Kai Laskos, CCC, November 14, 2020 » Engine Similarity
- CPU Vector Unit, the new jam for NNs... by Srdja Matovic, CCC, November 18, 2020 » SIMD
- You've trained a brilliant NN(UE) King-Piece Network. Now what? by Andrew Grant, CCC, November 19, 2020
- Pawn King Neural Network by Tamás Kuzmics, CCC, November 26, 2020
December
- Orion 0.8 + The Cerebrum release by David Carteau, CCC, December 01, 2020 » Orion, Cerebrum
- The NNUE split programmers are in by Ed Schröder, ProDeo Forum, December 02, 2020
- Introducing the "Cerebrum" library (NNUE-like trainer and inference code) by David Carteau, CCC, December 07, 2020
- Dispelling the Myth of NNUE with LazySMP: An Analysis by Andrew Grant, CCC, December 30, 2020 » Lazy SMP
2021
- Translation of Yu Nasu's NNUE paper by Dominik Klein, CCC, January 07, 2021
- Re: Pytorch NNUE training by Gary Linscott, CCC, January 09, 2021
- HalfKP Structure in NNUE by Roger Stephenson, CCC, January 12, 2021
- Andscacs nnue 0.1 by Daniel José Queraltó, CCC, January 17, 2021 » Andscacs
- NNUE and game phase by Dann Corbit, CCC, January 18, 2021 » Game Phases
External Links
NNUE
- Efficiently updatable neural network | Wikipedia
- 次世代の将棋思考エンジン、NNUE関数を学ぼう(その1.ネットワーク構造編) - コンピュータ将棋 Qhapaq, June 02, 2018 (Japanese)
- Learn Next Generation Shogi Thinking Engine, NNUE Function (Part 1. Network Structure) - Computer Shogi
- 次世代の将棋思考エンジン、NNUE関数を学ぼう(その2.改造/学習編) - コンピュータ将棋 Qhapaq, July 08, 2018 (Japanese)
- Let's Learn Next Generation Shogi Thinking Engine, NNUE Function (Part 2. Remodeling/Learning) - Computer Shogi
- Stockfish NNUE – The Complete Guide, June 19, 2020 (Japanese and English)
- 3 technologies in shogi AI that could be used for chess AI by Motohiro Isozaki, August 21, 2020 » Stockfish NNUE
- Stockfish NNUE Wiki
Source Code
- GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine
- GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです。
- GitHub - nodchip/Stockfish: UCI chess engine (Stockfish NNUE by Nodchip)
- A Leela NNUE? Night Nurse and Others · dkappe/leela-chess-weights Wiki · GitHub by Dietrich Kappe
- GitHub - david-carteau/cerebrum: The Cerebrum library by David Carteau » Cerebrum
- GitHub - DanielUranga/TensorFlowNNUE by Daniel Uranga
- GitHub - glinscott/nnue-pytorch: NNUE (Chess evaluation) trainer in Pytorch by Gary Linscott
- GitHub - connormcmonigle/seer-nnue: UCI chess engine using neural networks for position evaluation by Connor McMonigle » Seer
- GitHub - bmdanielsson/nnue-trainer: PyTorch trainer for NNUE style neural networks by Martin Danielsson » Marvin
Misc
References
- ↑ Nue (鵺) from the Konjaku Gazu Zoku Hyakki (今昔画図続百鬼) by Toriyama Sekien, circa 1779, Wikimedia Commons
- ↑ Re: What does NNUE actually mean by Henk Drost, CCC, July 29, 2020
- ↑ Yu Nasu (2018). ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi. Ziosoft Computer Shogi Club, pdf (Japanese with English abstract)
- ↑ GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine
- ↑ GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです。
- ↑ The Stockfish of shogi by Larry Kaufman, CCC, January 07, 2020
- ↑ Stockfish NN release (NNUE) by Henk Drost, CCC, May 31, 2020
- ↑ Can the sardine! NNUE clobbers SF by Henk Drost, CCC, July 16, 2020
- ↑ Orion 0.7 : NNUE experiment by David Carteau, CCC, August 19, 2020
- ↑ Re: New engine releases 2020...Don NNUE 2020? by supersharp77, CCC, August 19, 2020
- ↑ ... the last shall be first ... by MikeB, CCC, 19 Aug 2020
- ↑ Introducing Igel chess engine by Volodymyr Shcherbyna, CCC, 20 Aug 2020
- ↑ Night Nurse 0.2 by Dietrich Kappe, CCC, August 19, 2020
- ↑ Re: Hacking around CFish NNUE by Daniel Shawul, CCC, October 15, 2020
- ↑ Re: How to scale stockfish NNUE score? by Daniel Shawul, CCC, October 17, 2020
- ↑ Dragon by Komodo Chess by Larry Kaufman, CCC, November 10, 2020
- ↑ Image from Yu Nasu (2018). ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi. Ziosoft Computer Shogi Club, pdf (Japanese with English abstract)
- ↑ Translation of Yu Nasu's NNUE paper by Dominik Klein, CCC, January 07, 2021
- ↑ Outer product from Wikipedia
- ↑ Tensor product from Wikipedia
- ↑ PyTorch from Wikipedia
- ↑ TensorFlow from Wikipedia