NNUE

Home * Learning * Neural Networks * NNUE



NNUE, (&#398;U&#1048;&#1048; Efficiently Updatable Neural Networks) a Neural Network architecture intended to replace the evaluation of Shogi, chess and other board game playing alpha-beta searchers running on a CPU. Inspired by Kunihito Hoki's approach of piece-square tables indexed by king location, and further two-piece locations and side to move as applied in his Shogi engine Bonanza, NNUE was introduced in 2018 by Yu Nasu , and was used in Shogi adaptations of Stockfish such as YaneuraOu , and Kristallweizen, apparently with AlphaZero strength.

=Stockfish NNUE= As reported by Henk Drost , Nodchip incorporated NNUE into the chess playing Stockfish 10 as a proof of concept. Stockfish NNUE was born, and in summer 2020 the computer chess community bursts out enthusiastically due to its rapidly raising playing strength with different networks trained using a mixture of supervised and reinforcement learning methods - despite the approximately halved search speed, becoming stronger than its original, finally responsible for the huge strength improvement of Stockfish 12.

=NNUE Engines= see Category:NNUE

Being tempted by the success of Stockfish NNUE and attracted by how easy the method and small the code is, many engine developers have started testing and applying NNUE. For quick trials and evaluating before going into serious development, some of them borrowed and/or rewrote NNUE code and use networks from Stockfish NNUE. Most of them reported positive results, such as David Carteau with Orion, Ehsan Rashid with DON , various Stockfish derivatives by Michael Byrne , and Volodymyr Shcherbyna with Igel using the Night Nurse NNUE net by Dietrich Kappe. Daniel Shawul added NNUE support à la CFish into his egbbdll probing library of Scorpio, making it even easier to use NNUE. The promising engines Halogen 7 and 8 by Kieren Pearson, and Seer by Connor McMonigle came with their own, distinct NNUE implementations, and on November 10, 2020, the commercial Dragon by Komodo Chess aka Komodo NNUE appeared, trying to close the gap to Stockfish NNUE. The commercial Fat Fritz 2.0, based on a slightly modified Stockfish 12 using a customized, double sized network, was released by ChessBase in February 2021.

=NN Structure= The neural network of Stockfish NNUE consists of four layers, W1 through W4. The input layer W1 is heavily overparametrized, feeding in the board representation for various king configurations. The efficiency of the net is due to incremental update of W1 in make and unmake move, where only a fraction of its neurons need to be recalculated. The remaining three layers with 32x2x256, 32x32 and 32x1 weights are computational less expensive, best calculated using appropriate SIMD instructions like AVX2 on x86-64, or if available, AVX-512.

NNUE structure with incremental update

=See also=
 * Cerebrum
 * SANE
 * Stockfish HalfKAv2

=Publications=
 * Yu Nasu (2018). &#398;U&#1048;&#1048; Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi. Ziosoft Computer Shogi Club, pdf, pdf (Japanese with English abstract) GitHub - asdfjkl/nnue translation
 * Dominik Klein (2021). Neural Networks For Chess. Release Version 1.1 · GitHub

=Forum Posts=

January ...

 * The Stockfish of shogi by Larry Kaufman, CCC, January 07, 2020 » Stockfish, Shogi
 * Re: The Stockfish of shogi by Gian-Carlo Pascutto, CCC, January 18, 2020


 * Stockfish NNUE by Henk Drost, CCC, May 31, 2020 » Stockfish
 * Stockfish NN release (NNUE) by Henk Drost, CCC, May 31, 2020
 * NNUE shared library and tools by Adam Treat, CCC, June 10, 2020

July

 * Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation... by Nancy M Pichardo, CCC, July 15, 2020
 * Can the sardine! NNUE clobbers SF by Henk Drost, CCC, July 16, 2020
 * NNUE accessible explanation by Martin Fierz, CCC, July 21, 2020
 * Re: NNUE accessible explanation by Jonathan Rosenthal, CCC, July 23, 2020
 * Re: NNUE accessible explanation by Jonathan Rosenthal, CCC, July 24, 2020
 * Re: NNUE accessible explanation by Jonathan Rosenthal, CCC, August 03, 2020


 * BrainLearn NNUE 1.0 by Andrea Manzo, FishCooking, July 25, 2020 » BrainLearn
 * ShashChess NNUE 1.0 by Andrea Manzo, FishCooking, July 25, 2020 » ShashChess
 * LC0 vs. NNUE - some tech details... by Srdja Matovic, CCC, July 29, 2020 » Lc0
 * What does NNUE actually mean by Paloma, CCC, July 29, 2020

August

 * What happens with my hyperthreading? by Kai Laskos, CCC, August 06, 2020 » Stockfish NNUE
 * Re: Minic version 2 by Vivien Clauzon, CCC, August 08, 2020 » Minic
 * Neural Networks weights type by Fabio Gobbato, CCC, August 13, 2020 » Stockfish NNUE
 * Re: Introducing Igel chess engine - Igel and NNUE by Volodymyr Shcherbyna, CCC, August 19, 2020 » Igel
 * Orion 0.7 : NNUE experiment by David Carteau, CCC, August 19, 2020 » Orion
 * Night Nurse 0.2 by Dietrich Kappe, CCC, August 19, 2020 » A0lite, Igel
 * NNUE by Bert Tuyt, World Draughts Forum, August 19, 2020 » Draughts

September

 * Train a neural network evaluation by Fabio Gobbato, CCC, September 01, 2020 » Automated Tuning
 * RubiChess NNUE player implemented by Andreas Matthies, CCC, September 06, 2020 » RubiChess
 * Toga III 0.4 NNUE by Dietrich Kappe, CCC, September 07, 2020 » Toga
 * Neural network quantization by Fabio Gobbato, CCC, September 08, 2020 » Neural Networks
 * AVX-512 and NNUE by Gian-Carlo Pascutto, CCC, September 08, 2020 » AVX-512
 * First success with neural nets by Jonathan Kreuzer, CCC, September 23, 2020 » Neural Networks
 * Re: First success with neural nets by Jonathan Kreuzer, CCC, November 11, 2020 » Checkers


 * Nemorino 6 (NNUE) by Florentino, CCC, September 28, 2020 » Nemorino
 * A Crossroad in Computer Chess; Or Desperate Flailing for Relevance by Andrew Grant, CCC, September 29, 2020
 * NNUE variation by Ed Schröder, CCC, September 29, 2020

October

 * BONA_PIECE_ZERO by elcabesa, CCC, October 04, 2020
 * Re: Final Release of Ethereal, V12.75 by Andrew Grant, CCC, October 09, 2020 » Ethereal
 * Request for someone to train an NNUE for Ethereal by Andrew Grant, CCC, October 09, 2020
 * Ethereal Tuning - Data Dump by Andrew Grant, CCC, October 10, 2020
 * Dangerous turn by Dann Corbit, CCC, October 10, 2020
 * Black crushing white, weird ? by Vivien Clauzon, CCC, October 14, 2020 » MinicNNUE
 * Hacking around CFish NNUE by Maksim Korzh, CCC, October 15, 2020 » CFish
 * Re: Hacking around CFish NNUE by Daniel Shawul, CCC, October 15, 2020 » Scorpio


 * How to scale stockfish NNUE score? by Maksim Korzh, CCC, October 17, 2020
 * Re: How to scale stockfish NNUE score? by Daniel Shawul, CCC, October 17, 2020


 * Embedding Stockfish NNUE to ANY CHESS ENGINE: YouTube series by Maksim Korzh, CCC, October 17, 2020 » BBC
 * Seer by Gerd Isenberg, CCC, October 18, 2020 » Seer
 * BBC 1.3 + Stockfish NNUE released! by Maksim Korzh, CCC, October 21, 2020 » BBC
 * Mayhem NNUE - New NN engine by JohnWoe, CCC, October 22, 2020 » Mayhem
 * Centipawns vs Millipawns with NNUE by Madeleine Birchfield, CCC, October 23, 2020 » Centipawns, Millipawns
 * NNUE Question - King Placements by Andrew Grant, CCC, October 23, 2020 » Stockfish NNUE - NNUE Structure
 * July 01, 2021 continuation

November

 * Komodo 14.1 Release and Dragon Announcement by Larry Kaufman, CCC, November 02, 2020 » Komodo
 * NNUE outer product vs tensor product by Madeleine Birchfield, CCC, November 02, 2020
 * Pytorch NNUE training by Gary Linscott, CCC, November 08, 2020
 * TucaNNo: neural network research by Alcides Schulz, CCC, November 08, 2020
 * Dragon by Komodo Chess by Larry Kaufman, CCC, November 10, 2020 » Dragon by Komodo Chess
 * Tensorflow NNUE training by Daniel Shawul, CCC, November 10, 2020
 * Speculations about NNUE development (was New engine releases 2020) by Madeleine Birchfield, CCC, November 11, 2020
 * Re: Speculations about NNUE development (was New engine releases 2020) by Connor McMonigle, CCC, November 12, 2020
 * Re: Speculations about NNUE development (was New engine releases 2020) by Connor McMonigle, CCC, November 12, 2020 » Dragon by Komodo Chess, Seer, Halogen


 * Re: Final Release of Ethereal, V12.75 by Andrew Grant, CCC, November 12, 2020
 * Maybe not the best diversity of strongest chess engines under development by Kai Laskos, CCC, November 14, 2020 » Engine Similarity
 * CPU Vector Unit, the new jam for NNs... by Srdja Matovic, CCC, November 18, 2020 » SIMD
 * You've trained a brilliant NN(UE) King-Piece Network. Now what? by Andrew Grant, CCC, November 19, 2020
 * Pawn King Neural Network by Tamás Kuzmics, CCC, November 26, 2020

December

 * Orion 0.8 + The Cerebrum release by David Carteau, CCC, December 01, 2020 » Orion, Cerebrum
 * The NNUE split programmers are in by Ed Schröder, ProDeo Forum, December 02, 2020
 * Introducing the "Cerebrum" library (NNUE-like trainer and inference code) by David Carteau, CCC, December 07, 2020
 * Dispelling the Myth of NNUE with LazySMP: An Analysis by Andrew Grant, CCC, December 30, 2020 » Lazy SMP

January

 * Translation of Yu Nasu's NNUE paper by Dominik Klein, CCC, January 07, 2021
 * Re: Pytorch NNUE training by Gary Linscott, CCC, January 09, 2021
 * More experiments with neural nets by Jonathan Kreuzer, CCC, January 09, 2021 » Slow Chess
 * Shouldn't positional attributes drive SF's NNUE input features (rather than king position)? by Nick Pelling, FishCooking, January 10, 2021 » Stockfish NNUE
 * HalfKP Structure in NNUE by Roger Stephenson, CCC, January 12, 2021
 * Andscacs nnue 0.1 by Daniel José Queraltó, CCC, January 17, 2021 » Andscacs
 * It's NNUE era (sharing my thoughts) by Basti Dangca, CCC, January 18, 2021
 * NNUE and game phase by Dann Corbit, CCC, January 18, 2021 » Game Phases
 * correspondence chess in the age of NNUE by Larry Kaufman, CCC, January 21, 2021
 * One for Andrew Grant et al. - NNUE? by Srdja Matovic, CCC, January 21, 2021
 * 256 in NNUE? by Ted Wong, CCC, January 28, 2021
 * So what do we miss in the traditional evaluation? by Ferdinand Mosca, CCC, January 29, 2021 » Evaluation
 * Latest Night Nurse released by Dietrich Kappe, CCC, January 29, 2021
 * None-GPL NNUE probing code by Daniel Shawul, CCC, January 31, 2021

February

 * Fat Fritz 2 by Jouni Uski, CCC, February 09, 2021 » Fat Fritz 2.0
 * How much work is it to train an NNUE? by Gabor Szots, CCC, February 11, 2021
 * HCE and NNUE and vectorisation by Vivien Clauzon, CCC, February 11, 2021 » Minic
 * nnue reading code by Jon Dart, CCC, February 13, 2021
 * New net: The White Rose by Dietrich Kappe, CCC, February 20, 2021
 * Are neural nets (the weights file) copyrightable? by Adam Treat, CCC, February 21, 2021
 * My first NNUE nn-f0c1c3cbf2f1.nnue by MikeB, CCC, February 27, 2021
 * How to make a double-sized net as good as SF NNUE in a few easy steps by Chris Whittington, CCC, February 28, 2021 » Fat Fritz 2.0

March

 * A random walk down NNUE street … by MikeB, CCC, March 06, 2021
 * NNUE Research Project by Ed Schröder, CCC, March 10, 2021 » Engine Similarity
 * Simex including NNUE by jjoshua2, CCC, March 11, 2021 » Engine Similarity
 * NNUE ranking by Jim Logan, CCC, March 12, 2021 » Stockfish NNUE
 * FEN compression by lucasart, CCC, March 17, 2021 » FEN Compression, NNUE Training
 * Mabigat - hyperparameter optimizer for NNUE net by Ferdinand Mosca, CCC, March 22, 2021 » Automated Tuning
 * nnue-trainer by Jon Dart, CCC, March 27, 2021
 * Zeta with NNUE on GPU? by Srdja Matovic, CCC, March 31, 2021 » Zeta, GPU

April

 * Rubichess NN questions by Jon Dart, CCC, April 23, 2021 » RubiChess
 * Crafty NNUE Chess Engine? by supersharp77, CCC, April 29, 2021 » Crafty, Vafra

May

 * Komodo Dragon 2 released by Larry Kaufman, CCC, May 04, 2021 » Komodo Dragon
 * Stockfish with new NNUE architecture and bigger net released by Stefan Pohl, CCC, May 19, 2021 » Stockfish, Stockfish NNUE
 * NNUE scoring (egbb lib) by Desperado, CCC, May 19, 2021 » Scorpio NNUE

June

 * Re: Booot progress by Alex Morozov, CCC, June 01, 2021 » Booot
 * Commercial Release of Ethereal 13.00 (NNUE) for AVX2 Systems by Andrew Grant, CCC, June 04, 2021 » Ethereal 13.00 (NNUE)
 * Re: Commercial Release of Ethereal 13.00 (NNUE) for AVX2 Systems by Andrew Grant, CCC, June 04, 2021 » HalfKP


 * Dark Horse Update by Dietrich Kappe, CCC, June 11, 2021
 * Some more experiments with neural nets by Jonathan Kreuzer, CCC, June 15, 2021 » Slow Chess
 * Re: will Tcec allow Stockfish with a Leela net to play? by Connor McMonigle, CCC, June 17, 2021 » Stockfish, Komodo Dragon, Ethereal, Seer
 * Re: will Tcec allow Stockfish with a Leela net to play? by Daniel Shawul, CCC, June 18, 2021 » Scorpio
 * Re: will Tcec allow Stockfish with a Leela net to play? by Vivien Clauzon, CCC, June 18, 2021 » Minic


 * I declare that HCE is dead... by Andrew Grant, CCC, June 29, 2021 » Ethereal, HCE

July

 * Re: NNUE Question - King Placements by Tomasz Sobczyk, CCC, July 01, 2021 » NNUE Question
 * Re: NNUE Question - King Placements by Daniel Shawul, July 01, 2021 » ScorpioNNUE


 * Before things become more messy than they already are by Ed Schröder, CCC, July 02, 2021
 * NNUE training set generation by Edsel Apostol, CCC, July 03, 2021
 * Time to rethink what Albert Silver has done? by Srdja Matovic, CCC, July 03, 2021 » Fat Fritz 2
 * Would the ICGA have accepted today's NNUE engines? by Madeleine Birchfield, CCC, July 05, 2021 » ICGA
 * Koivisto 5.0 by Finn Eggers, CCC, July 07, 2021 » Koivisto
 * NNUE one year retrospective by Madeleine Birchfield, CCC, July 09, 2021

August ...

 * Basic NNUE questions by Amanj Sherwany, CCC, August 04, 2021
 * Alternatives to King-Pawn, King-All NNUE encoding by Andrew Grant, CCC, September 05, 2021
 * NNUE - Efficiently Updatable Network - understanding by Daniel Infuehr, CCC, October 11, 2021
 * NNUE - only from own engine? by Rebel, October 25, 2021
 * Regarding AVX2 by Rebel, CCC, November 03, 2021 » AVX2
 * Are NNUE Nets Specific to Chess Engines or They Universal to All Engines? by daniel71, CCC, December 26, 2021

2022 ...

 * Why NNUE trainer requires an online qsearch on each training position? by Chao Ma, CCC, January 01, 2022
 * Rebel 14 by Ed Schröder, CCC, January 12, 2022 » Rebel 14

=External Links=

NNUE

 * Efficiently updatable neural network | Wikipedia
 * 次世代の将棋思考エンジン、NNUE関数を学ぼう（その１．ネットワーク構造編） - コンピュータ将棋 Qhapaq, June 02, 2018 (Japanese)
 * Learn Next Generation Shogi Thinking Engine, NNUE Function (Part 1. Network Structure) - Computer Shogi


 * 次世代の将棋思考エンジン、NNUE関数を学ぼう（その2．改造/学習編） - コンピュータ将棋 Qhapaq, July 08, 2018 (Japanese)
 * Let's Learn Next Generation Shogi Thinking Engine, NNUE Function (Part 2. Remodeling/Learning) - Computer Shogi


 * Stockfish NNUE – The Complete Guide, June 19, 2020 (Japanese and English)
 * 3 technologies in shogi AI that could be used for chess AI by Motohiro Isozaki, August 21, 2020 » Stockfish NNUE
 * Stockfish NNUE Wiki
 * nnue | Home of the Dutch Rebel by Ed Schröder
 * NNUE Guide (nnue-pytorch/nnue.md at master · glinscott/nnue-pytorch · GitHub) hosted by Gary Linscott

NNUE libraries
Some developers disintegrate and rewrite the Stockfish NNUE code into independent libraries which can be much easier to embed into other chess engines.
 * GitHub - david-carteau/cerebrum: The Cerebrum library by David Carteau » Cerebrum
 * GitHub - dshawul/nncpu-probe by Daniel Shawul
 * GitHub - jdart1/nnue: NNUE reading code for chess by Jon Dart

Source Code

 * GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine
 * GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです.
 * GitHub - nodchip/Stockfish: UCI chess engine (Stockfish NNUE by Nodchip)
 * A Leela NNUE? Night Nurse and Others · dkappe/leela-chess-weights Wiki · GitHub by Dietrich Kappe
 * GitHub - DanielUranga/TensorFlowNNUE by Daniel Uranga
 * GitHub - glinscott/nnue-pytorch: NNUE (Chess evaluation) trainer in Pytorch by Gary Linscott
 * GitHub - connormcmonigle/seer-nnue: UCI chess engine using neural networks for position evaluation by Connor McMonigle » Seer
 * GitHub - bmdanielsson/nnue-trainer: PyTorch trainer for NNUE style neural networks by Martin Danielsson » Marvin
 * GitHub - fsmosca/Mabigat: NNUE parameter optimizer by Ferdinand Mosca » Automated Tuning

Misc

 * Lower Numerical Precision Deep Learning Inference and Training by Andres Rodriguez et al., Intel, January 19, 2018 » AVX-512
 * Nue from Wikipedia
 * Hiromi - Spectrum, 2019, YouTube Video

=References= Up one Level