Difference between revisions of "NNUE"

From Chessprogramming wiki
Jump to: navigation, search
 
(141 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 
'''[[Main Page|Home]] * [[Learning]] * [[Neural Networks]] * NNUE'''
 
'''[[Main Page|Home]] * [[Learning]] * [[Neural Networks]] * NNUE'''
  
[[FILE:SekienNue.jpg|border|right|thumb|250px| [[:Category:Toriyama Sekien|Toriyama Sekien]] - Nue (鵺) <ref>[https://en.wikipedia.org/wiki/Nue Nue] (鵺) from the [https://en.wikipedia.org/wiki/Konjaku_Gazu_Zoku_Hyakki Konjaku Gazu Zoku Hyakki] (今昔画図続百鬼) by [[:Category:Toriyama Sekien|Toriyama Sekien]], circa 1779, [https://en.wikipedia.org/wiki/Wikimedia_Commons Wikimedia Commons]</ref> <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74611&start=2 Re: What does NNUE actually mean] by [[ Henk Drost]], [[CCC]], July 29, 2020</ref> ]]  
+
[[FILE:SekienNue.jpg|border|right|thumb|250px| [[:Category:Toriyama Sekien|Toriyama Sekien]] - Nue (鵺) <ref>[https://en.wikipedia.org/wiki/Nue Nue] (鵺) from the [https://en.wikipedia.org/wiki/Konjaku_Gazu_Zoku_Hyakki Konjaku Gazu Zoku Hyakki] (今昔画図続百鬼) by [[:Category:Toriyama Sekien|Toriyama Sekien]], circa 1779, [https://en.wikipedia.org/wiki/Wikimedia_Commons Wikimedia Commons]</ref> <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74611&start=2 Re: What does NNUE actually mean] by [[ Henk Drost]], [[CCC]], July 29, 2020</ref> ]]  
  
 
'''NNUE''', (&#398;U&#1048;&#1048; Efficiently Updatable Neural Networks)<br/>  
 
'''NNUE''', (&#398;U&#1048;&#1048; Efficiently Updatable Neural Networks)<br/>  
a Neural Network architecture intended to replace the [[Evaluation|evaluation]] of [[Shogi]], [[Chess|chess]] and other board game playing [[Alpha-Beta|alpha-beta]] searchers running on a CPU.
+
a Neural Network architecture intended to replace the [[Evaluation|evaluation]] of [[Shogi]], [[Chess|chess]] and other board game playing [[Alpha-Beta|alpha-beta]] searchers running on a CPU. Inspired by [[Kunihito Hoki|Kunihito Hoki's]] approach of [[Piece-Square Tables|piece-square tables]] indexed by king location, and further two-piece locations and side to move as applied in his Shogi engine [[Bonanza]] <ref>[http://yaneuraou.yaneu.com/2020/05/03/%E6%A9%9F%E6%A2%B0%E5%AD%A6%E7%BF%92%E3%82%A8%E3%83%B3%E3%82%B8%E3%83%8B%E3%82%A2%E3%81%AE%E3%81%9F%E3%82%81%E3%81%AE%E5%B0%86%E6%A3%8Bai%E9%96%8B%E7%99%BA%E5%85%A5%E9%96%80%E3%81%9D%E3%81%AE1/ 機械学習エンジニアのための将棋AI開発入門その1 Introduction to Shogi AI development for machine learning engineers Part 1], May 03, 2020 (Japanese)</ref>, '''NNUE''' was introduced in 2018 by [[Yu Nasu]] <ref>[[Yu Nasu]] ('''2018'''). ''&#398;U&#1048;&#1048; Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''.  Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)</ref>, and was used in Shogi adaptations of [[Stockfish]] such as [[YaneuraOu]] <ref>[https://github.com/yaneurao/YaneuraOu GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine]</ref>,
NNUE was introduced in 2018 by [[Yu Nasu]] <ref>[[Yu Nasu]] ('''2018'''). ''&#398;U&#1048;&#1048; Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''.  Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)</ref>,
+
and [[Kristallweizen]] <ref>[https://github.com/Tama4649/Kristallweizen/ GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです。]</ref>, apparently with [[AlphaZero]] strength <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754 The Stockfish of shogi] by [[Larry Kaufman]], [[CCC]], January 07, 2020</ref>.  
and was used in Shogi adaptations of [[Stockfish]] such as [[YaneuraOu]] <ref>[https://github.com/yaneurao/YaneuraOu GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine]</ref>,
 
and [[Kristallweizen-kai]] <ref>[https://github.com/Tama4649/Kristallweizen/ GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです。]</ref>, apparently with [[AlphaZero]] strength <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754 The Stockfish of shogi] by [[Larry Kaufman]], [[CCC]], January 07, 2020</ref>.  
 
  
==[[Stockfish NNUE]]==
+
=[[Stockfish NNUE]]=
 
As reported by [[Henk Drost]] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74059 Stockfish NN release (NNUE)] by [[Henk Drost]], [[CCC]], May 31, 2020</ref>,  
 
As reported by [[Henk Drost]] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74059 Stockfish NN release (NNUE)] by [[Henk Drost]], [[CCC]], May 31, 2020</ref>,  
[[Hisayori Noda|Nodchip]] incorporated NNUE into the chess playing Stockfish 10 as a proof of concept.
+
[[Hisayori Noda|Nodchip]] incorporated NNUE into the chess playing [[Stockfish]] 10 as a proof of concept.
 
[[Stockfish NNUE]] was born, and in summer 2020 the computer chess community bursts out enthusiastically due to its rapidly raising [[Playing Strength|playing strength]] with different networks trained using a mixture of [[Supervised Learning|supervised]] and [[Reinforcement Learning|reinforcement learning]] methods -
 
[[Stockfish NNUE]] was born, and in summer 2020 the computer chess community bursts out enthusiastically due to its rapidly raising [[Playing Strength|playing strength]] with different networks trained using a mixture of [[Supervised Learning|supervised]] and [[Reinforcement Learning|reinforcement learning]] methods -
despite the approximately halved search speed, seemingly becoming stronger than its original <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74484 Can the sardine! NNUE clobbers SF] by [[Henk Drost]], [[CCC]], July 16, 2020</ref>.  
+
despite the approximately halved search speed, becoming stronger than its original <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74484 Can the sardine! NNUE clobbers SF] by [[Henk Drost]], [[CCC]], July 16, 2020</ref>, finally responsible for the huge [[Playing Strength|strength]] improvement of '''Stockfish 12'''.
  
=NN Structure=
+
==NN Structure==
The neural network consists of four layers, W1 through W4. The input layer W1 is heavily overparametrized, feeding in the [[Board Representation|board representation]] for various king configurations.
+
The neural network of Stockfish NNUE consists of four layers, W1 through W4. The input layer W1 is heavily overparametrized, feeding in the [[Board Representation|board representation]] for various king configurations.
 
The efficiency of the net is due to [[Incremental Updates|incremental update]] of W1 in [[Make Move|make]] and [[Unmake Move|unmake move]],
 
The efficiency of the net is due to [[Incremental Updates|incremental update]] of W1 in [[Make Move|make]] and [[Unmake Move|unmake move]],
where only a fraction of its neurons need to be recalculated. The remaining three layers with 256x2x32-32x32-32x1 neurons are computational less expensive, best calculated using appropriate [[SIMD and SWAR Techniques|SIMD instructions]] like [[AVX2]] on [[x86-64]], or if available, [[AVX-512]].
+
where only a fraction of its neurons need to be recalculated. The remaining three layers with 32x2x256, 32x32 and 32x1 weights are computational less expensive, best calculated using appropriate [[SIMD and SWAR Techniques|SIMD instructions]] like [[AVX2]] on [[x86-64]], or if available, [[AVX-512]].
  
 
[[FILE:NNUE.jpg|none|border|text-bottom]]  
 
[[FILE:NNUE.jpg|none|border|text-bottom]]  
NNUE structure with [[Incremental Updates|incremental update]] <ref>Image from [[Yu Nasu]] ('''2018'''). ''&#398;U&#1048;&#1048; Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''.  Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)</ref>
+
NNUE structure with [[Incremental Updates|incremental updates]] <ref>Image from [[Yu Nasu]] ('''2018'''). ''&#398;U&#1048;&#1048; Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''.  Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)</ref>
 +
 
 +
=Adoption=
 +
''see [[:Category:NNUE]]''
 +
 
 +
Being tempted by the success of [[Stockfish NNUE]] and attracted by how easy the method and small the code is, many engine developers have started testing and applying [[NNUE]]. For quick trials and evaluating before going into serious development, some of them borrowed and/or rewrote NNUE code and use networks from Stockfish NNUE. Most of them reported positive results, such as [[David Carteau]] with [[Orion]] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74828 Orion 0.7 : NNUE experiment] by [[David Carteau]], [[CCC]], August 19, 2020</ref>, [[Ehsan Rashid]] with [[DON]] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72613&start=320#p856640 Re: New engine releases 2020...Don NNUE 2020?] by supersharp77, [[CCC]], August 19, 2020</ref>, various [[Stockfish#Derivatives|Stockfish derivatives]] by [[Michael Byrne]] <ref>[http://talkchess.com/forum3/viewtopic.php?f=2&t=74825 ... the last shall be first ...] by [[Michael Byrne|MikeB]], [[CCC]], 19 Aug 2020</ref>, and [[Volodymyr Shcherbyna]] with [[Igel]] <ref>[http://talkchess.com/forum3/viewtopic.php?f=2&t=67890&start=10#p856742 Introducing Igel chess engine] by [[Volodymyr Shcherbyna]], [[CCC]], 20 Aug 2020</ref> using the ''Night Nurse'' NNUE net by [[Dietrich Kappe]] <ref>[http://talkchess.com/forum3/viewtopic.php?f=2&t=74837 Night Nurse 0.2] by [[Dietrich Kappe]], [[CCC]], August 19, 2020</ref>. [[Daniel Shawul]] added NNUE support à la [[CFish]] into his [[Scorpio#Bitbases|egbbdll]] probing library of [[Scorpio]] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75400&start=22 Re: Hacking around CFish NNUE] by [[Daniel Shawul]], [[CCC]], October 15, 2020</ref> <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75415&start=3 Re: How to scale stockfish NNUE score?] by [[Daniel Shawul]], [[CCC]], October 17, 2020</ref>, making it even easier to use NNUE. The promising engines [[Halogen]] 7 and 8 by [[Kieren Pearson]], and [[Seer]] by [[Connor McMonigle]] came with their own, distinct NNUE implementations, and on November 10, 2020, the commercial [[Dragon by Komodo Chess]] aka [[Komodo]] NNUE appeared <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75748 Dragon by Komodo Chess] by [[Larry Kaufman]], [[CCC]], November 10, 2020</ref>, trying to close the gap to Stockfish NNUE. The commercial [[Fat Fritz#Fat Fritz 2|Fat Fritz 2.0]], based on a slightly modified Stockfish 12 using a customized, double sized network, was released by [[ChessBase]] in February 2021.
 +
 
 +
=See also=
 +
* [[Cerebrum]]
 +
* [[David E. Moriarty#SANE|SANE]]
 +
* [[Stockfish NNUE#HalfKA|Stockfish HalfKAv2]]
  
 
=Publications=
 
=Publications=
* [[Yu Nasu]] ('''2018'''). ''&#398;U&#1048;&#1048; Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''.  Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)
+
* [[Yu Nasu]] ('''2018'''). ''&#398;U&#1048;&#1048; Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''.  Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf], [https://www.apply.computer-shogi.org/wcsc28/appeal/the_end_of_genesis_T.N.K.evolution_turbo_type_D/nnue.pdf pdf] (Japanese with English abstract) [https://github.com/asdfjkl/nnue GitHub - asdfjkl/nnue translation] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76250 Translation of Yu Nasu's NNUE paper] by [[Dominik Klein]], [[CCC]], January 07, 2021</ref>
 +
* [[Dominik Klein]] ('''2021'''). ''[https://github.com/asdfjkl/neural_network_chess Neural Networks For Chess]''. [https://github.com/asdfjkl/neural_network_chess/releases/tag/v1.1 Release Version 1.1 · GitHub] <ref>[https://www.talkchess.com/forum3/viewtopic.php?f=2&t=78283 Book about Neural Networks for Chess] by dkl, [[CCC]], September 29, 2021</ref>
  
 
=Forum Posts=
 
=Forum Posts=
 +
==2020==
 +
===January ...===
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754 The Stockfish of shogi] by [[Larry Kaufman]], [[CCC]], January 07, 2020 » [[Stockfish]], [[Shogi]]
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754 The Stockfish of shogi] by [[Larry Kaufman]], [[CCC]], January 07, 2020 » [[Stockfish]], [[Shogi]]
 
: [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754&start=18 Re: The Stockfish of shogi] by [[Gian-Carlo Pascutto]], [[CCC]], January 18, 2020
 
: [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754&start=18 Re: The Stockfish of shogi] by [[Gian-Carlo Pascutto]], [[CCC]], January 18, 2020
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74058 Stockfish NNUE] by [[Henk Drost]], [[CCC]], May 31, 2020 » [[Stockfish]]
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74058 Stockfish NNUE] by [[Henk Drost]], [[CCC]], May 31, 2020 » [[Stockfish]]
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74059 Stockfish NN release (NNUE)] by [[Henk Drost]], [[CCC]], May 31, 2020
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74059 Stockfish NN release (NNUE)] by [[Henk Drost]], [[CCC]], May 31, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74148 NNUE shared library and tools] by [[Adam Treat]], [[CCC]], June 10, 2020
 +
===July===
 +
* [http://talkchess.com/forum3/viewtopic.php?t=74480 Lizard-NNUE Experiment NOT bad with NNUE Net Evaluation...] by Nancy M Pichardo, [[CCC]], July 15, 2020
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74484 Can the sardine! NNUE clobbers SF] by [[Henk Drost]], [[CCC]], July 16, 2020
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74484 Can the sardine! NNUE clobbers SF] by [[Henk Drost]], [[CCC]], July 16, 2020
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531 NNUE accessible explanation] by [[Martin Fierz]], [[CCC]], July 21, 2020
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531 NNUE accessible explanation] by [[Martin Fierz]], [[CCC]], July 21, 2020
Line 36: Line 50:
 
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=5 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], July 24, 2020
 
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=5 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], July 24, 2020
 
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=8 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], August 03, 2020
 
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=8 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], August 03, 2020
 +
* [https://groups.google.com/d/msg/fishcooking/Wpk-9COzk64/ez643VTkAAAJ BrainLearn NNUE 1.0] by [[Andrea Manzo]], [[Computer Chess Forums|FishCooking]], July 25, 2020 » [[BrainLearn]]
 +
* [https://groups.google.com/d/msg/fishcooking/yWtpz_FY5_Y/RMTG56fkAAAJ ShashChess NNUE 1.0] by [[Andrea Manzo]], [[Computer Chess Forums|FishCooking]], July 25, 2020 » [[ShashChess]]
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74607 LC0 vs. NNUE - some tech details...] by [[Srdja Matovic]], [[CCC]], July 29, 2020 » [[Leela Chess Zero#Lc0|Lc0]]
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74607 LC0 vs. NNUE - some tech details...] by [[Srdja Matovic]], [[CCC]], July 29, 2020 » [[Leela Chess Zero#Lc0|Lc0]]
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74611 What does NNUE actually mean] by Paloma, [[CCC]], July 29, 2020
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74611 What does NNUE actually mean] by Paloma, [[CCC]], July 29, 2020
 +
===August===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74705 What happens with my hyperthreading?] by [[Kai Laskos]], [[CCC]], August 06, 2020 » [[Stockfish NNUE]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=73521&start=59 Re: Minic version 2] by [[Vivien Clauzon]], [[CCC]], August 08, 2020 » [[Minic]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74777 Neural Networks weights type] by [[Fabio Gobbato]], [[CCC]], August 13, 2020 » [[Stockfish NNUE]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=67890&start=17 Re: Introducing Igel chess engine - Igel and NNUE] by [[Volodymyr Shcherbyna]], [[CCC]], August 19, 2020 » [[Igel]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74828 Orion 0.7 : NNUE experiment] by [[David Carteau]], [[CCC]], August 19, 2020 » [[Orion]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74837 Night Nurse 0.2] by [[Dietrich Kappe]], [[CCC]], August 19, 2020 » [[A0lite]], [[Igel]]
 +
* [http://laatste.info/bb3/viewtopic.php?f=53&t=8298 NNUE] by [[Bert Tuyt]], [http://laatste.info/bb3/viewforum.php?f=53 World Draughts Forum], August 19, 2020 » [[Draughts]]
 +
===September===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74955 Train a neural network evaluation] by [[Fabio Gobbato]], [[CCC]], September 01, 2020 » [[Automated Tuning]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75016 RubiChess NNUE player implemented] by [[Andreas Matthies]], [[CCC]], September 06, 2020 » [[RubiChess]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75027 Toga III 0.4 NNUE] by [[Dietrich Kappe]], [[CCC]], September 07, 2020 » [[Toga]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75042 Neural network quantization] by [[Fabio Gobbato]], [[CCC]], September 08, 2020 » [[Neural Networks]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75049 AVX-512 and NNUE] by [[Gian-Carlo Pascutto]], [[CCC]], September 08, 2020 » [[AVX-512]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75190 First success with neural nets] by [[Jonathan Kreuzer]], [[CCC]], September 23, 2020 » [[Neural Networks]]
 +
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75190&start=21 Re: First success with neural nets] by [[Jonathan Kreuzer]], [[CCC]], November 11, 2020 » [[Checkers]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75241 Nemorino 6 (NNUE)] by [[Christian Günther|Florentino]], [[CCC]], September 28, 2020 » [[Nemorino]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75247 A Crossroad in Computer Chess; Or Desperate Flailing for Relevance] by [[Andrew Grant]], [[CCC]], September 29, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75248 NNUE variation] by [[Ed Schroder|Ed Schröder]], [[CCC]], September 29, 2020
 +
===October===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75296 BONA_PIECE_ZERO] by [[Marco Belli|elcabesa]], [[CCC]], October 04, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75335&start=91 Re: Final Release of Ethereal, V12.75] by [[Andrew Grant]], [[CCC]], October 09, 2020 » [[Ethereal]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75345 Request for someone to train an NNUE for Ethereal] by [[Andrew Grant]], [[CCC]], October 09, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75350 Ethereal Tuning - Data Dump] by [[Andrew Grant]], [[CCC]], October 10, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75358 Dangerous turn] by [[Dann Corbit]], [[CCC]], October 10, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75393 Black crushing white, weird ?] by [[Vivien Clauzon]], [[CCC]], October 14, 2020 » [[Minic#MinicNNUE|MinicNNUE]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75400 Hacking around CFish NNUE] by [[Maksim Korzh]], [[CCC]], October 15, 2020 » [[CFish]]
 +
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75400&start=22 Re: Hacking around CFish NNUE] by [[Daniel Shawul]], [[CCC]], October 15, 2020 » [[Scorpio]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75415 How to scale stockfish NNUE score?] by [[Maksim Korzh]], [[CCC]], October 17, 2020
 +
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75415&start=3 Re: How to scale stockfish NNUE score?] by [[Daniel Shawul]], [[CCC]], October 17, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75418 Embedding Stockfish NNUE to ANY CHESS ENGINE: YouTube series] by [[Maksim Korzh]], [[CCC]], October 17, 2020 » [[BBC]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75433 Seer] by [[Gerd Isenberg]], [[CCC]], October 18, 2020 » [[Seer]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75482 BBC 1.3 + Stockfish NNUE released!] by [[Maksim Korzh]], [[CCC]], October 21, 2020 » [[BBC]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75500 Mayhem NNUE - New NN engine] by [[Toni Helminen|JohnWoe]], [[CCC]], October 22, 2020 » [[Mayhem]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75501 Centipawns vs Millipawns with NNUE] by Madeleine Birchfield, [[CCC]], October 23, 2020 » [[Centipawns]], [[Millipawns]]
 +
*  <span id="KingPlacements"></span>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75506 NNUE Question - King Placements] by [[Andrew Grant]], [[CCC]], October 23, 2020 » [[Stockfish NNUE#NNUE Structure|Stockfish NNUE - NNUE Structure]]
 +
: [[#KingPlacementsCont|July 01, 2021 continuation]]
 +
===November===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75651 Komodo 14.1 Release and Dragon Announcement] by [[Larry Kaufman]], [[CCC]], November 02, 2020 » [[Komodo]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75653 NNUE outer product vs tensor product] by Madeleine Birchfield, [[CCC]], November 02, 2020 <ref>[https://en.wikipedia.org/wiki/Outer_product Outer product from Wikipedia]</ref> <ref>[https://en.wikipedia.org/wiki/Tensor_product Tensor product from Wikipedia]</ref>
 +
* <span id="Training"></span>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75724 Pytorch NNUE training] by [[Gary Linscott]], [[CCC]], November 08, 2020 <ref>[https://en.wikipedia.org/wiki/PyTorch PyTorch from Wikipedia]</ref>
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75725 TucaNNo: neural network research] by [[Alcides Schulz]], [[CCC]], November 08, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75748 Dragon by Komodo Chess] by [[Larry Kaufman]], [[CCC]], November 10, 2020 » [[Dragon by Komodo Chess]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75751 Tensorflow NNUE training] by [[Daniel Shawul]], [[CCC]], November 10, 2020 <ref>[https://en.wikipedia.org/wiki/TensorFlow TensorFlow from Wikipedia]</ref>
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75890 Speculations about NNUE development (was New engine releases 2020)] by Madeleine Birchfield, [[CCC]], November 11, 2020
 +
: [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75890&start=6 Re: Speculations about NNUE development (was New engine releases 2020)] by [[Connor McMonigle]], [[CCC]], November 12, 2020
 +
: [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75890&start=9 Re: Speculations about NNUE development (was New engine releases 2020)] by [[Connor McMonigle]], [[CCC]], November 12, 2020 » [[Dragon by Komodo Chess]], [[Seer]], [[Halogen]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75335&start=134 Re: Final Release of Ethereal, V12.75] by [[Andrew Grant]], [[CCC]], November 12, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75797 Maybe not the best diversity of strongest chess engines under development] by [[Kai Laskos]], [[CCC]], November 14, 2020 » [[Engine Similarity]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75862 CPU Vector Unit, the new jam for NNs...] by [[Srdja Matovic]], [[CCC]], November 18, 2020 » [[SIMD and SWAR Techniques|SIMD]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75870 You've trained a brilliant NN(UE) King-Piece Network. Now what?] by [[Andrew Grant]], [[CCC]], November 19, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75925 Pawn King Neural Network] by [[Tamás Kuzmics]], [[CCC]], November 26, 2020
 +
===December===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75953 Orion 0.8 + The Cerebrum release] by [[David Carteau]], [[CCC]], December 01, 2020 » [[Orion]], [[Cerebrum]]
 +
* [https://prodeo.actieforum.com/t104-the-nnue-split-programmers-are-in The NNUE split programmers are in] by [[Ed Schroder|Ed Schröder]], [[Computer Chess Forums|ProDeo Forum]], December 02, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76006 Introducing the "Cerebrum" library (NNUE-like trainer and inference code)] by [[David Carteau]], [[CCC]], December 07, 2020
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76190 Dispelling the Myth of NNUE with LazySMP: An Analysis] by [[Andrew Grant]], [[CCC]], December 30, 2020 » [[Lazy SMP]]
 +
==2021==
 +
===January===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76250 Translation of Yu Nasu's NNUE paper] by [[Dominik Klein]], [[CCC]], January 07, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75724&start=60 Re: Pytorch NNUE training] by [[Gary Linscott]], [[CCC]], January 09, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76263 More experiments with neural nets] by [[Jonathan Kreuzer]], [[CCC]], January 09, 2021 » [[Slow Chess]]
 +
* [https://groups.google.com/g/fishcooking/c/cad1MGSdpU4/m/Ury4iBqSBgAJ Shouldn't positional attributes drive SF's NNUE input features (rather than king position)?] by [[Nick Pelling]], [[Computer Chess Forums|FishCooking]], January 10, 2021 » [[Stockfish NNUE]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76285 HalfKP Structure in NNUE] by Roger Stephenson, [[CCC]], January 12, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76346 Andscacs nnue 0.1] by [[Daniel José Queraltó]], [[CCC]], January 17, 2021 » [[Andscacs]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76353 It's NNUE era (sharing my thoughts)] by Basti Dangca, [[CCC]], January 18, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76356 NNUE and game phase] by [[Dann Corbit]], [[CCC]], January 18, 2021 » [[Game Phases]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76382 correspondence chess in the age of NNUE] by [[Larry Kaufman]], [[CCC]], January 21, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76386 One for Andrew Grant et al. - NNUE?] by [[Srdja Matovic]], [[CCC]], January 21, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76437 256 in NNUE?] by Ted Wong, [[CCC]], January 28, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76446 So what do we miss in the traditional evaluation?] by [[Ferdinand Mosca]], [[CCC]], January 29, 2021 » [[Evaluation]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76447 Latest Night Nurse released] by [[Dietrich Kappe]], [[CCC]], January 29, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76456 None-GPL NNUE probing code] by [[Daniel Shawul]], [[CCC]], January 31, 2021
 +
===February===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76537 Fat Fritz 2] by [[Jouni Uski]], [[CCC]], February 09, 2021 » [[Fat Fritz#Fat Fritz 2|Fat Fritz 2.0]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76552 How much work is it to train an NNUE?] by [[Gabor Szots]], [[CCC]], February 11, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76556 HCE and NNUE and vectorisation] by [[Vivien Clauzon]], [[CCC]], February 11, 2021 » [[Minic]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76570 nnue reading code] by [[Jon Dart]], [[CCC]], February 13, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76648 New net: The White Rose] by [[Dietrich Kappe]], [[CCC]], February 20, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76664 Are neural nets (the weights file) copyrightable?] by [[Adam Treat]], [[CCC]], February 21, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76731 My first NNUE nn-f0c1c3cbf2f1.nnue] by [[Michael Byrne|MikeB]], [[CCC]], February 27, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76742 How to make a double-sized net as good as SF NNUE in a few easy steps] by [[Chris Whittington]], [[CCC]], February 28, 2021 » [[Fat Fritz#Fat Fritz 2|Fat Fritz 2.0]]
 +
===March===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76790 A random walk down NNUE street …] by [[Michael Byrne|MikeB]], [[CCC]], March 06, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76833 NNUE Research Project] by [[Ed Schroder|Ed Schröder]], [[CCC]], March 10, 2021 » [[Engine Similarity]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76840 Simex including NNUE] by jjoshua2, [[CCC]], March 11, 2021 » [[Engine Similarity]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76844 NNUE ranking] by Jim Logan, [[CCC]], March 12, 2021 » [[Stockfish NNUE]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76892 FEN compression] by lucasart, [[CCC]], March 17, 2021 » [[BMI2#FEN Compression|FEN Compression]], [[#Training|NNUE Training]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76917 Mabigat - hyperparameter optimizer for NNUE net] by [[Ferdinand Mosca]], [[CCC]], March 22, 2021 » [[Automated Tuning]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76964 nnue-trainer] by [[Jon Dart]], [[CCC]], March 27, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76986 Zeta with NNUE on GPU?] by [[Srdja Matovic]], [[CCC]], March 31, 2021 » [[Zeta]], [[GPU]]
 +
===April===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77157 Rubichess NN questions] by [[Jon Dart]], [[CCC]], April 23, 2021 » [[RubiChess]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77200 Crafty NNUE Chess Engine?] by supersharp77, [[CCC]], April 29, 2021 » [[Crafty]], [[Vafra]] <ref>[http://www.jurjevic.org.uk/chess/vafra/index.html Vafra] by [[Robert Jurjević]]</ref>
 +
===May===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77244 Komodo Dragon 2 released] by [[Larry Kaufman]], [[CCC]], May 04, 2021 » [[Dragon by Komodo Chess|Komodo Dragon]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77344 Stockfish with new NNUE architecture and bigger net released] by [[Stefan Pohl]], [[CCC]], May 19, 2021 » [[Stockfish]], [[Stockfish NNUE]] <ref>[https://github.com/official-stockfish/Stockfish/pull/3474 Update default net to nn-8a08400ed089.nnue by Sopel97 · Pull Request #3474 · official-stockfish/Stockfish · GitHub] by [[Tomasz Sobczyk]]</ref>
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77348 NNUE scoring (egbb lib)] by [[Michael Hoffmann|Desperado]], [[CCC]], May 19, 2021 » [[Scorpio#ScorpioNNUE|Scorpio NNUE]]
 +
===June===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77243&start=20 Re: Booot progress] by [[Alex Morozov]], [[CCC]], June 01, 2021 » [[Booot]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77438 Commercial Release of Ethereal 13.00 (NNUE) for AVX2 Systems] by [[Andrew Grant]], [[CCC]], June 04, 2021 » [[Ethereal#Ethereal 13 (NNUE)|Ethereal 13.00 (NNUE)]]
 +
: [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77438&start=17 Re: Commercial Release of Ethereal 13.00 (NNUE) for AVX2 Systems] by [[Andrew Grant]], [[CCC]], June 04, 2021 » [[Stockfish NNUE#NNUE Structure|HalfKP]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77467 Dark Horse Update] by [[Dietrich Kappe]], [[CCC]], June 11, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77492 Some more experiments with neural nets] by [[Jonathan Kreuzer]], [[CCC]], June 15, 2021 » [[Slow Chess]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77503&start=55 Re: will Tcec allow Stockfish with a Leela net to play?] by [[Connor McMonigle]], [[CCC]], June 17, 2021 » [[Stockfish]], [[Dragon by Komodo Chess|Komodo Dragon]], [[Ethereal]], [[Seer]]
 +
: [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77503&start=63 Re: will Tcec allow Stockfish with a Leela net to play?] by [[Daniel Shawul]], [[CCC]], June 18, 2021 » [[Scorpio]]
 +
: [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77503&start=68 Re: will Tcec allow Stockfish with a Leela net to play?] by [[Vivien Clauzon]], [[CCC]], June 18, 2021 » [[Minic]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77571 I declare that HCE is dead...] by [[Andrew Grant]], [[CCC]], June 29, 2021 » [[Ethereal]], [[Evaluation|HCE]]
 +
===July===
 +
* <span id="KingPlacementsCont"></span>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75506&start=39 Re: NNUE Question - King Placements] by [[Tomasz Sobczyk]], [[CCC]], July 01, 2021 » [[#KingPlacements|NNUE Question]]
 +
: [https://www.talkchess.com/forum3/viewtopic.php?f=7&t=75506&start=40 Re: NNUE Question - King Placements] by [[Daniel Shawul]], July 01, 2021 » [[Scorpio#ScorpioNNUE|ScorpioNNUE]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77602 Before things become more messy than they already are] by [[Ed Schroder|Ed Schröder]], [[CCC]], July 02, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77606 NNUE training set generation] by [[Edsel Apostol]], [[CCC]], July 03, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77612 Time to rethink what Albert Silver has done?] by [[Srdja Matovic]], [[CCC]], July 03, 2021 » [[Fat Fritz#Fat Fritz 2|Fat Fritz 2]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77639 Would the ICGA have accepted today's NNUE engines?] by Madeleine Birchfield, [[CCC]], July 05, 2021 » [[ICGA]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77664 Koivisto 5.0] by [[Finn Eggers]], [[CCC]], July 07, 2021 » [[Koivisto]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=77681 NNUE one year retrospective] by Madeleine Birchfield, [[CCC]], July 09, 2021
 +
===August ...===
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77869 Basic NNUE questions] by [[Amanj Sherwany]], [[CCC]], August 04, 2021
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=78109 Alternatives to King-Pawn, King-All NNUE encoding] by [[Andrew Grant]], [[CCC]], September 05, 2021
 +
* [https://www.talkchess.com/forum3/viewtopic.php?f=7&t=78394 NNUE - Efficiently Updatable Network - understanding] by [[Daniel Infuehr]], [[CCC]], October 11, 2021
 +
* [https://www.talkchess.com/forum3/viewtopic.php?f=2&t=78497 NNUE - only from own engine?] by [[Ed Schroder|Rebel]], October 25, 2021
 +
* [https://www.talkchess.com/forum3/viewtopic.php?f=2&t=78588 Regarding AVX2] by [[Ed Schroder|Rebel]], [[CCC]], November 03, 2021 » [[AVX2]]
 +
* [https://www.talkchess.com/forum3/viewtopic.php?f=2&t=78855 Mantissa 3.0.0] by [[Jeremy Wright]], [[CCC]], December 10, 2021 » [[Mantissa]]
 +
* [https://www.talkchess.com/forum3/viewtopic.php?f=2&t=78979 Are NNUE Nets Specific to Chess Engines or They Universal to All Engines?] by daniel71, [[CCC]], December 26, 2021
 +
==2022 ...==
 +
* [https://www.talkchess.com/forum3/viewtopic.php?f=7&t=79020 Why NNUE trainer requires an online qsearch on each training position?] by [[Chao Ma]], [[CCC]], January 01, 2022
 +
* [https://www.talkchess.com/forum3/viewtopic.php?f=2&t=79107 Rebel 14] by [[Ed Schroder|Ed Schröder]], [[CCC]], January 12, 2022 » [[Rebel#14|Rebel 14]]
 +
* [https://www.talkchess.com/forum3/viewtopic.php?f=2&t=79523 Koivisto 8.0] by [[Finn Eggers]], [[CCC]], March 15, 2022 » [[Koivisto]]
 +
* [https://www.talkchess.com/forum3/viewtopic.php?f=7&t=79742 NNUE + Pawn-King Network] by Alvin Peng, [[CCC]], April 22, 2022
 +
==2024 ...==
 +
* [https://talkchess.com/forum3/viewtopic.php?f=7&t=83170 How to get started with NNUE] by Arjun Basandrai, [[CCC]], January 12, 2024
  
 
=External Links=
 
=External Links=
* [https://github.com/yaneurao/YaneuraOu GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine]
+
==NNUE==
* [https://github.com/Tama4649/Kristallweizen/ GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです。]
+
* [https://en.wikipedia.org/wiki/Efficiently_updatable_neural_network Efficiently updatable neural network | Wikipedia]
 
* [http://qhapaq.hatenablog.com/entry/2018/06/02/221612 次世代の将棋思考エンジン、NNUE関数を学ぼう(その1.ネットワーク構造編) - コンピュータ将棋 Qhapaq], June 02, 2018 (Japanese)
 
* [http://qhapaq.hatenablog.com/entry/2018/06/02/221612 次世代の将棋思考エンジン、NNUE関数を学ぼう(その1.ネットワーク構造編) - コンピュータ将棋 Qhapaq], June 02, 2018 (Japanese)
 
: Learn Next Generation Shogi Thinking Engine, NNUE Function (Part 1. Network Structure) - Computer Shogi
 
: Learn Next Generation Shogi Thinking Engine, NNUE Function (Part 1. Network Structure) - Computer Shogi
Line 47: Line 195:
 
: Let's Learn Next Generation Shogi Thinking Engine, NNUE Function (Part 2. Remodeling/Learning) - Computer Shogi
 
: Let's Learn Next Generation Shogi Thinking Engine, NNUE Function (Part 2. Remodeling/Learning) - Computer Shogi
 
* [http://yaneuraou.yaneu.com/2020/06/19/stockfish-nnue-the-complete-guide/ Stockfish NNUE – The Complete Guide], June 19, 2020 (Japanese and English)
 
* [http://yaneuraou.yaneu.com/2020/06/19/stockfish-nnue-the-complete-guide/ Stockfish NNUE – The Complete Guide], June 19, 2020 (Japanese and English)
 +
* [http://yaneuraou.yaneu.com/2020/08/21/3-technologies-in-shogi-ai-that-could-be-used-for-chess-ai/ 3 technologies in shogi AI that could be used for chess AI] by [[Motohiro Isozaki]], August 21, 2020 » [[Stockfish NNUE]]
 +
* [https://www.qhapaq.org/shogi/shogiwiki/stockfish-nnue/ Stockfish NNUE Wiki]
 +
* [http://rebel13.nl/home/nnue.html nnue | Home of the Dutch Rebel] by [[Ed Schroder|Ed Schröder]]
 +
* [https://github.com/glinscott/nnue-pytorch/blob/master/docs/nnue.md NNUE Guide (nnue-pytorch/nnue.md at master · glinscott/nnue-pytorch · GitHub)] hosted by [[Gary Linscott]]
 +
==NNUE libraries==
 +
Some developers disintegrate and rewrite the Stockfish NNUE code into independent libraries which can be much easier to embed into other chess engines.
 +
* [https://github.com/david-carteau/cerebrum GitHub - david-carteau/cerebrum: The Cerebrum library] by [[David Carteau]] » [[Cerebrum]]
 +
* [https://github.com/dshawul/nncpu-probe GitHub - dshawul/nncpu-probe] by [[Daniel Shawul]]
 +
* [https://github.com/jdart1/nnue GitHub - jdart1/nnue: NNUE reading code for chess] by [[Jon Dart]]
 +
==Source Code==
 +
* [https://github.com/yaneurao/YaneuraOu GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine]
 +
* [https://github.com/Tama4649/Kristallweizen/ GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです。]
 
* [https://github.com/nodchip/Stockfish GitHub - nodchip/Stockfish: UCI chess engine] ([[Stockfish NNUE]] by [[Hisayori Noda|Nodchip]])
 
* [https://github.com/nodchip/Stockfish GitHub - nodchip/Stockfish: UCI chess engine] ([[Stockfish NNUE]] by [[Hisayori Noda|Nodchip]])
* [https://www.qhapaq.org/shogi/shogiwiki/stockfish-nnue/ Stockfish NNUE Wiki]
+
* [https://github.com/dkappe/leela-chess-weights/wiki/A-Leela-NNUE%3F-Night-Nurse-and-Others A Leela NNUE? Night Nurse and Others · dkappe/leela-chess-weights Wiki · GitHub] by [[Dietrich Kappe]]
 +
* [https://github.com/DanielUranga/TensorFlowNNUE GitHub - DanielUranga/TensorFlowNNUE] by [[Daniel Uranga]]
 +
* [https://github.com/glinscott/nnue-pytorch GitHub - glinscott/nnue-pytorch: NNUE (Chess evaluation) trainer in Pytorch] by [[Gary Linscott]]
 +
* [https://github.com/connormcmonigle/seer-nnue GitHub - connormcmonigle/seer-nnue: UCI chess engine using neural networks for position evaluation] by [[Connor McMonigle]] » [[Seer]]
 +
* [https://github.com/bmdanielsson/nnue-trainer GitHub - bmdanielsson/nnue-trainer: PyTorch trainer for NNUE style neural networks] by [[Martin Danielsson]] » [[Marvin]] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76964 nnue-trainer] by [[Jon Dart]], [[CCC]], March 27, 2021</ref>
 +
* [https://github.com/fsmosca/Mabigat GitHub - fsmosca/Mabigat: NNUE parameter optimizer] by [[Ferdinand Mosca]] » [[Automated Tuning]]
 +
==Misc==
 +
* [https://software.intel.com/content/www/us/en/develop/articles/lower-numerical-precision-deep-learning-inference-and-training.html Lower Numerical Precision Deep Learning Inference and Training] by [https://community.intel.com/t5/user/viewprofilepage/user-id/134067 Andres Rodriguez] et al., [[Intel]], January 19, 2018 » [[AVX-512]]
 
* [https://en.wikipedia.org/wiki/Nue Nue from Wikipedia]
 
* [https://en.wikipedia.org/wiki/Nue Nue from Wikipedia]
 +
* [[:Category:Hiromi Uehara|Hiromi]] - [https://en.wikipedia.org/wiki/Spectrum_(Hiromi_album) Spectrum], 2019, [https://en.wikipedia.org/wiki/YouTube YouTube] Video
 +
: {{#evu:https://www.youtube.com/watch?v=A8RCz_RoefM|alignment=left|valignment=top}}
  
 
=References=  
 
=References=  
Line 55: Line 224:
 
'''[[Neural Networks|Up one Level]]'''
 
'''[[Neural Networks|Up one Level]]'''
 
[[Category:Toriyama Sekien]]
 
[[Category:Toriyama Sekien]]
 +
[[Category:Hiromi Uehara]]

Latest revision as of 06:05, 22 September 2024

Home * Learning * Neural Networks * NNUE

Toriyama Sekien - Nue (鵺) [1] [2]

NNUE, (ƎUИИ Efficiently Updatable Neural Networks)
a Neural Network architecture intended to replace the evaluation of Shogi, chess and other board game playing alpha-beta searchers running on a CPU. Inspired by Kunihito Hoki's approach of piece-square tables indexed by king location, and further two-piece locations and side to move as applied in his Shogi engine Bonanza [3], NNUE was introduced in 2018 by Yu Nasu [4], and was used in Shogi adaptations of Stockfish such as YaneuraOu [5], and Kristallweizen [6], apparently with AlphaZero strength [7].

Stockfish NNUE

As reported by Henk Drost [8], Nodchip incorporated NNUE into the chess playing Stockfish 10 as a proof of concept. Stockfish NNUE was born, and in summer 2020 the computer chess community bursts out enthusiastically due to its rapidly raising playing strength with different networks trained using a mixture of supervised and reinforcement learning methods - despite the approximately halved search speed, becoming stronger than its original [9], finally responsible for the huge strength improvement of Stockfish 12.

NN Structure

The neural network of Stockfish NNUE consists of four layers, W1 through W4. The input layer W1 is heavily overparametrized, feeding in the board representation for various king configurations. The efficiency of the net is due to incremental update of W1 in make and unmake move, where only a fraction of its neurons need to be recalculated. The remaining three layers with 32x2x256, 32x32 and 32x1 weights are computational less expensive, best calculated using appropriate SIMD instructions like AVX2 on x86-64, or if available, AVX-512.

NNUE.jpg

NNUE structure with incremental updates [10]

Adoption

see Category:NNUE

Being tempted by the success of Stockfish NNUE and attracted by how easy the method and small the code is, many engine developers have started testing and applying NNUE. For quick trials and evaluating before going into serious development, some of them borrowed and/or rewrote NNUE code and use networks from Stockfish NNUE. Most of them reported positive results, such as David Carteau with Orion [11], Ehsan Rashid with DON [12], various Stockfish derivatives by Michael Byrne [13], and Volodymyr Shcherbyna with Igel [14] using the Night Nurse NNUE net by Dietrich Kappe [15]. Daniel Shawul added NNUE support à la CFish into his egbbdll probing library of Scorpio [16] [17], making it even easier to use NNUE. The promising engines Halogen 7 and 8 by Kieren Pearson, and Seer by Connor McMonigle came with their own, distinct NNUE implementations, and on November 10, 2020, the commercial Dragon by Komodo Chess aka Komodo NNUE appeared [18], trying to close the gap to Stockfish NNUE. The commercial Fat Fritz 2.0, based on a slightly modified Stockfish 12 using a customized, double sized network, was released by ChessBase in February 2021.

See also

Publications

Forum Posts

2020

January ...

Re: The Stockfish of shogi by Gian-Carlo Pascutto, CCC, January 18, 2020

July

Re: NNUE accessible explanation by Jonathan Rosenthal, CCC, July 23, 2020
Re: NNUE accessible explanation by Jonathan Rosenthal, CCC, July 24, 2020
Re: NNUE accessible explanation by Jonathan Rosenthal, CCC, August 03, 2020

August

September

Re: First success with neural nets by Jonathan Kreuzer, CCC, November 11, 2020 » Checkers

October

Re: Hacking around CFish NNUE by Daniel Shawul, CCC, October 15, 2020 » Scorpio
Re: How to scale stockfish NNUE score? by Daniel Shawul, CCC, October 17, 2020
July 01, 2021 continuation

November

Re: Speculations about NNUE development (was New engine releases 2020) by Connor McMonigle, CCC, November 12, 2020
Re: Speculations about NNUE development (was New engine releases 2020) by Connor McMonigle, CCC, November 12, 2020 » Dragon by Komodo Chess, Seer, Halogen

December

2021

January

February

March

April

May

June

Re: Commercial Release of Ethereal 13.00 (NNUE) for AVX2 Systems by Andrew Grant, CCC, June 04, 2021 » HalfKP
Re: will Tcec allow Stockfish with a Leela net to play? by Daniel Shawul, CCC, June 18, 2021 » Scorpio
Re: will Tcec allow Stockfish with a Leela net to play? by Vivien Clauzon, CCC, June 18, 2021 » Minic

July

Re: NNUE Question - King Placements by Daniel Shawul, July 01, 2021 » ScorpioNNUE

August ...

2022 ...

2024 ...

External Links

NNUE

Learn Next Generation Shogi Thinking Engine, NNUE Function (Part 1. Network Structure) - Computer Shogi
Let's Learn Next Generation Shogi Thinking Engine, NNUE Function (Part 2. Remodeling/Learning) - Computer Shogi

NNUE libraries

Some developers disintegrate and rewrite the Stockfish NNUE code into independent libraries which can be much easier to embed into other chess engines.

Source Code

Misc

References

  1. Nue (鵺) from the Konjaku Gazu Zoku Hyakki (今昔画図続百鬼) by Toriyama Sekien, circa 1779, Wikimedia Commons
  2. Re: What does NNUE actually mean by Henk Drost, CCC, July 29, 2020
  3. 機械学習エンジニアのための将棋AI開発入門その1 Introduction to Shogi AI development for machine learning engineers Part 1, May 03, 2020 (Japanese)
  4. Yu Nasu (2018). ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi. Ziosoft Computer Shogi Club, pdf (Japanese with English abstract)
  5. GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine
  6. GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです。
  7. The Stockfish of shogi by Larry Kaufman, CCC, January 07, 2020
  8. Stockfish NN release (NNUE) by Henk Drost, CCC, May 31, 2020
  9. Can the sardine! NNUE clobbers SF by Henk Drost, CCC, July 16, 2020
  10. Image from Yu Nasu (2018). ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi. Ziosoft Computer Shogi Club, pdf (Japanese with English abstract)
  11. Orion 0.7 : NNUE experiment by David Carteau, CCC, August 19, 2020
  12. Re: New engine releases 2020...Don NNUE 2020? by supersharp77, CCC, August 19, 2020
  13. ... the last shall be first ... by MikeB, CCC, 19 Aug 2020
  14. Introducing Igel chess engine by Volodymyr Shcherbyna, CCC, 20 Aug 2020
  15. Night Nurse 0.2 by Dietrich Kappe, CCC, August 19, 2020
  16. Re: Hacking around CFish NNUE by Daniel Shawul, CCC, October 15, 2020
  17. Re: How to scale stockfish NNUE score? by Daniel Shawul, CCC, October 17, 2020
  18. Dragon by Komodo Chess by Larry Kaufman, CCC, November 10, 2020
  19. Translation of Yu Nasu's NNUE paper by Dominik Klein, CCC, January 07, 2021
  20. Book about Neural Networks for Chess by dkl, CCC, September 29, 2021
  21. Outer product from Wikipedia
  22. Tensor product from Wikipedia
  23. PyTorch from Wikipedia
  24. TensorFlow from Wikipedia
  25. Vafra by Robert Jurjević
  26. Update default net to nn-8a08400ed089.nnue by Sopel97 · Pull Request #3474 · official-stockfish/Stockfish · GitHub by Tomasz Sobczyk
  27. nnue-trainer by Jon Dart, CCC, March 27, 2021

Up one Level