25,161
edits
Changes
NNUE
,Created page with "'''Home * Learning * Neural Networks * NNUE''' '''NNUE''', (ƎUИИ Efficiently Updatable Neural Networks)<br/> a Neural Network architec..."
'''[[Main Page|Home]] * [[Learning]] * [[Neural Networks]] * NNUE'''
'''NNUE''', (ƎUИИ Efficiently Updatable Neural Networks)<br/>
a Neural Network architecture intended to replace the [[Evaluation|evaluation]] of [[Shogi]], [[Chess|chess]] and other board game playing [[Alpha-Beta|alpha-beta]] searchers running on a CPU.
NNUE was introduced in 2018 by [[Yu Nasu]] <ref>[[Yu Nasu]] ('''2018'''). ''ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''. Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)</ref>,
and was used in Shogi adaptations of [[Stockfish]] such as [[YaneuraOu]] <ref>[https://github.com/yaneurao/YaneuraOu GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine]</ref>,
and [[Kristallweizen-kai]] <ref>[https://github.com/Tama4649/Kristallweizen/ GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです。]</ref>, apparently with [[AlphaZero]] strength <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754 The Stockfish of shogi] by [[Larry Kaufman]], [[CCC]], January 07, 2020</ref>.
==Stockfish NNUE==
As reported by [[Henk Drost]] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74058 Stockfish NNUE] by [[Henk Drost]], [[CCC]], May 31, 2020</ref>,
[[Nodchip]] incorporated NNUE into the chess playing Stockfish 10 as a proof of concept.
[[Stockfish NNUE]] was born incorporating recent search enhancements, and in summer 2020 the computer chess community burst out enthusiastically due to its rapidly raising [[Playing Strength|playing strength]] with different networks trained using a mixture of [[Supervised Learning|supervised]] and [[Reinforcement Learning|reinforcement learning]] methods -
despite the approximately halved search speed, even becoming stronger than its original <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74484 Can the sardine! NNUE clobbers SF] by [[Henk Drost]], [[CCC]], July 16, 2020</ref>.
=NN Structure=
The neural network consists of four layers, W1 through W4. The input layer W1 is heavily overparametrized, feeding in the [[Board Representation|board representation]] for various king and pawn configurations.
The efficiency of the net is due to [[Incremental Updates|incremental update]] of W1 in [[Make Move|make]] and [[Unmake Move|unmake move]],
where only a fractions of neurons need to be recalculated. The remaining three layers with 256x2-32-32 channels are computational less expensive, best calculated using appropriate [[SIMD and SWAR Techniques|SIMD instructions]] like [[AVX2]] on [[x86-64]], or if available, [[AVX-512]].
[[FILE:NNUE.jpg|none|border|text-bottom]]
NNUE structure with [[Incremental Updates|incremental update]] <ref>Image from [[Yu Nasu]] ('''2018'''). ''ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''. Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)</ref>
=Publications=
* [[Yu Nasu]] ('''2018'''). ''ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''. Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)
=Forum Posts=
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754 The Stockfish of shogi] by [[Larry Kaufman]], [[CCC]], January 07, 2020 » [[Stockfish]], [[Shogi]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74058 Stockfish NNUE] by [[Henk Drost]], [[CCC]], May 31, 2020 » [[Stockfish]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74059 Stockfish NN release (NNUE)] by [[Henk Drost]], [[CCC]], May 31, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74484 Can the sardine! NNUE clobbers SF] by [[Henk Drost]], [[CCC]], July 16, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531 NNUE accessible explanation] by [[Martin Fierz]], [[CCC]], July 21, 2020
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=1 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], July 23, 2020
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=5 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], July 24, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74565 Stockfish NNUE] by Lion, [[CCC]], July 25, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74585 SF-NNUE going forward...] by Zenmastur, [[CCC]], July 27, 2020
=External Links=
* [http://yaneuraou.yaneu.com/2020/06/19/stockfish-nnue-the-complete-guide/ Stockfish NNUE – The Complete Guide], June 19, 2020 (Japanese and English)
* [https://github.com/nodchip/Stockfish GitHub - nodchip/Stockfish: UCI chess engine] ([[Stockfish NNUE]] by [[Nodchip]])
* [https://www.qhapaq.org/shogi/shogiwiki/stockfish-nnue/ Stockfish NNUE Wiki]
* [https://github.com/official-stockfish/Stockfish/issues/2823 NNUE merge · Issue #2823 · official-stockfish/Stockfish · GitHub] by [[Joost VandeVondele]], July 25, 2020 <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74560 An info] by Sylwy, [[CCC]], July 25, 2020</ref>
=References=
<references />
'''[[Neural Networks|Up one Level]]'''
'''NNUE''', (ƎUИИ Efficiently Updatable Neural Networks)<br/>
a Neural Network architecture intended to replace the [[Evaluation|evaluation]] of [[Shogi]], [[Chess|chess]] and other board game playing [[Alpha-Beta|alpha-beta]] searchers running on a CPU.
NNUE was introduced in 2018 by [[Yu Nasu]] <ref>[[Yu Nasu]] ('''2018'''). ''ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''. Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)</ref>,
and was used in Shogi adaptations of [[Stockfish]] such as [[YaneuraOu]] <ref>[https://github.com/yaneurao/YaneuraOu GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine]</ref>,
and [[Kristallweizen-kai]] <ref>[https://github.com/Tama4649/Kristallweizen/ GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです。]</ref>, apparently with [[AlphaZero]] strength <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754 The Stockfish of shogi] by [[Larry Kaufman]], [[CCC]], January 07, 2020</ref>.
==Stockfish NNUE==
As reported by [[Henk Drost]] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74058 Stockfish NNUE] by [[Henk Drost]], [[CCC]], May 31, 2020</ref>,
[[Nodchip]] incorporated NNUE into the chess playing Stockfish 10 as a proof of concept.
[[Stockfish NNUE]] was born incorporating recent search enhancements, and in summer 2020 the computer chess community burst out enthusiastically due to its rapidly raising [[Playing Strength|playing strength]] with different networks trained using a mixture of [[Supervised Learning|supervised]] and [[Reinforcement Learning|reinforcement learning]] methods -
despite the approximately halved search speed, even becoming stronger than its original <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74484 Can the sardine! NNUE clobbers SF] by [[Henk Drost]], [[CCC]], July 16, 2020</ref>.
=NN Structure=
The neural network consists of four layers, W1 through W4. The input layer W1 is heavily overparametrized, feeding in the [[Board Representation|board representation]] for various king and pawn configurations.
The efficiency of the net is due to [[Incremental Updates|incremental update]] of W1 in [[Make Move|make]] and [[Unmake Move|unmake move]],
where only a fractions of neurons need to be recalculated. The remaining three layers with 256x2-32-32 channels are computational less expensive, best calculated using appropriate [[SIMD and SWAR Techniques|SIMD instructions]] like [[AVX2]] on [[x86-64]], or if available, [[AVX-512]].
[[FILE:NNUE.jpg|none|border|text-bottom]]
NNUE structure with [[Incremental Updates|incremental update]] <ref>Image from [[Yu Nasu]] ('''2018'''). ''ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''. Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)</ref>
=Publications=
* [[Yu Nasu]] ('''2018'''). ''ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''. Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)
=Forum Posts=
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754 The Stockfish of shogi] by [[Larry Kaufman]], [[CCC]], January 07, 2020 » [[Stockfish]], [[Shogi]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74058 Stockfish NNUE] by [[Henk Drost]], [[CCC]], May 31, 2020 » [[Stockfish]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74059 Stockfish NN release (NNUE)] by [[Henk Drost]], [[CCC]], May 31, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74484 Can the sardine! NNUE clobbers SF] by [[Henk Drost]], [[CCC]], July 16, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531 NNUE accessible explanation] by [[Martin Fierz]], [[CCC]], July 21, 2020
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=1 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], July 23, 2020
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=5 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], July 24, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74565 Stockfish NNUE] by Lion, [[CCC]], July 25, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74585 SF-NNUE going forward...] by Zenmastur, [[CCC]], July 27, 2020
=External Links=
* [http://yaneuraou.yaneu.com/2020/06/19/stockfish-nnue-the-complete-guide/ Stockfish NNUE – The Complete Guide], June 19, 2020 (Japanese and English)
* [https://github.com/nodchip/Stockfish GitHub - nodchip/Stockfish: UCI chess engine] ([[Stockfish NNUE]] by [[Nodchip]])
* [https://www.qhapaq.org/shogi/shogiwiki/stockfish-nnue/ Stockfish NNUE Wiki]
* [https://github.com/official-stockfish/Stockfish/issues/2823 NNUE merge · Issue #2823 · official-stockfish/Stockfish · GitHub] by [[Joost VandeVondele]], July 25, 2020 <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74560 An info] by Sylwy, [[CCC]], July 25, 2020</ref>
=References=
<references />
'''[[Neural Networks|Up one Level]]'''
