25,161
edits
Changes
no edit summary
The [[Neural Networks|neural network]] consists of four layers. The input layer is heavily overparametrized, feeding in the [[Board Representation|board representation]] for all king placements per side.
The efficiency of [[NNUE]] is due to [[Incremental Updates|incremental update]] of the input layer outputs in [[Make Move|make]] and [[Unmake Move|unmake move]],
where only a tiny fraction of its neurons need to be considered <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=1 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], July 23, 2020</ref>. The remaining three layers with 2x256x32, 32x32 and 32x1 weights are computational less expensive, hidden layer 1 and 2 with [https://en.wikipedia.org/wiki/Rectifier_(neural_networks) ReLu activation] <ref>[https://github.com/official-stockfish/Stockfish/blob/master/src/nnue/architectures/halfkp_256x2-32-32.h#L42 Stockfish/halfkp_256x2-32-32.h at master · official-stockfish/Stockfish · GitHub]</ref> <ref>[https://github.com/official-stockfish/Stockfish/blob/master/src/nnue/layers/clipped_relu.h#L82 Stockfish/clipped_relu.h at master · official-stockfish/Stockfish · GitHub]</ref>, best calculated using appropriate [[SIMD and SWAR Techniques|SIMD instructions]] performing fast [[Byte|8-bit]]/[[Word|16-bit]] integer vector arithmetic, like [[MMX]], [[SSE2]] or [[AVX2]] on [[x86]]/[[x86-64]], or, if available, [[AVX-512]].
[[FILE:StockfishNNUELayers.png|none|border|text-bottom|1024px]]
