25,161
edits
Changes
no edit summary
=AllieStein=
AllieStein is the combination of Allie with Leela's third party '''Leelenstein''' network by [[Mark Jordan]] <ref>[https://github.com/LeelaChessZero/lc0/wiki/Third-Party-Nets Third Party Nets · LeelaChessZero/lc0 Wiki · GitHub]</ref> <ref>[https://www.patreon.com/jjosh Jjosh is creating Leelenstein | Patreon]</ref>, which is trained by [[Supervised Learning|supervised learning]] feeding in games from [[CCRL]], supported by '''[[Ilya Loshchilov#SGDR''' |SGDR]] ([https://en.wikipedia.org/wiki/Stochastic_gradient_descent Stochastic Gradient Descent] with Warm Restarts) <ref>[[Ilya Loshchilov]], [[Frank Hutter]] ('''2016'''). ''SGDR: Stochastic Gradient Descent with Warm Restarts''. [https://arxiv.org/abs/1608.03983 arXiv:1608.03983]</ref>
and '''GGT''' (full-matrix adaptive [https://en.wikipedia.org/wiki/Regularization_(mathematics) regularization]) <ref>[[Naman Agarwal]], [[Brian Bullins]], [[Xinyi Chen]], [[Elad Hazan]], [[Karan Singh]], [[Cyril Zhang]], [[Yi Zhang]] ('''2018'''). ''The Case for Full-Matrix Adaptive Regularization''. [https://arxiv.org/abs/1806.02958 arXiv:1806.02958]</ref>,
using [https://en.wikipedia.org/wiki/Batch_normalization batch renormalization] <ref>[[Mathematician#SIoffe|Sergey Ioffe]] ('''2017'''). ''Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models''. [https://arxiv.org/abs/1702.03275 arXiv:1702.03275]</ref>,
