Changes

an [https://en.wikipedia.org/wiki/Automation automated] adjustment of [[Evaluation|evaluation]] parameters or weights, and less commonly, [[Search|search]] parameters <ref>[[Yngvi Björnsson]], [[Tony Marsland]] ('''2001'''). ''Learning Search Control in Adversary Games''. [[Advances in Computer Games 9]], pp. 157-174. [http://www.ru.is/faculty/yngvi/pdf/BjornssonM01b.pdf pdf]</ref>, with the aim to improve the [[Playing Strength|playing strength]] of a chess engine or game playing program. Evaluation tuning can be applied by [[Automated Tuning#Optimization|mathematical optimization]] or [[Learning|machine learning]], both fields with huge overlaps. Learning approaches are subdivided into [[Automated Tuning#SupervisedLearning|supervised learning]] using [https://en.wikipedia.org/wiki/Training_set labeled data], and [[Automated Tuning#ReinformentLearning|reinforcement learning]] to learn from trying, facing the exploration (of uncharted territory) and exploitation (of current knowledge) dilemma. [[Johannes Fürnkranz]] gives a comprehensive overview in ''Machine Learning in Games: A Survey'' published in 2000 <ref>[[Johannes Fürnkranz]] ('''2000'''). ''Machine Learning in Games: A Survey''. [https://en.wikipedia.org/wiki/Austrian_Research_Institute_for_Artificial_Intelligence Austrian Research Institute for Artificial Intelligence], OEFAI-TR-2000-3, [httphttps://wwwfmfi-uk.ofaihq.atsk/Informatika/cgi-binStrojove%20Ucenie/get-tr?download=1&paper=oefai-tr-2000-31.pdf pdf] - Chapter 4, Evaluation Function Tuning</ref>, covering evaluation tuning in chapter 4.

<span id="MoveAdaption"></span>One [[Supervised Learning|supervised learning]] method considers desired moves from a set of positions, likely from grandmaster games, and tries to adjust their evaluation weights so that for instance a one-ply search agrees with the desired move. Already pioneering in reinforcement learning some years before, move adaptation was described by [[Arthur Samuel]] in 1967 as used in the second version of his checkers player <ref>[[Arthur Samuel]] ('''1967'''). ''Some Studies in Machine Learning. Using the Game of Checkers. II-Recent Progress''. [http://researcher.watson.ibm.com/researcher/files/us-beygel/samuel-checkers.pdf pdf]</ref>, where a structure of stacked linear evaluation functions was trained by computing a correlation measure based on the number of times the feature rated an alternative move higher than the desired move played by an expert <ref>[[Johannes Fürnkranz]] ('''2000'''). ''Machine Learning in Games: A Survey''. [https://en.wikipedia.org/wiki/Austrian_Research_Institute_for_Artificial_Intelligence Austrian Research Institute for Artificial Intelligence], OEFAI-TR-2000-3, [httphttps://wwwfmfi-uk.ofaihq.atsk/Informatika/cgi-binStrojove%20Ucenie/get-tr?download=1&paper=oefai-tr-2000-31.pdf pdf]</ref>. In chess, move adaptation was first described by [[Thomas Nitsche]] in 1982 <ref>[[Thomas Nitsche]] ('''1982'''). ''A Learning Chess Program.'' [[Advances in Computer Chess 3]]</ref>, and with some extensions by [[Tony Marsland]] in 1985 <ref>[[Tony Marsland]] ('''1985'''). ''Evaluation-Function Factors''. [[ICGA Journal#8_2|ICCA Journal, Vol. 8, No. 2]], [http://webdocs.cs.ualberta.ca/~tony/OldPapers/evaluation.pdf pdf]</ref>. [[Eval Tuning in Deep Thought]] as mentioned by [[Feng-hsiung Hsu]] et al. in 1990 <ref>[[Feng-hsiung Hsu]], [[Thomas Anantharaman]], [[Murray Campbell]], [[Andreas Nowatzyk]] ('''1990'''). ''[http://www.disi.unige.it/person/DelzannoG/AI2/hsu.html A Grandmaster Chess Machine]''. [[Scientific American]], Vol. 263, No. 4, pp. 44-50. ISSN 0036-8733.</ref>, and later published by [[Andreas Nowatzyk]], is also based on an extended form of move adaptation <ref>see ''2.1 Learning from Desired Moves in Chess'' in [[Kunihito Hoki]], [[Tomoyuki Kaneko]] ('''2014'''). ''[https://www.jair.org/papers/paper4217.html Large-Scale Optimization for Evaluation Functions with Minimax Search]''. [https://www.jair.org/vol/vol49.html JAIR Vol. 49]</ref>. [[Jonathan Schaeffer|Jonathan Schaeffer's]] and [[Paul Lu|Paul Lu's]] efforts to make Deep Thought's approach work for [https://en.wikipedia.org/wiki/Chinook_%28draughts_player%29 Chinook] in 1990 failed <ref>[[Jonathan Schaeffer]], [[Joe Culberson]], [[Norman Treloar]], [[Brent Knight]], [[Paul Lu]], [[Duane Szafron]] ('''1992'''). ''A World Championship Caliber Checkers Program''. [https://en.wikipedia.org/wiki/Artificial_Intelligence_%28journal%29 Artificial Intelligence], Vol. 53, Nos. 2-3,[http://webdocs.cs.ualberta.ca/%7Ejonathan/Papers/Papers/chinook.ps ps]</ref> - nothing seemed to produce results that were as good than their hand-tuned effort <ref>[[Jonathan Schaeffer]] ('''1997, 2009'''). ''[http://www.springer.com/computer/ai/book/978-0-387-76575-4 One Jump Ahead]''. 7. The Case for the Prosecution, pp. 111-114</ref>.

{||-| style="vertical-align:top;" | The supervised problem of regression applied to [[Automated Tuning#MoveAdaption|move adaptation]] was used by [[Thomas Nitsche]] in 1982, minimizing the [https://en.wikipedia.org/wiki/Mean_squared_error mean squared error] of a cost function considering the program’s and a grandmaster’s choice of moves, as mentioned, extended by [[Tony Marsland]] in 1985, and later by the [[Deep Thought]] team. Regression used to [[Automated Tuning#ValueAdaption|adapt desired values]] was described by [[Donald H. Mitchell]] in his 1984 masters master thesis on evaluation features in [[Othello]], cited by [[Michael Buro]] <ref>[[Michael Buro]] ('''1995'''). ''[httphttps://www.jair.org/papersindex.php/jair/article/view/paper179.html 10146 Statistical Feature Combination for the Evaluation of Game Positions]''. [https://en.wikipedia.org/wiki/Journal_of_Artificial_Intelligence_Research JAIR], Vol. 3</ref> <ref>[[Donald H. Mitchell]] ('''1984'''). ''Using Features to Evaluate Positions in Experts' and Novices' Othello Games''. Masters Master thesis, Department of Psychology, [[Northwestern University]], Evanston, IL</ref>. [[Jens Christensen]] applied [https://en.wikipedia.org/wiki/Linear_regression linear regression] to chess in 1986 to learn [[Point Value|point values]] in the domain of [[Temporal Difference Learning|temporal difference learning]] <ref>[[Jens Christensen]] ('''1986'''). ''[http://link.springer.com/chapter/10.1007/978-1-4613-2279-5_9?no-access=true Learning Static Evaluation Functions by Linear Regression]''. in [[Tom Mitchell]], [[Jaime Carbonell]], [[Ryszard Michalski]] ('''1986'''). ''[http://link.springer.com/book/10.1007/978-1-4613-2279-5 Machine Learning: A Guide to Current Research]''. The Kluwer International Series in Engineering and Computer Science, Vol. 12</ref>. | [[FILE:Linear regression.svg|border|left|thumb|baseline|300px|[https://en.wikipedia.org/wiki/Linear_regression Linear Regression] <ref>Random data points and their [https://en.wikipedia.org/wiki/Linear_regression linear regression]. [https://commons.wikimedia.org/wiki/File:Linear_regression.svg Created] with [https://en.wikipedia.org/wiki/Sage_%28mathematics_software%29 Sage] by Sewaqu, November 5, 2010, [https://en.wikipedia.org/wiki/Wikimedia_Commons Wikimedia Commons]</ref> ]] |}

{ [[FILE:SigmoidTexelTune.gif|border|right|thumb|300px|link=http://wolfr.am/1al3d5B|[https://en.wikipedia.org/wiki/Logistic_function Logistic function] <ref>[http://wolfr.am/1al3d5B log-| stylelinear 1 / (1 + 10^(-s/4)) , s="vertical-align10 to 10] from [https:top;" //en.wikipedia.org/wiki/Wolfram_Alpha Wolfram| Alpha]</ref> ]]  Since the relationship between [[Pawn Advantage, Win Percentage, and Elo|win percentage and pawn advantage]] is assumed to follow a [https://en.wikipedia.org/wiki/Logistic_model logistic model], one may treat static evaluation as [[Neural Networks#Perceptron|single-layer perceptron]] or single [https://en.wikipedia.org/wiki/Artificial_neuron neuron] [[Neural Networks|ANN]] with the common [https://en.wikipedia.org/wiki/Logistic_function logistic] [https://en.wikipedia.org/wiki/Activation_function activation function], performing the perceptron algorithm to train it <ref>[http://www.talkchess.com/forum/viewtopic.php?t=56168&start=36 Re: Piece weights with regression analysis (in Russian)] by [[Fabien Letouzey]], [[CCC]], May 04, 2015</ref>. [https://en.wikipedia.org/wiki/Logistic_regression Logistic regression] in evaluation tuning was first elaborated by [[Michael Buro]] in 1995 <ref>[[Michael Buro]] ('''1995'''). ''[httphttps://www.jair.org/papersindex.php/paper179.html jair/article/view/10146 Statistical Feature Combination for the Evaluation of Game Positions]''. [https://en.wikipedia.org/wiki/Journal_of_Artificial_Intelligence_Research JAIR], Vol. 3</ref>, and proved successful in the game of [[Othello]] in comparison with [[Mathematician#RFisher|Fisher's]] [https://en.wikipedia.org/wiki/Kernel_Fisher_discriminant_analysis linear discriminant] and quadratic [https://en.wikipedia.org/wiki/Discriminant discriminant] function for [https://en.wikipedia.org/wiki/Normal_distribution normally distributed] features, and served as eponym of his Othello program ''Logistello'' <ref>[https://skatgame.net/mburo/log.html LOGISTELLO's Homepage]</ref>. In computer chess, logistic regression was applied by [[Arkadiusz Paterek]] with [[Gosu]] <ref>[[Arkadiusz Paterek]] ('''2004'''). ''Modelowanie funkcji oceniającej w grach''. [[University of Warsaw]], [https://www.mimuw.edu.pl/~paterek/mfog.ps.gz zipped ps] (Polish, Modeling of an evaluation function in games)</ref>, later proposed by [[Miguel A. Ballicora]] in 2009 as used by [[Gaviota]] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=27266&postdays=0&postorder=asc&topic_view=&start=11 Re: Insanity... or Tal style?] by [[Miguel A. Ballicora]], [[CCC]], April 02, 2009</ref>, independently described by [[Amir Ban]] in 2012 for [[Junior|Junior's]] evaluation learning <ref>[[Amir Ban]] ('''2012'''). ''[http://www.ratio.huji.ac.il/node/2362 Automatic Learning of Evaluation, with Applications to Computer Chess]''. Discussion Paper 613, [https://en.wikipedia.org/wiki/Hebrew_University_of_Jerusalem The Hebrew University of Jerusalem] - Center for the Study of Rationality, [https://en.wikipedia.org/wiki/Givat_Ram Givat Ram]</ref>, and explicitly mentioned by [[Álvaro Begué]] in a January 2014 [[CCC]] discussion <ref>[http://www.talkchess.com/forum/viewtopic.php?t=50823&start=10 Re: How Do You Automatically Tune Your Evaluation Tables] by [[Álvaro Begué]], [[CCC]], January 08, 2014</ref>, when [[Peter Österlund]] explained [[Texel's Tuning Method]] <ref>[http://www.talkchess.com/forum/viewtopic.php?topic_view=threads&p=555522&t=50823 The texel evaluation function optimization algorithm] by [[Peter Österlund]], [[CCC]], January 31, 2014</ref>, which subsequently popularized logistic regression tuning in computer chess. [[Vladimir Medvedev|Vladimir Medvedev's]] [[Point Value by Regression Analysis]] <ref>[http://habrahabr.ru/post/254753/ Определяем веса шахматных фигур регрессионным анализом / Хабрахабр] by [[Vladimir Medvedev|WinPooh]], April 27, 2015 (Russian)</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=56168 Piece weights with regression analysis (in Russian)] by [[Vladimir Medvedev]], [[CCC]], April 30, 2015</ref> experiments showed why the [https://en.wikipedia.org/wiki/Logistic_function logistic function] is appropriate, and further used [https://en.wikipedia.org/wiki/Cross_entropy cross-entropy] and [https://en.wikipedia.org/wiki/Regularization_%28mathematics%29 regularization].| [[FILE:SigmoidTexelTune.gif|border|left|thumb|baseline|300px|link=http://wolfr.am/1al3d5B|[https://en.wikipedia.org/wiki/Logistic_function Logistic function] <ref>[http://wolfr.am/1al3d5B log-linear 1 / (1 + 10^(-s/4)) , s=-10 to 10] from [https://en.wikipedia.org/wiki/Wolfram_Alpha Wolfram|Alpha]</ref> ]] |}

* [[Donald H. Mitchell]] ('''1984'''). ''Using Features to Evaluate Positions in Experts' and Novices' Othello Games''. Masters Master thesis, Department of Psychology, [[Northwestern University]], Evanston, IL

* [[Michael Buro]] ('''1995'''). ''[httphttps://www.jair.org/papersindex.php/jair/article/paper179.html view/10146 Statistical Feature Combination for the Evaluation of Game Positions]''. [https://en.wikipedia.org/wiki/Journal_of_Artificial_Intelligence_Research JAIR], Vol. 3

* [[Johannes Fürnkranz]] ('''2000'''). ''Machine Learning in Games: A Survey''. [https://en.wikipedia.org/wiki/Austrian_Research_Institute_for_Artificial_Intelligence Austrian Research Institute for Artificial Intelligence], OEFAI-TR-2000-3, [httphttps://wwwfmfi-uk.ofaihq.atsk/Informatika/cgi-binStrojove%20Ucenie/get-tr?download=1&paper=oefai-tr-2000-31.pdf pdf]

* [[Henk Mannen]], [[Marco Wiering]] ('''2004'''). ''[https://www.semanticscholar.org/paper/Learning-to-Play-Chess-using-TD(lambda)-learning-Mannen-Wiering/00a6f81c8ebe8408c147841f26ed27eb13fb07f3 Learning to play chess using TD(λ)-learning with database games]''. [http://students.uu.nl/en/hum/cognitive-artificial-intelligence Cognitive Artificial Intelligence], [https://en.wikipedia.org/wiki/Utrecht_University Utrecht University], Benelearn’04, [https://www.ai.rug.nl/~mwiering/GROUP/ARTICLES/learning-chess.pdf pdf]

* [[Joel Veness]], [[David Silver]], [[William Uther]], [[Alan Blair]] ('''2009'''). ''[http://papers.nips.cc/paper/3722-bootstrapping-from-game-tree-search Bootstrapping from Game Tree Search]''. [http://nips.cc/ Neural Information Processing Systems (NIPS), 2009], [http://booksjveness.nipsinfo/publications/nips2009%20-%20bootstrapping%20from%20game%20tree%20search.ccpdf pdf] » [[Meep]] <ref>[http:/papers/fileswww.talkchess.com/nips22forum/NIPS2009_0508viewtopic.pdf pdfphp?start=0&t=31667 A paper about parameter tuning] by [[Rémi Coulom]], [[CCC]], January 12, 2010</ref>

* [[Andrew Grant]] ('''2020'''). ''Evaluation & Tuning in Chess Engines''. [https://github.com/AndyGrant/Ethereal/blob/master/Tuning.pdf pdf] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74877 Evaluation & Tuning in Chess Engines] by [[Andrew Grant]], [[CCC]], August 24, 2020</ref>

: <span id="SGD"></span>[https://en.wikipedia.org/wiki/Stochastic_gradient_descent Stochastic gradient descent from Wikipedia](SGD)

* <span id="Rockstar"></span>[https://github.com/lantonovfacebookresearch/Rockstar Rockstarnevergrad GitHub - facebookresearch/nevergrad: Implementation of ROCKA Python toolbox for performing gradient-free optimization]* algorithm (Gaussian kernel regression + natural gradient descent) for optimisation | [https://github.com/fsmosca/Optuna-Game-Parameter-Tuner GitHub- fsmosca/Optuna-Game-Parameter-Tuner: A game search and evaluation parameter tuner using optuna framework] by [[Lyudmil Antonov]] and [[Joona KiiskiFerdinand Mosca]] » [[Automated Tuning#ROCK|ROCK*Deuterium]] <ref>[http://www.talkchess.com/forumforum3/viewtopic.php?f=2&t=65045 ROCK* black-box optimizer for chess75104 Yet another parameter tuner using optuna framework] by [[Jon DartFerdinand Mosca]], [[CCC]], August 31September 14, 20172020</ref>* [https://github.com/zamarfsmosca/spsa SPSA Tuner for Stockfish Chess Engine | Lakas GitHub- fsmosca/Lakas: Game parameter optimizer using nevergrad framework] by [[Joona KiiskiFerdinand Mosca]] » <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76024 Laskas parameter optimizer] by [[StockfishFerdinand Mosca]], [[Stockfish's Tuning MethodCCC]], December 09, 2020</ref>* [https://github.com/scikit-optimizefsmosca/scikit-optimize Mabigat GitHub - scikit-optimizefsmosca/scikit-optimizeMabigat: NNUE parameter optimizer] by [[Ferdinand Mosca]] » [[NNUE]] <ref>[http: Sequential model//www.talkchess.com/forum3/viewtopic.php?f=2&t=76917 Mabigat -based optimization with a `scipy.optimize` interfacehyperparameter optimizer for NNUE net] by [[Ferdinand Mosca]], [[CCC]], March 22, 2021</ref>

* [https://github.com/fsmoscascikit-optimize/Optunascikit-Game-Parameteroptimize GitHub -Tuner GitHub scikit- fsmoscaoptimize/Optunascikit-Gameoptimize: Sequential model-Parameter-Tunerbased optimization with a `scipy.optimize` interface]* <span id="Rockstar"></span>[https://github.com/lantonov/Rockstar Rockstar: A game search and evaluation parameter tuner using optuna frameworkImplementation of ROCK* algorithm (Gaussian kernel regression + natural gradient descent) for optimisation | GitHub] by [[Ferdinand MoscaLyudmil Antonov]] and [[Joona Kiiski]] » [[DeuteriumAutomated Tuning#ROCK|ROCK*]] <ref>[http://www.talkchess.com/forum3forum/viewtopic.php?f=2&t=75104 Yet another parameter tuner using optuna framework65045 ROCK* black-box optimizer for chess] by [[Ferdinand MoscaJon Dart]], [[CCC]], September 14August 31, 20202017</ref>* [https://github.com/zamar/spsa SPSA Tuner for Stockfish Chess Engine | GitHub] by [[Joona Kiiski]] » [[Stockfish]], [[Stockfish's Tuning Method]]