Changes

an [https://en.wikipedia.org/wiki/Automation automated] adjustment of [[Evaluation|evaluation]] parameters or weights, and less commonly, [[Search|search]] parameters <ref>[[Yngvi Björnsson]], [[Tony Marsland]] ('''2001'''). ''Learning Search Control in Adversary Games''. [[Advances in Computer Games 9]], pp. 157-174. [http://www.ru.is/faculty/yngvi/pdf/BjornssonM01b.pdf pdf]</ref>, with the aim to improve the [[Playing Strength|playing strength]] of a chess engine or game playing program. Evaluation tuning can be applied by [[Automated Tuning#Optimization|mathematical optimization]] or [[Learning|machine learning]], both fields with huge overlaps. Learning approaches are subdivided into [[Automated Tuning#SupervisedLearning|supervised learning]] using [https://en.wikipedia.org/wiki/Training_set labeled data], and [[Automated Tuning#ReinformentLearning|reinforcement learning]] to learn from trying, facing the exploration (of uncharted territory) and exploitation (of current knowledge) dilemma. [[Johannes Fürnkranz]] gives a comprehensive overview in ''Machine Learning in Games: A Survey'' published in 2000 <ref>[[Johannes Fürnkranz]] ('''2000'''). ''Machine Learning in Games: A Survey''. [https://en.wikipedia.org/wiki/Austrian_Research_Institute_for_Artificial_Intelligence Austrian Research Institute for Artificial Intelligence], OEFAI-TR-2000-3, [httphttps://wwwfmfi-uk.ofaihq.atsk/Informatika/cgi-binStrojove%20Ucenie/get-tr?download=1&paper=oefai-tr-2000-31.pdf pdf] - Chapter 4, Evaluation Function Tuning</ref>, covering evaluation tuning in chapter 4.

<span id="MoveAdaption"></span>One [[Supervised Learning|supervised learning]] method considers desired moves from a set of positions, likely from grandmaster games, and tries to adjust their evaluation weights so that for instance a one-ply search agrees with the desired move. Already pioneering in reinforcement learning some years before, move adaptation was described by [[Arthur Samuel]] in 1967 as used in the second version of his checkers player <ref>[[Arthur Samuel]] ('''1967'''). ''Some Studies in Machine Learning. Using the Game of Checkers. II-Recent Progress''. [http://researcher.watson.ibm.com/researcher/files/us-beygel/samuel-checkers.pdf pdf]</ref>, where a structure of stacked linear evaluation functions was trained by computing a correlation measure based on the number of times the feature rated an alternative move higher than the desired move played by an expert <ref>[[Johannes Fürnkranz]] ('''2000'''). ''Machine Learning in Games: A Survey''. [https://en.wikipedia.org/wiki/Austrian_Research_Institute_for_Artificial_Intelligence Austrian Research Institute for Artificial Intelligence], OEFAI-TR-2000-3, [httphttps://wwwfmfi-uk.ofaihq.atsk/Informatika/cgi-binStrojove%20Ucenie/get-tr?download=1&paper=oefai-tr-2000-31.pdf pdf]</ref>. In chess, move adaptation was first described by [[Thomas Nitsche]] in 1982 <ref>[[Thomas Nitsche]] ('''1982'''). ''A Learning Chess Program.'' [[Advances in Computer Chess 3]]</ref>, and with some extensions by [[Tony Marsland]] in 1985 <ref>[[Tony Marsland]] ('''1985'''). ''Evaluation-Function Factors''. [[ICGA Journal#8_2|ICCA Journal, Vol. 8, No. 2]], [http://webdocs.cs.ualberta.ca/~tony/OldPapers/evaluation.pdf pdf]</ref>. [[Eval Tuning in Deep Thought]] as mentioned by [[Feng-hsiung Hsu]] et al. in 1990 <ref>[[Feng-hsiung Hsu]], [[Thomas Anantharaman]], [[Murray Campbell]], [[Andreas Nowatzyk]] ('''1990'''). ''[http://www.disi.unige.it/person/DelzannoG/AI2/hsu.html A Grandmaster Chess Machine]''. [[Scientific American]], Vol. 263, No. 4, pp. 44-50. ISSN 0036-8733.</ref>, and later published by [[Andreas Nowatzyk]], is also based on an extended form of move adaptation <ref>see ''2.1 Learning from Desired Moves in Chess'' in [[Kunihito Hoki]], [[Tomoyuki Kaneko]] ('''2014'''). ''[https://www.jair.org/papers/paper4217.html Large-Scale Optimization for Evaluation Functions with Minimax Search]''. [https://www.jair.org/vol/vol49.html JAIR Vol. 49]</ref>. [[Jonathan Schaeffer|Jonathan Schaeffer's]] and [[Paul Lu|Paul Lu's]] efforts to make Deep Thought's approach work for [https://en.wikipedia.org/wiki/Chinook_%28draughts_player%29 Chinook] in 1990 failed <ref>[[Jonathan Schaeffer]], [[Joe Culberson]], [[Norman Treloar]], [[Brent Knight]], [[Paul Lu]], [[Duane Szafron]] ('''1992'''). ''A World Championship Caliber Checkers Program''. [https://en.wikipedia.org/wiki/Artificial_Intelligence_%28journal%29 Artificial Intelligence], Vol. 53, Nos. 2-3,[http://webdocs.cs.ualberta.ca/%7Ejonathan/Papers/Papers/chinook.ps ps]</ref> - nothing seemed to produce results that were as good than their hand-tuned effort <ref>[[Jonathan Schaeffer]] ('''1997, 2009'''). ''[http://www.springer.com/computer/ai/book/978-0-387-76575-4 One Jump Ahead]''. 7. The Case for the Prosecution, pp. 111-114</ref>.

* [[Johannes Fürnkranz]] ('''2000'''). ''Machine Learning in Games: A Survey''. [https://en.wikipedia.org/wiki/Austrian_Research_Institute_for_Artificial_Intelligence Austrian Research Institute for Artificial Intelligence], OEFAI-TR-2000-3, [httphttps://wwwfmfi-uk.ofaihq.atsk/Informatika/cgi-binStrojove%20Ucenie/get-tr?download=1&paper=oefai-tr-2000-31.pdf pdf]

* [[Henk Mannen]], [[Marco Wiering]] ('''2004'''). ''[https://www.semanticscholar.org/paper/Learning-to-Play-Chess-using-TD(lambda)-learning-Mannen-Wiering/00a6f81c8ebe8408c147841f26ed27eb13fb07f3 Learning to play chess using TD(λ)-learning with database games]''. [http://students.uu.nl/en/hum/cognitive-artificial-intelligence Cognitive Artificial Intelligence], [https://en.wikipedia.org/wiki/Utrecht_University Utrecht University], Benelearn’04, [https://www.ai.rug.nl/~mwiering/GROUP/ARTICLES/learning-chess.pdf pdf]

* [[Joel Veness]], [[David Silver]], [[William Uther]], [[Alan Blair]] ('''2009'''). ''[http://papers.nips.cc/paper/3722-bootstrapping-from-game-tree-search Bootstrapping from Game Tree Search]''. [http://nips.cc/ Neural Information Processing Systems (NIPS), 2009], [http://jveness.info/publications/nips2009%20-%20bootstrapping%20from%20game%20tree%20search.pdf pdf] » [[Meep]] <ref>[http://www.talkchess.com/forum/viewtopic.php?start=0&t=31667 A paper about parameter tuning] by [[Rémi Coulom]], [[CCC]], January 12, 2010</ref>

* [[Andrew Grant]] ('''2020'''). ''Evaluation & Tuning in Chess Engines''. [https://github.com/AndyGrant/Ethereal/blob/master/Tuning.pdf pdf] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74877 Evaluation & Tuning in Chess Engines] by [[Andrew Grant]], [[CCC]], August 24, 2020</ref>

: <span id="SGD"></span>[https://en.wikipedia.org/wiki/Stochastic_gradient_descent Stochastic gradient descent from Wikipedia](SGD)

* <span id="Rockstar"></span>[https://github.com/lantonovfacebookresearch/Rockstar Rockstarnevergrad GitHub - facebookresearch/nevergrad: Implementation of ROCKA Python toolbox for performing gradient-free optimization]* algorithm (Gaussian kernel regression + natural gradient descent) for optimisation | [https://github.com/fsmosca/Optuna-Game-Parameter-Tuner GitHub- fsmosca/Optuna-Game-Parameter-Tuner: A game search and evaluation parameter tuner using optuna framework] by [[Lyudmil Antonov]] and [[Joona KiiskiFerdinand Mosca]] » [[Automated Tuning#ROCK|ROCK*Deuterium]] <ref>[http://www.talkchess.com/forumforum3/viewtopic.php?f=2&t=65045 ROCK* black-box optimizer for chess75104 Yet another parameter tuner using optuna framework] by [[Jon DartFerdinand Mosca]], [[CCC]], August 31September 14, 20172020</ref>* [https://github.com/zamarfsmosca/spsa SPSA Tuner for Stockfish Chess Engine | Lakas GitHub- fsmosca/Lakas: Game parameter optimizer using nevergrad framework] by [[Joona KiiskiFerdinand Mosca]] » <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=76024 Laskas parameter optimizer] by [[StockfishFerdinand Mosca]], [[Stockfish's Tuning MethodCCC]], December 09, 2020</ref>* [https://github.com/scikit-optimizefsmosca/scikit-optimize Mabigat GitHub - scikit-optimizefsmosca/scikit-optimizeMabigat: NNUE parameter optimizer] by [[Ferdinand Mosca]] » [[NNUE]] <ref>[http: Sequential model//www.talkchess.com/forum3/viewtopic.php?f=2&t=76917 Mabigat -based optimization with a `scipy.optimize` interfacehyperparameter optimizer for NNUE net] by [[Ferdinand Mosca]], [[CCC]], March 22, 2021</ref>

* [https://github.com/fsmoscascikit-optimize/Optunascikit-Game-Parameteroptimize GitHub -Tuner GitHub scikit- fsmoscaoptimize/Optunascikit-Gameoptimize: Sequential model-Parameter-Tunerbased optimization with a `scipy.optimize` interface]* <span id="Rockstar"></span>[https://github.com/lantonov/Rockstar Rockstar: A game search and evaluation parameter tuner using optuna frameworkImplementation of ROCK* algorithm (Gaussian kernel regression + natural gradient descent) for optimisation | GitHub] by [[Ferdinand MoscaLyudmil Antonov]] and [[Joona Kiiski]] » [[DeuteriumAutomated Tuning#ROCK|ROCK*]] <ref>[http://www.talkchess.com/forum3forum/viewtopic.php?f=2&t=75104 Yet another parameter tuner using optuna framework65045 ROCK* black-box optimizer for chess] by [[Ferdinand MoscaJon Dart]], [[CCC]], September 14August 31, 20202017</ref>* [https://github.com/zamar/spsa SPSA Tuner for Stockfish Chess Engine | GitHub] by [[Joona Kiiski]] » [[Stockfish]], [[Stockfish's Tuning Method]]