25,161
edits
Changes
Created page with "'''Home * Engines * NeuroChess''' '''NeuroChess''',<br/> an experimental program which learns to play chess from the final outcome of..."
'''[[Main Page|Home]] * [[Engines]] * NeuroChess'''
'''NeuroChess''',<br/>
an experimental program which [[Learning#Programs|learns]] to play chess from the final outcome of games written by [[Sebastian Thrun]] in the early mid 90s, at that time at [[Carnegie Mellon University]] supported by [[Tom Mitchell]] and in particular on the domain of chess advised by [[Hans Berliner]] <ref>[[Sebastian Thrun]] ('''1995'''). ''[http://robots.stanford.edu/papers/thrun.nips7.neuro-chess.html Learning to Play the Game of Chess]''. Acknowledgment</ref> . NeuroChess took its [[Search|search algorithm]] from [[GNU Chess]] but its [[Evaluation|evaluation]] was based on [[Neural Networks|neural networks]] to integrate [https://en.wikipedia.org/wiki/Inductive_reasoning inductive] neural network learning, [[Temporal Difference Learning|temporal difference learning]], and a variant of [https://en.wikipedia.org/wiki/Explanation-based_learning explanation-based learning] <ref>[[Sebastian Thrun]], [[Tom Mitchell]] ('''1993'''). ''Integrating Inductive Neural Network Learning and Explanation-Based Learning''. [[Conferences#IJCAI1993|IJCAI 1993]], [http://robots.stanford.edu/papers/thrun.EBNN_ijcai93.ps.gz zipped ps]</ref> <ref>[[Sebastian Thrun]] ('''1995'''). ''Explanation-Based Neural Network Learning - A Lifelong Learning Approach''. Ph.D. thesis, [https://en.wikipedia.org/wiki/University_of_Bonn University of Bonn]</ref> .
=Architecture=
Using the raw [[Board Representation|board representaion]] as input representation of a neural network is a poor choice, since small changes on the board can cause huge differences in value contrasting the smooth nature of neural network representations. Therefore NeuroChess maps its internal board into a set of 175 carefully designed features. A specially trained, but otherwise conventional three layer feed forward neural network (V) with one final output maps the feature vector to an [[Score|evaluation score]].
The evaluation network V was trained by [[Neural Networks#Backpropagation|backpropagation]] and the [[Temporal Difference Learning|temporal difference method]] (TD(0)), utilizing a second neural network, an explanation based neural network (EBNN) dubbed the chess model M, which maps 175 inputs via 165 hidden units to 175 outputs, and aims to predict the (important) board features two plies ahead. This network M is trained independently prior to V to incorporate supervised domain knowledge.
[[FILE:NeuroChess.jpg|none|border|text-bottom]]
'''Learning an evaluation function in NeuroChess'''. Boards are mapped into a high-dimensional Feature vector, which forms the input
for both the evaluation network V and the chess model M. The evaluation network is trained by [[Neural Networks#Backpropagation|Backpropagation]] and the [[Temporal Difference Learning|TD(O)]] procedure.
Both networks are employed for analyzing training example in order to derive target slopes for V. <ref>Figure 2 from [[Sebastian Thrun]] ('''1995'''). ''[http://robots.stanford.edu/papers/thrun.nips7.neuro-chess.html Learning to Play the Game of Chess]''. [http://papers.nips.cc/paper/1007-learning-to-play-the-game-of-chess.pdf pdf]</ref>
=Conclusion=
After training M with 120,000 grandmaster games, and training V with a further 2400 games, NeuroChess managed to beat [[GNU Chess]] in about 13% of the time at fixed depth 2 games, but only in 10% without EBNN. Further, computing a large neural network function took two [https://en.wikipedia.org/wiki/Order_of_magnitude orders of magnitude] longer than evaluating the linear evaluation function of GNU Chess.
=See also=
* [[Neural Networks#engines|Chess Engines with Neural Networks]]
* [[Learning#Programs|Learning Chess Programs]]
=Publications=
* [[Sebastian Thrun]] ('''1995'''). ''[http://robots.stanford.edu/papers/thrun.nips7.neuro-chess.html Learning to Play the Game of Chess]''. in [[Gerald Tesauro]], [https://en.wikipedia.org/wiki/David_S._Touretzky David S. Touretzky], [http://mitpress.mit.edu/authors/todd-k-leen Todd K. Leen] (eds.) Advances in Neural Information Processing Systems 7, [https://en.wikipedia.org/wiki/MIT_Press MIT Press], [http://papers.nips.cc/paper/1007-learning-to-play-the-game-of-chess.pdf pdf]
* [[Johannes Fürnkranz]], [[Miroslav Kubat]] ('''2001'''). ''[https://www.novapublishers.com/catalog/product_info.php?products_id=720 Machines that Learn to Play Games]''. Advances in Computation: Theory and Practice, Vol. 8,. [https://en.wikipedia.org/wiki/Nova_Publishers NOVA Science Publishers]
* [[Jacek Mańdziuk]] ('''2007'''). ''[http://link.springer.com/chapter/10.1007/978-3-540-71984-7_15 Computational Intelligence in Mind Games]''. in [[Włodzisław Duch]], [[Jacek Mańdziuk]] (eds.) ''[http://link.springer.com/book/10.1007%2F978-3-540-71984-7 Challenges for Computational Intelligence]''. [http://link.springer.com/bookseries/7092 Studies in Computational Intelligence], Vol. 63, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Jacek Mańdziuk]] ('''2010'''). ''[http://link.springer.com/book/10.1007%2F978-3-642-11678-0 Knowledge-Free and Learning-Based Methods in Intelligent Game Playing]''. [http://link.springer.com/bookseries/7092 Studies in Computational Intelligence], Vol. 276, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], pp. 124
* [[István Szita]] ('''2012'''). ''[http://link.springer.com/chapter/10.1007%2F978-3-642-27645-3_17 Reinforcement Learning in Games]''. in [[Marco Wiering]], [http://martijnvanotterlo.nl/ Martijn Van Otterlo] (eds.). ''[https://scholar.google.com/citations?view_op=view_citation&hl=en&user=xVas0I8AAAAJ&citation_for_view=xVas0I8AAAAJ:abG-DnoFyZgC Reinforcement learning: State-of-the-art]''. [http://link.springer.com/book/10.1007/978-3-642-27645-3 Adaptation, Learning, and Optimization, Vol. 12], [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
=Forum Posts=
* [http://talkchess.com/forum/viewtopic.php?t=39342&start=1 Re: Chess Evaluation] by [[Srdja Matovic]], [[CCC]], June 11, 2011
* [http://www.talkchess.com/forum/viewtopic.php?t=40290 Sal or neurochess] by ethan ara, [[CCC]], September 06, 2011
=External Links=
* [http://satirist.org/learn-game/systems/neurochess.html Sebastian Thrun’s NeuroChess] from [http://satirist.org/learn-game/ Machine Learning in Games] by [[Jay Scott]], September 2000
=References=
<references />
'''[[Engines|Up one level]]'''
'''NeuroChess''',<br/>
an experimental program which [[Learning#Programs|learns]] to play chess from the final outcome of games written by [[Sebastian Thrun]] in the early mid 90s, at that time at [[Carnegie Mellon University]] supported by [[Tom Mitchell]] and in particular on the domain of chess advised by [[Hans Berliner]] <ref>[[Sebastian Thrun]] ('''1995'''). ''[http://robots.stanford.edu/papers/thrun.nips7.neuro-chess.html Learning to Play the Game of Chess]''. Acknowledgment</ref> . NeuroChess took its [[Search|search algorithm]] from [[GNU Chess]] but its [[Evaluation|evaluation]] was based on [[Neural Networks|neural networks]] to integrate [https://en.wikipedia.org/wiki/Inductive_reasoning inductive] neural network learning, [[Temporal Difference Learning|temporal difference learning]], and a variant of [https://en.wikipedia.org/wiki/Explanation-based_learning explanation-based learning] <ref>[[Sebastian Thrun]], [[Tom Mitchell]] ('''1993'''). ''Integrating Inductive Neural Network Learning and Explanation-Based Learning''. [[Conferences#IJCAI1993|IJCAI 1993]], [http://robots.stanford.edu/papers/thrun.EBNN_ijcai93.ps.gz zipped ps]</ref> <ref>[[Sebastian Thrun]] ('''1995'''). ''Explanation-Based Neural Network Learning - A Lifelong Learning Approach''. Ph.D. thesis, [https://en.wikipedia.org/wiki/University_of_Bonn University of Bonn]</ref> .
=Architecture=
Using the raw [[Board Representation|board representaion]] as input representation of a neural network is a poor choice, since small changes on the board can cause huge differences in value contrasting the smooth nature of neural network representations. Therefore NeuroChess maps its internal board into a set of 175 carefully designed features. A specially trained, but otherwise conventional three layer feed forward neural network (V) with one final output maps the feature vector to an [[Score|evaluation score]].
The evaluation network V was trained by [[Neural Networks#Backpropagation|backpropagation]] and the [[Temporal Difference Learning|temporal difference method]] (TD(0)), utilizing a second neural network, an explanation based neural network (EBNN) dubbed the chess model M, which maps 175 inputs via 165 hidden units to 175 outputs, and aims to predict the (important) board features two plies ahead. This network M is trained independently prior to V to incorporate supervised domain knowledge.
[[FILE:NeuroChess.jpg|none|border|text-bottom]]
'''Learning an evaluation function in NeuroChess'''. Boards are mapped into a high-dimensional Feature vector, which forms the input
for both the evaluation network V and the chess model M. The evaluation network is trained by [[Neural Networks#Backpropagation|Backpropagation]] and the [[Temporal Difference Learning|TD(O)]] procedure.
Both networks are employed for analyzing training example in order to derive target slopes for V. <ref>Figure 2 from [[Sebastian Thrun]] ('''1995'''). ''[http://robots.stanford.edu/papers/thrun.nips7.neuro-chess.html Learning to Play the Game of Chess]''. [http://papers.nips.cc/paper/1007-learning-to-play-the-game-of-chess.pdf pdf]</ref>
=Conclusion=
After training M with 120,000 grandmaster games, and training V with a further 2400 games, NeuroChess managed to beat [[GNU Chess]] in about 13% of the time at fixed depth 2 games, but only in 10% without EBNN. Further, computing a large neural network function took two [https://en.wikipedia.org/wiki/Order_of_magnitude orders of magnitude] longer than evaluating the linear evaluation function of GNU Chess.
=See also=
* [[Neural Networks#engines|Chess Engines with Neural Networks]]
* [[Learning#Programs|Learning Chess Programs]]
=Publications=
* [[Sebastian Thrun]] ('''1995'''). ''[http://robots.stanford.edu/papers/thrun.nips7.neuro-chess.html Learning to Play the Game of Chess]''. in [[Gerald Tesauro]], [https://en.wikipedia.org/wiki/David_S._Touretzky David S. Touretzky], [http://mitpress.mit.edu/authors/todd-k-leen Todd K. Leen] (eds.) Advances in Neural Information Processing Systems 7, [https://en.wikipedia.org/wiki/MIT_Press MIT Press], [http://papers.nips.cc/paper/1007-learning-to-play-the-game-of-chess.pdf pdf]
* [[Johannes Fürnkranz]], [[Miroslav Kubat]] ('''2001'''). ''[https://www.novapublishers.com/catalog/product_info.php?products_id=720 Machines that Learn to Play Games]''. Advances in Computation: Theory and Practice, Vol. 8,. [https://en.wikipedia.org/wiki/Nova_Publishers NOVA Science Publishers]
* [[Jacek Mańdziuk]] ('''2007'''). ''[http://link.springer.com/chapter/10.1007/978-3-540-71984-7_15 Computational Intelligence in Mind Games]''. in [[Włodzisław Duch]], [[Jacek Mańdziuk]] (eds.) ''[http://link.springer.com/book/10.1007%2F978-3-540-71984-7 Challenges for Computational Intelligence]''. [http://link.springer.com/bookseries/7092 Studies in Computational Intelligence], Vol. 63, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Jacek Mańdziuk]] ('''2010'''). ''[http://link.springer.com/book/10.1007%2F978-3-642-11678-0 Knowledge-Free and Learning-Based Methods in Intelligent Game Playing]''. [http://link.springer.com/bookseries/7092 Studies in Computational Intelligence], Vol. 276, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], pp. 124
* [[István Szita]] ('''2012'''). ''[http://link.springer.com/chapter/10.1007%2F978-3-642-27645-3_17 Reinforcement Learning in Games]''. in [[Marco Wiering]], [http://martijnvanotterlo.nl/ Martijn Van Otterlo] (eds.). ''[https://scholar.google.com/citations?view_op=view_citation&hl=en&user=xVas0I8AAAAJ&citation_for_view=xVas0I8AAAAJ:abG-DnoFyZgC Reinforcement learning: State-of-the-art]''. [http://link.springer.com/book/10.1007/978-3-642-27645-3 Adaptation, Learning, and Optimization, Vol. 12], [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
=Forum Posts=
* [http://talkchess.com/forum/viewtopic.php?t=39342&start=1 Re: Chess Evaluation] by [[Srdja Matovic]], [[CCC]], June 11, 2011
* [http://www.talkchess.com/forum/viewtopic.php?t=40290 Sal or neurochess] by ethan ara, [[CCC]], September 06, 2011
=External Links=
* [http://satirist.org/learn-game/systems/neurochess.html Sebastian Thrun’s NeuroChess] from [http://satirist.org/learn-game/ Machine Learning in Games] by [[Jay Scott]], September 2000
=References=
<references />
'''[[Engines|Up one level]]'''
