Difference between revisions of "Book Learning"

From Chessprogramming wiki
Jump to: navigation, search
 
(3 intermediate revisions by the same user not shown)
Line 4: Line 4:
  
 
'''Book Learning''',<br/>
 
'''Book Learning''',<br/>
is a technique of unsupervised [[Reinforcement Learning|reinforcement learning]] motivated by basic requirements for successfully playing a sequence of games, such as avoiding losing games in the same way from certain lines from the [[Opening Book|opening book]], or to recognize its own strong points and guide the engine into positions it does well <ref>[http://www.top-5000.nl/authors/rebel/hints.htm Learning Methods] by [[Ed Schroder|Ed Schröder]]</ref>.  
+
is a technique of unsupervised [[Reinforcement Learning|reinforcement learning]] motivated by basic requirements for successfully playing a sequence of games, such as avoiding losing games in the same way from certain lines from the [[Opening Book|opening book]], or to recognize its own strong points and guide the engine into positions it does well.  
  
 
Unsupervised reinforcement learning is characterized by [[Trial and Error|trial and error]] and feeding back reinforcement signals, in this context to modify move probabilities or to add new book moves. The signals are either based on the final outcome of the game, called result-driven learning, and/or by [[Search|search-driven]] learning, i.e. from the [[Score|score]] and trend of first few searches after leaving the book to determine how the program likes the position <ref>[[Robert Hyatt]] ('''1999'''). ''[http://www.craftychess.com/hyatt/learning.html Book Learning - a Methodology to Tune an Opening Book Automatically]''. [[ICGA Journal#22_1|ICCA Journal, Vol. 22, No. 1]]</ref>.
 
Unsupervised reinforcement learning is characterized by [[Trial and Error|trial and error]] and feeding back reinforcement signals, in this context to modify move probabilities or to add new book moves. The signals are either based on the final outcome of the game, called result-driven learning, and/or by [[Search|search-driven]] learning, i.e. from the [[Score|score]] and trend of first few searches after leaving the book to determine how the program likes the position <ref>[[Robert Hyatt]] ('''1999'''). ''[http://www.craftychess.com/hyatt/learning.html Book Learning - a Methodology to Tune an Opening Book Automatically]''. [[ICGA Journal#22_1|ICCA Journal, Vol. 22, No. 1]]</ref>.
Line 46: Line 46:
 
* [[Johannes Fürnkranz]] ('''2000''') ''Machine Learning in Games: A Survey''. Austrian Research Institute for Artificial Intelligence, Technical Report OEFAI-TR-2000-3, [http://www.ofai.at/cgi-bin/get-tr?download=1&paper=oefai-tr-2000-31.pdf pdf] - Chapter 2, Book Learning
 
* [[Johannes Fürnkranz]] ('''2000''') ''Machine Learning in Games: A Survey''. Austrian Research Institute for Artificial Intelligence, Technical Report OEFAI-TR-2000-3, [http://www.ofai.at/cgi-bin/get-tr?download=1&paper=oefai-tr-2000-31.pdf pdf] - Chapter 2, Book Learning
 
* [[Marty Hirsch]] ('''2001'''). ''Machine Learning in MChess Professional'' - [[Advances in Computer Games 9]]
 
* [[Marty Hirsch]] ('''2001'''). ''Machine Learning in MChess Professional'' - [[Advances in Computer Games 9]]
* [[Marek Strejczek]] ('''2004'''). ''Some aspects of chess programming''. [[Technical University of Łódź]] , Faculty of Electrical and Electronic Engineering, Department of Computer Science, 4.2 Experiments with book learning
+
* [[Marek Strejczek]] ('''2004'''). ''Some aspects of chess programming''. M.Sc. thesis, [[Technical University of Łódź]], 4.2 Experiments with book learning
 
* [[Thomas Widjaja]] ('''2004'''). ''Knowledge Engineering und Lernen in Spielen - Opening Book Learning''. [http://www.ke.tu-darmstadt.de/lehre/archiv/ss04/se-spiele/OpeningBookLearning.pdf slides as pdf] (German)
 
* [[Thomas Widjaja]] ('''2004'''). ''Knowledge Engineering und Lernen in Spielen - Opening Book Learning''. [http://www.ke.tu-darmstadt.de/lehre/archiv/ss04/se-spiele/OpeningBookLearning.pdf slides as pdf] (German)
 
* [[Chrilly Donninger]], [[Ulf Lorenz]] ('''2006'''). ''[http://www.springerlink.com/content/b01701g6kx317574/ Innovative Opening-Book Handling]''. [[Advances in Computer Games 11]], [http://www2.cs.uni-paderborn.de/cs/ag-monien/PERSONAL/FLULO/publications/icga_open_springer.pdf pdf]
 
* [[Chrilly Donninger]], [[Ulf Lorenz]] ('''2006'''). ''[http://www.springerlink.com/content/b01701g6kx317574/ Innovative Opening-Book Handling]''. [[Advances in Computer Games 11]], [http://www2.cs.uni-paderborn.de/cs/ag-monien/PERSONAL/FLULO/publications/icga_open_springer.pdf pdf]
Line 61: Line 61:
 
* [http://www.open-aurec.com/wbforum/viewtopic.php?f=18&t=37030 question about opening book and learning] by [[Uri Blass]],  [[Computer Chess Forums|Winboard Forum]], April 26, 2002
 
* [http://www.open-aurec.com/wbforum/viewtopic.php?f=18&t=37030 question about opening book and learning] by [[Uri Blass]],  [[Computer Chess Forums|Winboard Forum]], April 26, 2002
 
* [https://www.stmintz.com/ccc/index.php?id=226258 question about book and learning] by [[Uri Blass]], [[CCC]], April 26, 2002
 
* [https://www.stmintz.com/ccc/index.php?id=226258 question about book and learning] by [[Uri Blass]], [[CCC]], April 26, 2002
* [https://www.stmintz.com/ccc/index.php?id=294486 Shredder and Engine Book Learning] by Stephen Ham, [[CCC]], April 24, 2003 » [[Shredder]]
+
* [https://www.stmintz.com/ccc/index.php?id=294486 Shredder and Engine Book Learning] by [[Stephen Ham]], [[CCC]], April 24, 2003 » [[Shredder]]
 
* [http://www.talkchess.com/forum/viewtopic.php?t=21754 A simple book learning method] by [[Alvaro Cardoso]], [[CCC]], June 12, 2008 » [[Persistent Hash Table]]
 
* [http://www.talkchess.com/forum/viewtopic.php?t=21754 A simple book learning method] by [[Alvaro Cardoso]], [[CCC]], June 12, 2008 » [[Persistent Hash Table]]
 
==2010 ...==
 
==2010 ...==
Line 72: Line 72:
  
 
=External Links=
 
=External Links=
* [http://www.top-5000.nl/authors/rebel/hints.htm Learning Methods] by [[Ed Schroder|Ed Schröder]]
+
* [[:Category:Stanley Clarke|Stanley Clarke]] - [https://en.wikipedia.org/wiki/School_Days_(album) School Days] (1976), [https://en.wikipedia.org/wiki/YouTube YouTube] Video
 +
: {{#evu:https://www.youtube.com/watch?v=Xk_gzDQd4zw|alignment=left|valignment=top}}
  
 
=References=  
 
=References=  
Line 78: Line 79:
  
 
'''[[Learning|Up one Level]]'''
 
'''[[Learning|Up one Level]]'''
 +
[[Category:Stanley Clarke]]
 +
[[Category:Gerry Brown]]

Latest revision as of 23:48, 10 May 2021

Home * Learning * Book Learning

Book Learning [1]

Book Learning,
is a technique of unsupervised reinforcement learning motivated by basic requirements for successfully playing a sequence of games, such as avoiding losing games in the same way from certain lines from the opening book, or to recognize its own strong points and guide the engine into positions it does well.

Unsupervised reinforcement learning is characterized by trial and error and feeding back reinforcement signals, in this context to modify move probabilities or to add new book moves. The signals are either based on the final outcome of the game, called result-driven learning, and/or by search-driven learning, i.e. from the score and trend of first few searches after leaving the book to determine how the program likes the position [2].

Chess Engines

Crafty

Robert Hyatt on Book learning in Crafty [3]:

  • Crafty does two kinds of 'book learning'. Result learning sound just like what you are doing. If it loses a game, it won't play the last book move it played ever again. And this gets backed up several moves back as well to get it out of bad lines quickly.
  • Normal book learning uses the first ten non-book moves (search results) to determine the 'trend' of the game. This trend score is then used to adjust the book move that was played in this game...

Both are on by default... and further [4]:

  1. Crafty remembers the evaluations for the first 10 moves out of book, after each search has been completed. It uses these evaluations to detect a "trend". IE is the evaluation good and getting better? Is it bad and getting worse? Is it good but dropping (ie it grabbed a gambit pawn and is beginning to see that it was bad) or is it bad but getting better (IE it offered a gambit, the opponent took it, and the score is going up). It factors all of that together and marks the book line as good or bad.
  2. Crafty takes the result of a game when it loses, and updates the book line so that moves tried near the end of the line simply don't get played, and alternatives near the front of the book line get tried next.
  3. There is more to it than that, and you can look at the crafty.doc file [5] to at least see what I am doing in more detail. It is very effective.

MChess

Marty Hirsch in Advances in Computer Games 9 on machine learning in MChess Pro, excerpt [6]

Some sets of heuristics are employed to decide which moves to add to the opening book, and which to delete from it. Moves are deleted when the score just out of book is not too high, and the score later in the game is worse. Moves are added when the score just out of book is not too low, and the score later in the game is satisfactory. A further pre-condition for moves that may be added to the opening book is that the increase in score has been mainly monotonic. If this condition is not met, a future game may enable better moves to be found due to the general learning. In other words, only the best moves the program is able to discover (at a given level) are added to the opening book. Different thresholds are applied for the storage and deletion of moves for White and Black. 
...
Some programs now support a different type of book learning in which no moves are added to the book, but instead, unsuccessful moves are given a lower probability of selection. This approach narrows the book over the time and may optimize the results against a specific opponent, at the expense of the results against other opponents. The book learning described in this paper maintains the opening book variety and improves the book against multiple opponents. 

RomiChess

As explained by Michael Sherwin, RomiChess applies Monkey see Monkey do mimicry to learn book lines [7] :

Romi remembers and incorporates winning lines regardless of which side played the moves into the opening book and can play them back instantly up to 180 ply if the stats for that line remain good. 

See also

Selected Publications

Forum Posts

1998 ...

2000 ...

2010 ...

External Links

References

  1. France in 2000 year (XXI century). Future school. France, paper card, A reproduction of the early 20th century, scan, Learning from Wikipedia
  2. Robert Hyatt (1999). Book Learning - a Methodology to Tune an Opening Book Automatically. ICCA Journal, Vol. 22, No. 1
  3. Re: Book learning by Robert Hyatt, CCC, September 12, 1999
  4. Re: questions about book learning by Robert Hyatt, CCC, January 03, 2004
  5. Crafty Documentation
  6. Marty Hirsch (2001). Machine Learning in MChess Professional - Advances in Computer Games 9, pp. 133-142, 135
  7. Re: Positional learning by Michael Sherwin, CCC, December 18, 2010
  8. Anybody tried Logistello's book learning for chess? by Rémi Coulom, CCC, December 29, 2013

Up one Level