Book Learning

Home * Learning * Book Learning



Book Learning, is a technique of unsupervised reinforcement learning motivated by basic requirements for successfully playing a sequence of games, such as avoiding losing games in the same way from certain lines from the opening book, or to recognize its own strong points and guide the engine into positions it does well.

Unsupervised reinforcement learning is characterized by trial and error and feeding back reinforcement signals, in this context to modify move probabilities or to add new book moves. The signals are either based on the final outcome of the game, called result-driven learning, and/or by search-driven learning, i.e. from the score and trend of first few searches after leaving the book to determine how the program likes the position.

=Chess Engines=

Crafty
Robert Hyatt on Book learning in Crafty :

Both are on by default... and further :

MChess
Marty Hirsch in Advances in Computer Games 9 on machine learning in MChess Pro, excerpt Some sets of heuristics are employed to decide which moves to add to the opening book, and which to delete from it. Moves are deleted when the score just out of book is not too high, and the score later in the game is worse. Moves are added when the score just out of book is not too low, and the score later in the game is satisfactory. A further pre-condition for moves that may be added to the opening book is that the increase in score has been mainly monotonic. If this condition is not met, a future game may enable better moves to be found due to the general learning. In other words, only the best moves the program is able to discover (at a given level) are added to the opening book. Different thresholds are applied for the storage and deletion of moves for White and Black. ... Some programs now support a different type of book learning in which no moves are added to the book, but instead, unsuccessful moves are given a lower probability of selection. This approach narrows the book over the time and may optimize the results against a specific opponent, at the expense of the results against other opponents. The book learning described in this paper maintains the opening book variety and improves the book against multiple opponents.

RomiChess
As explained by Michael Sherwin, RomiChess applies Monkey see Monkey do mimicry to learn book lines : Romi remembers and incorporates winning lines regardless of which side played the moves into the opening book and can play them back instantly up to 180 ply if the stats for that line remain good.

=See also=
 * Brainfish
 * Match Statistics
 * Opening Book
 * Persistent Hash Table
 * PolyGlot
 * Reinforcement Learning
 * Trial and Error

=Selected Publications=
 * Arthur Samuel (1959). Some Studies in Machine Learning Using the Game of Checkers. IBM Journal July 1959
 * Arthur Samuel (1967). Some Studies in Machine Learning. Using the Game of Checkers. II-Recent Progress. pdf
 * Michael Byrne (1996). M-Chess Pro 5.0 Book Learning Feature. Computer Chess Reports, Vol. 5, Nos. 3+4, p. 85 » MChess
 * Robert Hyatt (1999). Book Learning - a Methodology to Tune an Opening Book Automatically. ICCA Journal, Vol. 22, No. 1
 * Michael Buro (1999). Toward Opening Book Learning. ICCA Journal, Vol. 22, No. 2, pdf
 * Johannes Fürnkranz (2000) Machine Learning in Games: A Survey. Austrian Research Institute for Artificial Intelligence, Technical Report OEFAI-TR-2000-3, pdf - Chapter 2, Book Learning
 * Marty Hirsch (2001). Machine Learning in MChess Professional - Advances in Computer Games 9
 * Marek Strejczek (2004). Some aspects of chess programming. M.Sc. thesis, Technical University of Łódź, 4.2 Experiments with book learning
 * Thomas Widjaja (2004). Knowledge Engineering und Lernen in Spielen - Opening Book Learning. slides as pdf (German)
 * Chrilly Donninger, Ulf Lorenz (2006). Innovative Opening-Book Handling. Advances in Computer Games 11, pdf

=Forum Posts=

1998 ...

 * Book learning and rating bias by Don Dailey, CCC, May 01, 1998
 * BookLearning Under the Microscope!!! by Robert Henry Durrett, CCC, August 31, 1998
 * Book learning? by Werner Inmann, CCC, December 31, 1998
 * Book learning by James Robertson, CCC, September 12, 1999

2000 ...

 * Opening book learning by Gordon Rattray, CCC, October 31, 2000
 * What information to store in book learning? by Christian Söderström, CCC, January 02, 2001
 * question about opening book and learning by Uri Blass, Winboard Forum, April 26, 2002
 * question about book and learning by Uri Blass, CCC, April 26, 2002
 * Shredder and Engine Book Learning by Stephen Ham, CCC, April 24, 2003 » Shredder
 * A simple book learning method by Alvaro Cardoso, CCC, June 12, 2008 » Persistent Hash Table

2010 ...

 * Computer based Opening theory by Lucas Braesch, CCC, December 28, 2013
 * Re: Computer based Opening theory by Marcel van Kervinck, CCC, December 28, 2013
 * Anybody tried Logistello's book learning for chess? by Rémi Coulom, CCC, December 29, 2013
 * Position learning and opening books by Forrest Hoch, CCC, May 11, 2015
 * Polyglot book learning by Günther Simon, CCC, August 24, 2016 » PolyGlot
 * Cumulative building of a shared search tree by Bojun Guo, CCC, December 28, 2016 » Chinese Chess, Opening Book, Persistent Hash Table

=External Links=
 * Learning Methods by Ed Schröder

=References=

Up one Level