Opening Book

Home * Knowledge * Opening Book



Chess programs often look up the positions at the beginning of the game in an Opening Book. The opening database can then be used as long as the opponent plays a new move from the database, so usually more common opening lines will be stored to a much higher depth than the uncommon ones. As soon as the program is "out of book" it has to continue using the normal search routines.

=Purposes=
 * Save time: Chess programs could use search from the beginning and look through all possible continuations in detail to find the best next move, but as this is quite time-consuming. In contrast, opening books can play opening moves almost instantly without searching nor heavy computing and help to save time.
 * Provide a higher quality of play: Searching even with large depths can't see deep tactics, strategy, compensation. It may lose temporarily material for long term advantage.
 * Provide variety: there already exists a lot of literature about different opening lines. As book moves are usually chosen randomly, whereas searches are more or less deterministic.

=Types= Opening books are typically stored in two main formats:

Text
The most advantage by using text form is that they are readable, understandable, and editable by humans and can be viewed, edited with normal text editors. However, the main disadvantage is that they may take too much space as well as time for searching since an item may take from 50 bytes (for FEN strings) to hundreds of bytes (for PGN text). If we store openings "continuously" (all possible opening positions) both data size and searching time become too huge and unacceptable for playing. They are usually missing weight values (to compare between items to know which ones are better) either. In practice, those opening books are usually very small with some lines or positions only, they are hard to be used for real game playing but for testing.

They are stored in typical ways for games and positions: (e4(e5)(c5))(d4d5)) etc.
 * EPD: They are the last positions of opening lines
 * PGN: Opening lines are stored as individual games
 * Special text formats: In 1999, Marc-Philippe Huget published a Book Builder in conjunction with his engine La Dame Blanche, a standalone open source program to convert a PGN file into an opening book of the format proposed by Kathe Spracklen :

Binary
Binary books have some important advantages, especially for computer chess engines: space-efficient, fast on accessing and searching, more useful information. At the most simple form, they are just a conversion of above text-opening books into binary, in the ready form for software and save text-binary converting steps. However, the major number of top books is built and worked with hash values from transpositions. Often each position is stored as a hash value (8 bytes) and some extra information such as the number of times occurred, number of games won by white/black / drawn with this position, average/maximum Elo of players playing to this opening position, chess program's success with the position. To save space those extra information typically about 2-8 bytes. For example, PolyGlot uses 16 bytes for each item, including hash, move, weight, learning values. Items of a book should be sorted by their hash. So when a chess program has to decide on a move, it will look up (by using binary-search) using the hash value of the current position to collect all information come with that hash value and then make a decision upon it.

To build and/or edit those opening books, users need to use special software.

The most popular format is PolyGlot.

=Generation= All in all the information stored in an opening database should guide the chess game into a type of middle game in which the chess program proves itself most successful. But to achieve this there are two main ways.

Handcrafted
If the opening database designer has some special preference of a certain opening line, he might create it fully manually. This is often the case if a chess program is about to play an important game against a grandmaster. Then the opening style will probably be chosen to be difficult and dangerous for the opponent. Another option is to first automatically generate a database and then fine-tune it manually.

From game collection
Another option is to select a number of games (the type of which will affect the openings played: e.g. drawn games only, GM games only, etc) and then up to a certain depth store all positions in the opening database. It is true that by using this way of database generation some severe errors might by copied as well, but usually the chess programs are set to stop playing according to the book if a position occurred less than a certain number times before.

By computing
Chess engines can be used to evaluate positions in a book opening tree to rank them, and/or decide to stop or expand.

=Book Building Tools=
 * Bookbuilder
 * Bookup
 * Chess Opening Wizard (COW)
 * Banksia GUI

=Book Playing=
 * Engines: They can directly use opening books to make move. Some engines use external books, some others integrate books inside their codes. Those engines need both opening book code and book data to run
 * Chess GUI: They can play opening books thus engines don't need to deal with and start computing from the middle period only. This way is quite popular because of some advantages: simplifier for chess engines, save space and easier to manage opening books, more various ways to select openings, more fair-play to compare engines...

=Quotes= Jonathan Schaeffer in One Jump Ahead : To solve the opening problems of his chess machine, Belle, Ken Thompson typed in opening lines from the Encyclopedia of Chess Openings (in five thick volumes). Religiously, he dedicated one hour a day for almost three years (!) to the tedious pursuit of entering lines of play from the books and having his Belle computer verify them. The result was an opening library of roughly three-hundred thousand moves. The results were immediate and obvious: Belle became a much stronger chess program, and Ken probably aged prematurely. Later Ken developed a program to automatically read the Encyclopedia, allowing him to do in a few days what had taken him three years to do manually.

=Formats=
 * ABK - Arena's book format
 * BIN - PolyGlot book fomat
 * CTG - ChessBase book format

=Classification of Chess Openings= =Book Issues=
 * Encyclopaedia of Chess Openings (ECO)
 * NIC-Key
 * Chaos vs. Nuchess @ WCCC 1980
 * Cray Blitz vs. Fidelity X @ ACM 1984

=See also=
 * Book Learning
 * Brainfish
 * CPW-Engine_book
 * Chess Databases
 * La Dame Blanche (Book Builder)
 * Opening
 * Opening Book Authors
 * Opening Suites
 * Portable Game Notation
 * Transposition

=Selected Publications=

1974 ...

 * Ya. Yu. Gol'fand, Aaron L. Futer (1974). Implementation of the opening book for the chess program. in Problems of Cybernetic, No. 29, pp. 201-210
 * Я.Ю. Гольфанд, А.Л. Футер (1974). Реализация дебютной справочной для шахматной программы./ Сб. Проблемы кибернетики №29, стр. 201-210

1980 ...

 * John F. White (1982). Chess-Book Openings. Your Computer, February 1982
 * Kathe Spracklen (1983). Tutorial: Representation of an Opening Tree. ICCA Newsletter, Vol. 6, No. 1
 * Joe Condon, Ken Thompson (1983). BELLE. Chess Skill in Man and Machine

1990 ...

 * John F. White (1990). The Amateur's Book-Opening Routine. ICCA Journal, Vol. 13, No. 1
 * Henry S. Baird, Ken Thompson (1990). Reading Chess. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 12, No. 6, pdf
 * Steven Walczak (1996). Improving Opening Book Performance Through Modeling of Chess Opponents. ACM Conference on Computer Science 1996: 53-57
 * Robert Hyatt (1999). Book Learning - a Methodology to Tune an Opening Book Automatically. ICCA Journal, Vol. 22, No. 1
 * Michael Buro (1999). Toward Opening Book Learning. ICCA Journal, Vol. 22, No. 2, pdf

2000 ...

 * Thomas Lincke (2000). Strategies for the Automatic Construction of Opening Books. CG 2000
 * Marty Hirsch (2001). Machine Learning in MChess Professional. Advances in Computer Games 9 » MChess
 * Thomas Lincke (2002). Exploring the Computational Limits of Large Exhaustive Search Problems. Ph.D thesis, ETH Zurich, pdf
 * Thomas Lincke (2002). Position-Value Representation in Opening Books. CG 2002
 * Marek Strejczek (2004). Some aspects of chess programming. M.Sc. thesis, Technical University of Łódź, 4.2 Experiments with book learning
 * Thomas Widjaja (2004). Knowledge Engineering und Lernen in Spielen - Opening Book Learning. slides as pdf (German)

2005 ...

 * Chrilly Donninger, Ulf Lorenz (2005). Innovative Opening-Book Handling. Advances in Computer Games 11, pdf
 * Jeff Rollason (2006). Fuzzy books - Approximate opening knowledge. AI Factory, Spring 2006
 * Mark Levene, Judit Bar-Ilan (2006). Comparing Typical Opening Move Choices Made by Humans and Chess Engines. arXiv:cs/0610060
 * Mark Levene, Judit Bar-Ilan (2007). Comparing Typical Opening Move Choices Made by Humans and Chess Engines. The Computer Journal, Vol. 50, No. 5
 * Jeff Rollason (2007). Creating Book Knowledge. AI Factory, Autumn 2007
 * Pierre Audouard, Guillaume Chaslot, Jean-Baptiste Hoock, Arpad Rimmel, Julien Pérez, Olivier Teytaud (2009). Grid co-evolution for adaptive simulations; application to the building of opening books in the game of Go. pdf
 * Guillaume Chaslot, Jean-Baptiste Hoock, Julien Pérez, Arpad Rimmel, Olivier Teytaud, Mark Winands (2009). Meta Monte-Carlo Tree Search for Automatic Opening Book Generation. pdf
 * Bernd Blasius, Ralf Tönjes (2009). Zipf's Law in the Popularity Distribution of Chess Openings. Physical Review Letters, 103, 218701, pdf

2010 ...

 * Romaric Gaudel, Jean-Baptiste Hoock, Julien Pérez, Nataliya Sokolovska, Olivier Teytaud (2010). A Principled Method for Exploiting Opening Books. CG 2010, pdf
 * Liang Li, Hong Huang, Litao Deng (2011). Dynamic Opening-Book in Computer Games. CCDC 2011 (Chinese)
 * Bo-Nian Chen, Tsan-sheng Hsu (2013). Automatic Generation of Chinese Dark Chess Opening Books. CG 2013

2015 ...

 * Ting-Han Wei, I-Chen Wu, Chao-Chin Liang, Bing-Tsung Chiang, Wen-Jie Tseng, Shi-Jim Yen, Chang-Shing Lee (2015). Job-Level Algorithms for Connect6 Opening Book Construction. ICGA Journal, Vol. 38, No. 3
 * John P. Fishburn (2016). Search-Based Opening Book Construction. pdf, 2017 edition
 * John Philip Fishburn (2018). Search-based opening book construction. ICGA Journal, Vol. 40, No. 1

=Forum Posts=

1990 ...

 * gnuchess.book in Lisp by Steven Edwards, gnu.chess, January 23, 1990

1995 ...

 * Opening taxonomy by Hugh S. Myers, rgcc, September 12, 1995
 * The MCHESS5 computer killer book... by Ed Schröder, rgcc, October 14, 1996
 * Killer Books by Andreas Mader, rgcc, October 29, 1996
 * Book learning and rating bias by Don Dailey, CCC, May 01, 1998
 * How Rebel plays at SSDF the bare facts, just statistics and thoughts by Vincent Diepeveen, CCC, June 15, 1998
 * Re: Killer Book by Ed Schröder, CCC, June 19, 1998
 * BookLearning Under the Microscope!!! by Robert Henry Durrett, CCC, August 31, 1998
 * MCP8 - Rebel8, SSDF by Tony Hedlund, CCC, December 04, 1998 » MChess
 * Book learning? by Werner Inmann, CCC, December 31, 1998
 * Re: A separate process for the chess engine - How do I do this? by Eugene Nalimov, CCC, February 20, 1999
 * Web site updated and new program Book Builder by Marc-Philippe Huget, CCC, February 24, 1999
 * Opening book by Frank Phillips, CCC, April 01, 1999
 * Book learning by James Robertson, CCC, September 12, 1999

2000 ...

 * random book moves/ random generator by Vincent Diepeveen, CCC, January 13, 2000 » Pseudorandom Number Generator
 * Gromitchess bookcheating (for Vincent DIEPEVEEN) by Gian-Carlo Pascutto, CCC, August 23, 2001
 * question about book and learning by Uri Blass, CCC, April 26, 2002
 * Book design by Sune Fischer, CCC, November 28, 2002
 * Opening Bug in Nejmet and Pharaon by Mainflame, Winboard Forum, May 31, 2003 » Nejmet, Pharaon
 * Why use opening books in machine-machine competitions? by Mig Greengard, CCC, November 25, 2003
 * Performance rating calculation by Dan Wulff, CCC, June 02, 2004

2005 ...

 * Nunn openings by Jouni Uski, CCC, November 07, 2005 » Opening Suites
 * Nunn2 openings by Kurt Utzinger, CCC, November 07, 2005
 * CTG specification by Sesse, Rybka Forum, September 30, 2007 » CTG
 * Opening books format by James Constance, CCC, April 14, 2008
 * Re: Opening books format by Jury Osipov, CCC, April 15, 2008 » ABK
 * Re: Opening books format by Richard Pijl, CCC, April 15, 2008
 * Re: Opening books format by Dann Corbit, CCC, April 15, 2008


 * Leaving/Returning to Book Annoyance by John Merlino, CCC, May 05, 2009
 * Book: KLO 150 ECO A00-E97 Variations by kingliveson, CCC, December 31, 2009

2010 ...
2013
 * Opening book formats and UIs by Vlad Stamate, CCC, June 08, 2010
 * Opening Books Battle.. by Swaminathan, OpenChess Forum, February 04, 2011
 * ICGA rule #2 / opening books / Diep-Crafty, Turino 2006 by Peter Berger, CCC, October 22, 2011 » WCCC 2006
 * Creating Books from .PGN files by David Nash, CCC, December 20, 2011
 * CookieCat's opening book implementation by Steven Edwards, CCC, January 05, 2012 » CookieCat
 * Bookbuilding 101 by Dan Honeycutt, CCC, August 06, 2012
 * Book format for the new engine by Dragan Zdravkovic, CCC, August 10, 2012
 * Ranking moves based on empirical information by Edmund Moshammer, CCC, January 10, 2013
 * Opening Book (for Aquarium) by andytl755, OpenChess Forum, January 21, 2013 » Aquarium
 * Search-based opening book by Harm Geert Muller, CCC, July 07, 2013
 * Opening Books - Something does not add up by Spacious Mind, Hiarcs Forum, July 17, 2013
 * Opening book cleanup by Dariusz Orzechowski, FishCooking, November 03, 2013
 * 2 Moves Engine Book by Lyudmil Tsvetkov, CCC, December 07, 2013
 * Re: 2 Moves Engine Book by Miguel A. Ballicora, CCC, December 08, 2013 » Ruby

2014
 * My new book by Daniel Shawul, CCC, January 02, 2014 » UCT
 * Perfect 2014 opening book is released by Sedat Canbaz, CCC, February 16, 2014
 * Scid.eco by Gregor Cramer, scid-users, April 19, 2014
 * UCI, ownbooks, and a potential problem by Miguel A. Ballicora, CCC, June 16, 2014 » UCI
 * Perfect 2014 Final (Full Package) by Sedat Canbaz, CCC, July 30, 2014

2015 ...
2016 2017 2018 2019
 * Re: ICGA's 2015 World Computer Chess Championship/Events by Jon Dart, CCC, February 27, 2015
 * On Opening books in 2015 by Peter Berger, CCC, March 06, 2015
 * creating an opening book by Folkert van Heusden, CCC, June 24, 2015
 * Making Symbolic's opening book by Steven Edwards, CCC, July 16, 2015 » Symbolic
 * The future of chess and elo ratings by Larry Kaufman, CCC, September 20, 2015 » Match Statistics
 * Introducing the *.EBF project by Ed Schroder, CCC, January 14, 2016
 * Statistical assessment of chess opening book moves by Edmund Moshammer, CCC, February 27, 2016
 * REBEL | ProDeo book available in Polyglot format by Ed Schröder, CCC, March 05, 2016 » Rebel, ProDeo, PolyGlot
 * How much benefit from opening book? by John Fishburn, CCC, March 06, 2016 » Playing Strength
 * reversed-color transpositions by John Fishburn, CCC, March 11, 2016 » Color Flipping
 * My new book is out: Noomen.ctg by Jeroen Noomen, CCC, May 22, 2016
 * How to use openings books? by Luis Babboni, CCC, July 25, 2016
 * CCRL - CEGT matches by Norbert Raimund Leisner, CCC, July 28, 2016 » CCRL, CEGT
 * Opening book from a statistical point of view by Stefano Gemma, CCC, July 29, 2016
 * opening book standards by Jef Kaan, CCC, August 10, 2016
 * Noomen.ctg: UPDATE by Jeroen Noomen, CCC, August 20, 2016
 * Properties of unbalanced openings using Bayeselo model by Kai Laskos, CCC, August 27, 2016 » Match Statistics
 * The scaling with time of opening books by Kai Laskos, CCC, September 23, 2016 » Match Statistics
 * Winboard book settings by Erin Dame, CCC, October 20, 2016 » WinBoard
 * Search-Based Opening Book Construction by John Fishburn, CCC, December 14, 2016
 * Cumulative building of a shared search tree by Bojun Guo, CCC, December 28, 2016 » Chinese Chess, Persistent Hash Table
 * Opening book trees by Robert Pope, CCC, February 27, 2017
 * Generate Opening Book from Scratch by Stefan Edlich, CCC, February 27, 2017
 * Book ChessGUI by Krzysztof Grzelak, CCC, March 17, 2017 » ChessGUI
 * Search-Based Opening Book Construction by John Fishburn, CCC, April 29, 2017
 * FEOBOS 2.0 is available ... by Frank Quisinsky, CCC, May 01, 2017
 * Opening testing suites efficiency by Kai Laskos, CCC, June 21, 2017 » Engine Testing, Opening, Match Statistics
 * HERT - brand new openings-set by Thomas Zipproth by Stefan Pohl, CCC, August 14, 2017
 * New: Noomen 2-move Testsuite by [Jeroen Noomen, Rybka Forum, October 22, 2017 » TCEC Season 10
 * SALC V5 openings and books launched by Stefan Pohl, CCC, January 13, 2018
 * Some opening book design questions by Nguyen Pham, CCC, February 21, 2018
 * Is there a program to build opening books based on engines? by Uri Blass, CCC, February 23, 2018
 * Rebel Book Draw? by John Merlino, CCC, October 03, 2018 » Draw
 * Book creation papers by Robert Pope, CCC, April 18, 2019
 * Database snapshot by noobpwnftw, CCC, July 27, 2019
 * ChessDBCN by noobpwnftw, CCC, September 09, 2019
 * Polyglot FRC/960 Opening Book by Dennis Sceviour, CCC, November 27, 2019 » PolyGlot, Chess960

2020 ...
2021
 * How many Elo points is a book? by Chris Whittington, CCC, September 25, 2020 » Playing Strength
 * Chess opening database with names by Peperoni, CCC, November 20, 2020 » ECO
 * Opening book implementation questions by Pedro Duran, CCC, June 18, 2021

=External Links=
 * Chess opening book from Wikipedia
 * Chess opening book (computers) from Wikipedia
 * Chess opening book (literature) from Wikipedia
 * List of chess openings from Wikipedia
 * Encyclopaedia of Chess Openings (ECO) from Wikipedia
 * Chess opening theory table from Wikipedia
 * Chess Opening Theory - Wikibooks
 * Opening Books from Strategy Game Programming by Martin Fierz
 * The use of openings books by the ICGA, September 15, 2008

Engine Books

 * HIARCS 13 – the Professional Openings Book by Mark Uniacke, ChessBase News, March 10, 2011 » HIARCS
 * Polyglot book format
 * Arena Chess GUI - Opening Books » Arena
 * Book Bilder of La Dame Blanche by Marc-Philippe Huget, hosted by Thomas Mayer
 * Perfect 2014 book by Sedat Canbaz
 * Fauzi's Opening Books by Fauzi Akram Dabat, G 6 site

Online Opening Tree

 * Chess Cloud Database Query Interface by noobpwnftw
 * ChessDB Online Opening Book by Ferdinand Mosca
 * Chess database with eight million games. Openings, players, tournaments - Online - ChessBase
 * Chess Opening Explorer from chessgames.com
 * NICBase Online - New In Chess - NICBase
 * Opening Tree Mode - ChessOK
 * OPEX - Chess Openings Explorer by Buğra Fırat
 * Shredder Computer Chess Download - Opening Database - Shredder

Misc

 * Jan Klare's The Dorf - Overtüre/Pose, Moers Festival 2013, YouTube Video

=References= Up one level