Difference between revisions of "Opening Book"

From Chessprogramming wiki
Jump to: navigation, search
(35 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
[[FILE:380px-Modern_Chess_Openings_Second_Edition.jpg|border|right|thumb|link=https://en.wikipedia.org/wiki/File:Modern_Chess_Openings_Second_Edition.jpg|[https://en.wikipedia.org/wiki/Modern_Chess_Openings Modern Chess Openings] <ref>Worn copy of second edition of Modern Chess Openings (1913) by [https://en.wikipedia.org/wiki/Richard_Griffith_%28chess_player%29 Richard Clewin Griffith] (1872–1955) and [https://en.wikipedia.org/wiki/John_Herbert_White John Herbert White] (1880–1920), with an introduction by [https://en.wikipedia.org/wiki/Henry_Ernest_Atkins Henry Ernest Atkins] (1872–1955), [https://en.wikipedia.org/wiki/Modern_Chess_Openings Modern Chess Openings from Wikipedia]</ref> ]]
 
[[FILE:380px-Modern_Chess_Openings_Second_Edition.jpg|border|right|thumb|link=https://en.wikipedia.org/wiki/File:Modern_Chess_Openings_Second_Edition.jpg|[https://en.wikipedia.org/wiki/Modern_Chess_Openings Modern Chess Openings] <ref>Worn copy of second edition of Modern Chess Openings (1913) by [https://en.wikipedia.org/wiki/Richard_Griffith_%28chess_player%29 Richard Clewin Griffith] (1872–1955) and [https://en.wikipedia.org/wiki/John_Herbert_White John Herbert White] (1880–1920), with an introduction by [https://en.wikipedia.org/wiki/Henry_Ernest_Atkins Henry Ernest Atkins] (1872–1955), [https://en.wikipedia.org/wiki/Modern_Chess_Openings Modern Chess Openings from Wikipedia]</ref> ]]
  
Chess programs often look up the positions at the [[Opening|beginning of the game]] in an '''Opening Book'''. Chess programs could use [[Search|search]] from the beginning and look through all possible continuations in detail to find the best next move, but as this is quite time-consuming and there already exists a lot of [https://en.wikipedia.org/wiki/Chess_opening_book_%28literature%29 literature] about different opening lines, opening books save time and provide higher quality of play. Another reason that opening books are used commonly is to provide variety, as book moves are usually chosen [[Pseudorandom Number Generator|randomly]], whereas searches are more or less deterministic.
+
Chess programs often look up the positions at the [[Opening|beginning of the game]] in an '''Opening Book'''. The opening database can then be used as long as the opponent plays a new move from the database, so usually more common opening lines will be stored to a much higher depth than the uncommon ones. As soon as the program is "out of book" it has to continue using the normal search routines.  
  
This opening database can then be used as long as the opponent plays a new move from the database, so usually more common opening lines will be stored to a much higher depth than the uncommon ones. As soon as the program is "out of book" it has to continue using the normal search routines.  
+
=Purposes=
 +
* Save time: Chess programs could use [[Search|search]] from the beginning and look through all possible continuations in detail to find the best next move, but as this is quite time-consuming and there already exists a lot of [https://en.wikipedia.org/wiki/Chess_opening_book_%28literature%29 literature] about different opening lines, opening books save time.
 +
* Provide a higher quality of play: Searching only even with large depths can't see deep tactics, strategy, compensation, lose temporarily material for long term advantage.
 +
* Provide variety: as book moves are usually chosen [[Pseudorandom Number Generator|randomly]], whereas searches are more or less deterministic.
  
=Types=  
+
=Types=
There are two ways to store an opening database:
+
Opening books are typically stored in two formats:
* Opening lines
+
==Text==
* Positions
+
The most advantage is that by using text form they are readable, understandable, and editable by humans without any special software. However, the main disadvantage is that they may take too much space as well as time for searching since an item may take from 50 bytes (for FEN strings) to hundreds of bytes (for PGN text). If we store openings "continuously" (all possible opening positions) both data size and searching time become too huge and become unacceptable for playing. In practice, those opening books are usually very small with some lines or positions only, they cannot be used for real game playing but for testing.
  
Some [[Databases|databases]] use a list of common opening lines, while others hold a list of positions that arise in common openings. The first option is probably easier to implement, while the second has some important advantages.
+
They are stored in typical ways for games and positions:
 +
* [[EPD]]: They are the last positions of opening lines
 +
* [[PGN]]: Opening lines are stored as individual games
 +
<span id="BookBuilder"></span>
 +
* Special formats: In 1999, [[Marc-Philippe Huget]] published a '''Book Builder''' <ref>[https://www.stmintz.com/ccc/index.php?id=44407 Web site updated and new program Book Builder] by [[Marc-Philippe Huget]], [[CCC]], February 24, 1999</ref> in conjunction with his engine [[La Dame Blanche]], a standalone open source program <ref>[http://www.quarkchess.de/ladameblanche/ Book Bilder of La Dame Blanche] by [[Marc-Philippe Huget]], hosted by [[Thomas Mayer]]</ref> to convert a [[Portable Game Notation|PGN file]] into an opening book of the format proposed by [[Kathe Spracklen]] <ref>[[Kathe Spracklen]] ('''1983'''). ''Tutorial: Representation of an Opening Tree''. [[ICGA Journal#6_1|ICCA Newsletter, Vol. 6, No. 1]]</ref>:
 +
<pre>
 +
(e4(e5)(c5))(d4d5)) etc.
 +
</pre>
  
Opening databases built on positions are usually more space efficient and overcome the problem of [[Transposition|transpositions]]. Often these positions are stored as hash value to allow fast access. Some opening database formats store with each positions as well some information about it. (e.g. number of times occurred, number of games won by white/black / drawn with this position, average/maximum Elo of players playing to this opening position, chess program's success with the position) So when a chess program has to decide on a move, it will look up all positions that could arise after moving in the database and then make a decision upon the information given with each position.
+
==Binary==
 +
Binary books have some important advantages, especially for computer chess engines. Opening databases built on all positions are usually more space-efficient and overcome the problem of [[Transposition|transpositions]]. Often these positions are stored as hash values to allow fast access. Some opening database formats store with each position as well as some information about it. (e.g. number of times occurred, number of games won by white/black / drawn with this position, average/maximum Elo of players playing to this opening position, chess program's success with the position). So when a chess program has to decide on a move, it will look up all positions that could arise after moving in the database and then make a decision upon the information given with each position. Depends on designs, an item may take 10 - 16 bytes. To build and/or edit those opening books, users need to use special software.
 +
 
 +
The most popular format is [[PolyGlot]].
  
 
=Generation=  
 
=Generation=  
Line 20: Line 33:
  
 
==Handcrafted==  
 
==Handcrafted==  
If the opening database designer has some special preference of a certain opening line, he might create it fully manually. This is often the case if a chess program is about to play an important game against a grandmaster. Then the opening style will probably be chosen to be difficult and dangerous for the opponent. Another option is to first automatically generate a database and then fine tune it manually.
+
If the opening database designer has some special preference of a certain opening line, he might create it fully manually. This is often the case if a chess program is about to play an important game against a grandmaster. Then the opening style will probably be chosen to be difficult and dangerous for the opponent. Another option is to first automatically generate a database and then fine-tune it manually.
  
==Automated==  
+
==From game collection==  
Another option is to select a number of games (the type of which will effect the openings played: e.g. drawn games only, GM games only, etc) and then up to a certain depth store all positions in the opening database. It is true that by using this way of database generation some severe errors might by copied as well, but usually the chess programs are set to stop playing according from the book if a position occurred less than a certain number times before.  
+
Another option is to select a number of games (the type of which will affect the openings played: e.g. drawn games only, GM games only, etc) and then up to a certain depth store all positions in the opening database. It is true that by using this way of database generation some severe errors might by copied as well, but usually the chess programs are set to stop playing according to the book if a position occurred less than a certain number times before.
  
==Book Building Tools==
+
==By computing==
 +
Chess [[engines]] can be used to evaluate positions in a book opening tree to rank them, and/or decide to stop or expand.
 +
 
 +
=Book Building Tools=
 
* [[Bookbuilder]]
 
* [[Bookbuilder]]
 
* [[Bookup]]
 
* [[Bookup]]
 
* [[Chess Opening Wizard]] (COW)
 
* [[Chess Opening Wizard]] (COW)
<span id="BookBuilder"></span>
+
* [[Banksia GUI]]
==Book Builder==
 
In 1999, [[Marc-Philippe Huget]] published a '''Book Builder''' <ref>[https://www.stmintz.com/ccc/index.php?id=44407 Web site updated and new program Book Builder] by [[Marc-Philippe Huget]], [[CCC]], February 24, 1999</ref> in conjunction with his engine [[La Dame Blanche]], a standalone open source program <ref>[http://www.quarkchess.de/ladameblanche/ Book Bilder of La Dame Blanche] by [[Marc-Philippe Huget]], hosted by [[Thomas Mayer]]</ref> to convert a [[Portable Game Notation|PGN file]] into an opening book of the format proposed by [[Kathe Spracklen]] <ref>[[Kathe Spracklen]] ('''1983'''). ''Tutorial: Representation of an Opening Tree''. [[ICGA Journal#6_1|ICCA Newsletter, Vol. 6, No. 1]]</ref>:
 
<pre>
 
(e4(e5)(c5))(d4d5)) etc.
 
</pre>
 
  
 
=Quotes=  
 
=Quotes=  
Line 40: Line 51:
 
  To solve the opening problems of his chess machine, [[Belle]], [[Ken Thompson]] typed in opening lines from the ''[https://en.wikipedia.org/wiki/Encyclopaedia_of_Chess_Openings Encyclopedia of Chess Openings]'' (in five thick volumes). Religiously, he dedicated one hour a day for almost three years (!) to the tedious pursuit of entering lines of play from the books and having his Belle computer verify them. The result was an opening library of roughly three-hundred thousand moves. The results were immediate and obvious: Belle became a much stronger chess program, and Ken probably aged prematurely. Later Ken developed a program to automatically read the ''Encyclopedia'', allowing him to do in a few days what had taken him three years to do manually. <ref>[[Mathematician#HSBaird|Henry S. Baird]], [[Ken Thompson]] ('''1990'''). ''[http://doc.cat-v.org/bell_labs/reading_chess/ Reading Chess]''. [[IEEE#TPAMI|IEEE Transactions on Pattern Analysis and Machine Intelligence]], Vol. 12, No. 6, [http://doc.cat-v.org/bell_labs/reading_chess/reading_chess.pdf pdf]</ref>  
 
  To solve the opening problems of his chess machine, [[Belle]], [[Ken Thompson]] typed in opening lines from the ''[https://en.wikipedia.org/wiki/Encyclopaedia_of_Chess_Openings Encyclopedia of Chess Openings]'' (in five thick volumes). Religiously, he dedicated one hour a day for almost three years (!) to the tedious pursuit of entering lines of play from the books and having his Belle computer verify them. The result was an opening library of roughly three-hundred thousand moves. The results were immediate and obvious: Belle became a much stronger chess program, and Ken probably aged prematurely. Later Ken developed a program to automatically read the ''Encyclopedia'', allowing him to do in a few days what had taken him three years to do manually. <ref>[[Mathematician#HSBaird|Henry S. Baird]], [[Ken Thompson]] ('''1990'''). ''[http://doc.cat-v.org/bell_labs/reading_chess/ Reading Chess]''. [[IEEE#TPAMI|IEEE Transactions on Pattern Analysis and Machine Intelligence]], Vol. 12, No. 6, [http://doc.cat-v.org/bell_labs/reading_chess/reading_chess.pdf pdf]</ref>  
 
<span id="Formats"></span>
 
<span id="Formats"></span>
 +
 
=Formats=  
 
=Formats=  
 
* [[ABK]] - [[Arena|Arena's]] book format  
 
* [[ABK]] - [[Arena|Arena's]] book format  
* [[Polyglot|BIN]] - [[Polyglot]] book fomat
+
* [[PolyGlot|BIN]] - [[PolyGlot]] book fomat
* [[CTG]] - [[ChessBase]] book format  
+
* [[CTG]] - [[ChessBase]] book format
  
 
=Classification of Chess Openings=
 
=Classification of Chess Openings=
Line 60: Line 72:
 
* [[La Dame Blanche]] (Book Builder)
 
* [[La Dame Blanche]] (Book Builder)
 
* [[Opening]]
 
* [[Opening]]
* [[Opening Book Authors]]
+
* [[:Category:Opening Book Author|Opening Book Authors]]
 
* [[Test-Positions#OpeningSuites|Opening Suites]]
 
* [[Test-Positions#OpeningSuites|Opening Suites]]
 
* [[Portable Game Notation]]
 
* [[Portable Game Notation]]
Line 66: Line 78:
  
 
=Selected Publications=  
 
=Selected Publications=  
 +
==1974 ...==
 +
* [http://www.mathnet.ru/php/person.phtml?option_lang=eng&personid=96114 Ya. Yu. Gol'fand], [[Aaron L. Futer]] ('''1974'''). ''Implementation of the opening book for the chess program.'' in Problems of Cybernetic, No. 29 , pp. 201-210
 +
: Я.Ю. Гольфанд, [[Aaron L. Futer|А.Л. Футер]] ('''1974'''). ''Реализация дебютной справочной для шахматной программы.''/ Сб. Проблемы кибернетики №29, стр. 201-210
 
==1980 ...==  
 
==1980 ...==  
 
* [[John F. White]] ('''1982'''). ''[http://yourcomputeronline.wordpress.com/2011/01/19/chess-book-openings/ Chess-Book Openings]''. [[Your Computer]], [http://yourcomputeronline.wordpress.com/2011/01/15/february-1982-contents-and-editorial/ February 1982]
 
* [[John F. White]] ('''1982'''). ''[http://yourcomputeronline.wordpress.com/2011/01/19/chess-book-openings/ Chess-Book Openings]''. [[Your Computer]], [http://yourcomputeronline.wordpress.com/2011/01/15/february-1982-contents-and-editorial/ February 1982]
 
* [[Kathe Spracklen]] ('''1983'''). ''Tutorial: Representation of an Opening Tree''. [[ICGA Journal#6_1|ICCA Newsletter, Vol. 6, No. 1]]
 
* [[Kathe Spracklen]] ('''1983'''). ''Tutorial: Representation of an Opening Tree''. [[ICGA Journal#6_1|ICCA Newsletter, Vol. 6, No. 1]]
* [[Joe Condon]] and [[Ken Thompson]] ('''1983'''). ''BELLE''. [[Chess Skill in Man and Machine]]
+
* [[Joe Condon]], [[Ken Thompson]] ('''1983'''). ''BELLE''. [[Chess Skill in Man and Machine]]
 
==1990 ...==  
 
==1990 ...==  
 
* [[John F. White]] ('''1990'''). ''The Amateur's Book-Opening Routine''. [[ICGA Journal#13_1|ICCA Journal, Vol. 13, No. 1]]
 
* [[John F. White]] ('''1990'''). ''The Amateur's Book-Opening Routine''. [[ICGA Journal#13_1|ICCA Journal, Vol. 13, No. 1]]
Line 81: Line 96:
 
* [[Thomas Lincke]] ('''2002'''). ''Exploring the Computational Limits of Large Exhaustive Search Problems''. Ph.D thesis, [[ETH Zurich]], [http://e-collection.library.ethz.ch/eserv/eth:25905/eth-25905-02.pdf pdf]
 
* [[Thomas Lincke]] ('''2002'''). ''Exploring the Computational Limits of Large Exhaustive Search Problems''. Ph.D thesis, [[ETH Zurich]], [http://e-collection.library.ethz.ch/eserv/eth:25905/eth-25905-02.pdf pdf]
 
* [[Thomas Lincke]] ('''2002'''). ''[http://www.springerlink.com/content/aqbcher49k48affn/ Position-Value Representation in Opening Books]''. [[CG 2002]]
 
* [[Thomas Lincke]] ('''2002'''). ''[http://www.springerlink.com/content/aqbcher49k48affn/ Position-Value Representation in Opening Books]''. [[CG 2002]]
* [[Marek Strejczek]] ('''2004'''). ''Some aspects of chess programming''. [[Technical University of Łódź]] , Faculty of Electrical and Electronic Engineering, Department of Computer Science, [http://nesik.republika.pl/download//SomeAspectsOfChessProgramming.zip zipped pdf], 4.2 Experiments with book learning
+
* [[Marek Strejczek]] ('''2004'''). ''Some aspects of chess programming''. M.Sc. thesis, [[Technical University of Łódź]], 4.2 Experiments with book learning
 
* [[Thomas Widjaja]] ('''2004'''). ''Knowledge Engineering und Lernen in Spielen - Opening Book Learning''. [http://www.ke.tu-darmstadt.de/lehre/archiv/ss04/se-spiele/OpeningBookLearning.pdf slides as pdf] (German)
 
* [[Thomas Widjaja]] ('''2004'''). ''Knowledge Engineering und Lernen in Spielen - Opening Book Learning''. [http://www.ke.tu-darmstadt.de/lehre/archiv/ss04/se-spiele/OpeningBookLearning.pdf slides as pdf] (German)
 
==2005 ...==  
 
==2005 ...==  
 
* [[Chrilly Donninger]], [[Ulf Lorenz]] ('''2005'''). ''[http://link.springer.com/chapter/10.1007/11922155_1 Innovative Opening-Book Handling]''. [[Advances in Computer Games 11]], [http://www2.cs.uni-paderborn.de/cs/ag-monien/PERSONAL/FLULO/publications/icga_open_springer.pdf pdf]
 
* [[Chrilly Donninger]], [[Ulf Lorenz]] ('''2005'''). ''[http://link.springer.com/chapter/10.1007/11922155_1 Innovative Opening-Book Handling]''. [[Advances in Computer Games 11]], [http://www2.cs.uni-paderborn.de/cs/ag-monien/PERSONAL/FLULO/publications/icga_open_springer.pdf pdf]
 
* [[Jeff Rollason]] ('''2006'''). ''[http://www.aifactory.co.uk/newsletter/2006_01_fuzzy_books.htm Fuzzy books - Approximate opening knowledge]''. [[AI Factory]], Spring 2006
 
* [[Jeff Rollason]] ('''2006'''). ''[http://www.aifactory.co.uk/newsletter/2006_01_fuzzy_books.htm Fuzzy books - Approximate opening knowledge]''. [[AI Factory]], Spring 2006
 +
* [[Mark Levene]], [[Judit Bar-Ilan]] ('''2006'''). ''Comparing Typical Opening Move Choices Made by Humans and Chess Engines''. [https://arxiv.org/abs/cs/0610060 arXiv:cs/0610060]
 +
* [[Mark Levene]], [[Judit Bar-Ilan]] ('''2007'''). ''Comparing Typical Opening Move Choices Made by Humans and Chess Engines''. [https://en.wikipedia.org/wiki/The_Computer_Journal The Computer Journal], Vol. 50, No. 5
 
* [[Jeff Rollason]] ('''2007'''). ''[http://www.aifactory.co.uk/newsletter/2007_03_creating_book.htm Creating Book Knowledge]''. [[AI Factory]], Autumn 2007
 
* [[Jeff Rollason]] ('''2007'''). ''[http://www.aifactory.co.uk/newsletter/2007_03_creating_book.htm Creating Book Knowledge]''. [[AI Factory]], Autumn 2007
* [[Mark Levene]], [[Judit Bar-Ilan]] ('''2007'''). ''[http://arxiv.org/abs/cs.AI/0610060 Comparing Typical Opening Move Choices Made by Humans and Chess Engines]''. [https://en.wikipedia.org/wiki/The_Computer_Journal Computer Journal], 50, [http://arxiv.org/PS_cache/cs/pdf/0610/0610060v1.pdf pdf]
 
 
* [http://gobase.org/information/players/?pp=Pierre%20Audouard Pierre Audouard], [[Guillaume Chaslot]], [[Jean-Baptiste Hoock]], [[Arpad Rimmel]], [[Julien Pérez]], [[Olivier Teytaud]] ('''2009'''). ''Grid co-evolution for adaptive simulations; application to the building of opening books in the game of Go''. [http://www.lri.fr/~rimmel/publi/ouvertures.pdf pdf]
 
* [http://gobase.org/information/players/?pp=Pierre%20Audouard Pierre Audouard], [[Guillaume Chaslot]], [[Jean-Baptiste Hoock]], [[Arpad Rimmel]], [[Julien Pérez]], [[Olivier Teytaud]] ('''2009'''). ''Grid co-evolution for adaptive simulations; application to the building of opening books in the game of Go''. [http://www.lri.fr/~rimmel/publi/ouvertures.pdf pdf]
 
* [[Guillaume Chaslot]], [[Jean-Baptiste Hoock]], [[Julien Pérez]], [[Arpad Rimmel]], [[Olivier Teytaud]], [[Mark Winands]] ('''2009'''). ''Meta Monte-Carlo Tree Search for Automatic Opening Book Generation''. [http://www.personeel.unimaas.nl/m-winands/documents/ouvertures9x9.pdf pdf]
 
* [[Guillaume Chaslot]], [[Jean-Baptiste Hoock]], [[Julien Pérez]], [[Arpad Rimmel]], [[Olivier Teytaud]], [[Mark Winands]] ('''2009'''). ''Meta Monte-Carlo Tree Search for Automatic Opening Book Generation''. [http://www.personeel.unimaas.nl/m-winands/documents/ouvertures9x9.pdf pdf]
Line 104: Line 120:
 
* [http://groups.google.com/group/gnu.chess/browse_frm/thread/329b9401059f679b gnuchess.book in Lisp] by [[Steven Edwards]], [[GNU Chess#NewsGroup|gnu.chess]], January 23, 1990
 
* [http://groups.google.com/group/gnu.chess/browse_frm/thread/329b9401059f679b gnuchess.book in Lisp] by [[Steven Edwards]], [[GNU Chess#NewsGroup|gnu.chess]], January 23, 1990
 
==1995 ...==  
 
==1995 ...==  
 +
* [https://groups.google.com/d/msg/rec.games.chess.computer/twuuIKTUqRw/bCN6Jn9zGOcJ Opening taxonomy] by [[Hugh S. Myers]], [[Computer Chess Forums|rgcc]], September 12, 1995
 
* [http://groups.google.com/group/rec.games.chess.computer/browse_frm/thread/656439670bd7c7fb The MCHESS5 computer killer book...] by [[Ed Schroder|Ed Schröder]], [[Computer Chess Forums|rgcc]], October 14, 1996
 
* [http://groups.google.com/group/rec.games.chess.computer/browse_frm/thread/656439670bd7c7fb The MCHESS5 computer killer book...] by [[Ed Schroder|Ed Schröder]], [[Computer Chess Forums|rgcc]], October 14, 1996
 
* [http://groups.google.com/group/rec.games.chess.computer/browse_frm/thread/f14614c6bdebff95 Killer Books] by [[Andreas Mader]], [[Computer Chess Forums|rgcc]], October 29, 1996
 
* [http://groups.google.com/group/rec.games.chess.computer/browse_frm/thread/f14614c6bdebff95 Killer Books] by [[Andreas Mader]], [[Computer Chess Forums|rgcc]], October 29, 1996
Line 121: Line 138:
 
* [https://www.stmintz.com/ccc/index.php?id=226258 question about book and learning] by [[Uri Blass]], [[CCC]], April 26, 2002
 
* [https://www.stmintz.com/ccc/index.php?id=226258 question about book and learning] by [[Uri Blass]], [[CCC]], April 26, 2002
 
* [https://www.stmintz.com/ccc/index.php?id=267805 Book design] by [[Sune Fischer]], [[CCC]], November 28, 2002
 
* [https://www.stmintz.com/ccc/index.php?id=267805 Book design] by [[Sune Fischer]], [[CCC]], November 28, 2002
 +
* [http://www.open-aurec.com/wbforum/viewtopic.php?f=18&t=42820 Opening Bug in Nejmet and Pharaon] by Mainflame, [[Computer Chess Forums|Winboard Forum]], May 31, 2003 » [[Nejmet]], [[Pharaon]]
 
* [https://www.stmintz.com/ccc/index.php?id=330362 Why use opening books in machine-machine competitions?] by [[Mig Greengard]], [[CCC]], November 25, 2003
 
* [https://www.stmintz.com/ccc/index.php?id=330362 Why use opening books in machine-machine competitions?] by [[Mig Greengard]], [[CCC]], November 25, 2003
 
* [https://www.stmintz.com/ccc/index.php?id=368697 Performance rating calculation] by [[Dan Wulff]], [[CCC]], June 02, 2004
 
* [https://www.stmintz.com/ccc/index.php?id=368697 Performance rating calculation] by [[Dan Wulff]], [[CCC]], June 02, 2004
Line 164: Line 182:
 
* [http://www.talkchess.com/forum/viewtopic.php?t=58913 Introducing the *.EBF project] by [[Ed Schroder]], [[CCC]], January 14, 2016
 
* [http://www.talkchess.com/forum/viewtopic.php?t=58913 Introducing the *.EBF project] by [[Ed Schroder]], [[CCC]], January 14, 2016
 
* [http://www.talkchess.com/forum/viewtopic.php?t=59374 Statistical assessment of chess opening book moves] by [[Edmund Moshammer]], [[CCC]], February 27, 2016
 
* [http://www.talkchess.com/forum/viewtopic.php?t=59374 Statistical assessment of chess opening book moves] by [[Edmund Moshammer]], [[CCC]], February 27, 2016
* [http://www.talkchess.com/forum/viewtopic.php?t=59435 REBEL | ProDeo book available in Polyglot format] by [[Ed Schroder|Ed Schröder]], [[CCC]], March 05, 2016 » [[Rebel]], [[Pro Deo]], [[PolyGlot]]
+
* [http://www.talkchess.com/forum/viewtopic.php?t=59435 REBEL | ProDeo book available in Polyglot format] by [[Ed Schroder|Ed Schröder]], [[CCC]], March 05, 2016 » [[Rebel]], [[ProDeo]], [[PolyGlot]]
 
* [http://www.talkchess.com/forum/viewtopic.php?p=662580 How much benefit from opening book?] by [[John Philip Fishburn|John Fishburn]], [[CCC]], March 06, 2016 » [[Playing Strength]]
 
* [http://www.talkchess.com/forum/viewtopic.php?p=662580 How much benefit from opening book?] by [[John Philip Fishburn|John Fishburn]], [[CCC]], March 06, 2016 » [[Playing Strength]]
 
* [http://www.talkchess.com/forum/viewtopic.php?p=663148 reversed-color transpositions] by [[John Philip Fishburn|John Fishburn]], [[CCC]], March 11, 2016 » [[Color Flipping]]
 
* [http://www.talkchess.com/forum/viewtopic.php?p=663148 reversed-color transpositions] by [[John Philip Fishburn|John Fishburn]], [[CCC]], March 11, 2016 » [[Color Flipping]]
Line 186: Line 204:
 
* [http://www.talkchess.com/forum/viewtopic.php?t=64358 Opening testing suites efficiency] by [[Kai Laskos]], [[CCC]], June 21, 2017 » [[Engine Testing]], [[Opening]], [[Match Statistics]]
 
* [http://www.talkchess.com/forum/viewtopic.php?t=64358 Opening testing suites efficiency] by [[Kai Laskos]], [[CCC]], June 21, 2017 » [[Engine Testing]], [[Opening]], [[Match Statistics]]
 
* [http://www.talkchess.com/forum/viewtopic.php?t=64894 HERT - brand new openings-set by Thomas Zipproth] by [[Stefan Pohl]], [[CCC]], August 14, 2017
 
* [http://www.talkchess.com/forum/viewtopic.php?t=64894 HERT - brand new openings-set by Thomas Zipproth] by [[Stefan Pohl]], [[CCC]], August 14, 2017
* [http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=32336 New: Noomen 2-move Testsuite ] by [[[Jeroen Noomen]], [[Computer Chess Forums|Rybka Forum]], October 22, 2017 » [[ TCEC Season 10]]
+
* [http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=32336 New: Noomen 2-move Testsuite] by [[[Jeroen Noomen]], [[Computer Chess Forums|Rybka Forum]], October 22, 2017 » [[ TCEC Season 10]]
 
'''2018'''
 
'''2018'''
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66311 SALC V5 openings and books launched] by [[Stefan Pohl]], [[CCC]], January 13, 2018
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66311 SALC V5 openings and books launched] by [[Stefan Pohl]], [[CCC]], January 13, 2018
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66657 Some opening book design questions] by [[Pham Hong Nguyen|Nguyen Pham]], [[CCC]], February 21, 2018
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66657 Some opening book design questions] by [[Pham Hong Nguyen|Nguyen Pham]], [[CCC]], February 21, 2018
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66680 Is there a program to build opening books based on engines?] by [[Uri Blass]], [[CCC]], February 23, 2018
 
* [http://www.talkchess.com/forum/viewtopic.php?t=66680 Is there a program to build opening books based on engines?] by [[Uri Blass]], [[CCC]], February 23, 2018
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=68558 Rebel Book Draw?] by [[John Merlino]], [[CCC]], October 03, 2018 » [[Draw]]
 +
'''2019'''
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70518 Book creation papers] by [[Robert Pope]], [[CCC]], April 18, 2019
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=71386 Database snapshot] by  [[Bojun Guo|noobpwnftw]], [[CCC]], July 27, 2019
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=71764 ChessDBCN] by [[Bojun Guo|noobpwnftw]], [[CCC]], September 09, 2019
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=72432 Polyglot FRC/960 Opening Book] by [[Dennis Sceviour]], [[CCC]], November 27, 2019 » [[PolyGlot]], [[Chess960]]
 +
==2020 ...==
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75205 How many Elo points is a book?] by [[Chris Whittington]], [[CCC]], September 25, 2020 » [[Playing Strength]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=75878 Chess opening database with names] by Peperoni, [[CCC]], November 20, 2020 » [[ECO]]
  
 
=External Links=  
 
=External Links=  
Line 203: Line 230:
 
* [https://www.game-ai-forum.org/icga-tournaments/news_item.php?id=35 The use of openings books] by the [[ICGA]], September 15, 2008
 
* [https://www.game-ai-forum.org/icga-tournaments/news_item.php?id=35 The use of openings books] by the [[ICGA]], September 15, 2008
 
==Engine Books==
 
==Engine Books==
* [http://www.top-5000.nl/tools.htm Source code Pro Deo Book] by [[Ed Schroder|Ed Schröder]] » [[Pro Deo]]
+
* [http://chessbase.com/newsdetail.asp?newsid=7069 HIARCS 13 – the Professional Openings Book] by [[Mark Uniacke]], [[ChessBase|ChessBase News]], March 10, 2011 » [[HIARCS]] <ref>[http://www.open-chess.org/viewtopic.php?f=3&t=1073#p11368 Re: Opening Books Battle..] by BB+ ([[Mark Watkins]]), [[Computer Chess Forums|OpenChess Forum]], March 12, 2011</ref>
* [http://www.top-5000.nl/eoc.htm Source code ProDeo EOC database] by [[Ed Schroder|Ed Schröder]]
 
* [http://chessbase.com/newsdetail.asp?newsid=7069 HIARCS 13 – the Professional Openings Book] by [[Mark Uniacke]], [[ChessBase|ChessBase News]], March 10, 2011 » [[Hiarcs]] <ref>[http://www.open-chess.org/viewtopic.php?f=3&t=1073#p11368 Re: Opening Books Battle..] by BB+ ([[Mark Watkins]]), [[Computer Chess Forums|OpenChess Forum]], March 12, 2011</ref>
 
 
* [http://hardy.uhasselt.be/Toga/book_format.html Polyglot book format]
 
* [http://hardy.uhasselt.be/Toga/book_format.html Polyglot book format]
 
* [http://www.playwitharena.com/?User_Files%2C_Engines:Opening_Books_%2821%29%26nbsp%3B Arena Chess GUI - Opening Books] » [[Arena]]
 
* [http://www.playwitharena.com/?User_Files%2C_Engines:Opening_Books_%2821%29%26nbsp%3B Arena Chess GUI - Opening Books] » [[Arena]]
Line 212: Line 237:
 
* [http://www.g-sei.org/wp-content/Users/Fauzi/fauzi.html Fauzi's Opening Books] by [[Fauzi Akram Dabat]], [[G 6]] site
 
* [http://www.g-sei.org/wp-content/Users/Fauzi/fauzi.html Fauzi's Opening Books] by [[Fauzi Akram Dabat]], [[G 6]] site
 
==Online Opening Tree==
 
==Online Opening Tree==
 +
* [https://www.chessdb.cn/queryc_en/ Chess Cloud Database Query Interface] by [[Bojun Guo|noobpwnftw]]
 
* [https://database.chessbase.com/?lang=en/ Chess database with eight million games. Openings, players, tournaments - Online] - [[ChessBase (Database)|ChessBase]]
 
* [https://database.chessbase.com/?lang=en/ Chess database with eight million games. Openings, players, tournaments - Online] - [[ChessBase (Database)|ChessBase]]
 
* [http://www.chessgames.com/perl/explorer Chess Opening Explorer] from [http://www.chessgames.com/index.html chessgames.com]
 
* [http://www.chessgames.com/perl/explorer Chess Opening Explorer] from [http://www.chessgames.com/index.html chessgames.com]
Line 220: Line 246:
  
 
==Misc==
 
==Misc==
* [[Videos#JanKlare|Jan Klare's]] [[Videos#TheDorf|The Dorf]] - Overtüre/Pose, [https://en.wikipedia.org/wiki/Moers_Festival Moers Festival] 2013, [https://en.wikipedia.org/wiki/YouTube YouTube] Video
+
* [[:Category:Jan Klare|Jan Klare's]] [[:Category:The Dorf|The Dorf]] - Overtüre/Pose, [https://en.wikipedia.org/wiki/Moers_Festival Moers Festival] 2013, [https://en.wikipedia.org/wiki/YouTube YouTube] Video
 
: {{#evu:https://www.youtube.com/watch?v=OhMwMEP52Os|alignment=left|valignment=top}}
 
: {{#evu:https://www.youtube.com/watch?v=OhMwMEP52Os|alignment=left|valignment=top}}
  
Line 226: Line 252:
 
<references />
 
<references />
 
'''[[Knowledge|Up one level]]'''
 
'''[[Knowledge|Up one level]]'''
 +
[[Category:Schaeffer Quotes]]
 +
[[Category:Jan Klare]]
 +
[[Category:Achim Zepezauer]]
 +
[[Category:The Dorf]]

Revision as of 10:19, 21 November 2020

Home * Knowledge * Opening Book

Chess programs often look up the positions at the beginning of the game in an Opening Book. The opening database can then be used as long as the opponent plays a new move from the database, so usually more common opening lines will be stored to a much higher depth than the uncommon ones. As soon as the program is "out of book" it has to continue using the normal search routines.

Purposes

  • Save time: Chess programs could use search from the beginning and look through all possible continuations in detail to find the best next move, but as this is quite time-consuming and there already exists a lot of literature about different opening lines, opening books save time.
  • Provide a higher quality of play: Searching only even with large depths can't see deep tactics, strategy, compensation, lose temporarily material for long term advantage.
  • Provide variety: as book moves are usually chosen randomly, whereas searches are more or less deterministic.

Types

Opening books are typically stored in two formats:

Text

The most advantage is that by using text form they are readable, understandable, and editable by humans without any special software. However, the main disadvantage is that they may take too much space as well as time for searching since an item may take from 50 bytes (for FEN strings) to hundreds of bytes (for PGN text). If we store openings "continuously" (all possible opening positions) both data size and searching time become too huge and become unacceptable for playing. In practice, those opening books are usually very small with some lines or positions only, they cannot be used for real game playing but for testing.

They are stored in typical ways for games and positions:

  • EPD: They are the last positions of opening lines
  • PGN: Opening lines are stored as individual games

(e4(e5)(c5))(d4d5)) etc.

Binary

Binary books have some important advantages, especially for computer chess engines. Opening databases built on all positions are usually more space-efficient and overcome the problem of transpositions. Often these positions are stored as hash values to allow fast access. Some opening database formats store with each position as well as some information about it. (e.g. number of times occurred, number of games won by white/black / drawn with this position, average/maximum Elo of players playing to this opening position, chess program's success with the position). So when a chess program has to decide on a move, it will look up all positions that could arise after moving in the database and then make a decision upon the information given with each position. Depends on designs, an item may take 10 - 16 bytes. To build and/or edit those opening books, users need to use special software.

The most popular format is PolyGlot.

Generation

All in all the information stored in an opening database should guide the chess game into a type of middle game in which the chess program proves itself most successful. But to achieve this there are two main ways.

Handcrafted

If the opening database designer has some special preference of a certain opening line, he might create it fully manually. This is often the case if a chess program is about to play an important game against a grandmaster. Then the opening style will probably be chosen to be difficult and dangerous for the opponent. Another option is to first automatically generate a database and then fine-tune it manually.

From game collection

Another option is to select a number of games (the type of which will affect the openings played: e.g. drawn games only, GM games only, etc) and then up to a certain depth store all positions in the opening database. It is true that by using this way of database generation some severe errors might by copied as well, but usually the chess programs are set to stop playing according to the book if a position occurred less than a certain number times before.

By computing

Chess engines can be used to evaluate positions in a book opening tree to rank them, and/or decide to stop or expand.

Book Building Tools

Quotes

Jonathan Schaeffer in One Jump Ahead [5] :

To solve the opening problems of his chess machine, Belle, Ken Thompson typed in opening lines from the Encyclopedia of Chess Openings (in five thick volumes). Religiously, he dedicated one hour a day for almost three years (!) to the tedious pursuit of entering lines of play from the books and having his Belle computer verify them. The result was an opening library of roughly three-hundred thousand moves. The results were immediate and obvious: Belle became a much stronger chess program, and Ken probably aged prematurely. Later Ken developed a program to automatically read the Encyclopedia, allowing him to do in a few days what had taken him three years to do manually. [6] 

Formats

Classification of Chess Openings

Book Issues

See also

Selected Publications

1974 ...

Я.Ю. Гольфанд, А.Л. Футер (1974). Реализация дебютной справочной для шахматной программы./ Сб. Проблемы кибернетики №29, стр. 201-210

1980 ...

1990 ...

2000 ...

2005 ...

2010 ...

2015 ...

Forum Posts

1990 ...

1995 ...

2000 ...

2005 ...

Re: Opening books format by Jury Osipov, CCC, April 15, 2008 » ABK
Re: Opening books format by Richard Pijl, CCC, April 15, 2008
Re: Opening books format by Dann Corbit, CCC, April 15, 2008

2010 ...

2013

Re: 2 Moves Engine Book by Miguel A. Ballicora, CCC, December 08, 2013 » Ruby

2014

2015 ...

2016

2017

2018

2019

2020 ...

External Links

Engine Books

Online Opening Tree

Misc

References

Up one level