Changes

Jump to: navigation, search

Transposition Table

3,622 bytes added, 08:46, 29 June 2021
no edit summary
A '''Transposition Table''',<br/>
first used in [[Richard Greenblatt|Greenblatt's]] program [[Mac Hack#HashTable|Mac Hack VI]] <ref>[[Richard Greenblatt]], [[Donald Eastlake]] and , [[https://en.wikipedia.org/wiki/Steve_Crocker Stephen D. Crocker]] ('''1967'''). ''The Greenblatt Chess Program''. Proceedings of the AfiPs Fall Joint Computer Conference, Vol. 31, pp. 801-810. Reprinted ('''1988''') in [[Computer Chess Compendium]], [http://archive.computerhistory.org/projects/chess/related_materials/text/2-4.Greenblatt_Chess_Program/The_Greenblatt_Chess_Program.Greenblatt_Eastlake_Crocker.1967.Fall_Joint_Computer_Conference.062303060.sm.pdf pdf] from [[The Computer History Museum]] or as [http://dspace.mit.edu/handle/1721.1/6176 pdf or ps] from [http://libraries.mit.edu/dspace-mit/ DSpace] at [[Massachusetts Institute of Technology|MIT]]</ref> <ref>[[Albert Zobrist]] ('''1970'''). ''A New Hashing Method with Application for Game Playing''. Technical Report #88, Computer Science Department, The University of Wisconsin, Madison, WI, USA. Reprinted ('''1990''') in [[ICGA Journal#13_2|ICCA Journal, Vol. 13, No. 2]], [http://www.cs.wisc.edu/techreports/1970/TR88.pdf pdf]</ref> <ref>[http://groups.google.com/group/rec.games.chess.computer/browse_frm/thread/259fbfd2b39b8ee4/80c64ba0f632fc31 Re: Berliner vs. Botvinnik Some interesting points], post 8 by [[Bradley Kuszmaul|Bradley C. Kuszmaul]], [[Computer Chess Forums|rgcc]], November 06, 1996 » Transposition Table in [[Mac Hack]]</ref> , is a database that stores results of previously performed searches. It is a way to greatly reduce the search space of a [[Search Tree|chess tree]] with little negative impact. Chess programs, during their [[Brute-Force|brute-force]] search, encounter the same [[Chess Position|positions]] again and again, but from different sequences of [[Moves|moves]], which is called a [[Transposition|transposition]]. Transposition (and [[Refutation Table|refutation]]) tables are techniques derived from [[Dynamic Programming|dynamic programming]] <ref>[https://en.wikipedia.org/wiki/Dynamic_programming#Algorithms_that_use_dynamic_programming Algorithms that use dynamic programming from Wikipedia]</ref> , a term coined by [[Richard E. Bellman]] in the 1950s, when programming meant planning, and dynamic programming was conceived to optimally plan multistage processes <ref>[[Mathematician#SDasgupta|Sanjoy Dasgupta]], [[Mathematician#CHPapadimitriou|Christos H. Papadimitriou]], [[Mathematician#UVVazirani|Umesh Vazirani]] ('''2006'''). ''[http://www.cs.berkeley.edu/%7Evazirani/algorithms.html Algorithms]''. [https://en.wikipedia.org/wiki/McGraw-Hill McGraw-Hill], ISBN: 978-0073523408, [http://www.amazon.com/gp/product/0073523402?ie=UTF8&tag=ebookdire-20&link_code=as3&camp=211189&creative=373489&creativeASIN=0073523402 amazon], Chapter 6, Dynamic programming</ref> .
=How it works=
<span id="Entry"></span>
=What Information is Stored=
Typically, the following information is stored as determined by the [[Search|search]] <ref>[[Dennis Breuker]], [[Jos Uiterwijk]], and [[Jaap van den Herik]] ('''1997'''). ''Information in Transposition Tables''. [[Advances in Computer Chess 8]]</ref> :
* [[Zobrist Hashing|Zobrist-]] or [[BCH Hashing|BCH-key]], to look whether the position is the right one while probing
* [[Best Move|Best-]] or [[Refutation Move|Refutation move]]
<span id="ReplacementStrategies"></span>
=Replacement Strategies=
Because there are a limited number of entries in a transposition table, and because in modern chess programs they can fill up very quickly, it is necessary to have a scheme by which the program can decide which entries would be most valuable to keep, i.e. a replacement scheme <ref>[[Dennis Breuker]], [[Jos Uiterwijk]] and , [[Jaap van den Herik]] ('''1994'''). ''Replacement Schemes for Transposition Tables''. [[ICGA Journal#17_4|ICCA Journal, Vol. 17, No. 4]]</ref> . Replacement schemes are used to solve an index collision, when a program attempts to store a position in a table slot that already has a different entry in it. There are two opposing considerations to replacement schemes:
* Entries that were searched to a high depth save more work per table hit than those searched to a low depth.
* Entries that are closer to the leaves of the tree are more likely to be searched multiple times, making the table hits of them higher. Also, entries that were searched recently are more likely to be searched again.
<span id="TwoTier"></span>
==Two-tier System==
This strategy, devised by [[Ken Thompson]] and [[Joe Condon]] <ref>[[Joe Condon]] and , [[Ken Thompson]] ('''1983'''). ''BELLE Chess Hardware''. [[Advances in Computer Chess 3]]</ref> , uses two tables, side by side. For each table slot, there is a depth-preferred and an always-replace entry <ref>[[Dennis Breuker]], [[Jos Uiterwijk]] and , [[Jaap van den Herik]] ('''1996'''). ''Replacement Schemes and Two-Level Tables''. [[ICGA Journal#19_3|ICCA Journal, Vol. 19, No. 3]]</ref> .
<span id="Bucket"></span>
==Bucket Systems==
This family of strategies is similar to the two-tier system, but any number of tiers (known as "buckets") can be used (typically the number is based on the size of a cacheline). The difference is that the buckets are not specific to one consideration, but rather the new entry overwrites the entry in the bucket with the lowest depth <ref>[[Don Beal]] and , [[Martin C. Smith]] ('''1996'''). ''Multiple Probes of Transposition Tables''. [[ICGA Journal#19_4|ICCA Journal, Vol. 19, No. 4]]</ref> .
<span id="Aging"></span>
==Aging==
=Publications=
==1967 ...==
* [[Richard Greenblatt]], [[Donald Eastlake]], [https://en.wikipedia.org/wiki/Steve_Crocker [Stephen D. Crocker]] ('''1967'''). ''The Greenblatt Chess Program''. Proceedings of the AfiPs Fall Joint Computer Conference, Vol. 31, pp. 801-810. Reprinted ('''1988''') in [[Computer Chess Compendium]], [http://archive.computerhistory.org/projects/chess/related_materials/text/2-4.Greenblatt_Chess_Program/The_Greenblatt_Chess_Program.Greenblatt_Eastlake_Crocker.1967.Fall_Joint_Computer_Conference.062303060.sm.pdf pdf] from [[The Computer History Museum]] or as [http://dspace.mit.edu/handle/1721.1/6176 pdf or ps] from [http://libraries.mit.edu/dspace-mit/ DSpace] at [[Massachusetts Institute of Technology|MIT]]
==1970 ...==
* [[Albert Zobrist]] ('''1970'''). ''A New Hashing Method with Application for Game Playing''. Technical Report #88, Computer Science Department, The University of Wisconsin, Madison, WI, USA. Reprinted ('''1990''') in [[ICGA Journal#13_2|ICCA Journal, Vol. 13, No. 2.]], [http://www.cs.wisc.edu/techreports/1970/TR88.pdf pdf]
==1990 ...==
* [[Dennis Breuker]], [[Jos Uiterwijk]], [[Jaap van den Herik]] ('''1996'''). ''Replacement Schemes and Two-Level Tables''. [[ICGA Journal#19_3|ICCA Journal, Vol. 19, No. 3]].
* [[Don Beal]] and , [[Martin C. Smith]] ('''1996'''). ''Multiple Probes of Transposition Tables''. [[ICGA Journal#19_4|ICCA Journal, Vol. 19, No. 4]]
* [[Dennis Breuker]], [[Jos Uiterwijk]], [[Jaap van den Herik]] ('''1997'''). ''Information in Transposition Tables''. [[Advances in Computer Chess 8]]
* [[Dennis Breuker]] ('''1998'''). ''[http://www.dennisbreuker.nl/thesis/ Memory versus Search in Games]''. Ph.D. Thesis, [[Maastricht University|Universiteit Maastricht]], The Netherlands. ISBN 90-9012006-8.
* [https://www.stmintz.com/ccc/index.php?id=22816 draw by repetition and hash tables] by [[Werner Inmann]], [[CCC]], July 24, 1998
'''1999'''
* [https://www.stmintz.com/ccc/index.php?id=41612 Hash Tables - Should one store EP, Castling rights etc?] by [[Steve Maughan]], [[CCC]], January 30, 1999 » [[Castling Rights]], [[En passant]]
* [https://www.stmintz.com/ccc/index.php?id=43613 Hash collisions and a little maths] by [[Rémi Coulom]], [[CCC]], February 18, 1999
* [https://www.stmintz.com/ccc/index.php?id=43990 Hash Collisions] by [[KarinsDad]], [[CCC]], February 21, 1999
'''2004'''
* [https://www.stmintz.com/ccc/index.php?id=339934 I need your opinion about this hash entry structure] by [[Federico Andrés Corigliano|Federico Corigliano]], [[CCC]], January 02, 2004
* [https://www.stmintz.com/ccc/index.php?id=344036 Question about details of hashing (olithink)] by [[Michel Langeveld]], [[CCC]], January 22, 2004 » [[OliThink]]
* [https://www.stmintz.com/ccc/index.php?id=354012 Fruit - Question for Fabien] by [[Dan Honeycutt]], [[CCC]], March 11, 2004 » [[Fruit]], [[Node Types]], [[Principal Variation]], [[Principal Variation Search]]
* [https://www.stmintz.com/ccc/index.php?id=358836 Hashkey collisions (typical numbers)] by [[Jan Renze Steenhuisen|Renze Steenhuisen]], [[CCC]], April 07, 2004 » [[Transposition Table#Collisions|Transposition Table - Collisions]]
* [http://www.open-chess.org/viewtopic.php?f=5&t=2703 transposition table implementation help] by lazyguy123, [[Computer Chess Forums|OpenChess Forum]], August 09, 2014
* [http://www.talkchess.com/forum/viewtopic.php?t=53462 Transposition table chaining and replacement strategy] by [[Alex Ferguson]], [[CCC]], August 28, 2014
* [https://groups.google.com/d/msg/fishcooking/6nNXAQQAXOE/FXs2chqDargJ Using the Transposition Table for long searches] by Theodr Elwurtz, [[Computer Chess Forums|FishCooking]], September 22, 2014 » [[Stockfish]]
* [http://www.talkchess.com/forum/viewtopic.php?t=53849 Speculative prefetch] by [[Peter Österlund]], [[CCC]], September 27, 2014 » [[Memory]]
* [http://www.talkchess.com/forum/viewtopic.php?t=53859 Transposition Table Oddity] by [[Thomas Kolarik]], [[CCC]], September 28, 2014
* [http://www.talkchess.com/forum/viewtopic.php?t=64688 cutechess-cli: not restarting an engine because of tt] by [[Folkert van Heusden]], [[CCC]], July 22, 2017 » [[Cutechess-cli]]
* [http://www.talkchess.com/forum/viewtopic.php?t=65327 Size of Transposition Table Entry] by [[Jason Fernandez]], [[CCC]], September 29, 2017
* [http://www.talkchess.com/forum/viewtopic.php?t=65526 Transposition table and Fine#70] by [[Vincent Tang]], [[CCC]], October 23, 2017 » [[Lasker-Reichhelm Position]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=65903 TT in Qsearch] by [[Laurie Tunnicliffe]], [[CCC]], December 05, 2017 » [[Quiescence Search]]
* [http://www.talkchess.com/forum/viewtopic.php?t=66135 Transposition table based pruning idea] by [[Jerry Donald]], [[CCC]], December 25, 2017 » [[Pruning]]
* [http://www.talkchess.com/forum/viewtopic.php?t=67102 Draw scores in TT] by [[Srdja Matovic]], [[CCC]], April 14, 2018 » [[Draw]], [[Score#DrawScore|Draw Score]]
* [http://www.talkchess.com/forum/viewtopic.php?t=67131 Depth extensions and effect on transposition queries] by [[Kenneth Jones]], [[CCC]], April 16, 2018 » [[Extensions]], [[Check Extensions]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=67387 Not detected collisions in tt probing] by [[Andreas Matthies]], [[CCC]], May 09, 2018 » [[#Collisions|Collisions]]
'''2019'''
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=69629 Playing transposition table moves in the Quiescence search] by [[Andrew Grant]], [[CCC]], January 17, 2019 » [[Quiescence Search]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70586 Prefetch and Threading] by [[Dennis Sceviour]], [[CCC]], April 25, 2019 » [[Memory]], [[Thread]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=67599 Debugging a transposition table] by [[Vincent Tang]], [[CCC]], May 29, 2018 » [[Debugging]], [[Lasker-Reichhelm Position]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70931 Hash collision?] by [[Tom King]], [[CCC]], June 05, 2019
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=71994 Looking for TT policy advice] by [[Vivien Clauzon]], [[CCC]], October 04, 2019 » [[#ReplacementStrategies|Replacement Strategy]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=73110 Zobrist key independence] by [[Harm Geert Muller]], [[CCC]], February 17, 2020 » [[Zobrist Hashing]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=73126 SIMD methods in TT probing and replacement] by [[Harm Geert Muller]], [[CCC]], February 20, 2020 » [[SIMD and SWAR Techniques]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=73202 Measuring Hash Collisions (once again)] by [[Ed Schroder|Ed Schröder]], [[CCC]], February 27, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=73232 Hash size, hash fullness, strength] by [[Vivien Clauzon]], [[CCC]], February 29, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=73493 Where to enter/read position into hash table in perft?] by [[Marcel Vanthoor]], [[CCC]], March 28, 2020 » [[Perft]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=73516 Hash entry/bucket memory usage optimization] by [[Marcel Vanthoor]], [[CCC]], March 30, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74081 asymmetric evaluation and TT] by [[Vivien Clauzon]], [[CCC]], June 02, 2020 » [[Asymmetric Evaluation]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74411 Correct way to store and extract mate scores] by Ian Mitchell, [[CCC]], July 08, 2020 » [[Checkmate]], [[Score]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75549 Principal Variation Search vs. Transposition Table] by [[Marcel Vanthoor]], [[CCC]], October 26, 2020 » [[Principal Variation]], [[Principal Variation Search]]
'''2021'''
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76499 Transposition table replacement scheme] by [[Niels Abildskov]], [[CCC]], February 05, 2021
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76508 Best practices for transposition tables] by Brian Adkins, [[CCC]], February 06, 2021
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76575 TT: key collisions] by Brian Adkins, [[CCC]], February 13, 2021 » [[#KeyCollisions|Key Collisions]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76887 Hash move ordering vs. Hash cuts: savings in number of nodes visited] by [[Marcel Vanthoor]], [[CCC]], March 16, 2021
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77054 PERFT transposition table funny?!] by [[Martin Bryant]], [[CCC]], April 10, 2021 » [[Perft]], [[Memory]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77286 For or against the transposition table probe in quiet search?] by [[Eugene Kotlov]], [[CCC]], May 11, 2021 » [[Quiescence Search]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77470 Saving moves into the TT] by [[Marcel Vanthoor]], [[CCC]], June 11, 2021
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77504 Hash value composition] by [[Yves De Billoëz]], [[CCC]], June 17, 2021
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77569 When to clear the Transposition table?] by [[Thomas Jahn]], [[CCC]], June 28, 2021
=External Links=

Navigation menu