Syzygy Bases

Home * Knowledge * Endgame Tablebases * Syzygy Bases



Syzygy Bases, a compact six piece endgame database developed by Ronald de Man, published on April 01, 2013. Since August 2018, seven piece Syzygy Bases are available after an effort by Bojun Guo started in March 2018. The generator is released under the GNU General Public License Version 2, the thread safe probing code is released without restrictions.

=Data=

File Types
Syzygy Bases consist of two sets of files, WDL files (extension .rtbw) storing win/draw/loss information considering the fifty-move rule for access during search, and DTZ files (extension .rtbz) with distance-to-zero information for access at the root. WDL has full data for two sides but DTZ50 omitted data of one side to save space. Each endgame has a pair of those types.

Comparision
Syzygy EGTB is significantly smaller than any DTM EGTB. It is 7 times as small as Gaviota for 5 men, 8 times as small as Nalimov for 6 men, 8 times as small as Lomonosov for 7 men. However, when all DTM EGTBs have full data of two sides, Syzygy EGTB omits data of one side for DTZ data to save space. Ronald de Man estimated if keep them all, the Syzygy's 6 men size may increase 158 GB, become 307 GB in total, double on size, 4 times as small as Nalimov 6 men.

=Generation=

Up to 6-man
On the first release (Apr 01, 2013) the generator was ready to generate all endgames up to 6 men. The generation is multithreaded and processes completely in RAM. To generate all 6 men, it requires a system with at least 32 GB of RAM and may run in 5 days (the period was measured with a computer 6-core i3930K @ 4.2Ghz, 64 GB).

7-man
Ronald de Man wasn't initially interested in the creation of 7-men Syzygy Based since the generation would require about 1 TB of RAM, too expensive at that time. Generation time would be about 64 x per table, which means around 175 x time total .His original generator could not create them. But in 2018, he supported Bojun Guo in his 5-month attempt to generated them. His hardware was estimated at over US$ 90K. In August of 2018 their creation was completed.

8-man
After the completion of 7-man, many people start being curious about the feasibleness of building 8-man. Ronald de Man made an estimation that task requires computers with 64 TB RAM and 2000 TB hard disks (cost about $640K and $40K respectively in 2020 ).

Checksums
Syzygy endgame files may contain 128-bit checksum keys at the end of those files. It also has its own code for checksums (based on Google's cityhash library).

=Search=

During the Search
During the search, with the WDL tables stored on SSD, it is possible to probe the tables at all depths without much slowdown. They have been tested in Ronald de Man's engine Sjaak (playing on FICS as TrojanKnight(C)) a couple of months quite successfully, don't probing in quiescence search.

At the Root
At the root, since pure DTZ50-optimal play (i.e. minimaxing the number of moves to the next capture or pawn move by either side) can be very unnatural, it might be desirable to let the engine search on the winning moves until it becomes clear that insufficient progress is being made and only then switch to DTZ-optimal play (e.g. by detecting repetitions and monitoring the halfmove clock).

=Pros & Cons=

Pros

 * Small sizes. It is about 8 times as small as the second-best EGTBs. Having small sizes is the main success key of Syzygy Bases since it is much easier to create, store, provide, download than other EGTBs
 * Free and more popular (than other EGTBs) to find on the Internet
 * Support DTZ50 metric. That metric can help engines to have better results than DTM which is supported widely by other EGTBs

Cons

 * Hard to integrate with chess engines. Ronald de Man has not provided probing code as an independent library but as a part of Stockfish chess engines. The probe code has integrated too deeply with that chess engine code and it requires a lot of effort to de-integrate, modify and integrate with other chess projects
 * Hard to understand and contribute to the project. ETGB itself is a hard topic. Syzygy EGTB has also integrated with many advanced techniques/tricks. It is written in old-style C language. All make it become very hard to understand and/or modify to improve or for other purposes
 * DTZ50 metric may lead the engines to win in much longer and ugly ways, compared with DTM one

=Data publish= Ronald de Man has provided only open source code for generators/probers but not endgame files themselves. Using his tools some people have generated endgames and published them via DVD or online.

DVD
As of February 2015, all 3-5 and most important 6-men Syzygy Bases are commercially available on 4 DVDs by ChessBase as Endgame Turbo 4 for their products Deep Fritz 14, Komodo Chess 8, Houdini 4 or ChessBase 12/13.

Free Download
There are some free FTP servers for downloading such as Bojun Guo and Lichess servers.

3-6 men
The sizes of those men are small enough to download and store on modern computers. Users should download them in full sets (3, 4, 5, 6 men).

7 men
All 7 men files' size is over 16.7 TiB, over storages of typical modern computers. They also require a long time to download too. Thus some users choose to download one or a few endgames only, based on their statistics of use in endgames. Bellow is the top 20 of those endgames by their order. The first one, KRPPvKRP, has a significantly higher frequency of use than the others and should be always downloaded:

=Probe Code and Tools=

Stockfish
Ronald de Man did not provide the probe code as an independent library. Instead, he published it firstly as an already integrated code for Stockfish chess engines. It is c++ code and it has been rewritten, updated several times by Stockfish team.

Fathom
Fathom is a stand-alone Syzygy based probing tool and API by Basil Falcinelli, introduced in November 2015 along with his Gull 3 release. Unlike the original tbprobe code, Fathom does not necessarily require the callee to provide move generation functionality. The new modifications and extensions to Ronald de Man's original code which can be "redistributed and/or modified without restrictions", are released under the permissive MIT License. The API consists of three functions :
 * tb_init initializes the tablebase
 * tb_probe_wdl probes the Win-Draw-Loss (WDL) table for a given position
 * tb_probe_root probes the Distance-To-Zero (DTZ) table for the given position.

Jon Dart has a fork of Fathom with some bug fixes and enhancements, also supporting 7-man.

Pyrrhic
Pyrrhic is a cleaned up Fathom by Andrew Grant, introduced in August 2020.

=Elo gain= Fishtest team revealed a test with Stockfish (SF10dev) at 10+0.1, with all Syzygy WDL files on RAM, testing using none(0), 4, 5, and 6 man TB in a round-robin tournament.

=Quotes= by Ronald de Man in a reply to Guy Haworth, April 06, 2013 : I create both WDL and DTZ in one go, so I don't use WDL in the creation of DTZ. The algorithm used is the grandfather algorithm with 2 plies per iteration (I think HGM calls this leapfrogging, but I might be wrong). I tried the outcounting method, but it didn't seem to be competitive (and it makes things more complicated). A pure WDL/DTZ pair is not of much use for creating WDL50+/DTZ50+. I create tables in RAM that have all the information necessary for WDL50+ and DTZ50+, then permute them to different indexing schemes and compress. I do test runs on subsets of the data to find good permutations. (The idea to try permutations is from Jesper Torp Kristensen's master thesis.)

Ronald de Man in a reply to Nguyen Pham, April 15, 2020 : Syzygy WDL is double sided, DTZ is single sided. So to know whether a 7-piece position is winning, losing or drawn (or cursed), the engine needs to do only a single probe of a 7-piece WDL table. (It may in addition have to do some probes of 6-piece WDL tables if any direct captures are available.) If the engine needs to know the DTZ value (which is only necessary when a TB root position has been reached), the probing code may have to do a 1-ply search to get to the "right" side of the DTZ table. For 6-piece TBs, DTZ is 81.9GB when storing only the smaller side of each table. Storing both sides might require perhaps 240GB.

Morgan Houppin explained why his chess engine Stash doesn't have Syzygy probe code, March 25, 2021 : Syzygy probing is a hell of a mess, and I don't want to plug two thousand lines of foreign code that I don't understand, nor do I have the motivation to fully understand how Syzygy files are stored, and then write the additional two thousand lines of code to read them for a mere 5 Elo gain at TCEC.

=Endgame News= In his 2014 Chess Endgame News in ICGA Journal, Vol. 37, No. 2, Guy Haworth classified Syzygy Bases as new data in three ways:
 * 1) 5-valued scale for evaluating positions in the context of the FIDE 50-move rule (50mr) which constrains the length of phases of play
 * 2) +2 ≡ unconditional win for the side to move
 * 3) +1 ≡ ‘win’ which can be frustrated by best play and a 50mr draw-claim
 * 4) _ 0 ≡ unconditional draw
 * 5) -1 ≡ ‘loss’ saved by a 50mr draw-claim
 * 6) -2 ≡ unconditional loss
 * 7) depths for ‘50mr draw’ positions with value ±1
 * 8) depths in symmetric, information-preserving ply ‘p’

and further gives some news about early software bugs and glitches concerning ChessBase products, and the importance of MD5 to check the EGT integrity.

=See also=
 * Bitbases
 * Edwards' Tablebases
 * Gaviota Tablebases
 * Lomonosov Tablebases
 * Nalimov Tablebases
 * python-chess
 * Scorpio Bitbases
 * Thompson's Databases

=Publications=
 * Guy Haworth (2014). Chess Endgame News. ICGA Journal, Vol. 37, No. 1
 * Guy Haworth (2014). Chess Endgame News. ICGA Journal, Vol. 37, No. 2
 * Guy Haworth (2014). Chess Endgame News. ICGA Journal, Vol. 37, No. 3 » Fritz 14
 * Guy Haworth (2018). Chess Endgame News: 7-man ‘Syzygy’ DTZ50 EGTs. ICGA Journal, Vol. 40, No. 4

=Forum Posts=

2013 ...
2014
 * New 6-piece tablebases by Ronald de Man, CCC, April 01, 2013
 * New 6-piece tablebase generator by syzygy, CCRL Discussion Board, April 01, 2013
 * Re: PEXT Bitboards by Ronald de Man, CCC, June 07, 2013 » BMI2 - PDEP, BMI2 - PEXT
 * Syzygy EGTB's via Torrent Thread by Joshua Shriver, CCC, September 11, 2013
 * Syzygy tablebases, work in Stockfish? by Jose Mº Velasco, CCC, September 23, 2013 » Stockfish
 * Building Syzygy bases by higgs, Rybka Forum, October 12, 2013
 * tablebase caching / mmap / page cache by Ronald de Man, CCC, October 13, 2013 » Memory
 * Syzygy endgame tables: Generation and first impressions by Mike Scheidl, CCC, October 15, 2013
 * deMan TB Path and Cache by Matthias Gemuh, CCC, October 19, 2013
 * syzygy TB (3-4-5 men only) download link ? by MarshallArts, Rybka Forum, October 21, 2013
 * rkiss and other dependencies in syzygy by Don Dailey, CCC, October 23, 2013
 * Syzygy / egbb discussion by Daniel Shawul, CCC, October 23, 2013 » Scorpio Bitbases
 * Manual: How to use Syzygy (or any other) 6-men without SSD by Milos Stanisavljevic, CCC, November 16, 2013
 * potential deadlock in syzygy reference implementation by Richard Vida, CCC, November 23, 2013
 * Re: A note for C programmers by Marcel van Kervinck, CCC, November 27, 2013
 * Stockfish Syzygy: how to interpret mates? by Jouni Uski, CCC, December 01, 2013 » Stockfish, Mate Scores
 * Problem with 6-piece syzygy-bases using wine by Bernhard Bauer, CCC, December 05, 2013
 * ChessGUI 0.245f is available by Matthias Gemuh, CCC, December 14, 2013 » ChessGUI
 * Syzygybases suitable for win32-systems? by Norbert Raimund Leisner, CCC, December 17, 2013
 * Syzygy Tablebases list of importance by chri$, OpenChess Forum, December 21, 2013
 * Syzygy options by Harm Geert Muller, CCC, December 27, 2013
 * Ideal Syzygy Probe Depth ? (using SSD) by Anil V Dharan, CCC, January 14, 2014
 * SYZYGY Base question by Ingo Bauer, CCC, January 19, 2014
 * problem with syzygy tablebases by Youri Matiounine, CCC, February 01, 2014
 * Performance of Syzygy and Scorpio by Kai Laskos, CCC, February 04, 2014 » Scorpio Bitbases
 * A question about syzygy by Enrico Tognoni, CCC, February 26, 2014
 * A question about syzygy 6 men and partial use by Enrico Tognoni, CCC, February 26, 2014
 * Syzygy on RAM Drive by Kai Laskos, CCC, May 23, 2014 » Stockfish, Komodo 7, Houdini 4
 * Re: Syzygy tb generator for windows by syzygy, CCRL Discussion Board, June 01, 2014
 * Question about syzygy bases by Gabor Szots, CCC, June 02, 2014
 * Re: 7-piece syzygy by syzygy, CCRL Discussion Board, July 03, 2014
 * Question on Stockfish and SyzygyCache UCI option by Erfuk Neuni, CCC, December 07, 2014
 * USB 3 Storage for Syzygy WDL files by Louis Zulli, CCC, December 13, 2014 » USB 3.0

2015 ...
2016
 * Komodo 8 - 5-men Syzygy tablebases by Andreas Strangmüller, CCC, January 10, 2015 » Komodo 8
 * Problem with SF6 and Syzygy TB by Forrest Hoch, CCC, April 01, 2015 » Stockfish
 * SD Syzygy by Ted Summers, CCC, April 26, 2015
 * 5 men Syzygy on USB 3.0 Flash Drive by Kai Laskos, CCC, May 09, 2015
 * Re: how to probe egtb from console? by Ronald de Man, CCC, May 15, 2015 » python-chess
 * Gull 3 Linux+Syzygy and Fathom released by Basil Falcinelli, CCC, November 20, 2015 » Gull, Fathom
 * Syzygy probing code: DTZ in some cursed endgames off by one? by Niklas Fiekas, CCC, December 06, 2015
 * Re: Squash anyone? by Ronald de Man, CCC, February 07, 2016
 * My troubles with MultiPV and Syzygy in Stockfish 7 by Árpád Rusz, CCC, February 16, 2016
 * Stockfish 7 and partial 6 piece syzygy problem? by Jouni Uski, CCC, March 01, 2016 » Stockfish
 * Re: Stockfish 7 and partial 6 piece syzygy problem? by Marco Costalba, CCC, September 01, 2016

2017 2018
 * Arasan Syzygy support (working with Windows, too) by Jon Dart, CCC, March 10, 2016 » Arasan
 * Question to syzygy author by Marco Costalba, CCC, April 24, 2016
 * syzygy question by Robert Hyatt, CCC, May 04, 2016
 * question about syzygy probing by Marco Belli, CCC, May 21, 2016
 * Natural TB by Marco Costalba, CCC, May 29, 2016 » Stockfish
 * syzygy questions by Robert Hyatt, CCC, July 06, 2016
 * How texel probes endgame tablebases by Peter Österlund, CCC, July 16, 2016 » Gaviota Tablebases, Texel
 * Syzygy and draw by repetition by Jon Dart, CCC, July 22, 2016 » Draw, Repetitions
 * Syzygy question by J. Wesley Cleveland, CCC, September 03, 2016
 * Suicide chess tablebases (stalemated player wins) by Niklas Fiekas, CCC, October 25, 2016 » Losing Chess
 * Syzygy tablebases by Andy Leese, CCC, December 01, 2016
 * Help for Syzygy probe? by Ted Wong, CCC, December 04, 2016 » Fathom
 * 6-men Syzygy from HDD and USB 3.0 by Kai Laskos, CCC, April 04, 2017 » Komodo, Playing Strength, USB 3.0
 * Fathom memory usage by Álvaro Begué, CCC, June 22, 2017» Fathom
 * RuyDos with support for syzygy tablebases by Álvaro Begué, CCC, June 23, 2017 » RuyDos
 * Natural TB (take 2) by Marco Costalba, CCC, August 22, 2017 » Stockfish
 * Probing tablebases through USB 3.0 by Jon Fredrik Åsvang, CCC, September 25, 2017 » USB 3.0
 * understanding DTZ by Alexandru Mosoi, CCC, October 06, 2017 » DTZ, Fathom
 * Is there now coming changes to syzygy databases? by Jouni Uski, CCC, November 13, 2017 » DTM, CFish
 * How to Download Syzygy Endgame Tablebase Files by Daniel Johnson, CCC, December 23, 2017
 * The history of Syzygy tablebases by Isaac Haïk Dunn, CCC, March 06, 2018
 * 7-men Syzygy attempt by Bojun Guo, CCC, March 10, 2018
 * Re: 7-men Syzygy attempt by Bojun Guo, CCC, August 19, 2018

2019
 * Syzygy implementations of top engines by Kai Laskos, CCC, March 14, 2018
 * Probing the Syzygy tablebase - beginners question by Andreas Matthies, CCC, April 16, 2018
 * DTM50 by Ronald de Man, CCC, May 22, 2018
 * Re-Pair compression questions by Rein Halbersma, CCC, August 17, 2018
 * BIG NEWS! The 7 man syzygy tablebase files are complete by Dann Corbit, CCC, August 20, 2018
 * Technical reason why probing N-men syzygy will also probe N-1 men? by Sven Schüle, CCC, October 28, 2018
 * 7 Man Syzygy and SSD by Michael B, CCC, December 18, 2018
 * Testing the implementation of Syzygy by Vincent Tang, CCC, March 02, 2019
 * 7-man Syzygy support in Fathom by Jon Dart, CCC, April 23, 2019 » Fathom
 * Simplest use of syzygy table by Vivien Clauzon, CCC, July 28, 2019
 * SYZYGY question by Robert Hyatt, CCC, August 11, 2019 » Crafty, En passant
 * Syzygy 7-piece - several questions by Andreas Matthies, CCC, August 19, 2019
 * Syzygy 7 man advice please by Barry Clements, CCC, August 21, 2019
 * Syzygy DTZ data explaination? by Nguyen Pham, CCC, September 23, 2019

2020 ...
2021
 * Almost perfect DTM tablebase by Dann Corbit, CCC, April 08, 2020
 * What is the best way to obtain the 7-piece tablebases? by Mark Thellen, CCC, June 15, 2020
 * Pyrrhic, Fathom for Humanoids by Andrew Grant, CCC, August 16, 2020 » Pyrrhic
 * EGTB compression by Dann Corbit, CCC, October 14, 2020
 * Syzygy bases ... question to "Syzygy Probe Depth" by Frank Quisinsky, CCC, October 21, 2020
 * Fathom and 7-men by Joshua Shriver, CCC, November 24, 2020 » Fathom
 * Problem with Syzygy tablebase by Elias Nilsson, CCC, December 03, 2020
 * Can EGTB storage requirements be reduced using this scheme? by mmt, CCC, December 07, 2020
 * Syzygy Tablebase Names: A very stupid exercise by Andrew Grant, CCC, May 08, 2021
 * syzygy implementation by Desperado, CCC, May 23, 2021
 * When will 8 piece tablebase be ready? by Agustin Jorge Pichardo, CCC, May 29, 2021
 * Syzygy bases from memory by Ed Schröder, CCC, June 16, 2021 » KPK
 * Syzygy benefit for current SF by Jouni, CCC, September 02, 2021

=External Links=

Tablebase

 * syzygy1/tb · GitHub by Ronald de Man
 * jromang · GitHub by Jean-Francois Romang has a fork from syzygy1/tb
 * python-chess/syzygy.py at master · niklasf/python-chess · GitHub by Niklas Fiekas, Python implementation of probing code » python-chess
 * niklasf/syzygy-tables.info · GitHub by Niklas Fiekas, GUI and public API for Syzygy probing
 * OICS Chess and EGTB Tracker Statistics by Joshua Shriver
 * Endgame Tablebases Online by Kirill Kryukov
 * Index of /tablebases/syzygy (3,4,5) by kingliveson
 * tablebase.sesse.net by Sesse

Fathom

 * GitHub - basil00/Fathom: Syzygy TB probe tool by Basil Falcinelli
 * jdart1/Fathom · GitHub by Jon Dart (with some bug fixes and enhancements)
 * GitHub - ljgw/syzygy-bridge: Java bridge to use the Syzygy Tablebases via JNI by Laurens Winkelhagen » FrankWalter

Pyrrhic

 * GitHub - AndyGrant/Pyrrhic: Fathom, for Humanoids by Andrew Grant

Online Lookup

 * Chess Cloud Database Query Interface by noobpwnftw
 * Syzygy endgame tablebases Web Interface

ChessBase

 * Syzygy Tablebases: newest, fastest, smallest by Albert Silver, ChessBase News, February 08, 2015
 * Syzygy tablebases: maximizing performance by Albert Silver, ChessBase News, February 10, 2015
 * Endgame Turbo 4 by ChessBase

Misc

 * syzygy - Wiktionary
 * Syzygy (disambiguation) from Wikipedia
 * Syzygy (astronomy) from Wikipedia
 * Syzygy (mathematics) from Wikipedia
 * Caledonian Antisyzygy from Wikipedia
 * Michael Brecker Band - Syzygy, BJD 1987, Bratislava, YouTube Videos
 * Michael Brecker, Mike Stern, Jeff Andrews, Adam Nussbaum, Joey Calderazzo

=References= Up one level