Changes

Jump to: navigation, search

Engine Testing

1,242 bytes added, 08:41, 3 May 2020
no edit summary
=Tuning=
* [[Automated Tuning]]
* [[Engine Similarity]]
=Test-Positions=
During testing the engines should ideally play the same style of openings they would play in a normal tournament, so not to optimize them for different types of positions. One option is to use the engines own [[Opening Book|opening book]] or one can use [[Test-Positions#OpeningSuites|opening suites]], a set of quiet test positions. In the latter case the same opening suit would be used for each tournament conducted and furthermore each position is played a second time with colors reversed. With these measures one can try to minimize the disparity between tests caused by different openings.
==InterfacesTournament Manager== Free [[GUIUser Interface|graphical user User interfaces]] or [[CLI|command line tools]] for [[UCI]] and [[Chess Engine Communication Protocol]] compatible engines in engine-engine matches are:* mentioned under [[Amoeba#TournamentManager|Amoeba Tournament Manager]]* [[Arena]] by [[Martin Blume]] <ref>[http://www.playwitharena.com/ Free chess graphical user interface (GUI) Arena for chess engines]</ref>* [[Cute Chess]] by [[Arto Jonsson]] and [[Ilari Pihlajisto]]: [[Cutechess-cli]]* [[LittleBlitzer]] by [[Nathan Thom]] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=35249 Fast Games] by [[Manuel Diaz]], [[CCC]], July 02, 2010</ref>
==Frameworks==
* [[Stockfish#TestingFramework|Fishtest]]
* [[OpenBench]]
<span id="ChessServer"></span>
==Chess Server==
* [http://www.open-aurec.com/wbforum/viewtopic.php?f=4&t=5866 test methodology] by [[Giuseppe Cannella]], [[Computer Chess Forums|Winboard Forum]], November 13, 2006
* [http://www.open-aurec.com/wbforum/viewtopic.php?f=4&t=5955 Testing and debugging chess engines] by [[Patrice Duhamel]], [[Computer Chess Forums|Winboard Forum]], December 03, 2006
'''2007'''
* [http://www.talkchess.com/forum/viewtopic.php?t=13557 Programmer bug hunt challenge] by [[Ed Schroder|Ed Schröder]], [[CCC]], May 04, 2007 » [[Portable Game Notation]], [[En passant]]
* [http://www.talkchess.com/forum/viewtopic.php?t=13800 a beat b,b beat c,c beat a question] by [[Uri Blass]], [[CCC]], May 16, 2007 » [[Playing Strength]]
* [http://www.talkchess.com/forum/viewtopic.php?t=16412 An objective test process for the rest of us?] by [[Nicolai Czempin]], [[CCC]], September 12, 2007
* [http://www.talkchess.com/forum/viewtopic.php?t=17947 My new testing scheme] by [[Zach Wegner]], [[CCC]], November 20, 2007
'''2008'''
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=20082 Test you engine] by [[Fermin Serrano]], [[CCC]], March 10, 2008
* [http://www.talkchess.com/forum/viewtopic.php?t=22832 New testing thread] by [[Robert Hyatt]], [[CCC]], August 07, 2008
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=24590 Comparing two version of the same engine] by [[Fermin Serrano]], [[CCC]], October 26, 2008
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=25461 Debate: testing at fast time controls] by [[Fermin Serrano]], [[CCC]], December 15, 2008
'''2009'''
* [http://www.talkchess.com/forum/viewtopic.php?t=27024 Cutechess-cli: A command line tool for engine-engine matches], by [[Ilari Pihlajisto]], [[CCC]], March 16, 2009
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=28130 Testing procedure] by [[Matt Gingell]], [[CCC]], May 27, 2009
* [http://www.talkchess.com/forum/viewtopic.php?topic_view=threads&p=293506&t=27024 Cutechess-cli version 0.1.8 released] by [[Ilari Pihlajisto]], [[CCC]], September 29, 2009
* [http://www.talkchess.com/forum/viewtopic.php?t=30513 A reason for testing at fixed number of nodes] by [[J. Wesley Cleveland]], [[CCC]], November 06, 2009
* [http://www.talkchess.com/forum/viewtopic.php?t=61422 Testing using many computers and architectures] by [[Andrew Grant]], [[CCC]], September 14, 2016
* [http://www.talkchess.com/forum/viewtopic.php?t=61988 command line engine match?] by [[Erin Dame]], [[CCC]], November 06, 2016 » [[CLI]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=61991 Looking for an epd file for sanity checks...] by [[Fermin Serrano]], [[CCC]], November 06, 2016
* [http://www.talkchess.com/forum/viewtopic.php?t=62576 Testing with different EPD suits for search vs eval changes] by [[Michael Sherwin]], [[CCC]], December 23, 2016
'''2017'''
'''2018'''
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=67485 Issue with self play testing] by [[Charles Roberson]], [[CCC]], May 18, 2018
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=68531 Basic automated testing] by Josh Odom, [[CCC]], September 28, 2018
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=68531&start=6 Re:Basic automated testing] by [[Andrew Grant]], [[CCC]], September 30, 2018 » [[OpenBench]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=69284 testing consistency] by [[Jon Dart]], [[CCC]], December 16, 2018
'''2019'''
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70383 Any testing framwork similair to Fishtest that can be run locally ?] by [[Mahmoud Uthman]], [[CCC]], April 02, 2019
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=71848 self test] by [[Vivien Clauzon]], [[CCC]], September 18, 2019
==2020 ...==
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=73129 EPD destruction tests] by [[Chris Whittington]], [[CCC]], February 19, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=73208 EPD destruction tests, part 2] by [[Chris Whittington]], [[CCC]], February 27, 2020
=External Links=
* [http://ajonsson.kapsi.fi/cutechess.html Cute Chess]
* [https://github.com/cutechess/cutechess cutechess · GitHub]
* [httphttps://wwwgithub.topcom/AndyGrant/OpenBench GitHub -5000.nl/tuning.htm OpenBench a Distributed SPRT Testing a chess engine from the ground upFramework for Chess Engines] from by [http://www.top-5000.nl/ Home of the Dutch Rebel[Andrew Grant]] by » [[Ed SchroderMatch Statistics#SPRT|Ed SchröderSPRT]] <ref>* [httphttps://www.talkchessgithub.com/forumChrisWhittington/Chess-EPDs GitHub - ChrisWhittington/viewtopic.php?t=46759 A poor man's testing environmentChess-EPDs: Various EPD test suites] by [[Ed Schroder|Ed Schröder]], [[CCC]], January 04, 2013</ref> » [[Match StatisticsChris Whittington]]
* [https://en.wikipedia.org/wiki/Regression_testing Regression testing from Wikipedia]
* [http://www.sp-cc.de/index.htm SPCC] by [[Stefan Pohl]]

Navigation menu