OpenBench

Home * Engine Testing * OpenBench

OpenBench, an open source chess engine testing framework authored by Andrew Grant, written in Python and licensed under the GNU GPL. It is inspired Fishtest and makes use of distributed computing, allowing anyone to contribute CPU time to further improve the development of open source chess engines. OpenBench uses the Django Web Framework and Cutechess and applies the sequential probability ratio test in engine versus engine matches, and supports Chess960. OpenBench was the primary testing framework used for the development of Ethereal.

=Clients= OpenBench client PCs need to be POSIX compliant - that is Windows clients require MinGW - to build chess engine binaries on the fly assuming that gcc/g++ and make are on the system path. To get the sources, OpenBench will pull down a zip file from GitHub. Further, the client PC requires Python 3 and finally the OpenBench Client software itself. The client is started with four arguments specifying the registered user name and password, the server URL and the number of threads used for engine matches : python3 OpenBench.py -U username -P password -S http://chess.grantnet.us/ -T 4

=See also=
 * Ethereal
 * Fishtest
 * SPRT
 * Engines using OpenBench

=Forum Posts=

2016 ...

 * SPRT when not used for self testing by Andrew Grant, CCC, October 21, 2016 » SPRT
 * A question about SPRT by Andrew Grant, CCC, December 25, 2016
 * Testing A against B by playing a pool of others by Andrew Grant, CCC, June 24, 2017 » Match Statistics
 * Basic automated testing by odomobo, CCC, September 28, 2018
 * Re:Basic automated testing by Andrew Grant, CCC, September 30, 2018


 * Any testing framwork similair to Fishtest that can be run locally ? by Mahmoud Uthman, CCC, April 02, 2019

2020 ...

 * Re: Looking for automatic Engine Testing Software by Jon Dart, CCC, July 19, 2020
 * Final Release of Ethereal, V12.75 by Andrew Grant, CCC, October 09, 2020 » Ethereal
 * OpenBench question by Ed Schröder, CCC, April 13, 2021

=External Links=

OpenBench

 * GitHub - AndyGrant/OpenBench: OpenBench is a Distributed SPRT Testing Framework for Chess Engines
 * OpenBench/Client.py at master · AndyGrant/OpenBench · GitHub
 * OpenBench

Misc

 * Benchmark (computing) from Wikipedia

=References= Up one Level