Engine Similarity

Home * Engines * Similarity



Engine Similarity, a loosely defined relation between (two) chess engines concerning their positional playing style, therefor mostly dependent on their evaluation features and weights. One approach to assess engine similarity is to count how often these engines agree with the same move after a shallow search over a set of carefully selected, quiet test positions with apparently several possible best moves - another, in k-best mode, how often they propose the same ranking of k moves. A relative high similarity measure between two engines could be a symptom of using similar evaluation ideas, features and weights, along with automated tuning (or trainig), or even code copying aka cloning. Similarity testing is intended as first automated "screening" to trigger further investigations in case moves are too similar.

=Similarity Testers= In December 2010, Don Dailey published SIM03 to test the similary with a set of UCI compliant engines with 8238 in-build positions - still freely available from the Komodo site. A pair of different chess engines could then be compared for similarity by matching up and counting the number of positions for which each engine chose the same move. SIMEX is a more user friendly successor by Ed Schröder using external EPD-files. Overall similarity of multiple engines may be visualized as hierarchical clustering dendrogram.

=See also=
 * Category:Clone
 * Category:Derivative
 * ICGA Investigations
 * Engine Testing
 * Match Statistics
 * Playing Strength

=Publications=
 * Ingo Althöfer (1997). On the k-best Mode in Computer Chess: Measuring the Similarity of Move Proposals. ICCA Journal, Vol. 20, No. 3
 * Mark Levene, Judit Bar-Ilan (2005). Comparing Move Choices of Chess Search Engines. ICGA Journal, Vol. 28, No. 2, pdf
 * Paolo Ciancarini, Gian Piero Favini (2009). Detecting clones in game-playing software. Entertainment Computing, Vol. 1, No. 1
 * Don Dailey, Adam Hair, Mark Watkins (2014). Move Similarity Analysis in Chess Programs. Entertainment Computing, Vol. 5, No. 3, preprint as pdf

=Forum Posts=

1999

 * Chess program similarity experiment by Bruce Moreland, CCC, January 24, 1999
 * This test is not scientific! by Don Dailey, CCC, January 26, 1999
 * Re: This test is not scientific! by Don Dailey, CCC, January 26, 1999

2000 ...

 * question to vas on similarity of rybka 1.0 to fruit by duncan, Rybka Forum, August 25, 2008

2010 ...

 * Similarity Detector Available by Don Dailey, CCC, December 26, 2010
 * Cluster analysis of similarity test by Miguel A. Ballicora, CCC, December 31, 2010
 * Similarity testing for source code by Mark Lefler, CCC, February 16, 2011
 * Pairwise Analysis of Chess Engine Move Selections by Adam Hair, CCC, April 17, 2011
 * Fritz and Naum shown as Rybka/Strelka clones by Kai Laskos, CCC (Engine Origins), October 17, 2011
 * Pairwise Analysis of Chess Engine Move Selections Revisited by Adam Hair, CCC, March 04, 2012
 * Similarity tool myth - debunked by Don Dailey, CCC, August 23, 2012
 * Fritz 11 by Rebel, OpenChess Forum, December 10, 2013

2015 ...

 * Similarity test by Martin Sedlak, CCC, January 24, 2015 » Cheng
 * Similarity test for Capivara > version 0.0.8 available? by Günther Simon, CCC, May 01, 2016 » Capivara
 * New Similarity Dendrogram by Kai Laskos, CCC, December 03, 2016 (Images broken links)
 * Experiment 11 - Similarity between the top engines back then and now by Ed Schröder, CCC, March 14, 2019
 * How to measure overall similarity by Ferdinand Mosca, CCC, April 02, 2019
 * Why all Lc0 runs result in such similarity of quiet moves selection? by Kai Laskos, CCC, May 01, 2019
 * Similarity tester - 2nd generation - BETA by Ed Schröder, CCC, August 09, 2019
 * What the heck happens here? by Kai Laskos, CCC, August 20, 2019
 * SIMEX 2.1 by Ed Schröder, CCC, September 01, 2019
 * Similarity Report 2019 by Ed Schröder, CCC (Engine Origins), September 23, 2019

=External Links=

Chess Engines

 * Komodo Chess Engine - Official Site with Similary tester version 03 download
 * Similary tester 2nd generation (SIMEX) by Ed Schröder
 * Similarity Report 2019 by Ed Schröder

Misc

 * Similarity from Wikipedia
 * Similarity (geometry) from Wikipedia
 * Lexical similarity from Wikipedia
 * Semantic similarity from Wikipedia
 * Structural similarity from Wikipedia
 * Similarity measure from Wikipedia
 * Dendrogram from Wikipedia
 * Hierarchical clustering from Wikipedia

=References= Up one Level