Engine Similarity

From Chessprogramming wiki
Jump to: navigation, search

Home * Engines * Similarity

Engine Similarity,
a loosely defined relation between (two) chess engines concerning their positional playing style, therefor mostly dependent on their evaluation features and weights. One approach to assess engine similarity is to count how often these engines agree with the same move after a shallow search over a set of carefully selected, quiet test positions with apparently several possible best moves - another, in k-best mode, how often they propose the same ranking of k moves. A relative high similarity measure between two engines [2] could be a symptom of using similar evaluation ideas, features and weights, along with automated tuning (or trainig), or even code copying aka cloning. Similarity testing is intended as first automated "screening" to trigger further investigations in case moves are too similar.

Similarity Testers

In December 2010, Don Dailey published SIM03 to test the similary with a set of UCI compliant engines with 8238 in-build positions - still freely available from the Komodo site [3]. A pair of different chess engines could then be compared for similarity by matching up and counting the number of positions for which each engine chose the same move. SIMEX is a more user friendly successor by Ed Schröder [4] using external EPD-files. Overall similarity of multiple engines may be visualized as hierarchical clustering dendrogram [5].

See also

Publications

Forum Posts

1999

Re: This test is not scientific! by Don Dailey, CCC, January 26, 1999
Comparing Bionic-Impakt vs Bionic by Albrecht Heeffer, CCC, January 27, 1999 » Bionic Impakt, Bionic

2000 ...

2010 ...

2015 ...

External Links

Chess Engines

Misc

References

  1. A phylogenetic tree of living things, based on RNA data and proposed by Carl Woese, showing the separation of bacteria, archaea, and eukaryotes. Trees constructed with other genes are generally similar, although they may place some early-branching groups very differently, thanks to long branch attraction. The exact relationships of the three domains are still being debated, as is the position of the root of the tree. It has also been suggested that due to lateral gene transfer, a tree may not be the best representation of the genetic relationships of all organisms. For instance some genetic evidence suggests that eukaryotes evolved from the union of some bacteria and archaea (one becoming an organelle and the other the main cell). Autor: Eric Gaba, September 2006, Wikimedia Commons
  2. i.e. > 65% in SIM03
  3. Komodo Chess Engine - Official Site with Similary tester version 03 download
  4. SIMEX 2.1 provided by Ed Schröder, CCC, September 01, 2019
  5. How to measure overall similarity by Ferdinand Mosca, CCC, April 02, 2019
  6. Pairwise Analysis of Chess Engine Move Selections by Adam Hair, CCC, April 17, 2011

Up one Level