25,161
edits
Changes
Created page with "'''Home * Engine Testing * Search Statistics''' '''Search statistics''' refers to counting various appearances of interest inside the search or [..."
'''[[Main Page|Home]] * [[Engine Testing]] * Search Statistics'''
'''Search statistics''' refers to counting various appearances of interest inside the [[Search|search]] or [[Evaluation|evaluation]] routines and to analyze their relationships to eventually spot [[Engine Testing#bugs|bugs]] or unfavorable conditions in [[Move Ordering|move ordering]].
=Node Statistics=
Of interest is the appearance of searched [[Node|nodes]] inside an [[Iterative Deepening|iterative deepening framework]] and the ratio of [[Quiescent Node|quiescent nodes]], per [[Depth|depth]] and/or aggregated, in conjunction with [[Branching Factor#EffectiveBranchingFactor|effective branching factor]] (EBF) and of course [[Nodes per second|nodes per second]].
==Cutoffs==
Inside [[Alpha-Beta|Alpha-beta]]- or [[Principal Variation Search|PV-search]] it is of particular interest to count how often a node [[Fail-High|failes high]] or not, in relation to its [[Node Types|node types]], that is expected [[Node Types#CUT|cut-Nodes]] or expected [[Node Types#ALL|All-nodes]]. If a fail-high occurs, it is illuminative to know whether the fail-high move it was tried first, early or late and what [[Move Ordering|move ordering]] classification like move from the [[Transposition Table|transposition table]], [[Captures|winning captures]], [[Killer Move|killer moves]], etc. was applied.
==Re-Searches==
At [[Node Types#PV|PV-nodes]] inside a [[Alpha-Beta|alpha-beta]]- or [[Principal Variation Search|PV-search]] the number of re-searches does concern, and how often a re-search does improve alpha (or even performs a cutoff) or not.
==Root Statistics==
The [[Root]] as distinguished PV-node specially with [[Aspiration Windows|aspiration window]] often has its own statistics related to how much relative time (nodes) of the whole [[Iteration|iteration]] is performed per each root-move, and how often another best move was found. In conjunction with static move properties, score graph by searched depth so far, etc., these statistic based informations may be considered in [[Time Management|time management]] to possibly decide about a new iteration.
==Selectivity==
Counting the various [[Extensions|extensions]], [[Reductions|reductions]] and [[Pruning|forward pruning]] decisions in relation with [[Branching Factor#EffectiveBranchingFactor|effective branching factor]] might also spot some deficiency inside the search.
<span id="TTStatistics"></span>
=TT Statistics=
Statistics of the [[Transposition Table|transposition table]] covers number of probes and stores, fill level, and number of probe hits, likely differentiated by sufficient [[Depth|draft]] and [[Node Types|type of stored node]].
=See also=
* [[Profiling]]
* [[Match Statistics]]
=Publications=
* [[Hermann Kaindl]] ('''1988'''). ''Useful Statistics from Tournament Programs''. [[ICGA Journal#11_4|ICCA Journal, Vol. 11, No. 4]] » [[Merlin]]
=Forum Posts=
==1994 ...==
* [http://groups.google.com/group/rec.games.chess/browse_frm/thread/32af80f05abdc805 COMPUTER CHESS: statistics] by [[Deniz Yuret]], [[Computer Chess Forums|rgcc]], October 27, 1994
* [https://www.stmintz.com/ccc/index.php?id=13052 Cutoff Statistics] by [[Roland Pfister]], [[CCC]], December, 18, 1997
==2000 ...==
* [https://www.stmintz.com/ccc/index.php?id=92156 A matter of statistics] by [[Tijs van Dam]], [[CCC]], January 26, 2000
* [https://www.stmintz.com/ccc/index.php?id=115020 Re: The Scalable Search Test / Results of "Der Bringer"] by [[Gerrit Reubold]], [[CCC]], June 18, 2000
* [https://www.stmintz.com/ccc/index.php?id=129504 Faster, deeper and more of such...] by [[Ed Schroder|Ed Schröder]], [[CCC]], September 14, 2000 » [[Depth#DiminishingReturns|Diminishing Returns]]
* [https://www.stmintz.com/ccc/index.php?id=143022 Hash table efficiency] by [[Miguel A. Ballicora]], [[CCC]], December 05, 2000 » [[Gaviota]], [[Transposition Table]]
* [https://www.stmintz.com/ccc/index.php?id=359148 Hash table statistics] by [[Andrew Wagner]], [[CCC]], April 08, 2004
* [http://www.talkchess.com/forum/viewtopic.php?t=20562 What is the Ideal Output for Understanding a Chess Engine?] by Rick Fadden, [[CCC]], April 07, 2008
* [http://www.talkchess.com/forum/viewtopic.php?t=27737 Search statistics] by henkf, [[CCC]], May 04, 2009
==2010 ...==
* [http://www.talkchess.com/forum/viewtopic.php?t=45264 Probabilistic approach to optimize search tree] by [[Sergei Markoff|Sergei S. Markoff]], [[CCC]], September 22, 2012
* [http://www.talkchess.com/forum/viewtopic.php?t=49871 Search statistics] by [[Robert Pope]], [[CCC]], October 30, 2013
* [http://www.talkchess.com/forum/viewtopic.php?t=51075 pruning statistics] by [[Jon Dart]], [[CCC]], January 27, 2014 » [[Pruning]]
* [http://www.talkchess.com/forum/viewtopic.php?t=61401 Best move statistics] by [[Matthew Lai]], [[CCC]], September 12, 2016
* [http://www.talkchess.com/forum/viewtopic.php?t=63280 Move ordering statistics] by [[Sander Maassen vd Brink]], [[CCC]], February 26, 2017 » [[Move Ordering]]
* [http://www.talkchess.com/forum/viewtopic.php?t=63555 Testing for Move Ordering Improvements] by [[Cheney Nattress]], [[CCC]], March 25, 2017 » [[Move Ordering]]
* [http://www.talkchess.com/forum/viewtopic.php?t=66335 Komodo 11.2.2 - Initial position until depth 54] by [[Andreas Strangmüller]], [[CCC]], January 15, 2018 » [[Komodo]], [[Initial Position]]
* [http://www.talkchess.com/forum/viewtopic.php?t=66340 Stockfish 8 - Initial position until depth 59] by [[Andreas Strangmüller]], [[CCC]], January 16, 2018 » [[Stockfish]]
'''[[Engine Testing|Up one level]]'''
'''Search statistics''' refers to counting various appearances of interest inside the [[Search|search]] or [[Evaluation|evaluation]] routines and to analyze their relationships to eventually spot [[Engine Testing#bugs|bugs]] or unfavorable conditions in [[Move Ordering|move ordering]].
=Node Statistics=
Of interest is the appearance of searched [[Node|nodes]] inside an [[Iterative Deepening|iterative deepening framework]] and the ratio of [[Quiescent Node|quiescent nodes]], per [[Depth|depth]] and/or aggregated, in conjunction with [[Branching Factor#EffectiveBranchingFactor|effective branching factor]] (EBF) and of course [[Nodes per second|nodes per second]].
==Cutoffs==
Inside [[Alpha-Beta|Alpha-beta]]- or [[Principal Variation Search|PV-search]] it is of particular interest to count how often a node [[Fail-High|failes high]] or not, in relation to its [[Node Types|node types]], that is expected [[Node Types#CUT|cut-Nodes]] or expected [[Node Types#ALL|All-nodes]]. If a fail-high occurs, it is illuminative to know whether the fail-high move it was tried first, early or late and what [[Move Ordering|move ordering]] classification like move from the [[Transposition Table|transposition table]], [[Captures|winning captures]], [[Killer Move|killer moves]], etc. was applied.
==Re-Searches==
At [[Node Types#PV|PV-nodes]] inside a [[Alpha-Beta|alpha-beta]]- or [[Principal Variation Search|PV-search]] the number of re-searches does concern, and how often a re-search does improve alpha (or even performs a cutoff) or not.
==Root Statistics==
The [[Root]] as distinguished PV-node specially with [[Aspiration Windows|aspiration window]] often has its own statistics related to how much relative time (nodes) of the whole [[Iteration|iteration]] is performed per each root-move, and how often another best move was found. In conjunction with static move properties, score graph by searched depth so far, etc., these statistic based informations may be considered in [[Time Management|time management]] to possibly decide about a new iteration.
==Selectivity==
Counting the various [[Extensions|extensions]], [[Reductions|reductions]] and [[Pruning|forward pruning]] decisions in relation with [[Branching Factor#EffectiveBranchingFactor|effective branching factor]] might also spot some deficiency inside the search.
<span id="TTStatistics"></span>
=TT Statistics=
Statistics of the [[Transposition Table|transposition table]] covers number of probes and stores, fill level, and number of probe hits, likely differentiated by sufficient [[Depth|draft]] and [[Node Types|type of stored node]].
=See also=
* [[Profiling]]
* [[Match Statistics]]
=Publications=
* [[Hermann Kaindl]] ('''1988'''). ''Useful Statistics from Tournament Programs''. [[ICGA Journal#11_4|ICCA Journal, Vol. 11, No. 4]] » [[Merlin]]
=Forum Posts=
==1994 ...==
* [http://groups.google.com/group/rec.games.chess/browse_frm/thread/32af80f05abdc805 COMPUTER CHESS: statistics] by [[Deniz Yuret]], [[Computer Chess Forums|rgcc]], October 27, 1994
* [https://www.stmintz.com/ccc/index.php?id=13052 Cutoff Statistics] by [[Roland Pfister]], [[CCC]], December, 18, 1997
==2000 ...==
* [https://www.stmintz.com/ccc/index.php?id=92156 A matter of statistics] by [[Tijs van Dam]], [[CCC]], January 26, 2000
* [https://www.stmintz.com/ccc/index.php?id=115020 Re: The Scalable Search Test / Results of "Der Bringer"] by [[Gerrit Reubold]], [[CCC]], June 18, 2000
* [https://www.stmintz.com/ccc/index.php?id=129504 Faster, deeper and more of such...] by [[Ed Schroder|Ed Schröder]], [[CCC]], September 14, 2000 » [[Depth#DiminishingReturns|Diminishing Returns]]
* [https://www.stmintz.com/ccc/index.php?id=143022 Hash table efficiency] by [[Miguel A. Ballicora]], [[CCC]], December 05, 2000 » [[Gaviota]], [[Transposition Table]]
* [https://www.stmintz.com/ccc/index.php?id=359148 Hash table statistics] by [[Andrew Wagner]], [[CCC]], April 08, 2004
* [http://www.talkchess.com/forum/viewtopic.php?t=20562 What is the Ideal Output for Understanding a Chess Engine?] by Rick Fadden, [[CCC]], April 07, 2008
* [http://www.talkchess.com/forum/viewtopic.php?t=27737 Search statistics] by henkf, [[CCC]], May 04, 2009
==2010 ...==
* [http://www.talkchess.com/forum/viewtopic.php?t=45264 Probabilistic approach to optimize search tree] by [[Sergei Markoff|Sergei S. Markoff]], [[CCC]], September 22, 2012
* [http://www.talkchess.com/forum/viewtopic.php?t=49871 Search statistics] by [[Robert Pope]], [[CCC]], October 30, 2013
* [http://www.talkchess.com/forum/viewtopic.php?t=51075 pruning statistics] by [[Jon Dart]], [[CCC]], January 27, 2014 » [[Pruning]]
* [http://www.talkchess.com/forum/viewtopic.php?t=61401 Best move statistics] by [[Matthew Lai]], [[CCC]], September 12, 2016
* [http://www.talkchess.com/forum/viewtopic.php?t=63280 Move ordering statistics] by [[Sander Maassen vd Brink]], [[CCC]], February 26, 2017 » [[Move Ordering]]
* [http://www.talkchess.com/forum/viewtopic.php?t=63555 Testing for Move Ordering Improvements] by [[Cheney Nattress]], [[CCC]], March 25, 2017 » [[Move Ordering]]
* [http://www.talkchess.com/forum/viewtopic.php?t=66335 Komodo 11.2.2 - Initial position until depth 54] by [[Andreas Strangmüller]], [[CCC]], January 15, 2018 » [[Komodo]], [[Initial Position]]
* [http://www.talkchess.com/forum/viewtopic.php?t=66340 Stockfish 8 - Initial position until depth 59] by [[Andreas Strangmüller]], [[CCC]], January 16, 2018 » [[Stockfish]]
'''[[Engine Testing|Up one level]]'''
