Difference between revisions of "Depth"

From Chessprogramming wiki
Jump to: navigation, search
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
'''[[Main Page|Home]] * [[Search]] * Depth'''
 
'''[[Main Page|Home]] * [[Search]] * Depth'''
  
[[FILE:EschersDepth.jpg|border|right|thumb|231px|link=http://www.mcescher.com/Gallery/recogn-bmp/LW403.jpg|[[Arts#Escher|M. C. Escher]], Depth, 1955 <ref>[http://www.mcescher.com/Gallery/gallery-recogn.htm Picture gallery "Recognition and Success 1955 - 1972"] from [http://www.mcescher.com/ The Official M.C. Escher Website]</ref>  
+
[[FILE:EschersDepth.jpg|border|right|thumb|231px|link=http://www.mcescher.com/Gallery/recogn-bmp/LW403.jpg|[[:Category:M. C. Escher|M. C. Escher]], Depth, 1955 <ref>[http://www.mcescher.com/Gallery/gallery-recogn.htm Picture gallery "Recognition and Success 1955 - 1972"] from [http://www.mcescher.com/ The Official M.C. Escher Website]</ref>  
 
]]  
 
]]  
  
 
'''Depth''' is the height or ''nominal'' depth in [[Ply|plies]] between the [[Root|root]] and so called [[Horizon Node|horizon nodes]] (depth 0), where a heuristic value is assigned to. Thus, depth is the number of half moves the search ''nominally'' looks ahead.
 
'''Depth''' is the height or ''nominal'' depth in [[Ply|plies]] between the [[Root|root]] and so called [[Horizon Node|horizon nodes]] (depth 0), where a heuristic value is assigned to. Thus, depth is the number of half moves the search ''nominally'' looks ahead.
  
Despite [[Quiescence Search|quiescence search]], where usually winning captures and even some checks are tried at or behind the search horizon, until positions become sufficiently quite, [[Selectivity|selectivity]] of modern chess programs, caused by [[Extensions|extensions]], [[Pruning|pruning]] and [[Reductions|reductions]], notably [[Check Extensions|check extensions]], [[Null Move Pruning|nNMP]] and [[Late Move Reductions|LMR]], leads to bushy, non-uniform [[Search Tree|trees]] where some branches are searched deeper than nominal, but others shallower. A [[Depth Reduction R|depth reduction R]] of multiple plies is often performed in forward pruning techniques like [[Null Move Pruning|null move pruning]] and [[Multi-Cut|multi-cut]].  
+
Despite [[Quiescence Search|quiescence search]], where usually winning captures and even some checks are tried at or behind the search horizon, until positions become sufficiently quite, [[Selectivity|selectivity]] of modern chess programs, caused by [[Extensions|extensions]], [[Pruning|pruning]] and [[Reductions|reductions]], notably [[Check Extensions|check extensions]], [[Null Move Pruning|NMP]] and [[Late Move Reductions|LMR]], leads to bushy, non-uniform [[Search Tree|trees]] where some branches are searched deeper than nominal, but others shallower. A [[Depth Reduction R|depth reduction R]] of multiple plies is often performed in forward pruning techniques like [[Null Move Pruning|null move pruning]] and [[Multi-Cut|multi-cut]].  
  
 
=Draft versus Ply-Index=  
 
=Draft versus Ply-Index=  
Line 200: Line 200:
  
 
=See also=  
 
=See also=  
 +
* [[Alpha-Beta Conspiracy Search]]
 
* [[Extensions#FractionalExtensions|Fractional Extensions]]
 
* [[Extensions#FractionalExtensions|Fractional Extensions]]
 
* [[Iterative Deepening]]
 
* [[Iterative Deepening]]
Line 210: Line 211:
 
* [[Selectivity]]
 
* [[Selectivity]]
 
* [[SEX Algorithm]]
 
* [[SEX Algorithm]]
 +
* [[Alexander Szabo#TechnologyCurve|The Technology Curve]]
  
 
=Publications=  
 
=Publications=  
Line 223: Line 225:
 
* [[David Levy]], [[David Broughton]], [[Mark Taylor]] ('''1989'''). ''The SEX Algorithm in Computer Chess''. [[ICGA Journal#12_1|ICCA Journal, Vol. 12, No. 1]]
 
* [[David Levy]], [[David Broughton]], [[Mark Taylor]] ('''1989'''). ''The SEX Algorithm in Computer Chess''. [[ICGA Journal#12_1|ICCA Journal, Vol. 12, No. 1]]
 
==1990 ...==  
 
==1990 ...==  
 +
* [[David McAllester]], [[Deniz Yuret]] ('''1993'''). ''Alpha-Beta Conspiracy Search''. [http://ttic.uchicago.edu/~dmcallester/abc.ps ps (draft)] » [[Alpha-Beta Conspiracy Search]]
 
* [[Robert Hyatt]], [[Monroe Newborn]] ('''1997'''). ''CRAFTY Goes Deep''. [[ICGA Journal#20_2|ICCA Journal, Vol. 20, No. 2]]
 
* [[Robert Hyatt]], [[Monroe Newborn]] ('''1997'''). ''CRAFTY Goes Deep''. [[ICGA Journal#20_2|ICCA Journal, Vol. 20, No. 2]]
 
* [[Andreas Junghanns]], [[Jonathan Schaeffer]], [[Mark Brockington]], [[Yngvi Björnsson]], [[Tony Marsland]] ('''1997'''). ''Diminishing Returns for Additional Search in Chess''. [[Advances in Computer Chess 8]], [http://www.ru.is/faculty/yngvi/pdf/JunghannsSBBM97.pdf pdf]
 
* [[Andreas Junghanns]], [[Jonathan Schaeffer]], [[Mark Brockington]], [[Yngvi Björnsson]], [[Tony Marsland]] ('''1997'''). ''Diminishing Returns for Additional Search in Chess''. [[Advances in Computer Chess 8]], [http://www.ru.is/faculty/yngvi/pdf/JunghannsSBBM97.pdf pdf]
Line 235: Line 238:
 
* [[Ernst A. Heinz]] ('''2003'''). ''Follow-Up on Self-Play, Deep Search, and Diminishing Returns.'' [[ICGA Journal#26_2|ICGA Journal, Vol. 26, No. 2]]
 
* [[Ernst A. Heinz]] ('''2003'''). ''Follow-Up on Self-Play, Deep Search, and Diminishing Returns.'' [[ICGA Journal#26_2|ICGA Journal, Vol. 26, No. 2]]
 
* [[Jonathan Schaeffer]] ('''2004'''). ''8. Search Depth''. in AI- and Search, Online Course, [http://webdocs.cs.ualberta.ca/%7Ejonathan/Courses/657/Notes/8.SearchDepth.pdf slides as pdf]
 
* [[Jonathan Schaeffer]] ('''2004'''). ''8. Search Depth''. in AI- and Search, Online Course, [http://webdocs.cs.ualberta.ca/%7Ejonathan/Courses/657/Notes/8.SearchDepth.pdf slides as pdf]
* [[Jan Renze Steenhuisen]] ('''2005'''). ''New Results in Deep-Search Behaviour''. [[ICGA Journal#28_4|ICGA Journal, Vol. 28, No. 4]], [http://www.st.ewi.tudelft.nl/%7Erenze/doc/ICGA_2005_4_DeepSearch.pdf pdf]
+
* [[Jan Renze Steenhuisen]] ('''2005'''). ''New Results in Deep-Search Behaviour''. [[ICGA Journal#28_4|ICGA Journal, Vol. 28, No. 4]], [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.104.9527 CiteSeerX]
 
* [[Matej Guid]], [[Ivan Bratko]] ('''2007'''). ''Factors affecting diminishing returns for searching deeper''. [[CGW 2007]] » [[Crafty]], [[Rybka]], [[Shredder]], [[Depth#DiminishingReturns|Diminishing Returns]]
 
* [[Matej Guid]], [[Ivan Bratko]] ('''2007'''). ''Factors affecting diminishing returns for searching deeper''. [[CGW 2007]] » [[Crafty]], [[Rybka]], [[Shredder]], [[Depth#DiminishingReturns|Diminishing Returns]]
 
* [[Matej Guid]], [[Ivan Bratko]] ('''2007'''). ''Factors affecting diminishing returns for searching deeper''. [[ICGA Journal#30_2|ICGA Journal, Vol. 30, No. 2]], [http://www.ailab.si/matej/doc/Factors_Affecting_Diminishing_Returns.pdf pdf]  
 
* [[Matej Guid]], [[Ivan Bratko]] ('''2007'''). ''Factors affecting diminishing returns for searching deeper''. [[ICGA Journal#30_2|ICGA Journal, Vol. 30, No. 2]], [http://www.ailab.si/matej/doc/Factors_Affecting_Diminishing_Returns.pdf pdf]  
 
==2010 ...==  
 
==2010 ...==  
* [[Diogo R. Ferreira]] ('''2013'''). ''The Impact of the Search Depth on Chess Playing Strength''. [[ICGA Journal#36_2|ICGA Journal, Vol. 36, No. 2]]
+
* [[Diogo R. Ferreira]] ('''2013'''). ''The Impact of the Search Depth on Chess Playing Strength''. [[ICGA Journal#36_2|ICGA Journal, Vol. 36, No. 2]] » [[#DiminishingReturns|Diminishing Returns]], [[Match Statistics]], [[Playing Strength]], [[Houdini]] <ref>[https://www.hiarcs.net/forums/viewtopic.php?t=10004 Ply versus ELO] by Greg, [[Computer Chess Forums|HIARCS Forum]], May 30, 2020 » [[Diogo R. Ferreira#Impact|Diogo R. Ferreira - Impact of the Search Depth ...]]</ref>
 +
 
 
* [[Tamal T. Biswas]], [[Kenneth W. Regan]] ('''2015'''). ''Quantifying Depth and Complexity of Thinking and Knowledge''. [http://www.icaart.org/EuropeanProjectSpace.aspx?y=2015 ICAART 2015], [http://www.cse.buffalo.edu/~regan/papers/pdf/BiReICAART15CR.pdf pdf]
 
* [[Tamal T. Biswas]], [[Kenneth W. Regan]] ('''2015'''). ''Quantifying Depth and Complexity of Thinking and Knowledge''. [http://www.icaart.org/EuropeanProjectSpace.aspx?y=2015 ICAART 2015], [http://www.cse.buffalo.edu/~regan/papers/pdf/BiReICAART15CR.pdf pdf]
 
* [[Tamal T. Biswas]], [[Kenneth W. Regan]] ('''2015'''). ''Measuring Level-K Reasoning, Satisficing, and Human Error in Game-Play Data''. [[IEEE]] [http://www.icmla-conference.org/icmla15/ ICMLA 2015], [http://www.cse.buffalo.edu/~regan/papers/pdf/BiRe15_ICMLA2015.pdf pdf preprint]
 
* [[Tamal T. Biswas]], [[Kenneth W. Regan]] ('''2015'''). ''Measuring Level-K Reasoning, Satisficing, and Human Error in Game-Play Data''. [[IEEE]] [http://www.icmla-conference.org/icmla15/ ICMLA 2015], [http://www.cse.buffalo.edu/~regan/papers/pdf/BiRe15_ICMLA2015.pdf pdf preprint]
 
* [[Matej Guid]], [[Ivan Bratko]] ('''2017'''). ''Influence of Search Depth on Position Evaluation''. [[Advances in Computer Games 15]]
 
* [[Matej Guid]], [[Ivan Bratko]] ('''2017'''). ''Influence of Search Depth on Position Evaluation''. [[Advances in Computer Games 15]]
  
=Forum Posts=  
+
=Postings=  
 +
==1983 ...==
 +
* [http://quux.org:70/Archives/usenet-a-news/NET.chess/82.01.07_alice.349_net.chess.txt chess strength] by [[Ken Thompson]], [http://quux.org:70/Archives/usenet-a-news/NET.chess net.chess], January 7, 1982
 
==1996 ...==  
 
==1996 ...==  
 
* [https://groups.google.com/d/msg/rec.games.chess.computer/1uVIWZFB41k/VUcAUkzyFd0J Fractional depth increments] by S. Read, [[Computer Chess Forums|rgcc]], January 18, 1996
 
* [https://groups.google.com/d/msg/rec.games.chess.computer/1uVIWZFB41k/VUcAUkzyFd0J Fractional depth increments] by S. Read, [[Computer Chess Forums|rgcc]], January 18, 1996
Line 252: Line 258:
 
* [https://groups.google.com/d/msg/rec.games.chess.computer/aqg3nQsm9qM/8UIE5idQoJUJ Suggested chess experiment] by Henri H. Arsenault, [[Computer Chess Forums|rgcc]], February 17, 1999
 
* [https://groups.google.com/d/msg/rec.games.chess.computer/aqg3nQsm9qM/8UIE5idQoJUJ Suggested chess experiment] by Henri H. Arsenault, [[Computer Chess Forums|rgcc]], February 17, 1999
 
==2000 ...==  
 
==2000 ...==  
* [https://www.stmintz.com/ccc/index.php?id=92700 diminishing returns w/ increased search depth?] by Peter Kappler, [[CCC]], January 27, 2000
+
* [https://www.stmintz.com/ccc/index.php?id=92700 diminishing returns w/ increased search depth?] by [[Peter Kappler]], [[CCC]], January 27, 2000
 
* [https://www.stmintz.com/ccc/index.php?id=112359 A New Self-Play Experiment - Diminishing Returns Shown with 95% Conf.] by [[Ernst A. Heinz]], [[CCC]], May 24, 2000 » [[Depth#DiminishingReturns|Diminishing Returns]]
 
* [https://www.stmintz.com/ccc/index.php?id=112359 A New Self-Play Experiment - Diminishing Returns Shown with 95% Conf.] by [[Ernst A. Heinz]], [[CCC]], May 24, 2000 » [[Depth#DiminishingReturns|Diminishing Returns]]
 
* [https://www.stmintz.com/ccc/index.php?id=129504 Faster, deeper and more of such...] by [[Ed Schroder|Ed Schröder]], [[CCC]], September 14, 2000 » [[Search Statistics]]
 
* [https://www.stmintz.com/ccc/index.php?id=129504 Faster, deeper and more of such...] by [[Ed Schroder|Ed Schröder]], [[CCC]], September 14, 2000 » [[Search Statistics]]
Line 268: Line 274:
 
==2010 ...==  
 
==2010 ...==  
 
* [http://www.open-chess.org/viewtopic.php?f=5&t=1403 Node counts at a given depth/iteration in search] by [[Mark Watkins|BB+]], [[Computer Chess Forums|OpenChess Forum]], May 23, 2011
 
* [http://www.open-chess.org/viewtopic.php?f=5&t=1403 Node counts at a given depth/iteration in search] by [[Mark Watkins|BB+]], [[Computer Chess Forums|OpenChess Forum]], May 23, 2011
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=42677 Counting depth as a function of number of legal moves] by [[Pio Korinth]], [[CCC]], February 28, 2012
 
* [http://www.talkchess.com/forum/viewtopic.php?t=43134 Elo versus speed] by [[Peter Österlund]], [[CCC]], April 02, 2012
 
* [http://www.talkchess.com/forum/viewtopic.php?t=43134 Elo versus speed] by [[Peter Österlund]], [[CCC]], April 02, 2012
 
* [http://www.talkchess.com/forum/viewtopic.php?t=43596 From 5 ply to 6....] by [[Fernando Villegas]], [[CCC]], May 06, 2012
 
* [http://www.talkchess.com/forum/viewtopic.php?t=43596 From 5 ply to 6....] by [[Fernando Villegas]], [[CCC]], May 06, 2012
Line 285: Line 292:
 
* [http://www.talkchess.com/forum/viewtopic.php?t=63326 Ridiculous QSearch Depth] by [[Jonathan Rosenthal]], [[CCC]], March 03, 2017 » [[Quiescence Search]]
 
* [http://www.talkchess.com/forum/viewtopic.php?t=63326 Ridiculous QSearch Depth] by [[Jonathan Rosenthal]], [[CCC]], March 03, 2017 » [[Quiescence Search]]
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70217 Depth reduced but ELO increased] by [[Tom King]], [[CCC]], March 16, 2019 » [[Countermove Heuristic]], [[Playing Strength]]
 
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70217 Depth reduced but ELO increased] by [[Tom King]], [[CCC]], March 16, 2019 » [[Countermove Heuristic]], [[Playing Strength]]
 +
==2020 ...==
 +
* [https://www.hiarcs.net/forums/viewtopic.php?t=10004 Ply versus ELO] by Greg, [[Computer Chess Forums|HIARCS Forum]], May 30, 2020 » [[Diogo R. Ferreira#Impact|Diogo R. Ferreira - Impact of the Search Depth ...]]
 +
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=77202 On reaching maximum ply] by [[Martin Bryant]], [[CCC]], April 29, 2021 » [[#MaxPly|Maximum Search Depth]], [[Ply]], [[Search]]
  
 
=External Links=  
 
=External Links=  
Line 297: Line 307:
 
=References=  
 
=References=  
 
<references />
 
<references />
 
 
'''[[Search|Up one Level]]'''
 
'''[[Search|Up one Level]]'''
 
[[Category:M. C. Escher]]
 
[[Category:M. C. Escher]]
 
[[Category:Roy Hargrove]]
 
[[Category:Roy Hargrove]]
 +
[[Category:Ban Quotes]]

Revision as of 08:06, 30 April 2021

Home * Search * Depth

M. C. Escher, Depth, 1955 [1]

Depth is the height or nominal depth in plies between the root and so called horizon nodes (depth 0), where a heuristic value is assigned to. Thus, depth is the number of half moves the search nominally looks ahead.

Despite quiescence search, where usually winning captures and even some checks are tried at or behind the search horizon, until positions become sufficiently quite, selectivity of modern chess programs, caused by extensions, pruning and reductions, notably check extensions, NMP and LMR, leads to bushy, non-uniform trees where some branches are searched deeper than nominal, but others shallower. A depth reduction R of multiple plies is often performed in forward pruning techniques like null move pruning and multi-cut.

Draft versus Ply-Index

Most likely inside the search routine, a ply-index is used to index stacks or arrays with pre-saved search information. This index is initialized with zero at the root, and is then incremented after making a move each time the recursive search is called. This index measures the ply-distance from the current node to the root and would therefor be sufficient to determine the remaining depth to the horizon, also called draft:

draft ::= depth at the root - ply index

However, there are various reasons to decouple the depth to horizon from the ply-index or depth from root, which are often passed as independent parameters to a recursive search routine (see code below). While the ply-index is incremented by one each time, the draft may be independently altered by various extension- or reduction-schemes and may also consider fractional extensions [2] [3] .

Fractional Plies

Some programs extend or reduce in fractions of one ply. Inside an iterative deepening framework, the search depth is incremented, usually by one ply - or by a fraction of one ply, for instance 1/2 ply.

Amir Ban on Junior in rgcc, March 1998 [4] :

The brute-force ply depth is indeed half the publicized depth. All the rest are extensions (in conventional terminology, I don't think of them this way). If you set Junior to depth 12, e.g., then you should be able to find a 7-ply combination where it fails. If I am doing a good job, then you should have a hard time finding one.
The question of what this is equivalent to in terms of other programs, e.g. a null-mover with "standard" extensions is interesting, but I don't know the answer. In tournament conditions middlegame Junior typically gets 14-16 depths, and it looks competitive tactically. 

Depth Comparison of different programs

Due to different implementations, the reported search depth of chess programs is not comparable in general. Programs like The King (Chessmaster), Junior and Rybka are known for interpreting depth differently for whatever reasons.

Selective Search Depth

Some programs also report a selective search depth beside the nominal search depth, most often much greater than the nominal search depth. Some programs determine the highest distance to the root at any node, others only at the horizon.

int highestDepth;

int iterativeDeepening() {
   ...
   highestDepth = 0;
   for (depth = 0; depth <= maxdepth; depth += DEPTH_OF_ONE_PLY) {
      score = abSearch( -oo, +oo, depth, 0 );
      if (timeIsOver (...) )
         break;
   }
   ...
}

int abSearch( int alpha, int beta, int depth, int ply ) {
   depth += determineExtensions(...);
   depth -= determineReductions(...);
   if( depth <= 0 ) return quiesce( alpha, beta );
   if ( ply > highestDepth )
      highestDepth = ply;

   for ( all moves)  {
      score = -abSearch( -beta, -alpha, depth - DEPTH_OF_ONE_PLY, ply + 1 );
      if( score >= beta )
         return beta;   // beta cutoff
      if( score > alpha )
         alpha = score; // alpha acts like max in MiniMax
   }
   return alpha;
}

Maximum Search Depth

The Maximum Search Depth of a depth-first search is usually determined by a compile time constant in ply units (MAX_PLAY). It is used to statically allocate arrays like a Triangular PV-Table, or search stacks inside the programs data- or bss segment. While 64 was quite common, todays programs tend to use higher values, e.g. 128. A search routine should nevertheless check the upper bound of the search stack to immediate return a lazy evaluation score or material balance when the ply index threatens overflow.

Diminishing Returns

Despite the existence of pathology in searching some trees, where a deeper minimax search results in worse play, it is quite consensus in Chess that deeper search yields in stronger play. Strength improvement from depth d to depth d+1 was first systematically examined by Ken Thompson with Belle in Computer Chess Strength, as introduced at the Advances in Computer Chess 3 conference in 1981 [5] . Thompson found Belle (n+1) scored about 80% versus Belle (n), which roughly translates to a 200 Elo improvement playing one ply deeper, while the improvement seemed constant independent from the used depths from 3 to 8, while a second experiment [6] indicated a falloff beyond depth 7.

P4 P5 P6 P7 P8 P9 Ratings Improvement
P4 5 ½ 0 0 0 1235 -
P5 15 3 ½ 0 1570 235
P6 19½ 16½ 4 1826 256
P7 20 17 16 5 4 2031 205
P8 20 19½ 18½ 15 2208 167
P9 20 20 18½ 16 14½ 2328 120

Also, in other board games such as Othello and Checkers, additional plies of search translated into decreasing benefits, giving rise to Diminishing returns for deeper searching. In their 1997 paper Diminishing Returns for Additional Search in Chess [7] , Junghanns, Schaeffer, Brockington, Björnsson and Marsland conclude the existence of Diminishing returns in Chess as well, somehow hidden by the high percentage of errors made by chess programs for lower search depth.

In self-play experiments with Crafty, Robert Hyatt, Monroe Newborn [8] and later Ernst A. Heinz with DarkThought [9] steadily discovered new best moves while searching deeper. In further experiments [10] , Heinz found indications of decreasing returns from increasing search in chess. In his 2001 ICGA Journal paper Self-Play, Deep Search and Diminishing Returns [11] he gave following match results (3,000 games each) [12] :

  • 12-ply was 84 Elo points better than 11 ply
  • 11-ply was 92 Elo points better than 10 ply
  • 10-ply was 115 Elo points better than 9 ply

Tony van Roon-Werten made following statement on Diminishing Returns [13] :

If two programs play with 5 vs 6 ply search, the second engine has a 20% depth advantage. With 10 vs 11 it's only 10%. So of course the difference in wins is smaller. ...
Diminishing returns are only proven (IMO) if 6 vs 5 wins more games than 12 vs 10 because only then are you comparing something linear and you give a linear advantage. 

Ed Schröder conducted self-play experiments with ProDeo 1.74 playing different depths. Schröder also suggests that ProDeo has a branching-factor of roughly 2, in other words an additional ply corresponds to a doubling of time. In the following table the values indicate the Elo advantage of ProDeo playing with depth A against itself with depth B. The exact tournament conditions can be studied on his webpage [14] .

depth A vs B 7 8 9 10 11
6 180 321 401
7 0 147 281 389
8 0 151 255 386
9 0 129 255
10 0 127

See also

Publications

1978 ...

1980 ...

1990 ...

2000 ...

2010 ...

Postings

1983 ...

1996 ...

2000 ...

Re: Shredder 8 secret: search depth? by Vasik Rajlich, CCC, March 23, 2004 » Shredder, Junior, Fritz

2010 ...

2015 ...

2020 ...

External Links

References

  1. Picture gallery "Recognition and Success 1955 - 1972" from The Official M.C. Escher Website
  2. David Levy, David Broughton, Mark Taylor (1989). The SEX Algorithm in Computer Chess. ICCA Journal, Vol. 12, No. 1
  3. David Levy (2002). SOME COMMENTS ON REALIZATION PROBABILITIES AND THE SEX ALGORITHM. ICGA Journal, Vol. 25, No. 3
  4. Funny Junior Engine in CBLight / Junior Engine ply depth by Wolfgang Krietsch, rgcc, February 27, 1998, post 7 and 16 by Amir Ban
  5. Ken Thompson (1982). Computer Chess Strength. Advances in Computer Chess 3
  6. Joe Condon, Ken Thompson (1983). BELLE. Chess Skill in Man and Machine
  7. Andreas Junghanns, Jonathan Schaeffer, Mark Brockington, Yngvi Björnsson, Tony Marsland (1997). Diminishing Returns for Additional Search in Chess. Advances in Computer Chess 8, pdf
  8. Robert Hyatt, Monroe Newborn (1997). CRAFTY Goes Deep. ICCA Journal, Vol. 20, No. 2
  9. Ernst A. Heinz (1998). DarkThought Goes Deep. ICCA Journal, Vol. 21, No. 4
  10. A New Self-Play Experiment - Diminishing Returns Shown with 95% Conf. by Ernst A. Heinz, CCC, May 24, 2000
  11. Ernst A. Heinz (2001). Self-Play, Deep Search and Diminishing Returns. ICGA Journal, Vol. 24, No. 2
  12. ICGA_J (June) self-play information by Guy Haworth, CCC, September 05, 2001
  13. Re: In chess we will reach diminishing returns just like in Checkers 1994 by Tony Werten, CCC, October 30, 2003
  14. Experiments in computer chess: The value of depth and diminishing return effects by Ed Schröder, June 2012
  15. Ply versus ELO by Greg, HIARCS Forum, May 30, 2020 » Diogo R. Ferreira - Impact of the Search Depth ...
  16. Ernst A. Heinz (2001). Self-Play, Deep Search and Diminishing Returns. ICGA Journal, Vol. 24, No. 2
  17. Perfection in checkers, ChessBase News, October 29, 2003
  18. Regan's latest: Depth of Satisficing by Carl Lumma, CCC, October 09, 2015

Up one Level