Internal Iterative Deepening

Home * Search * Move Ordering * Internal Iterative Deepening



Internal Iterative Deepening (IID), used in nodes of the search tree in a iterative deepening depth-first alpha-beta framework, where a program has no best move available from a previous search PV or from the transposition table. IID is used to find a good move to search first by searching the current position to a reduced depth, and using the best move of that search as the first move at the real depth. IID was already introduced by John J. Scott in 1969 inside his program Lancaster, and further elaborated by Thomas Anantharaman in 1991 , as used in Deep Thought with a reduction of 2 plies at PV-nodes and cut-nodes with no best-move information from the transposition table. While it is pretty much a washout on average, Deep Though used this heuristic since it makes the search times more predictable by avoiding those isolated instances when the search time suddenly becomes 10 times larger than expected.

=Reduction= The main implementation detail of IID comes with how to reduce the search. Typical reductions include -1, -2, /2, or /4. Some also use either the minimum or the maximum of a subtraction and a division.

=Conditions= Where to use IID is also an important question. Limiting the depth is an obvious condition (only use IID if depth > 5, say). Most only use IID in PV-Nodes, but it is also possible to use it at predicted Cut-Nodes.

=Insurance= Harm Geert Muller in a reply to Stan Arts on the insurance aspect of IID, in a CCC discussion why IID does not work some programs like Movei and RomiChess I completely agree. IID is like an insurance. Most of the time it was not needed, but then it also costs very little. And now and then it saves you big time.

Of course the lousier your basic move ordering, the more you benefit. But then it can be really spectacular, to the point where a program like micro-Max (which does have no move ordering at all, not even a move list, and can only search moves as it generates them) is able to compete with 'serious' engines. IID really works miracles there.

One thing still on my to-do list is to investigate if even more drastic IID would not even be better. I am thinking of situations where a previously best move in a PV node experiences a dramatic drop in score (or even any decrease in score) at high search depth. If that move has been best move for many ID or IID iterations, you will know next to nothing about the other moves. They have always been scoring below alpha, all their hashed scores are upper bounds, and in many cases the upper bounds are no longer usable. (With hard fail they would never be usable!) Disastrously poor moves might very well have the highest upper bounds. To prevent wasting lots of time on such a disastrous move at high search depth, it would make sense to start searching for moves that might beat the new alpha at small depth first, resetting the IID depth back to 1 in PV nodes each time the value of alpha after search of the previous best move drops compared to the previous iteration.

=Publications=
 * John J. Scott (1969). A chess-playing program. Machine Intelligence 4
 * Thomas Anantharaman (1991). Extension Heuristics. 1.12 Internal iterative deepening, ICCA Journal, Vol. 14, No. 2

=See also=
 * Double Null Move
 * Iterative Deepening
 * Multi–ProbCut
 * Node Types

=Forum Posts=

1998 ...

 * Internal Iterative Deepening by Roberto Waldteufel, CCC, August 12, 1998

2000 ...

 * Crafty internal iterative deepening by Tijs van Dam, CCC, January 26, 2000 » Crafty
 * MTD, IID, fail-low, root-research by Juergen Wolf, CCC, August 14, 2003 » MTD(f), Fail-Low, Root

2005 ...

 * Something obious on IID that not everybody does by Dann Corbit, Winboard Forum, March 04, 2006
 * self deepening: an improved implementation of IID by Harm Geert Muller, Winboard Forum, April 24, 2006
 * Re: movei hash report by Harm Geert Muller, CCC, August 13, 2007 » Movei
 * Internal Iterative Deepening by Andrew Short, CCC, September 24, 2008
 * Null driven IID by Marco Costalba, CCC, December 08, 2008

2010 ...

 * iid and singular extensions by Daniel Shawul, CCC, July 05, 2010 » Singular Extensions
 * Trouble with IID by kenny stanley, CCC, February 20, 2011
 * On internal iterative deepening by Onno Garms, CCC, March 13, 2011 » Onno, Node Types
 * Internal Iterative Deepening questions by Alcides Schulz, CCC, September 21, 2011
 * Reductions from internal iterative deepening by Evert Glebbeek, CCC, August 20, 2012 » Reductions
 * IID and move sorting by Daniel Shawul, CCC, May 09, 2013 » ABDADA
 * What do you use IID for by Daniel Shawul, CCC, December 10, 2013
 * IFD vs IID by Sergei S. Markoff, CCC, January 30, 2014
 * Internal Iterative Deepening Payoffs by CDaley11, OpenChess Forum, February 01, 2014
 * Fail soft vs fail hard by Sergei S. Markoff, CCC, February 15, 2014 » Fail-Soft, Fail-Hard, Fail-Low

2015 ...

 * Hash Refutation by Dennis Sceviour, CCC, August 24, 2015 » Schooner
 * Mate scores from IID by Matthew R. Brades, CCC, August 16, 2016 » Mate Score
 * Value of IID by Thomas Kolarik, CCC, August 25, 2016
 * improving iid by Folkert van Heusden, CCC, January 06, 2017
 * (I)ID and PV dropout by Harm Geert Muller, CCC, June 17, 2017 » Aspiration Windows, Fail-Low, Iterative Deepening
 * IID (skipped when in check and Score greater than alpha) by Tamás Kuzmics, CCC, July 24, 2017

2020 ...

 * An alternative to IID by Ed Schröder, CCC, August 13, 2020 » Reductions

=External Links=
 * µ-Max Search: Iterative Deepening by Harm Geert Muller

=References= Up one level