From Chessprogramming wiki
Revision as of 08:47, 19 May 2018 by GerdIsenberg (talk | contribs) (Created page with "'''Home * Learning''' FILE:Laurentius de Voltolina 001.jpg|border|right|thumb| Learning <ref>A depiction of the world's oldest continually operating universit...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Home * Learning

Learning [1]

the process of acquiring new knowledge which involves synthesizing different types of information. Machine learning as aspect of computer chess programming deals with algorithms that allow the program to change its behavior based on data, which for instance occurs during game playing against a variety of opponents considering the final outcome and/or the game record for instance as history score chart indexed by ply. Related to Machine learning is evolutionary computation and its sub-areas of genetic algorithms, and genetic programming, that mimics the process of natural evolution, as further mentioned in automated tuning. The process of learning often implies understanding, perception or reasoning. So called Rote learning avoids understanding and focuses on memorization. Inductive learning takes examples and generalizes rather than starting with existing knowledge. Deductive learning takes abstract concepts to make sense of examples [2].

Learning inside a Chess Program

Learning inside a chess program may address several disjoint issues. A persistent hash table remembers "important" positions from earlier games inside the search with its exact score [3]. Worse positions may be avoided in advance. Learning opening book moves, that is appending successful novelties or modify the probability of already stored moves from the book based on the outcome of a game [4]. Another application is learning evaluation weights of various features, f. i. piece- [5] or piece-square [6] values or mobility. Programs may also learn to control search [7] or time usage [8].

Learning Paradigms

There are three major learning paradigms, each corresponding to a particular abstract learning task. These are supervised learning, unsupervised learning and reinforcement learning. Usually any given type of neural network architecture can be employed in any of those tasks.

Supervised Learning

Supervised learning is learning from examples provided by a knowledgable external supervisor. In machine learning, supervised learning is a technique for deducing a function from training data. The training data consist of pairs of input objects and desired outputs, f.i. in computer chess a sequence of positions associated with the outcome of a game [9] .

Unsupervised Learning

Unsupervised machine learning seems much harder: the goal is to have the computer learn how to do something that we don't tell it how to do. The learner is given only unlabeled examples, f. i. a sequence of positions of a running game but the final result (still) unknown. A form of reinforcement learning can be used for unsupervised learning, where an agent bases its actions on the previous rewards and punishments without necessarily even learning any information about the exact ways that its actions affect the world. Clustering is another method of unsupervised learning.

Reinforcement Learning

see main page Reinforcement Learning

Reinforcement learning is defined not by characterizing learning methods, but by characterizing a learning problem. Reinforcement learning is learning what to do - how to map situations to actions - so as to maximize a numerical reward signal. The learner is not told which actions to take, as in most forms of machine learning, but instead must discover which actions yield the most reward by trying them. The reinforcement learning problem is deeply indebted to the idea of Markov decision processes (MDPs) from the field of optimal control.

Learning Topics


See also

Selected Publications


1940 ...

1950 ...

Claude Shannon, John McCarthy (eds.) (1956). Automata Studies. Annals of Mathematics Studies, No. 34
Alan Turing, Jack Copeland (editor) (2004). The Essential Turing, Seminal Writings in Computing, Logic, Philosophy, Artificial Intelligence, and Artificial Life plus The Secrets of Enigma. Oxford University Press, amazon, google books

1955 ...

  • John von Neumann (1956). Probabilistic Logic and the Synthesis of Reliable Organisms From Unreliable Components. in
Claude Shannon, John McCarthy (eds.) (1956). Automata Studies. Annals of Mathematics Studies, No. 34, pdf

1960 ...

1965 ...

1970 ...

1975 ...

  • Jacques Pitrat (1976). A Program to Learn to Play Chess. Pattern Recognition and Artificial Intelligence, pp. 399-419. Academic Press Ltd. London, UK. ISBN 0-12-170950-7.
  • Jacques Pitrat (1976). Realization of a Program Learning to Find Combinations at Chess. Computer Oriented Learning Processes (ed. J. Simon). Noordhoff, Groningen, The Netherlands.
  • Pericles Negri (1977). Inductive Learning in a Hierarchical Model for Representing Knowledge in Chess End Games. pdf
  • Ryszard Michalski, Pericles Negri (1977). An experiment on inductive learning in chess endgames. Machine Intelligence 8, pdf
  • Boris Stilman (1977). The Computer Learns. in 1976 US Computer Chess Championship, by David Levy, Computer Science Press, Woodland Hills, CA, pp. 83-90
  • Richard Sutton (1978). Single channel theory: A neuronal theory of learning. Brain Theory Newsletter 3, No. 3/4, pp. 72-75.
  • Ross Quinlan (1979). Discovering Rules by Induction from Large Collections of Examples. Expert Systems in the Micro-electronic Age, pp. 168-201. Edinburgh University Press (Introducing ID3)

1980 ...

1985 ...





1990 ...





1995 ...




Miroslav Kubat, Ivan Bratko, Ryszard Michalski (1998). A Review of Machine Learning Methods. pdf


2000 ...





2005 ...





2010 ...



István Szita (2012). Reinforcement Learning in Games. Chapter 17



2015 ...



Forum Posts

1998 ...

2000 ...

2005 ...

2010 ...

2015 ...

External Links

Machine Learning


Learning I
Learning II


Supervised Learning

AdaBoost from Wikipedia

Unsupervised Learning

Reinforcement Learning

TD Learning


Naive Bayes classifier from Wikipedia
Probabilistic classification from Wikipedia
Outline of regression analysis from Wikipedia
Linear regression from Wikipedia
Logistic regression from Wikipedia
Normal distribution from Wikipedia
Pseudorandom number generator from Wikipedia
Pseudo-random number sampling from Wikipedia
Statistical randomness from Wikipedia

Markov Models






The Single Layer Perceptron
Hidden Neurons and Feature Space
Training Neural Networks Using Back Propagation in C#
Data Mining with Artificial Neural Networks (ANN)



  1. A depiction of the world's oldest continually operating university, the University of Bologna, Italy, by Laurentius de Voltolina, second half of 14th century, Learning from Wikipedia
  2. Inductive learning vs Deductive learning
  3. David Slate (1987). A Chess Program that uses its Transposition Table to Learn from Experience. ICCA Journal, Vol. 10, No. 2
  4. Robert Hyatt (1999). Book Learning - a Methodology to Tune an Opening Book Automatically. ICCA Journal, Vol. 22, No. 1
  5. Don Beal, Martin C. Smith (1997). Learning Piece Values Using Temporal Differences. ICCA Journal, Vol. 20, No. 3
  6. Don Beal, Martin C. Smith (1999). Learning Piece-Square Values using Temporal Differences. ICCA Journal, Vol. 22, No. 4
  7. Yngvi Björnsson, Tony Marsland (2001). Learning Search Control in Adversary Games. Advances in Computer Games 9, pdf
  8. Levente Kocsis, Jos Uiterwijk, Jaap van den Herik (2000). Learning Time Allocation using Neural Networks. CG 2000, postscript
  9. AI Horizon: Machine Learning, Part II: Supervised and Unsupervised Learning
  10. online papers from Machine Learning in Games by Jay Scott
  11. Rosenblatt's Contributions
  12. Ratio Club from Wikipedia
  13. Royal Radar Establishment from Wikipedia
  14. see Swap-off by Helmut Richter
  15. The abandonment of connectionism in 1969 - Wikipedia
  16. Frank Rosenblatt (1962). Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books
  17. Long short term memory from Wikipedia
  18. Tsumego from Wikipedia
  19. Learnable Evolution Model from Wikipedia
  20. University of Bristol - Department of Computer Science - Technical Reports
  21. Generalized Hebbian Algorithm from Wikipedia
  22. Dap Hartmann (2010). Mimicking the Black Box - Genetically evolving evaluation functions and search algorithms. Review on Omid David's Ph.D. Thesis, ICGA Journal, Vol 33, No. 1
  23. Monte-Carlo Simulation Balancing - videolectures.net by David Silver
  24. MATLAB from Wikipedia
  25. Ms. Pac-Man from Wikipedia
  26. Demystifying Deep Reinforcement Learning by Tambet Matiisen, Nervana, December 21, 2015
  27. Patent US20150100530 - Methods and apparatus for reinforcement learning - Google Patents
  28. Jaap van den Herik wint Humies Award 2014 - LIACS - Leiden Institute of Advanced Computer Science
  29. 2048 (video game) from Wikipedia
  30. Teaching Deep Convolutional Neural Networks to Play Go by Hiroshi Yamashita, The Computer-go Archives, December 14, 2014
  31. Teaching Deep Convolutional Neural Networks to Play Go by Michel Van den Bergh, CCC, December 16, 2014
  32. Convolutional neural network from Wikipedia
  33. Best Paper Awards | TAAI 2014
  34. DeepChess: Another deep-learning based chess program by Matthew Lai, CCC, October 17, 2016
  35. ICANN 2016 | Recipients of the best paper awards
  36. Using GAN to play chess by Evgeniy Zheltonozhskiy, CCC, February 23, 2017
  37. Naive Bayes classifier from Wikipedia
  38. Amir Ban (2012). Automatic Learning of Evaluation, with Applications to Computer Chess. Discussion Paper 613, The Hebrew University of Jerusalem - Center for the Study of Rationality, Givat Ram
  39. Christopher Clark, Amos Storkey (2014). Teaching Deep Convolutional Neural Networks to Play Go. arXiv:1412.3409
  40. Chris J. Maddison, Aja Huang, Ilya Sutskever, David Silver (2014). Move Evaluation in Go Using Deep Convolutional Neural Networks. arXiv:1412.6564v1

Up one Level