25,161
edits
Changes
no edit summary
The [https://en.wikipedia.org/wiki/Perceptron perceptron] is an algorithm for [[Supervised Learning|supervised learning]] of [https://en.wikipedia.org/wiki/Binary_classification binary classifiers]. It was the first artificial neural network, introduced in 1957 by [https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] <ref>[https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] ('''1957'''). ''The Perceptron - a Perceiving and Recognizing Automaton''. Report 85-460-1, [https://en.wikipedia.org/wiki/Calspan#History Cornell Aeronautical Laboratory]</ref>, implemented in custom hardware. In its basic form it consists of a single neuron with multiple inputs and associated weights.
[[Supervised Learning|Supervised learning]] is applied using a set D of labeled [https://en.wikipedia.org/wiki/Test_set training data] with pairs of [https://en.wikipedia.org/wiki/Feature_vector feature vectors] (x) and given results as desired output (d), usually started with cleared or randomly initialized weight vector w. The output is calculated by all inputs of a sample, multiplied by its corresponding weights, passing the sum to the activation function f. The difference of desired and actual value is then immediately used modify the weights for all features using a learning rate 0.0 < α <= 1.0:
<pre>
for (j=0, Σ = 0.0; j < nSamples; ++j) {
==Backpropagation==
In 1974, [https://en.wikipedia.org/wiki/Paul_Werbos [Mathematician#PWerbos|Paul Werbos]] started to end the AI winter concerning neural networks, when he first described the mathematical process of training [https://en.wikipedia.org/wiki/Multilayer_perceptron multilayer perceptrons] through [https://en.wikipedia.org/wiki/Backpropagation backpropagation] of errors <ref>[https://en.wikipedia.org/wiki/Paul_Werbos [Mathematician#PWerbos|Paul Werbos]] ('''1974'''). ''[http://aitopics.org/publication/beyond-regression-new-tools-prediction-and-analysis-behavioral-sciences Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences]''. Ph. D. thesis, [[Harvard University]]</ref>, derived in the context of [https://en.wikipedia.org/wiki/Control_theory control theory] by [https://en.wikipedia.org/wiki/Henry_J._Kelley Henry J. Kelley] in 1960 <ref>[https://en.wikipedia.org/wiki/Henry_J._Kelley Henry J. Kelley] ('''1960'''). ''[http://arc.aiaa.org/doi/abs/10.2514/8.5282?journalCode=arsj& Gradient Theory of Optimal Flight Paths]''. [http://arc.aiaa.org/loi/arsj ARS Journal, Vol. 30, No. 10</ref> and by [https://en.wikipedia.org/wiki/Arthur_E._Bryson Arthur E. Bryson] in 1961 <ref>[https://en.wikipedia.org/wiki/Arthur_E._Bryson Arthur E. Bryson] ('''1961'''). ''A gradient method for optimizing multi-stage allocation processes''. In Proceedings of the [[Harvard University]] Symposium on digital computers and their applications</ref> using principles of [[Dynamic Programming|dynamic programming]], simplified by [https://en.wikipedia[Mathematician#SEDreyfus|Stuart E.org/wiki/Stuart_Dreyfus Stuart Dreyfus]] in 1961 applying the [https://en.wikipedia.org/wiki/Chain_rule chain rule] <ref>[https://en[Mathematician#SEDreyfus|Stuart E.wikipedia.org/wiki/Stuart_Dreyfus Stuart Dreyfus]] ('''1961'''). ''[http://www.rand.org/pubs/papers/P2374.html The numerical solution of variational problems]''. RAND paper P-2374</ref>. It was in 1982, when Werbos applied a [https://en.wikipedia.org/wiki/Automatic_differentiation automatic differentiation] method described in 1970 by [[Mathematician#SLinnainmaa|Seppo Linnainmaa]] <ref>[[Mathematician#SLinnainmaa|Seppo Linnainmaa]] ('''1970'''). ''The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors''. Master's thesis, [https://en.wikipedia.org/wiki/University_of_Helsinki University of Helsinki]</ref> to neural networks in the way that is widely used today <ref>[https://en.wikipedia.org/wiki/Paul_Werbos [Mathematician#PWerbos|Paul Werbos]] ('''1982'''). ''Applications of advances in nonlinear sensitivity analysis''. [http://link.springer.com/book/10.1007%2FBFb0006119 System Modeling and Optimization], [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [http://werbos.com/Neural/SensitivityIFIPSeptember1981.pdf pdf]</ref> <ref>[https://en.wikipedia.org/wiki/Paul_Werbos [Mathematician#PWerbos|Paul Werbos]] ('''1994'''). ''[http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471598976.html The Roots of Backpropagation. From Ordered Derivatives to Neural Networks and Political Forecasting]''. [https://en.wikipedia.org/wiki/John_Wiley_%26_Sons John Wiley & Sons]</ref> <ref>[http://www.scholarpedia.org/article/Deep_Learning#Backpropagation Deep Learning - Scholarpedia | Backpropagation] by [[Jürgen Schmidhuber]]</ref> <ref>[http://people.idsia.ch/~juergen/who-invented-backpropagation.html Who Invented Backpropagation?] by [[Jürgen Schmidhuber]] (2014, 2015)</ref>.
Backpropagation is a generalization of the [https://en.wikipedia.org/wiki/Delta_rule delta] rule to multilayered [https://en.wikipedia.org/wiki/Feedforward_neural_network feedforward networks], made possible by using the [https://en.wikipedia.org/wiki/Chain_rule chain rule] to iteratively compute [https://en.wikipedia.org/wiki/Gradient gradients] for each layer. Backpropagation requires that the [https://en.wikipedia.org/wiki/Activation_function activation function] used by the artificial neurons be [https://en.wikipedia.org/wiki/Differentiable_function differentiable], which is true for the common [https://en.wikipedia.org/wiki/Sigmoid_function sigmoid] [https://en.wikipedia.org/wiki/Logistic_function logistic function] or its [https://en.wikipedia.org/wiki/Softmax_function softmax] generalization in [https://en.wikipedia.org/wiki/Multiclass_classification multiclass classification].
===Alpha Zero===
In December 2017, the [[Google]] [[DeepMind]] team along with former [[Giraffe]] author [[Matthew Lai]] reported on their generalized [[AlphaZero]] algorithm, combining [[Deep Learning|Deep learning]] with [[Monte-Carlo Tree Search]]. AlphaZero can achieve, tabula rasa, superhuman performance in many challenging domains with some training effort. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved a superhuman level of play in the games of chess and [[Shogi]] as well as Go, and convincingly defeated a world-champion program in each case <ref>[[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2017'''). ''Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm''. [https://arxiv.org/abs/1712.01815 arXiv:1712.01815]</ref>.
<span id="engines"></span>
===NN Chess Programs===
* [[:Category:NN]]
=See also=
* [[Pattern Recognition]]
* [[Temporal Difference Learning]]
=Selected Publications=
* [https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] ('''1957'''). ''The Perceptron - a Perceiving and Recognizing Automaton''. Report 85-460-1, [https://en.wikipedia.org/wiki/Calspan#History Cornell Aeronautical Laboratory] <ref>[http://csis.pace.edu/~ctappert/srd2011/rosenblatt-contributions.htm Rosenblatt's Contributions]</ref>
==1960 ...==
* [[Mathematician#BWidrow|Bernard Widrow]], [[Mathematician#MTHoff|Marcian Hoff]] ('''1960'''). ''Adaptive switching circuits''. [https://catalog.hathitrust.org/Record/009671379 IRE WESCON Convention Record], Vol. 4, [http://www-isl.stanford.edu/~widrow/papers/c1960adaptiveswitching.pdf pdf]* [https://en.wikipedia.org/wiki/Henry_J._Kelley Henry J. Kelley] ('''1960'''). ''[http://arc.aiaa.org/doi/abs/10.2514/8.5282?journalCode=arsj& Gradient Theory of Optimal Flight Paths]''. [http://arc.aiaa.org/loi/arsj ARS Journal, Vol. 30, No. 10 » [[Neural Networks#Backpropagation|Backpropagation]]
* [https://en.wikipedia.org/wiki/Arthur_E._Bryson Arthur E. Bryson] ('''1961'''). ''A gradient method for optimizing multi-stage allocation processes''. In Proceedings of the [[Harvard University]] Symposium on digital computers and their applications » [[Neural Networks#Backpropagation|Backpropagation]]
* [https://en[Mathematician#SEDreyfus|Stuart E.wikipedia.org/wiki/Stuart_Dreyfus Stuart Dreyfus]] ('''1961'''). ''[http://www.rand.org/pubs/papers/P2374.html The numerical solution of variational problems]''. RAND paper P-2374 » [[Neural Networks#Backpropagation|Backpropagation]]
* [https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] ('''1962'''). ''[http://catalog.hathitrust.org/Record/000203591 Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms]''. Spartan Books
* [https://en.wikipedia.org/wiki/Alexey_Grigorevich_Ivakhnenko Alexey G. Ivakhnenko] ('''1965'''). ''Cybernetic Predicting Devices''. [https://en.wikipedia.org/wiki/Naukova_Dumka Naukova Dumka]
* [[Mathematician#SGrossberg|Stephen Grossberg]] ('''1973'''). ''Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks''. [https://en.wikipedia.org/wiki/Studies_in_Applied_Mathematics Studies in Applied Mathematics], Vol. 52, [http://cns.bu.edu/~steve/Gro1973StudiesAppliedMath.pdf pdf]
* [[Mathematician#SGrossberg|Stephen Grossberg]] ('''1974'''). ''[http://techlab.bu.edu/resources/article_view/classical_and_instrumental_learning_by_neural_networks/ Classical and instrumental learning by neural networks]''. Progress in Theoretical Biology. [https://en.wikipedia.org/wiki/Academic_Press Academic Press]
* [https://en.wikipedia.org/wiki/Paul_Werbos [Mathematician#PWerbos|Paul Werbos]] ('''1974'''). ''[http://aitopics.org/publication/beyond-regression-new-tools-prediction-and-analysis-behavioral-sciences Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences]''. Ph. D. thesis, [[Harvard University]] <ref>[https://en.wikipedia.org/wiki/Backpropagation Backpropagation from Wikipedia]</ref> <ref>[https://en.wikipedia.org/wiki/Paul_Werbos [Mathematician#PWerbos|Paul Werbos]] ('''1994'''). ''[http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471598976.html The Roots of Backpropagation. From Ordered Derivatives to Neural Networks and Political Forecasting]''. [https://en.wikipedia.org/wiki/John_Wiley_%26_Sons John Wiley & Sons]</ref>
* [[Richard Sutton]] ('''1978'''). ''Single channel theory: A neuronal theory of learning''. Brain Theory Newsletter 3, No. 3/4, pp. 72-75. [http://www.cs.ualberta.ca/%7Esutton/papers/sutton-78-BTN.pdf pdf]
==1980 ...==
* [http://www.scholarpedia.org/article/User:Kunihiko_Fukushima Kunihiko Fukushima] ('''1980'''). ''Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position''. [http://link.springer.com/journal/422 Biological Cybernetics], Vol. 36 <ref>[http://www.scholarpedia.org/article/Neocognitron Neocognitron - Scholarpedia] by [http://www.scholarpedia.org/article/User:Kunihiko_Fukushima Kunihiko Fukushima]</ref>
* [[Richard Sutton]], [[Andrew Barto]] ('''1981'''). ''Toward a modern theory of adaptive networks: Expectation and prediction''. Psychological Review, Vol. 88, pp. 135-170. [http://www.cs.ualberta.ca/%7Esutton/papers/sutton-barto-81-PsychRev.pdf pdf]
* [https://en.wikipedia.org/wiki/Paul_Werbos [Mathematician#PWerbos|Paul Werbos]] ('''1982'''). ''Applications of advances in nonlinear sensitivity analysis''. [http://link.springer.com/book/10.1007%2FBFb0006119 System Modeling and Optimization], [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [http://werbos.com/Neural/SensitivityIFIPSeptember1981.pdf pdf]
* [[A. Harry Klopf]] ('''1982'''). ''The Hedonistic Neuron: A Theory of Memory, Learning, and Intelligence''. Hemisphere Publishing Corporation, [[University of Michigan]]
* [[Mathematician#DHAckley|David H. Ackley]], [[Mathematician#GEHinton|Geoffrey E. Hinton]], [[Terrence J. Sejnowski]] ('''1985'''). ''A Learning Algorithm for Boltzmann Machines''. Cognitive Science, Vol. 9, No. 1, [https://web.archive.org/web/20110718022336/http://learning.cs.toronto.edu/~hinton/absps/cogscibm.pdf pdf]
* [https://en.wikipedia.org/wiki/David_Rumelhart [Mathematician#DERumelhart|David E. Rumelhart]], [[Mathematician#GEHinton|Geoffrey E. Hinton]], [https://en.wikipedia.org/wiki/Ronald_J._Williams Ronald J. Williams] ('''1986'''). ''Learning representations by back-propagating errors''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 323, [http://www.iro.umontreal.ca/~vincentp/ift3395/lectures/backprop_old.pdf pdf]
'''1987'''
* [[Gerald Tesauro]], [[Terrence J. Sejnowski]] ('''1987'''). ''A 'Neural' Network that Learns to Play Backgammon''. [http://www.informatik.uni-trier.de/~ley/db/conf/nips/nips1987.html#TesauroS87 NIPS 1987]
* [[Eric B. Baum]] ('''1989'''). ''[http://papers.nips.cc/paper/226-the-perceptron-algorithm-is-fast-for-non-malicious-distributions The Perceptron Algorithm Is Fast for Non-Malicious Distributions]''. [http://papers.nips.cc/book/advances-in-neural-information-processing-systems-2-1989 NIPS 1989]
* [[Eric B. Baum]] ('''1989'''). ''[http://www.mitpressjournals.org/doi/abs/10.1162/neco.1989.1.2.201#.VfGX0JdpluM A Proposal for More Powerful Learning Algorithms]''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 1, No. 2
* [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/i/Irani:E=_A=.html Erach A. Irani], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/m/Matts:John_P=.html John P. Matts], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/l/Long:John_M=.html John M. Long], [[James R. Slagle]], POSCH group ('''1989'''). ''Using Artificial Neural Nets for Statistical Discovery: Observations after Using Backpropogation, Expert Systems, and Multiple-Linear Regression on Clinical Trial Data''. [[University of Minnesota]], Minneapolis, MN 55455, USA, Complex Systems 3, [http://www.complex-systems.com/pdf/03-3-5.pdf pdf]
* [[Gerald Tesauro]], [[Terrence J. Sejnowski]] ('''1989'''). ''A Parallel Network that Learns to Play Backgammon''. [https://en.wikipedia.org/wiki/Artificial_Intelligence_%28journal%29 Artificial Intelligence], Vol. 39, No. 3
* [[Mathematician#EGelenbe|Erol Gelenbe]] ('''1989'''). ''[http://cognet.mit.edu/journal/10.1162/neco.1989.1.4.502 Random Neural Networks with Negative and Positive Signals and Product Form Solution]''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 1, No. 4
* [[Mathematician#XZhang|Xiru Zhang]], [https://dblp.uni-trier.de/pers/hd/m/McKenna:Michael Michael McKenna], [[Mathematician#JPMesirov|Jill P. Mesirov]], [[David Waltz]] ('''1989'''). ''[http://papers.neurips.cc/paper/281-an-efficient-implementation-of-the-back-propagation-algorithm-on-the-connection-machine-cm-2 An Efficient Implementation of the Back-propagation Algorithm on the Connection Machine CM-2]''. [https://dblp.uni-trier.de/db/conf/nips/nips1989.html NIPS 1989]
==1990 ...==
* [https://en.wikipedia.org/wiki/Paul_Werbos [Mathematician#PWerbos|Paul Werbos]] ('''1990'''). ''Backpropagation Through Time: What It Does and How to Do It''. Proceedings of the [[IEEE]], Vol. 78, No. 10, [http://deeplearning.cs.cmu.edu/pdfs/Werbos.backprop.pdf pdf]
* [[Gordon Goetsch]] ('''1990'''). ''Maximization of Mutual Information in a Context Sensitive Neural Network''. Ph.D. thesis
* [[Vadim Anshelevich]] ('''1990'''). ''Neural Networks''. Review. in Multi Component Systems (Russian)
* [[Eric B. Baum]] ('''1990'''). ''Polynomial Time Algorithms for Learning Neural Nets''. [http://dblp.uni-trier.de/db/conf/colt/colt1990.html#Baum90 COLT 1990]
* [https://dblp.uni-trier.de/pers/hd/r/Ruck:Dennis_W= Dennis W. Ruck], [http://spie.org/profile/Steven.Rogers-5480?SSO=1 Steven K. Rogers], [https://dblp.uni-trier.de/pers/hd/k/Kabrisky:Matthew Matthew Kabrisky], [[Mathematician#MEOxley|Mark E. Oxley]], [[Bruce W. Suter]] ('''1990'''). ''[https://ieeexplore.ieee.org/document/80266 The multilayer perceptron as an approximation to a Bayes optimal discriminant function]''. [[IEEE#NN|IEEE Transactions on Neural Networks]], Vol. 1, No. 4
* [https://dblp.uni-trier.de/pers/hd/h/Hellstrom:Benjamin_J= Benjamin J. Hellstrom], [[Laveen Kanal|Laveen N. Kanal]] ('''1990'''). ''[https://ieeexplore.ieee.org/document/5726889 The definition of necessary hidden units in neural networks for combinatorial optimization]''. [https://dblp.uni-trier.de/db/conf/ijcnn/ijcnn1990.html IJCNN 1990]
* [[Mathematician#XZhang|Xiru Zhang]], [https://dblp.uni-trier.de/pers/hd/m/McKenna:Michael Michael McKenna], [[Mathematician#JPMesirov|Jill P. Mesirov]], [[David Waltz]] ('''1990'''). ''[https://www.sciencedirect.com/science/article/pii/016781919090084M The backpropagation algorithm on grid and hypercube architectures]''. [https://www.journals.elsevier.com/parallel-computing Parallel Computing], Vol. 14, No. 3
'''1991'''
* [[Mathematician#SHochreiter|Sepp Hochreiter]] ('''1991'''). ''Untersuchungen zu dynamischen neuronalen Netzen''. Diploma thesis, [[Technical University of Munich|TU Munich]], advisor [[Jürgen Schmidhuber]], [http://people.idsia.ch/~juergen/SeppHochreiter1991ThesisAdvisorSchmidhuber.pdf pdf] (German) <ref>[http://people.idsia.ch/~juergen/fundamentaldeeplearningproblem.html Sepp Hochreiter's Fundamental Deep Learning Problem (1991)] by [[Jürgen Schmidhuber]], 2013</ref>
* [[Justin A. Boyan]] ('''1992'''). ''Modular Neural Networks for Learning Context-Dependent Game Strategies''. Master's thesis, [https://en.wikipedia.org/wiki/University_of_Cambridge University of Cambridge], [http://www.cs.cmu.edu/~jab/cv/pubs/boyan.backgammon-thesis.pdf pdf]
* [https://en.wikipedia.org/wiki/Patricia_Churchland Patricia Churchland], [[Terrence J. Sejnowski]] ('''1992'''). ''[https://mitpress.mit.edu/books/computational-brain The Computational Brain]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [https://dblp.uni-trier.de/pers/hd/h/Hellstrom:Benjamin_J= Benjamin J. Hellstrom], [[Laveen Kanal|Laveen N. Kanal]] ('''1992'''). ''[https://ieeexplore.ieee.org/document/125871 Knapsack packing networks]''. [[IEEE#NN|IEEE Transactions on Neural Networks]], Vol. 3, No. 2
* [https://dblp.uni-trier.de/pers/hd/h/Hellstrom:Benjamin_J= Benjamin J. Hellstrom], [[Laveen Kanal|Laveen N. Kanal]] ('''1992'''). ''Asymmetric mean-field neural networks for multiprocessor scheduling''. [https://en.wikipedia.org/wiki/Neural_Networks_(journal) Neural Networks], Vol. 5, No. 4
'''1993'''
* [[Jacek Mańdziuk]], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/m/Macuk:Bohdan.html Bohdan Macukow] ('''1993'''). ''A Neural Network performing Boolean Logic Operations''. [http://www.springerlink.com/content/1060-992x/ Optical Memory and Neural Networks], Vol. 2, No. 1, [http://www.mini.pw.edu.pl/~mandziuk/PRACE/omnn93.pdf pdf]
* [[Sebastian Thrun]], [[Tom Mitchell]] ('''1993'''). ''Integrating Inductive Neural Network Learning and Explanation-Based Learning''. Proceedings of the 13th IJCAI, pp. 930-936. Morgan Kaufmann, San Mateo, CA, [http://robots.stanford.edu/papers/thrun.EBNN_ijcai93.ps.gz zipped ps]
* [[Byoung-Tak Zhang]], [[Mathematician#HMuehlenbein|Heinz Mühlenbein]] ('''1993'''). ''Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor''. [https://en.wikipedia.org/wiki/Complex_Systems_(journal) Complex Systems], Vol. 7, [http://www.complex-systems.com/pdf/07-3-2.pdf pdf]
* [[Martin Riedmiller]], [[Heinrich Braun]] ('''1993'''). ''A direct adaptive method for faster backpropagation learning: The RPROP algorithm''. [http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=1059 IEEE International Conference On Neural Networks], [http://paginas.fe.up.pt/~ee02162/dissertacao/RPROP%20paper.pdf pdf]
* [[Nicol N. Schraudolph]], [[Peter Dayan]], [[Terrence J. Sejnowski]] ('''1993'''). ''[https://papers.nips.cc/paper/820-temporal-difference-learning-of-position-evaluation-in-the-game-of-go Temporal Difference Learning of Position Evaluation in the Game of Go]''. [https://papers.nips.cc/book/advances-in-neural-information-processing-systems-6-1993 NIPS 1993] <ref>[http://satirist.org/learn-game/systems/go-net.html Nici Schraudolph’s go networks], review by [[Jay Scott]]</ref>
'''1994'''
* [https://en.wikipedia.org/wiki/Paul_Werbos [Mathematician#PWerbos|Paul Werbos]] ('''1994'''). ''[http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471598976.html The Roots of Backpropagation. From Ordered Derivatives to Neural Networks and Political Forecasting]''. [https://en.wikipedia.org/wiki/John_Wiley_%26_Sons John Wiley & Sons]
* [[David E. Moriarty]], [[Risto Miikkulainen]] ('''1994'''). ''Evolving Neural Networks to focus Minimax Search''. [[AAAI|AAAI-94]], [http://www.cs.utexas.edu/~ai-lab/pubs/moriarty.focus.pdf pdf]
* [[Eric Postma]] ('''1994'''). ''SCAN: A Neural Model of Covert Attention''. Ph.D. thesis, [[Maastricht University]], advisor [[Jaap van den Herik]]
* [[Sebastian Thrun]] ('''1994'''). ''Neural Network Learning in the Domain of Chess''. Machines That Learn, [http://snowbird.djvuzone.org/ Snowbird], Extended abstract
* [[Christian Posthoff]], S. Schawelski, [[Michael Schlosser]] ('''1994'''). ''Neural Network Learning in a Chess Endgame Positions''. IEEE World Congress on Computational Intelligence
* [[Alois Heinz]] ('''1994'''). ''[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.55.3994 Efficient Neural Net α-β-Evaluators]''. [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.55.3994&rep=rep1&type=pdf pdf] <ref>[https://www.stmintz.com/ccc/index.php?id=11893 Re: Evaluation by neural network ?] by [[Jay Scott]], [[CCC]], November 10, 1997</ref>
* [[Alois Heinz]] ('''1994'''). ''Fast bounded smooth regression with lazy neural trees''. [http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=3013 ICNN 1994], DOI: [http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=374421 10.1109/ICNN.1994.374421]
* [[Martin Riedmiller]] ('''1994'''). ''Rprop - Description and Implementation Details''. Technical Report, [https://en.wikipedia.org/wiki/Karlsruhe_Institute_of_Technology University of Karlsruhe], [http://www.inf.fu-berlin.de/lehre/WS06/Musterererkennung/Paper/rprop.pdf pdf]
* [[Igor Kononenko]] ('''1994'''). ''On Bayesian Neural Networks''. [https://dblp.uni-trier.de/db/journals/informaticaSI/informaticaSI18.html Informatica (Slovenia), Vol. 18], No. 2
'''1995'''
* [https://peterbraspenning.wordpress.com/ Peter J. Braspenning], [[Frank Thuijsman]], [https://scholar.google.com/citations?user=Ba9L7CAAAAAJ Ton Weijters] (eds) ('''1995'''). ''[http://link.springer.com/book/10.1007%2FBFb0027019 Artificial neural networks: an introduction to ANN theory and practice]''. [https://de.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science LNCS] 931, [https://de.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Pieter Spronck]] ('''1996'''). ''Elegance: Genetic Algorithms in Neural Reinforcement Control''. Master thesis, [[Delft University of Technology]], [http://ticc.uvt.nl/~pspronck/pubs/Elegance.pdf pdf]
* [[Raúl Rojas]] ('''1996'''). ''Neural Networks - A Systematic Introduction''. Springer, available as [http://www.inf.fu-berlin.de/inst/ag-ki/rojas_home/documents/1996/NeuralNetworks/neuron.pdf pdf ebook]
* [[Ida Sprinkhuizen-Kuyper]], [https://dblp.org/pers/hd/b/Boers:Egbert_J=_W= Egbert J. W. Boers] ('''1996'''). ''[https://ieeexplore.ieee.org/abstract/document/6796246 The Error Surface of the Simplest XOR Network Has Only Global Minima]''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 8, No. 6, [http://www.socsci.ru.nl/idak/publications/papers/NeuralComputation.pdf pdf]
'''1997'''
* [[Mathematician#SHochreiter|Sepp Hochreiter]], [[Jürgen Schmidhuber]] ('''1997'''). ''Long short-term memory''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 9, No. 8, [http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf pdf] <ref>[https://en.wikipedia.org/wiki/Long_short_term_memory Long short term memory from Wikipedia]</ref>
* [[Don Beal]], [[Martin C. Smith]] ('''1999'''). ''Learning Piece-Square Values using Temporal Differences.'' [[ICGA Journal#22_4|ICCA Journal, Vol. 22, No. 4]]
* [https://en.wikipedia.org/wiki/Simon_Haykin Simon S. Haykin] ('''1999'''). ''[http://dl.acm.org/citation.cfm?id=521706 Neural Networks: A Comprehensive Foundation]''. 2nd Edition, [https://en.wikipedia.org/wiki/Prentice_Hall Prentice-Hall]
* [[https://en.wikipedia.org/wiki/Larry_Abbott Laurence F. Abbott]], [[Terrence J. Sejnowski]] (eds.) ('''1999'''). ''[https://mitpress.mit.edu/books/neural-codes-and-distributed-representations Neural Codes and Distributed Representations]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Mathematician#GEHinton|Geoffrey E. Hinton]], [[Terrence J. Sejnowski]] (eds.) ('''1999'''). ''[https://mitpress.mit.edu/books/unsupervised-learning Unsupervised Learning: Foundations of Neural Computation]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Peter Dayan]] ('''1999'''). ''Recurrent Sampling Models for the Helmholtz Machine''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 11, No. 3, [http://www.gatsby.ucl.ac.uk/~dayan/papers/rechelm99.pdf pdf] <ref>[https://en.wikipedia.org/wiki/Helmholtz_machine Helmholtz machine from Wikipedia]</ref>
* [[Ida Sprinkhuizen-Kuyper]], [https://dblp.org/pers/hd/b/Boers:Egbert_J=_W= Egbert J. W. Boers] ('''1999'''). ''[https://ieeexplore.ieee.org/document/774274 A local minimum for the 2-3-1 XOR network]''. [[IEEE#NN|IEEE Transactions on Neural Networks]], Vol. 10, No. 4
==2000 ...==
* [[Levente Kocsis]], [[Jos Uiterwijk]], [[Jaap van den Herik]] ('''2000'''). ''[http://link.springer.com/chapter/10.1007%2F3-540-45579-5_11 Learning Time Allocation using Neural Networks]''. [[CG 2000]]
* [[Jonathan Schaeffer]], [[Markian Hlynka]], [[Vili Jussila]] ('''2001'''). ''Temporal Difference Learning Applied to a High-Performance Game-Playing Program''. [http://www.informatik.uni-trier.de/~ley/db/conf/ijcai/ijcai2001.html#SchaefferHJ01 IJCAI 2001]
* [[Don Beal]], [[Martin C. Smith]] ('''2001'''). ''Temporal difference learning applied to game playing and the results of application to Shogi''. Theoretical Computer Science, Volume 252, Issues 1-2, pp. 105-119
* [[Nicol N. Schraudolph]], [[Peter Dayan]], [[Terrence J. Sejnowski]] ('''2001'''). ''[http://nic.schraudolph.org/bib2html/b2hd-SchDaySej01.html Learning to Evaluate Go Positions via Temporal Difference Methods]''. in [[Norio Baba]], [[Lakhmi C. Jain]] (eds.) ('''2001'''). ''[http://jasss.soc.surrey.ac.uk/7/1/reviews/takama.html Computational Intelligence in Games, Studies in Fuzziness and Soft Computing]''. , [http://www.springer.com/economics?SGWID=1-165-6-73481-0 Physica-Verlag] * [[Peter Dayan]], [[https://en.wikipedia.org/wiki/Larry_Abbott Laurence F. Abbott]] ('''2001, 2005'''). ''[http://www.gatsby.ucl.ac.uk/~dayan/book/index.html Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
'''2002'''
* [[Levente Kocsis]], [[Jos Uiterwijk]], [[Eric Postma]], [[Jaap van den Herik]] ('''2002'''). ''[http://link.springer.com/chapter/10.1007%2F978-3-540-40031-8_11 The Neural MoveMap Heuristic in Chess]''. [[CG 2002]]
* [[Moshe Sipper]] ('''2002''') ''[http://books.google.com/books/about/Machine_Nature.html?id=fbFQAAAAMAAJ&redir_esc=y Machine Nature: The Coming Age of Bio-Inspired Computing]''. [https://en.wikipedia.org/wiki/McGraw-Hill_Financial McGraw-Hill, New York]
* [[Paul E. Utgoff]], [[David J. Stracuzzi]] ('''2002'''). ''Many-Layered Learning''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 14, No. 10, [http://people.cs.umass.edu/~utgoff/papers/neco-stl.pdf pdf]
* [[Mathematician#MIJordan|Michael I. Jordan]], [[Terrence J. Sejnowski]] (eds.) ('''2002'''). ''[https://mitpress.mit.edu/books/graphical-models Graphical Models: Foundations of Neural Computation]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
'''2003'''
* [[Levente Kocsis]] ('''2003'''). ''Learning Search Decisions''. Ph.D thesis, [[Maastricht University]], [https://project.dke.maastrichtuniversity.nl/games/files/phd/Kocsis_thesis.pdf pdf]
* [[Holk Cruse]] ('''2006'''). ''[http://www.brains-minds-media.org/archive/615 Neural Networks as Cybernetic Systems]''. 2nd and revised edition, [http://www.uni-bielefeld.de/biologie/Kybernetik/ Department of Biological Cybernetics], [https://en.wikipedia.org/wiki/Bielefeld_University Bielefeld University]
* [[Mathematician#GEHinton|Geoffrey E. Hinton]], [https://www.linkedin.com/in/osindero Simon Osindero], [https://scholar.google.com/citations?user=y-nUzMwAAAAJ Yee Whye Teh] ('''2006'''). ''[http://www.mitpressjournals.org/doi/abs/10.1162/neco.2006.18.7.1527 A Fast Learning Algorithm for Deep Belief Nets]''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 18, No. 7, [https://www.cs.toronto.edu/~hinton/absps/fastnc.pdf pdf]
* [[Mathematician#GEHinton|Geoffrey E. Hinton]], [[Mathematician#RRSalakhutdinov|Ruslan R. Salakhutdinov]] ('''2006'''). ''Reducing the Dimensionality of Data with Neural Networks''. [https://en.wikipedia.org/wiki/Science_(journal) Science], Vol. 313, [https://www.cs.toronto.edu/~hinton/science.pdf pdf]
'''2007'''
* [[Edward P. Manning]] ('''2007'''). ''[http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=4219046 Temporal Difference Learning of an Othello Evaluation Function for a Small Neural Network with Shared Weights]''. [[IEEE#CIG|IEEE Symposium on Computational Intelligence and AI in Games]]
* [[Ilya Sutskever]], [https://research.google.com/pubs/OriolVinyals.html Oriol Vinyals], [https://www.linkedin.com/in/quoc-v-le-319b5a8 Quoc V. Le] ('''2014'''). ''Sequence to Sequence Learning with Neural Networks''. [https://arxiv.org/abs/1409.3215 arXiv:1409.3215]
'''2015'''
* [https://scholar.google.nl/citations?user=yyIoQu4AAAAJ Diederik P. Kingma], [https://scholar.google.ca/citations?user=ymzxRhAAAAAJ&hl=en Jimmy Lei Ba] ('''2015'''). ''Adam: A Method for Stochastic Optimization''. [https://arxiv.org/abs/1412.6980v8 arXiv:1412.6980v8], [http://www.iclr.cc/doku.php?id=iclr2015:main ICLR 2015] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=61948 Arasan 19.2] by [[Jon Dart]], [[CCC]], November 03, 2016 » [[Arasan#Tuning|Arasan's Tuning]]</ref>
* [http://michaelnielsen.org/ Michael Nielsen] ('''2015'''). ''[http://neuralnetworksanddeeplearning.com/ Neural networks and deep learning]''. Determination Press
* [[Mathematician#SIoffe|Sergey Ioffe]], [[Mathematician#CSzegedy|Christian Szegedy]] ('''2015'''). ''Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift''. [https://arxiv.org/abs/1502.03167 arXiv:1502.03167]
* [[Mathematician#GEHinton|Geoffrey E. Hinton]], [https://research.google.com/pubs/OriolVinyals.html Oriol Vinyals], [https://en.wikipedia.org/wiki/Jeff_Dean_(computer_scientist) Jeff Dean] ('''2015'''). ''Distilling the Knowledge in a Neural Network''. [https://arxiv.org/abs/1503.02531 arXiv:1503.02531]
* [[James L. McClelland]] ('''2015'''). ''[https://web.stanford.edu/group/pdplab/pdphandbook/handbook3.html#handbookch10.html Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises]''. Second Edition, [https://web.stanford.edu/group/pdplab/pdphandbook/handbookli1.html Contents]
* [[Gábor Melis]] ('''2015'''). ''[http://jmlr.org/proceedings/papers/v42/meli14.html Dissecting the Winning Solution of the HiggsML Challenge]''. [https://nips.cc/Conferences/2014 NIPS 2014]
* [[Volodymyr Mnih]], [[Koray Kavukcuoglu]], [[David Silver]], [[Andrei A. Rusu]], [[Joel Veness]], [[Marc G. Bellemare]], [[Alex Graves]], [[Martin Riedmiller]], [[Andreas K. Fidjeland]], [[Georg Ostrovski]], [[Stig Petersen]], [[Charles Beattie]], [[Amir Sadik]], [[Ioannis Antonoglou]], [[Helen King]], [[Dharshan Kumaran]], [[Daan Wierstra]], [[Shane Legg]], [[Demis Hassabis]] ('''2015'''). ''[http://www.nature.com/nature/journal/v518/n7540/abs/nature14236.html Human-level control through deep reinforcement learning]''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 518
* [[Jürgen Schmidhuber]] ('''2015'''). ''[http://people.idsia.ch/~juergen/deep-learning-overview.html Deep Learning in Neural Networks: An Overview]''. [https://en.wikipedia.org/wiki/Neural_Networks_(journal) Neural Networks], Vol. 61
* [[Matthew Lai]] ('''2015'''). ''Giraffe: Using Deep Reinforcement Learning to Play Chess''. M.Sc. thesis, [https://en.wikipedia.org/wiki/Imperial_College_London Imperial College London], [http://arxiv.org/abs/1509.01549v1 arXiv:1509.01549v1] » [[Giraffe]]
* [[Nikolai Yakovenko]], [[Liangliang Cao]], [[Colin Raffel]], [[James Fan]] ('''2015'''). ''Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games''. [https://arxiv.org/abs/1509.06731 arXiv:1509.06731]
* [[Ilya Loshchilov]], [[Frank Hutter]] ('''2015'''). ''Online Batch Selection for Faster Training of Neural Networks''. [https://arxiv.org/abs/1511.06343 arXiv:1511.06343]
* [[Yuandong Tian]], [[Yan Zhu]] ('''2015'''). ''Better Computer Go Player with Neural Network and Long-term Prediction''. [http://arxiv.org/abs/1511.06410 arXiv:1511.06410] <ref>[http://www.technologyreview.com/view/544181/how-facebooks-ai-researchers-built-a-game-changing-go-engine/?utm_campaign=socialsync&utm_medium=social-post&utm_source=facebook How Facebook’s AI Researchers Built a Game-Changing Go Engine | MIT Technology Review], December 04, 2015</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=58514 Combining Neural Networks and Search techniques (GO)] by Michael Babigian, [[CCC]], December 08, 2015</ref> » [[Go]]
* [[Peter H. Jin]], [[Kurt Keutzer]] ('''2015'''). ''Convolutional Monte Carlo Rollouts in Go''. [http://arxiv.org/abs/1512.03375 arXiv:1512.03375]
* [[Ian Goodfellow]], [[Yoshua Bengio]], [[Aaron Courville]] ('''2016'''). ''[http://www.deeplearningbook.org/ Deep Learning]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Volodymyr Mnih]], [[Adrià Puigdomènech Badia]], [[Mehdi Mirza]], [[Alex Graves]], [[Timothy Lillicrap]], [[Tim Harley]], [[David Silver]], [[Koray Kavukcuoglu]] ('''2016'''). ''Asynchronous Methods for Deep Reinforcement Learning''. [https://arxiv.org/abs/1602.01783 arXiv:1602.01783v2]
* [https://scholar.google.ca/citations?user=mZfgLA4AAAAJ&hl=en Vincent Dumoulin], [https://scholar.google.it/citations?user=kaAnZw0AAAAJ&hl=en Francesco Visin] ('''2016'''). ''A guide to convolution arithmetic for deep learning''. [https://arxiv.org/abs/1603.07285 arXiv:1603.07285]
* [https://en.wikipedia.org/wiki/Patricia_Churchland Patricia Churchland], [[Terrence J. Sejnowski]] ('''2016'''). ''[https://mitpress.mit.edu/books/computational-brain-0 The Computational Brain, 25th Anniversary Edition]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Ilya Loshchilov]], [[Frank Hutter]] ('''2016'''). ''CMA-ES for Hyperparameter Optimization of Deep Neural Networks''. [https://arxiv.org/abs/1604.07269 arXiv:1604.07269] <ref>[https://en.wikipedia.org/wiki/CMA-ES CMA-ES from Wikipedia]</ref>
* [[Audrūnas Gruslys]], [[Rémi Munos]], [[Ivo Danihelka]], [[Marc Lanctot]], [[Alex Graves]] ('''2016'''). ''Memory-Efficient Backpropagation Through Time''. [https://arxiv.org/abs/1606.03401v1 arXiv:1606.03401]
* [[Andrei A. Rusu]], [[Neil C. Rabinowitz]], [[Guillaume Desjardins]], [[Hubert Soyer]], [[James Kirkpatrick]], [[Koray Kavukcuoglu]], [[Razvan Pascanu]], [[Raia Hadsell]] ('''2016'''). ''Progressive Neural Networks''. [https://arxiv.org/abs/1606.04671 arXiv:1606.04671]
* [[George Rajna]] ('''2016'''). ''Deep Neural Networks''. [http://vixra.org/abs/1609.0126 viXra:1609.0126]
* [[James Kirkpatrick]], [[Razvan Pascanu]], [[Neil C. Rabinowitz]], [[Joel Veness]], [[Guillaume Desjardins]], [[Andrei A. Rusu]], [[Kieran Milan]], [[John Quan]], [[Tiago Ramalho]], [[Agnieszka Grabska-Barwinska]], [[Demis Hassabis]], [[Claudia Clopath]], [[Dharshan Kumaran]], [[Raia Hadsell]] ('''2016'''). ''Overcoming catastrophic forgetting in neural networks''. [https://arxiv.org/abs/1612.00796 arXiv:1612.00796] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70704 catastrophic forgetting] by [[Daniel Shawul]], [[CCC]], May 09, 2019</ref>* [https://dblp.uni-trier.de/pers/hd/n/Niu:Zhenxing Zhenxing Niu], [https://dblp.uni-trier.de/pers/hd/z/Zhou:Mo Mo Zhou], [https://dblp.uni-trier.de/pers/hd/w/Wang_0003:Le Le Wang], [[Xinbo Gao]], [https://dblp.uni-trier.de/pers/hd/h/Hua_0001:Gang Gang Hua] ('''2016'''). ''Ordinal Regression with Multiple Output CNN for Age Estimation''. [https://dblp.uni-trier.de/db/conf/cvpr/cvpr2016.html CVPR 2016], [https://www.cv-foundation.org/openaccess/content_cvpr_2016/app/S21-20.pdf pdf]
'''2017'''
* [[Yutian Chen]], [[Matthew W. Hoffman]], [[Sergio Gomez Colmenarejo]], [[Misha Denil]], [[Timothy Lillicrap]], [[Matthew Botvinick]], [[Nando de Freitas]] ('''2017'''). ''Learning to Learn without Gradient Descent by Gradient Descent''. [https://arxiv.org/abs/1611.03824v6 arXiv:1611.03824v6], [http://dblp.uni-trier.de/db/conf/icml/icml2017.html ICML 2017]
* [[Raúl Rojas]] ('''2017'''). ''Deepest Neural Networks''. [https://arxiv.org/abs/1707.02617 arXiv:1707.02617]
* [[Matej Moravčík]], [[Martin Schmid]], [[Neil Burch]], [[Viliam Lisý]], [[Dustin Morrill]], [[Nolan Bard]], [[Trevor Davis]], [[Kevin Waugh]], [[Michael Johanson]], [[Michael Bowling]] ('''2017'''). ''[http://science.sciencemag.org/content/356/6337/508 DeepStack: Expert-level artificial intelligence in heads-up no-limit poker]''. [https://en.wikipedia.org/wiki/Science_(journal) Science], Vol. 356, No. 6337
* [[Xinqi Zhu]], [[Michael Bain]] ('''2017'''). ''B-CNN: Branch Convolutional Neural Network for Hierarchical Classification''. [https://arxiv.org/abs/1709.09890 arXiv:1709.09890], [https://github.com/zhuxinqimac/B-CNN GitHub - zhuxinqimac/B-CNN: Sample code of B-CNN paper]
* [[David Silver]], [[Julian Schrittwieser]], [[Karen Simonyan]], [[Ioannis Antonoglou]], [[Shih-Chieh Huang|Aja Huang]], [[Arthur Guez]], [[Thomas Hubert]], [[Lucas Baker]], [[Matthew Lai]], [[Adrian Bolton]], [[Yutian Chen]], [[Timothy Lillicrap]], [[Fan Hui]], [[Laurent Sifre]], [[George van den Driessche]], [[Thore Graepel]], [[Demis Hassabis]] ('''2017'''). ''[https://www.nature.com/nature/journal/v550/n7676/full/nature24270.html Mastering the game of Go without human knowledge]''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 550 <ref>[https://deepmind.com/blog/alphago-zero-learning-scratch/ AlphaGo Zero: Learning from scratch] by [[Demis Hassabis]] and [[David Silver]], [[DeepMind]], October 18, 2017</ref>
* [[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2017'''). ''Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm''. [https://arxiv.org/abs/1712.01815 arXiv:1712.01815] » [[Neural Networks#AlphaZero|AlphaZero]] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=65909 Google's AlphaGo team has been working on chess] by [[Peter Kappler]], [[CCC]], December 06, 2017</ref>
* [[Kei Takada]], [[Hiroyuki Iizuka]], [[Masahito Yamamoto]] ('''2017'''). ''Reinforcement Learning for Creating Evaluation Function Using Convolutional Neural Network in Hex''. TAAI 2017 » [[Hex]]
* [[Chao Gao]], [[Martin Müller]], [[Ryan Hayward]] ('''2017'''). ''Focused Depth-first Proof Number Search using Convolutional Neural Networks for the Game of Hex''. [[Conferences#IJCAI2017|IJCAI 2017]]
* [[Thomas Elsken]], [[Jan Hendrik Metzen]], [[Frank Hutter]] ('''2017'''). ''Simple And Efficient Architecture Search for Convolutional Neural Networks''. [https://arxiv.org/abs/1711.04528 arXiv:1711.04528]
* [[Joel Veness]], [[Tor Lattimore]], [https://github.com/avishkar58 Avishkar Bhoopchand], [https://scholar.google.co.uk/citations?user=mB4yebIAAAAJ&hl=en Agnieszka Grabska-Barwinska], [https://dblp.org/pers/hd/m/Mattern:Christopher Christopher Mattern], [https://dblp.org/pers/hd/t/Toth:Peter Peter Toth] ('''2017'''). ''Online Learning with Gated Linear Networks''. [https://arxiv.org/abs/1712.01897 arXiv:1712.01897]
* [https://dblp.uni-trier.de/pers/hd/c/Chen:Qiming Qiming Chen], [[Ren Wu]] ('''2017'''). ''CNN Is All You Need''. [https://arxiv.org/abs/1712.09662 arXiv:1712.09662]
* [https://dblp.org/pers/hd/s/Serb:Alexander Alexantrou Serb], [[Edoardo Manino]], [https://dblp.org/pers/hd/m/Messaris:Ioannis Ioannis Messaris], [https://dblp.org/pers/hd/t/Tran=Thanh:Long Long Tran-Thanh], [https://www.orc.soton.ac.uk/people/tp1f12 Themis Prodromakis] ('''2017'''). ''[https://eprints.soton.ac.uk/425616/ Hardware-level Bayesian inference]''. [https://nips.cc/Conferences/2017 NIPS 2017] » [[Analog Evaluation]]
'''2018'''
* [[Kei Takada]], [[Hiroyuki Iizuka]], [[Masahito Yamamoto]] ('''2018'''). ''[https://link.springer.com/chapter/10.1007%2F978-3-319-75931-9_2 Computer Hex Algorithm Using a Move Evaluation Method Based on a Convolutional Neural Network]''. [https://link.springer.com/bookseries/7899 Communications in Computer and Information Science] » [[Hex]]
* [[Matthia Sabatelli]], [[Francesco Bidoia]], [[Valeriu Codreanu]], [[Marco Wiering]] ('''2018'''). ''Learning to Evaluate Chess Positions with Deep Neural Networks and Limited Lookahead''. ICPRAM 2018, [http://www.ai.rug.nl/~mwiering/GROUP/ARTICLES/ICPRAM_CHESS_DNN_2018.pdf pdf]
* [[Ashwin Srinivasan]], [[Lovekesh Vig]], [[Michael Bain]] ('''2018'''). ''Logical Explanations for Deep Relational Machines Using Relevance Information''. [https://arxiv.org/abs/1807.00595 arXiv:1807.00595]
* [[Thomas Elsken]], [[Jan Hendrik Metzen]], [[Frank Hutter]] ('''2018'''). ''Neural Architecture Search: A Survey''. [https://arxiv.org/abs/1808.05377 arXiv:1808.05377]
* [[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2018'''). ''[http://science.sciencemag.org/content/362/6419/1140 A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play]''. [https://en.wikipedia.org/wiki/Science_(journal) Science], Vol. 362, No. 6419 <ref>[https://deepmind.com/blog/alphazero-shedding-new-light-grand-games-chess-shogi-and-go/ AlphaZero: Shedding new light on the grand games of chess, shogi and Go] by [[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]] and [[Demis Hassabis]], [[DeepMind]], December 03, 2018</ref>
* [[Chao Gao]], [[Siqi Yan]], [[Ryan Hayward]], [[Martin Müller]] ('''2018'''). ''A transferable neural network for Hex''. [[ICGA Journal#40_3|ICGA Journal, Vol. 40, No. 3]]
'''2019'''
* [[Marius Lindauer]], [[Frank Hutter]] ('''2019'''). ''Best Practices for Scientific Research on Neural Architecture Search''. [https://arxiv.org/abs/1909.02453 arXiv:1909.02453]
=Blog & Forum Posts=
* [http://www.talkchess.com/forum/viewtopic.php?t=64096 Is AlphaGo approach unsuitable to chess?] by Mel Cooper, [[CCC]], May 27, 2017 » [[AlphaGo]], [[Deep Learning]], [[Giraffe]]
: [http://www.talkchess.com/forum/viewtopic.php?t=64096&start=12 Re: Is AlphaGo approach unsuitable to chess?] by [[Peter Österlund]], [[CCC]], May 31, 2017 » [[Texel]]
* [https://groups.google.com/d/msg/computer-go-archive/WImAk15gRN4/bhA7kSAnBgAJ Neural nets for Go - chain pooling?] by [[David J. Wu|David Wu]], [https://groups.google.com/forum/#!forum/computer-go-archive Computer Go Archive], August 18, 2017
* [https://deepmind.com/blog/alphago-zero-learning-scratch/ AlphaGo Zero: Learning from scratch] by [[Demis Hassabis]] and [[David Silver]], [[DeepMind]], October 18, 2017
* [http://www.talkchess.com/forum/viewtopic.php?t=65481 We are doomed - AlphaGo Zero, learning only from basic rules] by [[Vincent Lejeune]], [[CCC]], October 18, 2017
* [http://www.talkchess.com/forum/viewtopic.php?t=66681 3 million games for training neural networks] by [[Álvaro Begué]], [[CCC]], February 24, 2018 » [[Automated Tuning]]
* [http://www.talkchess.com/forum/viewtopic.php?t=66791 Looking inside NNs] by [[J. Wesley Cleveland]], [[CCC]], March 09, 2018
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=67347 GPU ANN, how to deal with host-device latencies?] by [[Srdja Matovic]], [[CCC]], May 06, 2018 » [[GPU]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=67524 Poor man's neurones] by [[Pawel Koziol]], [[CCC]], May 21, 2018 » [[Evaluation]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=67600 Egbb dll neural network support] by [[Daniel Shawul]], [[CCC]], May 29, 2018 » [[Scorpio Bitbases]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=68119 Instruction for running Scorpio with neural network on linux] by [[Daniel Shawul]], [[CCC]], August 01, 2018 » [[Scorpio]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=69069 Are draws hard to predict?] by [[Daniel Shawul]], [[CCC]], November 27, 2018 » [[Draw]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=69393 neural network architecture] by jackd, [[CCC]], December 26, 2018
'''2019'''
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=69795 So, how many of you are working on neural networks for chess?] by [[Srdja Matovic]], [[CCC]], February 01, 2019
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=69942 categorical cross entropy for value] by [[Chris Whittington]], [[CCC]], February 18, 2019
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70504 Google's bfloat for neural networks] by [[Srdja Matovic]], [[CCC]], April 16, 2019 » [[Float]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70704 catastrophic forgetting] by [[Daniel Shawul]], [[CCC]], May 09, 2019 » [[Nebiyu]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=71269 Wouldn’t it be nice if there was a ChessNet50] by [[Chris Whittington]], [[CCC]], July 13, 2019
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=71301 A question to MCTS + NN experts] by [[Maksim Korzh]], [[CCC]], July 17, 2019 » [[Monte-Carlo Tree Search]]
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=71301&start=3 Re: A question to MCTS + NN experts] by [[Daniel Shawul]], [[CCC]], July 17, 2019
=External Links=
* [https://en.wikipedia.org/wiki/Neuron Neuron from Wikipedia]
* [https://en.wikipedia.org/wiki/Neural_pathway Neural pathway from Wikipedia]
* [https://en.wikipedia.org/wiki/Artificial_neural_network Artificial neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Types_of_artificial_neural_networks Types of artificial neural networks from Wikipedia]
* [https://en.wikipedia.org/wiki/Helmholtz_machine Helmholtz machine from Wikipedia]
* [http://de.slideshare.net/piuprabhu/chess-end-games-using-neural-networks-presentation Chess end games using Neural Networks]
* [https://en.wikipedia.org/wiki/Artificial_neuron Artificial neuron from Wikipedia]
* [https://en.wikipedia.org/wiki/Connectionism Connectionism from Wikipedia]
* [https://en.wikipedia.org/wiki/Deep_learning Deep Learning from Wikipeadia]
* [http://www.scholarpedia.org/article/Deep_Learning Deep Learning - Scholarpedia] by [[Jürgen Schmidhuber]]
* [https://en.wikipedia.org/wiki/Grossberg_network Grossberg network from Wikipedia]
* [https://en.wikipedia.org/wiki/Modular_neural_network Modular neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Neocognitron Neocognitron from Wikipedia]
* [http://www.scholarpedia.org/article/Neocognitron Neocognitron - Scholarpedia] by [http://www.scholarpedia.org/article/User:Kunihiko_Fukushima Kunihiko Fukushima]
* [https://en.wikipedia.org/wiki/Neural_architecture_search Neural architecture search from Wikipedia]
* [https://en.wikipedia.org/wiki/Neuromorphic_engineering Neuromorphic engineering from Wikipedia]
: [https://en.wikipedia.org/wiki/Neurogrid Neurogrid from Wikipedia]
* [https://en.wikipedia.org/wiki/Physical_neural_network Physical neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Radial_basis_function_network Radial basis function network from Wikipedia]
* [https://en.wikipedia.org/wiki/Spiking_neural_network Spiking neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Time_delay_neural_network Time delay neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Recurrent_neural_network Recurrent neural network from Wikipedia]
* [http://www.scholarpedia.org/article/Recurrent_neural_networks Recurrent neural networks - Scholarpedia]
* [http://www.scholarpedia.org/article/Hopfield_network Hopfield network - Scholarpedia]
* [https://en.wikipedia.org/wiki/Long_short_term_memory Long short term memory from Wikipedia]
* [https://en.wikipedia.org/wiki/Activation_function Activation function from Wikipedia]
* [https://en.wikipedia.org/wiki/Rectifier_(neural_networks) Rectifier (neural networks) from Wikipedia]
* [https://en.wikipedia.org/wiki/Sigmoid_function Sigmoid function from Wikipedia]
* [https://en.wikipedia.org/wiki/Backpropagation Backpropagation from Wikipedia]
* [https://en.wikipedia.org/wiki/Backpropagation_through_structure Backpropagation through structure from Wikipedia]
* [https://en.wikipedia.org/wiki/Rprop Rprop from Wikipedia]
* [http://people.idsia.ch/~juergen/who-invented-backpropagation.html Who Invented Backpropagation?] by [[Jürgen Schmidhuber]] (2014, 2015)
* [https://en.wikipedia.org/wiki/Gradient Gradient from Wikipedia]
* [https://en.wikipedia.org/wiki/Del Del from Wikipedia]
* [https://blogs.princeton.edu/imabandit/2014/03/06/nesterovs-accelerated-gradient-descent-for-smooth-and-strongly-convex-optimization/ Nesterov’s Accelerated Gradient Descent for Smooth and Strongly Convex Optimization] by [[Sébastien Bubeck]], [https://blogs.princeton.edu/imabandit/ I’m a bandit], March 6, 2014
* [https://blogs.princeton.edu/imabandit/2015/06/30/revisiting-nesterovs-acceleration/ Revisiting Nesterov’s Acceleration] by [[Sébastien Bubeck]], [https://blogs.princeton.edu/imabandit/ I’m a bandit], June 30, 2015
* [https://theneural.wordpress.com/ Neural Networks Blog] by [[Ilya Sutskever]]
* [https://software.intel.com/en-us/articles/an-introduction-to-neural-networks-with-an-application-to-games An Introduction to Neural Networks with an Application to Games] by [https://www.linkedin.com/pub/dean-p-macri/a/762/68b Dean Macri], [https://en.wikipedia.org/wiki/Intel_Developer_Zone Intel Developer Zone], September 9, 2011
* [https://blog.waya.ai/deep-residual-learning-9610bb62c355 Understand Deep Residual Networks — a simple, modular learning framework that has redefined state-of-the-art] by [https://blog.waya.ai/@waya.ai Michael Dietz], [https://blog.waya.ai/ Waya.ai], May 02, 2017
* [https://medium.com/applied-data-science/how-to-build-your-own-alphazero-ai-using-python-and-keras-7f664945c188 How to build your own AlphaZero AI using Python and Keras] by [https://www.linkedin.com/in/davidtfoster/ David Foster], January 26, 2018 » [[AlphaZero]], [[Connect Four]], [[Python]]
* [http://www.cedar.buffalo.edu/~srihari/CSE574/index.html Machine Learning and Probabilistic Graphical Models: Course Materials - 5. Neural Networks] by [https://en.wikipedia.org/wiki/Sargur_Srihari Sargur Srihari], [https://en.wikipedia.org/wiki/University_at_Buffalo University at Buffalo]
* [http://www.holehouse.org/mlclass/08_Neural_Networks_Representation.html Neural Networks - Representation] from [http://www.holehouse.org/mlclass/index.html Stanford Machine Learning] by [[Andrew Ng]]
: [https://youtu.be/Ilg3gGewQ5U What is backpropagation really doing? | Chapter 3]
: [https://youtu.be/tIeHLnjs5U8 Backpropagation calculus | Appendix to Chapter 3]
* [https://en.wikipedia.org/wiki/John_Zorn#The_Dreamers The Dreamers] & [[:Category:John Zorn|John Zorn]] - Gormenghast, [https://en.wikipedia.org/wiki/Pellucidar:_A_Dreamers_Fantabula Pellucidar: A Dreamers Fantabula] (2015), [https://en.wikipedia.org/wiki/YouTube YouTube] Video
: [[:Category:Marc Ribot|Marc Ribot]], [https://en.wikipedia.org/wiki/Kenny_Wollesen Kenny Wollesen], [https://en.wikipedia.org/wiki/Joey_Baron Joey Baron], [https://en.wikipedia.org/wiki/Jamie_Saft Jamie Saft], [https://en.wikipedia.org/wiki/Trevor_Dunn Trevor Dunn], [https://en.wikipedia.org/wiki/Cyro_Baptista Cyro Baptista], John Zorn
[[Category:Marc Ribot]]
[[Category:John Zorn]]
[[Category:Videos]]
