Changes

Newer edit →

Neural Networks

126,690 bytes added, 10:55, 19 May 2018

Created page with "'''Home * Learning * Neural Networks''' FILE:Artificial neural network.svg|border|right|thumb|Artificial Neural Network <ref>An example artificial neural..."

'''[[Main Page|Home]] * [[Learning]] * Neural Networks'''

[[FILE:Artificial neural network.svg|border|right|thumb|Artificial Neural Network <ref>An example artificial neural network with a hidden layer, Image by [https://en.wikipedia.org/wiki/User:Cburnett Colin M.L. Burnett] with [https://en.wikipedia.org/wiki/Inkscape Inkscape], December 27, 2006, [https://creativecommons.org/licenses/by-sa/3.0/deed.en CC BY-SA 3.0], [https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Neural_Network_Basics Artificial Neural Networks/Neural Network Basics - Wikibooks], [https://en.wikipedia.org/wiki/Wikimedia_Commons Wikimedia Commons]</ref> ]]

'''Neural Networks''', 
a series of connected [https://en.wikipedia.org/wiki/Neuron neurons] which communicate due to [https://en.wikipedia.org/wiki/Neurotransmitter neurotransmission]. The interface through which neurons interact with their neighbors consists of [https://en.wikipedia.org/wiki/Axon_terminal axon terminals] connected via [https://en.wikipedia.org/wiki/Synapse synapses] to [https://en.wikipedia.org/wiki/Dendrite dendrites] on other neurons. If the sum of the input signals into one neuron surpasses a certain threshold, the neuron sends an [https://en.wikipedia.org/wiki/Action_potential action potential] at the [https://en.wikipedia.org/wiki/Axon_hillock axon hillock] and transmits this electrical signal along the [https://en.wikipedia.org/wiki/Axon axon].

In 1949, [https://en.wikipedia.org/wiki/Donald_O._Hebb Donald O. Hebb] introduced his [https://en.wikipedia.org/wiki/Hebbian_theory theory] in ''[https://en.wikipedia.org/wiki/The_Organization_of_Behavior The Organization of Behavior]'', stating that [[Learning|learning]] is about to adapt weight vectors (persistent [https://en.wikipedia.org/wiki/Synaptic_plasticity synaptic plasticity]) of the neuron pre-synaptic inputs, whose dot-product activates or controls the post-synaptic output, which is the base of Neural network learning <ref>[https://en.wikipedia.org/wiki/Biological_neural_network#Early_study Biological neural network - Early study - from Wikipedia]</ref>.

=AN=
Already in the early 40s, [https://en.wikipedia.org/wiki/Warren_Sturgis_McCulloch Warren S. McCulloch] and [https://en.wikipedia.org/wiki/Walter_Pitts Walter Pitts] introduced the [https://en.wikipedia.org/wiki/Artificial_neuron artificial neuron] as a logical element with multiple analogue inputs and a single digital output with a boolean result. The output fired "true", if the sum of the inputs exceed a threshold. In their 1943 paper ''A Logical Calculus of the Ideas Immanent in Nervous Activity'' <ref>[https://en.wikipedia.org/wiki/Warren_Sturgis_McCulloch Warren S. McCulloch], [https://en.wikipedia.org/wiki/Walter_Pitts Walter Pitts] ('''1943'''). ''[http://link.springer.com/article/10.1007%2FBF02478259 A Logical Calculus of the Ideas Immanent in Nervous Activity]''. [http://link.springer.com/journal/11538 Bulletin of Mathematical Biology], Vol. 5, No. 1, [http://www.aemea.org/math/McCulloch_Pitts_1943.pdf pdf]</ref>, they attempted to demonstrate that a [[Alan Turing#TuringMachine|Turing machine]] program could be implemented in a finite network of such neurons of [[Combinatorial Logic|combinatorial logic]] functions of [[Combinatorial Logic#AND|AND]], [[Combinatorial Logic#OR|OR]] and [[Combinatorial Logic#NOT|NOT]].

=ANNs=
[https://en.wikipedia.org/wiki/Artificial_neural_network Artificial Neural Networks] ('''ANNs''') are a family of [https://en.wikipedia.org/wiki/Machine_learning statistical learning] devices or algorithms used in [https://en.wikipedia.org/wiki/Regression_analysis regression], and [https://en.wikipedia.org/wiki/Binary_classification binary] or [[multiclass classification|multiclass classification]], implemented in [[Hardware|hardware]] or [[Software|software]] inspired by their biological counterparts. The [https://en.wikipedia.org/wiki/Artificial_neuron artificial neurons] of one or more layers receive one or more inputs (representing dendrites), and after being weighted, sum them to produce an output (representing a neuron's axon). The sum is passed through a [https://en.wikipedia.org/wiki/Nonlinear_system nonlinear] function known as an [https://en.wikipedia.org/wiki/Activation_function activation function] or transfer function. The transfer functions usually have a [https://en.wikipedia.org/wiki/Sigmoid_function sigmoid shape], but they may also take the form of other non-linear functions, [https://en.wikipedia.org/wiki/Piecewise piecewise] linear functions, or [https://en.wikipedia.org/wiki/Artificial_neuron#Step_function step functions] <ref>[https://en.wikipedia.org/wiki/Artificial_neuron Artificial neuron from Wikipedia]</ref>. The weights of the inputs of each layer are tuned to minimize a [https://en.wikipedia.org/wiki/Loss_function cost or loss function], which is a task in [https://en.wikipedia.org/wiki/Mathematical_optimization mathematical optimization] and machine learning.

==Perceptron==
[[FILE:Perceptron.svg|border|right|thumb|300px| Perceptron <ref>The appropriate weights are applied to the inputs, and the resulting weighted sum passed to a function that produces the output y, image created by [https://en.wikipedia.org/wiki/User:Mat_the_w mat_the_w], based on [https://en.wikipedia.org/wiki/Raster_graphics raster image] [http://commons.wikimedia.org/wiki/File:Perceptron.gif Perceptron.gif] by '[https://en.wikipedia.org/wiki/User:Paskari Paskari]', using [https://en.wikipedia.org/wiki/Inkscape Inkscape] 0.46 for [[Mac OS|OSX]], [https://en.wikipedia.org/wiki/Wikimedia_Commons Wikimedia Commons], [https://en.wikipedia.org/wiki/Perceptron Perceptron from Wikipedia]</ref> ]]
The [https://en.wikipedia.org/wiki/Perceptron perceptron] is an algorithm for [[Supervised Learning|supervised learning]] of [https://en.wikipedia.org/wiki/Binary_classification binary classifiers]. It was the first artificial neural network, introduced in 1957 by [https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] <ref>[https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] ('''1957'''). ''The Perceptron - a Perceiving and Recognizing Automaton''. Report 85-460-1, [https://en.wikipedia.org/wiki/Calspan#History Cornell Aeronautical Laboratory]</ref>, implemented in custom hardware. In its basic form it consists of a single neuron with multiple inputs and associated weights.

[[Supervised learning]] is applied using a set D of labeled [https://en.wikipedia.org/wiki/Test_set training data] with pairs of [https://en.wikipedia.org/wiki/Feature_vector feature vectors] (x) and given results as desired output (d), usually started with cleared or randomly initialized weight vector w. The output is calculated by all inputs of a sample, multiplied by its corresponding weights, passing the sum to the activation function f. The difference of desired and actual value is then immediately used modify the weights for all features using a learning rate 0.0 < α <= 1.0:
<pre>
for (j=0, Σ = 0.0; j < nSamples; ++j) {
for (i=0, X = bias; i < nFeatures; ++i)
X += w[i]*x[j][i];
y = f ( X );
Σ += abs(Δ = d[j] - y);
for (i=0; i < nFeatures; ++i)
w[i] += α*Δ*x[j][i];
}
</pre>

==AI Winter==
[[FILE:XOR perceptron net.png|border|right|thumb|Three layer, XOR capable Perceptron <ref>A two-layer neural network capable of calculating XOR. The numbers within the neurons represent each neuron's explicit threshold (which can be factored out so that all neurons have the same threshold, usually 1). The numbers that annotate arrows represent the weight of the inputs. This net assumes that if the threshold is not reached, zero (not -1) is output. Note that the bottom layer of inputs is not always considered a real neural network layer, [https://en.wikipedia.org/wiki/Feedforward_neural_network Feedforward neural network from Wikipedia]</ref> ]]
Although the perceptron initially seemed promising, it was proved that perceptrons could not be trained to recognise many classes of patterns. This led to neural network research stagnating for many years, the [https://en.wikipedia.org/wiki/AI_winter#The_abandonment_of_connectionism_in_1969 AI-winter], before it was recognised that a [https://en.wikipedia.org/wiki/Feedforward_neural_network feedforward neural network] with two or more layers had far greater processing power than with one layer. Single layer perceptrons are only capable of learning [https://en.wikipedia.org/wiki/Linear_separability linearly separable] patterns. In their 1969 book ''[https://en.wikipedia.org/wiki/Perceptrons_%28book%29 Perceptrons]'', [[Marvin Minsky]] and [[Mathematician#SPapert|Seymour Papert]] wrote that it was impossible for these classes of network to learn the [[Combinatorial Logic#XOR|XOR function]]. It is often believed that they also conjectured (incorrectly) that a similar result would hold for a [https://en.wikipedia.org/wiki/Multilayer_perceptron multilayer perceptron] <ref>[https://en.wikipedia.org/wiki/Multilayer_perceptron multilayer perceptron] is a misnomer for a more complicated neural network</ref>. However, this is not true, as both Minsky and Papert already knew that multilayer perceptrons were capable of producing an XOR function <ref>[https://en.wikipedia.org/wiki/Perceptron#History Perceptron from Wikipedia]</ref>-

==Backpropagation==
In 1974, [https://en.wikipedia.org/wiki/Paul_Werbos Paul Werbos] started to end the AI winter concerning neural networks, when he first described the mathematical process of training [https://en.wikipedia.org/wiki/Multilayer_perceptron multilayer perceptrons] through [https://en.wikipedia.org/wiki/Backpropagation backpropagation] of errors <ref>[https://en.wikipedia.org/wiki/Paul_Werbos Paul Werbos] ('''1974'''). ''[http://aitopics.org/publication/beyond-regression-new-tools-prediction-and-analysis-behavioral-sciences Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences]''. Ph. D. thesis, [[Harvard University]]</ref>, derived in the context of [https://en.wikipedia.org/wiki/Control_theory control theory] by [https://en.wikipedia.org/wiki/Henry_J._Kelley Henry J. Kelley] in 1960 <ref>[https://en.wikipedia.org/wiki/Henry_J._Kelley Henry J. Kelley] ('''1960'''). ''[http://arc.aiaa.org/doi/abs/10.2514/8.5282?journalCode=arsj& Gradient Theory of Optimal Flight Paths]''. [http://arc.aiaa.org/loi/arsj ARS Journal, Vol. 30, No. 10</ref> and by [[https://en.wikipedia.org/wiki/Arthur_E._Bryson|Arthur E. Bryson] in 1961 <ref>[https://en.wikipedia.org/wiki/Arthur_E._Bryson Arthur E. Bryson] ('''1961'''). ''A gradient method for optimizing multi-stage allocation processes''. In Proceedings of the [[Harvard University]] Symposium on digital computers and their applications</ref> using principles of [[Dynamic Programming|dynamic programming]], simplified by [https://en.wikipedia.org/wiki/Stuart_Dreyfus Stuart Dreyfus] in 1961 applying the [https://en.wikipedia.org/wiki/Chain_rule chain rule] <ref>[https://en.wikipedia.org/wiki/Stuart_Dreyfus Stuart Dreyfus] ('''1961'''). ''[http://www.rand.org/pubs/papers/P2374.html The numerical solution of variational problems]''. RAND paper P-2374</ref>. It was in 1982, when Werbos applied a [https://en.wikipedia.org/wiki/Automatic_differentiation automatic differentiation] method described in 1970 by [[Mathematician#SLinnainmaa|Seppo Linnainmaa]] <ref>[[Mathematician#SLinnainmaa|Seppo Linnainmaa]] ('''1970'''). ''The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors''. Master's thesis, [https://en.wikipedia.org/wiki/University_of_Helsinki University of Helsinki]</ref> to neural networks in the way that is widely used today <ref>[https://en.wikipedia.org/wiki/Paul_Werbos Paul Werbos] ('''1982'''). ''Applications of advances in nonlinear sensitivity analysis''. [http://link.springer.com/book/10.1007%2FBFb0006119 System Modeling and Optimization], [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [http://werbos.com/Neural/SensitivityIFIPSeptember1981.pdf pdf]</ref> <ref>[https://en.wikipedia.org/wiki/Paul_Werbos Paul Werbos] ('''1994'''). ''[http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471598976.html The Roots of Backpropagation. From Ordered Derivatives to Neural Networks and Political Forecasting]''. [https://en.wikipedia.org/wiki/John_Wiley_%26_Sons John Wiley & Sons]</ref> <ref>[http://www.scholarpedia.org/article/Deep_Learning#Backpropagation Deep Learning - Scholarpedia | Backpropagation] by [[Jürgen Schmidhuber]]</ref> <ref>[http://people.idsia.ch/~juergen/who-invented-backpropagation.html Who Invented Backpropagation?] by [[Jürgen Schmidhuber]] (2014, 2015)</ref>.

Backpropagation is a generalization of the [https://en.wikipedia.org/wiki/Delta_rule delta] rule to multilayered [https://en.wikipedia.org/wiki/Feedforward_neural_network feedforward networks], made possible by using the [https://en.wikipedia.org/wiki/Chain_rule chain rule] to iteratively compute [https://en.wikipedia.org/wiki/Gradient gradients] for each layer. Backpropagation requires that the [https://en.wikipedia.org/wiki/Activation_function activation function] used by the artificial neurons be [https://en.wikipedia.org/wiki/Differentiable_function differentiable], which is true for the common [https://en.wikipedia.org/wiki/Sigmoid_function sigmoid] [https://en.wikipedia.org/wiki/Logistic_function logistic function] or its [https://en.wikipedia.org/wiki/Softmax_function softmax] generalization in [https://en.wikipedia.org/wiki/Multiclass_classification multiclass classification].

Along with an [https://en.wikipedia.org/wiki/Mathematical_optimization optimization method] such as [https://en.wikipedia.org/wiki/Gradient_descent gradient descent], it calculates the gradient of a [https://en.wikipedia.org/wiki/Loss_function cost or loss function] with respect to all the weights in the neural network. The gradient is fed to the optimization method which in turn uses it to update the weights, in an attempt to minimize the loss function, which choice depends on the learning type (supervised, unsupervised, reinforcement) and the activation function - [https://en.wikipedia.org/wiki/Mean_squared_error mean squared error] or [https://en.wikipedia.org/wiki/Cross_entropy#Cross-entropy_error_function_and_logistic_regression cross-entropy error function] are used in [https://en.wikipedia.org/wiki/Binary_classification binary classification] <ref>"Using [https://en.wikipedia.org/wiki/Cross_entropy#Cross-entropy_error_function_and_logistic_regression cross-entropy error function] instead of [https://en.wikipedia.org/wiki/Mean_squared_error sum of squares] leads to faster training and improved generalization", from [https://en.wikipedia.org/wiki/Sargur_Srihari Sargur Srihari], [http://www.cedar.buffalo.edu/~srihari/CSE574/Chap5/Chap5.2-Training.pdf Neural Network Training] (pdf)</ref>. The gradient is almost always used in a simple [https://en.wikipedia.org/wiki/Stochastic_gradient_descent stochastic gradient descent] algorithm. In 1983, [[Mathematician#YNesterov|Yurii Nesterov]] contributed an accelerated version of gradient descent that converges considerably faster than ordinary gradient descent <ref>[https://en.wikipedia.org/wiki/Yurii_Nesterov Yurii Nesterov from Wikipedia]</ref> <ref>[https://blogs.princeton.edu/imabandit/2013/04/01/acceleratedgradientdescent/ ORF523: Nesterov’s Accelerated Gradient Descent] by [[Sébastien Bubeck]], [https://blogs.princeton.edu/imabandit/ I’m a bandit], April 1, 2013</ref> <ref>[https://blogs.princeton.edu/imabandit/2014/03/06/nesterovs-accelerated-gradient-descent-for-smooth-and-strongly-convex-optimization/ Nesterov’s Accelerated Gradient Descent for Smooth and Strongly Convex Optimization] by [[Sébastien Bubeck]], [https://blogs.princeton.edu/imabandit/ I’m a bandit], March 6, 2014</ref> <ref>[https://blogs.princeton.edu/imabandit/2015/06/30/revisiting-nesterovs-acceleration/ Revisiting Nesterov’s Acceleration] by [[Sébastien Bubeck]], [https://blogs.princeton.edu/imabandit/ I’m a bandit], June 30, 2015</ref>.

Backpropagation algorithm for a 3-layer network <ref>[https://en.wikipedia.org/wiki/Backpropagation#Algorithm Backpropagation algorithm from Wikipedia]</ref>:
<pre>
initialize the weights in the network (often small random values)
do
for each example e in the training set
O = neural-net-output(network, e) // forward pass
T = teacher output for e
compute error (T - O) at the output units
compute delta_wh for all weights from hidden layer to output layer // backward pass
compute delta_wi for all weights from input layer to hidden layer // backward pass continued
update the weights in the network
until all examples classified correctly or stopping criterion satisfied
return the network
</pre>

==Deep Learning==
[[Deep Learning|Deep learning]] has been characterized as a [https://en.wikipedia.org/wiki/Buzzword buzzword], or a rebranding of neural networks. A [https://en.wikipedia.org/wiki/Deep_learning#Deep_neural_networks deep neural network] (DNN) is an ANN with multiple hidden layers of units between the input and output layers which can be [https://en.wikipedia.org/wiki/Discriminative_model discriminatively] trained with the standard backpropagation algorithm. Two common issues if naively trained are [https://en.wikipedia.org/wiki/Overfitting overfitting] and computation time.

==Convolutional NNs==
[https://en.wikipedia.org/wiki/Convolutional_neural_network Convolutional neural networks] form a subclass of feedforward neural networks that have special weight constraints, individual neurons are tiled in such a way that they respond to overlapping regions. A neuron of a convolutional layer is connected to a correspondent [https://en.wikipedia.org/wiki/Receptive_field receptive field] of the previous layer, a small subset of their neurons. Convolutional NNs are suited for deep learning and are highly suitable for parallelization on [[GPU|GPUs]] <ref>[http://parse.ele.tue.nl/education/cluster2 PARsE | Education | GPU Cluster | Efficient mapping of the training of Convolutional Neural Networks to a CUDA-based cluster]</ref>. They were [[Go#CNN|research topic]] in the game of [[Go]] since 2008 <ref>[[Ilya Sutskever]], [[Vinod Nair]] ('''2008'''). ''Mimicking Go Experts with Convolutional Neural Networks''. [http://dblp.uni-trier.de/db/conf/icann/icann2008-2.html#SutskeverN08 ICANN 2008], [http://www.cs.utoronto.ca/~ilya/pubs/2008/go_paper.pdf pdf]</ref>, and along with the [[Neural Networks#Residual|residual]] modification successful applied in [[Go]] and other [[Games|games]], most spectacular due to [[AlphaGo]] in 2015 and [[AlphaZero]] in 2017.

[[FILE:Typical_cnn.png|none|border|text-bottom|link=https://commons.wikimedia.org/wiki/File:Typical_cnn.png]]
Typical CNN <ref>Typical [https://en.wikipedia.org/wiki/Convolutional_neural_network CNN] architecture, Image by Aphex34, December 16, 2015, [https://creativecommons.org/licenses/by-sa/4.0/deed.en CC BY-SA 4.0], [https://en.wikipedia.org/wiki/Wikimedia_Commons Wikimedia Commons]</ref>

==Residual Nets==
[[FILE:ResiDualBlock.png|border|right|thumb|link=https://arxiv.org/abs/1512.03385| A residual block <ref>The fundamental building block of residual networks. Figure 2 in [https://scholar.google.com/citations?user=DhtAFkwAAAAJ Kaiming He], [https://scholar.google.com/citations?user=yuB-cfoAAAAJ&hl=en Xiangyu Zhang], [http://shaoqingren.com/ Shaoqing Ren], [http://www.jiansun.org/ Jian Sun] ('''2015'''). ''Deep Residual Learning for Image Recognition''. [https://arxiv.org/abs/1512.03385 arXiv:1512.03385]</ref> <ref>[https://blog.waya.ai/deep-residual-learning-9610bb62c355 Understand Deep Residual Networks — a simple, modular learning framework that has redefined state-of-the-art] by [https://blog.waya.ai/@waya.ai Michael Dietz], [https://blog.waya.ai/ Waya.ai], May 02, 2017</ref> ]]
'''Residual nets''' add the input of a layer, typically composed of a convolutional layer and of a [https://en.wikipedia.org/wiki/Rectifier_(neural_networks) ReLU] layer, to its output. This modification, like convolutional nets inspired from image classification, enables faster training and deeper networks <ref>[[Tristan Cazenave]] ('''2017'''). ''[http://ieeexplore.ieee.org/document/7875402/ Residual Networks for Computer Go]''. [[IEEE#TOCIAIGAMES|IEEE Transactions on Computational Intelligence and AI in Games]], Vol. PP, No. 99, [http://www.lamsade.dauphine.fr/~cazenave/papers/resnet.pdf pdf]</ref> <ref>[https://wiki.tum.de/display/lfdv/Deep+Residual+Networks Deep Residual Networks] from [https://wiki.tum.de/ TUM Wiki], [[Technical University of Munich]]</ref>.

=ANNs in Games=
Applications of neural networks in computer games and chess are [[Learning|learning]] of [[Evaluation|evaluation]] and [[Search|search]] control. Evaluation topics include [https://en.wikipedia.org/wiki/Feature_selection feature selection] and [[Automated Tuning|automated tuning]], search control [[Move Ordering|move ordering]], [[Selectivity|selectivity]] and [[Time Management|time management]]. The [[Neural Networks#Perceptron|perceptron]] looks like the ideal learning algorithm for [[Automated Tuning|automated evaluation tuning]].

==Backgammon==
In the late 80s, [[Gerald Tesauro]] pioneered in applying ANNs to the game of [[Backgammon]]. His program [https://en.wikipedia.org/wiki/Neurogammon Neurogammon] won the Gold medal at the [[1st Computer Olympiad]] 1989 - and was further improved by ''TD-Lambda'' based [[Temporal Difference Learning]] within [https://en.wikipedia.org/wiki/TD-Gammon TD-Gammon] <ref>[[Richard Sutton]], [[Andrew Barto]] ('''1998'''). ''[http://www.incompleteideas.net/sutton/book/the-book.html Reinforcement Learning: An Introduction]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press], [http://www.incompleteideas.net/sutton/book/ebook/node108.html 11.1 TD-Gammon]</ref>. Today all strong backgammon programs rely on heavily trained neural networks.

==Go==
In 2014, two teams independently investigated whether deep [[Neural Networks#Convolutional|convolutional neural networks]] could be used to directly represent and [[Learning|learn]] a move evaluation function for the game of [[Go]]. [[Christopher Clark]] and [[Amos Storkey]] trained an 8-layer convolutional neural network by [[Supervised Learning|supervised learning]] from a database of human professional games, which without any [[Search|search]], defeated the traditional search program [[Gnu Go]] in 86% of the games <ref>[[Christopher Clark]], [[Amos Storkey]] ('''2014'''). ''Teaching Deep Convolutional Neural Networks to Play Go''. [http://arxiv.org/abs/1412.3409 arXiv:1412.3409]</ref> <ref>[http://computer-go.org/pipermail/computer-go/2014-December/007010.html Teaching Deep Convolutional Neural Networks to Play Go] by [[Hiroshi Yamashita]], [http://computer-go.org/pipermail/computer-go/ The Computer-go Archives], December 14, 2014</ref> <ref>[http://www.technologyreview.com/view/533496/why-neural-networks-look-set-to-thrash-the-best-human-go-players-for-the-first-time/ Why Neural Networks Look Set to Thrash the Best Human Go Players for the First Time] | [https://en.wikipedia.org/wiki/MIT_Technology_Review MIT Technology Review], December 15, 2014</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=54663 Teaching Deep Convolutional Neural Networks to Play Go] by [[Michel Van den Bergh]], [[CCC]], December 16, 2014</ref>. In their paper ''Move Evaluation in Go Using Deep Convolutional Neural Networks'' <ref>[[Chris J. Maddison]], [[Shih-Chieh Huang|Aja Huang]], [[Ilya Sutskever]], [[David Silver]] ('''2014'''). ''Move Evaluation in Go Using Deep Convolutional Neural Networks''. [http://arxiv.org/abs/1412.6564v1 arXiv:1412.6564v1]</ref>, [[Chris J. Maddison]], [[Shih-Chieh Huang|Aja Huang]], [[Ilya Sutskever]], and [[David Silver]] report they trained a large 12-layer convolutional neural network in a similar way, to beat Gnu Go in 97% of the games, and matched the performance of a state-of-the-art [[Monte-Carlo Tree Search]] that simulates a million positions per move <ref>[http://computer-go.org/pipermail/computer-go/2014-December/007046.html Move Evaluation in Go Using Deep Convolutional Neural Networks] by [[Shih-Chieh Huang|Aja Huang]], [http://computer-go.org/pipermail/computer-go/ The Computer-go Archives], December 19, 2014</ref>.

In 2015, a team affiliated with [[Google]] [[DeepMind]] around [[David Silver]] and [[Shih-Chieh Huang|Aja Huang]], supported by [[Google]] researchers [[John Nham]] and [[Ilya Sutskever]], build a Go playing program dubbed [[AlphaGo]] <ref>[http://deepmind.com/alpha-go.html AlphaGo | Google DeepMind]</ref>, combining Monte-Carlo tree search with their 12-layer networks <ref>[[David Silver]], [[Shih-Chieh Huang|Aja Huang]], [[Chris J. Maddison]], [[Arthur Guez]], [[Laurent Sifre]], [[George van den Driessche]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Veda Panneershelvam]], [[Marc Lanctot]], [[Sander Dieleman]], [[Dominik Grewe]], [[John Nham]], [[Nal Kalchbrenner]], [[Ilya Sutskever]], [[Timothy Lillicrap]], [[Madeleine Leach]], [[Koray Kavukcuoglu]], [[Thore Graepel]], [[Demis Hassabis]] ('''2016'''). ''[http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html Mastering the game of Go with deep neural networks and tree search]''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 529</ref>.

==Chess==
[[Automated Tuning#LogisticRegression|Logistic regression]] as applied in [[Texel's Tuning Method]] may be interpreted as [[Supervised Learning|supervised learning]] application of the single-layer perceptron with one neuron. This is also true for [[Reinforcement Learning|reinforcement learning]] approaches, such as [[Temporal Difference Learning#TDLeaf|TD-Leaf]] in [[KnightCap]] or [[Meep|Meep's]] [[Meep#TreeStrap|TreeStrap]], where the evaluation consists of a weighted linear combination of features. Despite these similarities with the perceptron, these engines are not considered using ANNs - since they use manually selected chess specific feature construction concepts like [[Material|material]], [[Piece-Square Tables|piece square tables]], [[Pawn Structure|pawn structure]], [[Mobility|mobility]] etc..

More sophisticated attempts to replace static evaluation by neural networks and perceptrons feeding in more unaffiliated feature sets like [[Board Representation|board representation]] and [[Attack and Defend Maps|attack tables]] etc., where not yet that successful like in other games. Chess evaluation seems not that well suited for neural nets, but there are also aspects of too weak models and feature recognizers as addressed by [[Gian-Carlo Pascutto]] with [[Stoofvlees]] <ref>[http://www.talkchess.com/forum/viewtopic.php?topic_view=threads&p=316511&t=31545 Re: Chess program with Artificial Neural Networks (ANN)?] by [[Gian-Carlo Pascutto]], [[CCC]], January 07, 2010</ref>, huge training effort, and weak [[Float|floating point]] performance - but there is still hope due to progress in hardware and parallelization using [[SIMD and SWAR Techniques|SIMD instructions]] and [[GPU|GPUs]], and deeper and more powerful neural network structures and methods successful in other domains. In December 2017, [[Google]] [[DeepMind]] published about their generalized [[Neural Networks#AlphaZero|AlphaZero]] algorithm.

===Move Ordering===
Concerning [[Move Ordering|move ordering]] - there were interesting NN proposals like the [[Chessmaps Heuristic]] by [[Kieran Greer]] et al. <ref>[[Kieran Greer]], [[Piyush Ojha]], [[David A. Bell]] ('''1999'''). ''A Pattern-Oriented Approach to Move Ordering: the Chessmaps Heuristic''. [[ICGA Journal#22_1|ICCA Journal, Vol. 22, No. 1]]</ref>, and the [[Neural MoveMap Heuristic]] by [[Levente Kocsis]] et al. <ref>[[Levente Kocsis]], [[Jos Uiterwijk]], [[Eric Postma]], [[Jaap van den Herik]] ('''2002'''). ''[http://link.springer.com/chapter/10.1007%2F978-3-540-40031-8_11 The Neural MoveMap Heuristic in Chess]''. [[CG 2002]]</ref>.

===Giraffe & Zurichess===
In 2015, [[Matthew Lai]] trained [[Giraffe|Giraffe's]] [[Neural Networks#Deep|deep neural network]] by [[Temporal Difference Learning#TDLeaf|TD-Leaf]] <ref>[http://talkchess.com/forum/viewtopic.php?t=56913 *First release* Giraffe, a new engine based on deep learning] by [[Matthew Lai]], [[CCC]], July 08, 2015</ref>. [[Zurichess]] by [[Alexandru Mosoi|Alexandru Moșoi]] uses the [https://en.wikipedia.org/wiki/TensorFlow TensorFlow] library for [[Automated Tuning|automated tuning]] - in a two layers neural network, the second layer is responsible for a [[Tapered Eval|tapered eval]] to phase [[Endgame|endgame]] and [[Middlegame|middlegame]] [[Score|scores]] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=60883&start=1 Re: Deep Learning Chess Engine ?] by [[Alexandru Mosoi]], [[CCC]], July 21, 2016</ref>.

===DeepChess===
In 2016, [[Omid David|Omid E. David]], [[Nathan S. Netanyahu]], and [[Lior Wolf]] introduced [[DeepChess]] obtaining a grandmaster-level chess playing performance using a learning method incorporating two deep neural networks, which are trained using a combination of unsupervised pretraining and supervised training. The unsupervised training extracts high level features from a given [[Chess Position|chess position]], and the supervised training learns to compare two chess positions to select the more favorable one. In order to use DeepChess inside a chess program, a novel version of [[Alpha-Beta|alpha-beta]] is used that does not require [[Bound|bounds]] but positions αpos and βpos <ref>[[Omid David|Omid E. David]], [[Nathan S. Netanyahu]], [[Lior Wolf]] ('''2016'''). ''[http://link.springer.com/chapter/10.1007%2F978-3-319-44781-0_11 DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess]''. [http://icann2016.org/ ICAAN 2016], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science], Vol. 9887, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [http://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf pdf preprint]</ref>.

===Alpha Zero===
In December 2017, the [[Google]] [[DeepMind]] team along with former [[Giraffe]] author [[Matthew Lai]] reported on their generalized [[AlphaZero]] algorithm, combining [[Deep Learning|Deep learning]] with [[Monte-Carlo Tree Search]]. AlphaZero can achieve, tabula rasa, superhuman performance in many challenging domains with some training effort. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved a superhuman level of play in the games of chess and [[Shogi]] as well as Go, and convincingly defeated a world-champion program in each case <ref>[[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2017'''). ''Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm''. [https://arxiv.org/abs/1712.01815 arXiv:1712.01815]</ref>.

=See also=
* [[Analog Evaluation]]
* [[Backgammon]]
* [[Chessmaps Heuristic]]
* [[Go#CNN|Convolutional Neural Networks in Go]]
: [[AlphaGo]]
: [[CG 2016#Keynote|Keynote Lecture CG 2016 Conference]] by [[Shih-Chieh Huang|Aja Huang]]
* [[Cognition]]
* [[Deep Learning]]
* [[DeepChess]]
* [[Genetic Programming]]
* [[Memory]]
* [[Neural MoveMap Heuristic]]
* [[Pattern Recognition]]
* [[Temporal Difference Learning]]

=NN Chess Programs=
* [[Alexs]]
* [[AlphaZero]]
* [[Arminius]]
* [[Blondie25]]
* [[ChessMaps]]
* [[Chessterfield]]
* [[Deep Pink]]
* [[Giraffe]]
* [[Golch]]
* [[Gosu]]
* [[Hermann]]
* [[LCZero]]
* [[Morph]]
* [[NeuroChess]]
* [[Octavius]]
* [[SAL]]
* [[Spawkfish]]
* [[Stoofvlees]]
* [[Tempo (engine)|Tempo]]
* [[Zurichess]]

=Selected Publications=
==1940 ...==
* [https://en.wikipedia.org/wiki/Walter_Pitts Walter Pitts] ('''1942'''). ''[http://link.springer.com/article/10.1007%2FBF02477942 Some observations on the simple neuron circuit]''. [http://link.springer.com/journal/11538 Bulletin of Mathematical Biology], Vol. 4, No. 3
* [https://en.wikipedia.org/wiki/Warren_Sturgis_McCulloch Warren S. McCulloch], [https://en.wikipedia.org/wiki/Walter_Pitts Walter Pitts] ('''1943'''). ''[http://link.springer.com/article/10.1007%2FBF02478259 A Logical Calculus of the Ideas Immanent in Nervous Activity]''. [http://link.springer.com/journal/11538 Bulletin of Mathematical Biology], Vol. 5, No. 1, [http://www.aemea.org/math/McCulloch_Pitts_1943.pdf pdf]
* [https://en.wikipedia.org/wiki/Donald_O._Hebb Donald O. Hebb] ('''1949'''). ''[https://en.wikipedia.org/wiki/The_Organization_of_Behavior The Organization of Behavior]''. [https://en.wikipedia.org/wiki/John_Wiley_%26_Sons Wiley & Sons]
==1950 ...==
* [[Mathematician#SCKleene|Stephen C. Kleene]] ('''1951''') ''Representation of Events in Nerve Nets and Finite Automata''. RM-704, [https://en.wikipedia.org/wiki/RAND_Corporation RAND paper], [http://www.rand.org/content/dam/rand/pubs/research_memoranda/2008/RM704.pdf pdf], reprinted in
: [[Claude Shannon]], [[John McCarthy]] (eds.) ('''1956'''). ''Automata Studies''. [http://press.princeton.edu/math/series/amh.html Annals of Mathematics Studies], No. 34
* [[Marvin Minsky]] ('''1954'''). ''Neural Nets and the Brain Model Problem''. Ph.D. dissertation, [https://en.wikipedia.org/wiki/Princeton_University Princeton University]
* [http://dblp.uni-trier.de/pers/hd/f/Farley:B=_G= B. G. Farley], [http://dblp.uni-trier.de/pers/hd/c/Clark:W=_A= W. A. Clark] ('''1954'''). ''Simulation of Self-Organizing Systems by Digital Computer''. [http://dblp.uni-trier.de/db/journals/tit/tit4.html#FarleyC54 IRE Transactions on Information Theory, Vol. 4]
* [[John von Neumann]] ('''1956'''). ''Probabilistic Logic and the Synthesis of Reliable Organisms From Unreliable Components''. in
: [[Claude Shannon]], [[John McCarthy]] (eds.) ('''1956'''). ''Automata Studies''. [http://press.princeton.edu/math/series/amh.html Annals of Mathematics Studies], No. 34, [http://www.dna.caltech.edu/courses/cs191/paperscs191/VonNeumann56.pdf pdf]
* [[Nathaniel Rochester]], [[Mathematician#Holland|John H. Holland]], [http://dblp.uni-trier.de/pers/hd/h/Haibt:L=_H= L. H. Haibt], [http://dblp.uni-trier.de/pers/hd/d/Duda:W=_L= W. L. Duda] ('''1956'''). ''Tests on a Cell Assembly Theory of the Action of the Brain, Using a Large Digital Computer''. [http://dblp.uni-trier.de/db/journals/tit/tit2n.html#RochesterHHD56 IRE Transactions on Information Theory, Vol. 2], No. 3
* [https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] ('''1957'''). ''The Perceptron - a Perceiving and Recognizing Automaton''. Report 85-460-1, [https://en.wikipedia.org/wiki/Calspan#History Cornell Aeronautical Laboratory] <ref>[http://csis.pace.edu/~ctappert/srd2011/rosenblatt-contributions.htm Rosenblatt's Contributions]</ref>
==1960 ...==
* [https://en.wikipedia.org/wiki/Henry_J._Kelley Henry J. Kelley] ('''1960'''). ''[http://arc.aiaa.org/doi/abs/10.2514/8.5282?journalCode=arsj& Gradient Theory of Optimal Flight Paths]''. [http://arc.aiaa.org/loi/arsj ARS Journal, Vol. 30, No. 10 » [[Neural Networks#Backpropagation|Backpropagation]
* [https://en.wikipedia.org/wiki/Arthur_E._Bryson Arthur E. Bryson] ('''1961'''). ''A gradient method for optimizing multi-stage allocation processes''. In Proceedings of the [[Harvard University]] Symposium on digital computers and their applications » [[Neural Networks#Backpropagation|Backpropagation]]
* [https://en.wikipedia.org/wiki/Stuart_Dreyfus Stuart Dreyfus] ('''1961'''). ''[http://www.rand.org/pubs/papers/P2374.html The numerical solution of variational problems]''. RAND paper P-2374 » [[Neural Networks#Backpropagation|Backpropagation]]
* [https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] ('''1962'''). ''[http://catalog.hathitrust.org/Record/000203591 Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms]''. Spartan Books
* [https://en.wikipedia.org/wiki/Alexey_Grigorevich_Ivakhnenko Alexey G. Ivakhnenko] ('''1965'''). ''Cybernetic Predicting Devices''. [https://en.wikipedia.org/wiki/Naukova_Dumka Naukova Dumka]
* [[Marvin Minsky]], [[Mathematician#SPapert|Seymour Papert]] ('''1969'''). ''[https://en.wikipedia.org/wiki/Perceptrons_%28book%29 Perceptrons]''. <ref>[https://en.wikipedia.org/wiki/AI_winter#The_abandonment_of_connectionism_in_1969 The abandonment of connectionism in 1969 - Wikipedia]</ref> <ref>[https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] ('''1962'''). ''[http://catalog.hathitrust.org/Record/000203591 Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms]''. Spartan Books</ref>
==1970 ...==
* [[Mathematician#SLinnainmaa|Seppo Linnainmaa]] ('''1970'''). ''The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors''. Master's thesis, [https://en.wikipedia.org/wiki/University_of_Helsinki University of Helsinki] » [[Neural Networks#Backpropagation|Backpropagation]] <ref>[[Mathematician#SLinnainmaa|Seppo Linnainmaa]] ('''1976'''). ''[http://link.springer.com/article/10.1007/BF01931367 Taylor expansion of the accumulated rounding error]''. [https://en.wikipedia.org/wiki/BIT_Numerical_Mathematics BIT Numerical Mathematics], Vol. 16, No. 2</ref>
* [https://en.wikipedia.org/wiki/Alexey_Grigorevich_Ivakhnenko Alexey G. Ivakhnenko] ('''1971'''). ''Polynomial theory of complex systems''. [[IEEE#SMC|IEEE Transactions on Systems, Man, and Cybernetics]], Vol. 1, No. 4
* [[A. Harry Klopf]] ('''1972'''). ''Brain Function and Adaptive Systems - A Heterostatic Theory''. [https://en.wikipedia.org/wiki/Air_Force_Cambridge_Research_Laboratories Air Force Cambridge Research Laboratories], Special Reports, No. 133, [http://www.dtic.mil/dtic/tr/fulltext/u2/742259.pdf pdf]
* [[Marvin Minsky]], [[Mathematician#SPapert|Seymour Papert]] ('''1972'''). ''[https://en.wikipedia.org/wiki/Perceptrons_%28book%29 Perceptrons: An Introduction to Computational Geometry]''. [https://en.wikipedia.org/wiki/MIT_Press The MIT Press], 2nd edition with corrections
* [[Mathematician#SGrossberg|Stephen Grossberg]] ('''1973'''). ''Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks''. [https://en.wikipedia.org/wiki/Studies_in_Applied_Mathematics Studies in Applied Mathematics], Vol. 52, [http://cns.bu.edu/~steve/Gro1973StudiesAppliedMath.pdf pdf]
* [[Mathematician#SGrossberg|Stephen Grossberg]] ('''1974'''). ''[http://techlab.bu.edu/resources/article_view/classical_and_instrumental_learning_by_neural_networks/ Classical and instrumental learning by neural networks]''. Progress in Theoretical Biology. [https://en.wikipedia.org/wiki/Academic_Press Academic Press]
* [https://en.wikipedia.org/wiki/Paul_Werbos Paul Werbos] ('''1974'''). ''[http://aitopics.org/publication/beyond-regression-new-tools-prediction-and-analysis-behavioral-sciences Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences]''. Ph. D. thesis, [[Harvard University]] <ref>[https://en.wikipedia.org/wiki/Backpropagation Backpropagation from Wikipedia]</ref> <ref>[https://en.wikipedia.org/wiki/Paul_Werbos Paul Werbos] ('''1994'''). ''[http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471598976.html The Roots of Backpropagation. From Ordered Derivatives to Neural Networks and Political Forecasting]''. [https://en.wikipedia.org/wiki/John_Wiley_%26_Sons John Wiley & Sons]</ref>
* [[Richard Sutton]] ('''1978'''). ''Single channel theory: A neuronal theory of learning''. Brain Theory Newsletter 3, No. 3/4, pp. 72-75. [http://www.cs.ualberta.ca/%7Esutton/papers/sutton-78-BTN.pdf pdf]
==1980 ...==
* [http://www.scholarpedia.org/article/User:Kunihiko_Fukushima Kunihiko Fukushima] ('''1980'''). ''Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position''. [http://link.springer.com/journal/422 Biological Cybernetics], Vol. 36 <ref>[http://www.scholarpedia.org/article/Neocognitron Neocognitron - Scholarpedia] by [http://www.scholarpedia.org/article/User:Kunihiko_Fukushima Kunihiko Fukushima]</ref>
* [[Richard Sutton]], [[Andrew Barto]] ('''1981'''). ''Toward a modern theory of adaptive networks: Expectation and prediction''. Psychological Review, Vol. 88, pp. 135-170. [http://www.cs.ualberta.ca/%7Esutton/papers/sutton-barto-81-PsychRev.pdf pdf]
* [https://en.wikipedia.org/wiki/Paul_Werbos Paul Werbos] ('''1982'''). ''Applications of advances in nonlinear sensitivity analysis''. [http://link.springer.com/book/10.1007%2FBFb0006119 System Modeling and Optimization], [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [http://werbos.com/Neural/SensitivityIFIPSeptember1981.pdf pdf]
* [[A. Harry Klopf]] ('''1982'''). ''The Hedonistic Neuron: A Theory of Memory, Learning, and Intelligence''. Hemisphere Publishing Corporation, [[University of Michigan]]
* [[Mathematician#DHAckley|David H. Ackley]], [[Mathematician#GEHinton|Geoffrey E. Hinton]], [[Terrence J. Sejnowski]] ('''1985'''). ''A Learning Algorithm for Boltzmann Machines''. Cognitive Science, Vol. 9, No. 1, [https://web.archive.org/web/20110718022336/http://learning.cs.toronto.edu/~hinton/absps/cogscibm.pdf pdf]
* [https://en.wikipedia.org/wiki/David_Rumelhart David E. Rumelhart], [[Mathematician#GEHinton|Geoffrey E. Hinton]], [https://en.wikipedia.org/wiki/Ronald_J._Williams Ronald J. Williams] ('''1986'''). ''Learning representations by back-propagating errors''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 323, [http://www.iro.umontreal.ca/~vincentp/ift3395/lectures/backprop_old.pdf pdf]
'''1987'''
* [[Gerald Tesauro]], [[Terrence J. Sejnowski]] ('''1987'''). ''A 'Neural' Network that Learns to Play Backgammon''. [http://www.informatik.uni-trier.de/~ley/db/conf/nips/nips1987.html#TesauroS87 NIPS 1987]
* [[Eric B. Baum]], [https://en.wikipedia.org/wiki/Frank_Wilczek Frank Wilczek] ('''1987'''). ''[http://papers.nips.cc/paper/3-supervised-learning-of-probability-distributions-by-neural-networks Supervised Learning of Probability Distributions by Neural Networks]''. [http://papers.nips.cc/book/neural-information-processing-systems-1987 NIPS 1987]
* [[A. Harry Klopf]] ('''1987'''). ''[http://www.dtic.mil/docs/citations/ADA188378 A Neuronal Model of Classical Conditioning]''. Technical Report, [https://en.wikipedia.org/wiki/Wright_Laboratory Air Force Wright Aeronautical Laboratories] <ref>[https://en.wikipedia.org/wiki/Classical_conditioning Classical conditioning from Wikipedia]</ref>
'''1988'''
* [[Richard Sutton]] ('''1988'''). ''Learning to Predict by the Methods of Temporal Differences''. Machine Learning, Vol. 3, No. 1, pp. 9-44. Kluwer Academic Publishers, Boston. ISSN 0885-6125.
* [[Gerald Tesauro]] ('''1988'''). ''Neural network defeats creator in backgammon match''. Technical report no. CCSR-88-6, Center for Complex Systems Research, [[University of Illinois at Urbana-Champaign]]
* [[Eric B. Baum]], [[Mathematician#DHHaussler|David Haussler]] ('''1988'''). ''[http://papers.nips.cc/paper/154-what-size-net-gives-valid-generalization What Size Net Gives Valid Generalization?]'' [http://papers.nips.cc/book/advances-in-neural-information-processing-systems-1-1988 NIPS 1988]
* [[Eric B. Baum]] ('''1988'''). ''[http://www.sciencedirect.com/science/article/pii/0885064X88900209 On the capabilities of multilayer perceptrons]''. [https://en.wikipedia.org/wiki/Complexity_%28journal%29 Complexity], Vol. 4, No. 3
* [[Mathematician#AlanLapedes|Alan Lapedes]], [http://www.techenablement.com/rob-farber/ Robert Farber] ('''1988'''). ''How Neural Nets Work''. [https://papers.nips.cc/paper/59-how-neural-nets-work.pdf pdf]
'''1989'''
* [[Eric B. Baum]] ('''1989'''). ''[http://papers.nips.cc/paper/226-the-perceptron-algorithm-is-fast-for-non-malicious-distributions The Perceptron Algorithm Is Fast for Non-Malicious Distributions]''. [http://papers.nips.cc/book/advances-in-neural-information-processing-systems-2-1989 NIPS 1989]
* [[Eric B. Baum]] ('''1989'''). ''[http://www.mitpressjournals.org/doi/abs/10.1162/neco.1989.1.2.201#.VfGX0JdpluM A Proposal for More Powerful Learning Algorithms]''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 1, No. 2
* [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/i/Irani:E=_A=.html Erach A. Irani], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/m/Matts:John_P=.html John P. Matts], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/l/Long:John_M=.html John M. Long], [[James R. Slagle]], POSCH group ('''1989'''). ''Using Artificial Neural Nets for Statistical Discovery: Observations after Using Backpropogation, Expert Systems, and Multiple-Linear Regression on Clinical Trial Data''. University of Minnesota, Minneapolis, MN 55455, USA, Complex Systems 3, [http://www.complex-systems.com/pdf/03-3-5.pdf pdf]
* [[Gerald Tesauro]], [[Terrence J. Sejnowski]] ('''1989'''). ''A Parallel Network that Learns to Play Backgammon''. [https://en.wikipedia.org/wiki/Artificial_Intelligence_%28journal%29 Artificial Intelligence], Vol. 39, No. 3
* [[Mathematician#EGelenbe|Erol Gelenbe]] ('''1989'''). ''[http://cognet.mit.edu/journal/10.1162/neco.1989.1.4.502 Random Neural Networks with Negative and Positive Signals and Product Form Solution]''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 1, No. 4
==1990 ...==
* [https://en.wikipedia.org/wiki/Paul_Werbos Paul Werbos] ('''1990'''). ''Backpropagation Through Time: What It Does and How to Do It''. Proceedings of the [[IEEE]], Vol. 78, No. 10, [http://deeplearning.cs.cmu.edu/pdfs/Werbos.backprop.pdf pdf]
* [[Gordon Goetsch]] ('''1990'''). ''Maximization of Mutual Information in a Context Sensitive Neural Network''. Ph.D. thesis
* [[Vadim Anshelevich]] ('''1990'''). ''Neural Networks''. Review. in Multi Component Systems (Russian)
* [[Eric B. Baum]] ('''1990'''). ''Polynomial Time Algorithms for Learning Neural Nets''. [http://dblp.uni-trier.de/db/conf/colt/colt1990.html#Baum90 COLT 1990]
'''1991'''
* [[Mathematician#SHochreiter|Sepp Hochreiter]] ('''1991'''). ''Untersuchungen zu dynamischen neuronalen Netzen''. Diploma thesis, [[Technical University of Munich|TU Munich]], advisor [[Jürgen Schmidhuber]], [http://people.idsia.ch/~juergen/SeppHochreiter1991ThesisAdvisorSchmidhuber.pdf pdf] (German) <ref>[http://people.idsia.ch/~juergen/fundamentaldeeplearningproblem.html Sepp Hochreiter's Fundamental Deep Learning Problem (1991)] by [[Jürgen Schmidhuber]], 2013</ref>
* [[Alex van Tiggelen]] ('''1991'''). ''Neural Networks as a Guide to Optimization - The Chess Middle Game Explored''. [[ICGA Journal#14_3|ICCA Journal, Vol. 14, No. 3]]
* [[Jürgen Schmidhuber]], [[Rudolf Huber]] ('''1991'''). ''Using sequential adaptive Neuro-control for efficient Learning of Rotation and Translation Invariance''. In [http://www.cis.hut.fi/research/som-research/teuvo.html Teuvo Kohonen], [http://www.metla.fi/pp/KMak/index-en.htm Kai Mäkisara], [http://users.ics.tkk.fi/ollis/ Olli Simula], and [http://cis.legacy.ics.tkk.fi/jari/ Jari Kangas], editors, ''[http://www.abebooks.com/servlet/BookDetailsPL?bi=3453347721&searchurl=isbn%3D0444891781 Artificial Neural Networks]''. [https://en.wikipedia.org/wiki/Elsevier Elsevier]
* [[Jürgen Schmidhuber]] ('''1991'''). ''[http://www.idsia.ch/%7Ejuergen/promotion/ Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem]'' (Dynamic Neural Nets and the Fundamental Spatio-Temporal Credit Assignment Problem). Ph.D. thesis
* [[Yoav Freund]], [[Mathematician#DHHaussler|David Haussler]] ('''1991'''). ''Unsupervised Learning of Distributions of Binary Vectors Using 2-Layer Networks''. [http://dblp.uni-trier.de/db/conf/nips/nips1991.html#FreundH91 NIPS 1991]
* [[Byoung-Tak Zhang]], [[Gerd Veenker]] ('''1991'''). ''[http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=170480&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D170480 Neural networks that teach themselves through genetic discovery of novel examples]''. [http://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000500 IEEE International Joint Conference on Neural Networks]
'''1992'''
* [[Michael Reiss]] ('''1992'''). ''Temporal Sequence Processing in Neural Networks''. Ph.D. thesis, [https://en.wikipedia.org/wiki/King%27s_College_London King's College London], advisor [[Mathematician#JGTaylor|John G. Taylor]], [http://www.reiss.demon.co.uk/misc/m_reiss_phd.pdf pdf]
* [[Jacek Mańdziuk]], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/m/Macuk:Bohdan.html Bohdan Macukow] ('''1992'''). ''A Neural Network designed to solve the N-Queens Problem''. [http://www.informatik.uni-trier.de/~ley/db/journals/bc/bc72.html#Mandziuk95 Biological Cybernetics, Vol. 66 No. 4], [http://www.mini.pw.edu.pl/~mandziuk/PRACE/bc92.pdf pdf]
* [[Gerald Tesauro]] ('''1992'''). ''Temporal Difference Learning of Backgammon Strategy''. [http://www.informatik.uni-trier.de/~ley/db/conf/icml/ml1992.html#Tesauro92 ML 1992]
* [[Gerald Tesauro]] ('''1992'''). ''[http://dl.acm.org/citation.cfm?id=139616 Practical Issues in Temporal Difference Learning]''. [http://www.informatik.uni-trier.de/~ley/db/journals/ml/ml8.html#Tesauro92 Machine Learning, Vol. 8, No. 3-4]
* [[Egbert Boers]], [[Herman Kuiper]] ('''1992'''). ''Biological metaphors and the design of modular artificial neural networks''. Master’s thesis, [[Leiden University]], [http://liacs.leidenuniv.nl/assets/PDF/boers-kuiper.92.pdf pdf]
* [[Martin Riedmiller]], [[Heinrich Braun]] ('''1992'''). ''[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.52.4576 Rprop - A Fast Adaptive Learning Algorithm]''. Proceedings of the International Symposium on Computer and Information Science
* [[Justin A. Boyan]] ('''1992'''). ''Modular Neural Networks for Learning Context-Dependent Game Strategies''. Master's thesis, [https://en.wikipedia.org/wiki/University_of_Cambridge University of Cambridge], [http://www.cs.cmu.edu/~jab/cv/pubs/boyan.backgammon-thesis.pdf pdf]
* [https://en.wikipedia.org/wiki/Patricia_Churchland Patricia Churchland], [[Terrence J. Sejnowski]] ('''1992'''). ''[https://mitpress.mit.edu/books/computational-brain The Computational Brain]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
'''1993'''
* [[Jacek Mańdziuk]], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/m/Macuk:Bohdan.html Bohdan Macukow] ('''1993'''). ''A Neural Network performing Boolean Logic Operations''. [http://www.springerlink.com/content/1060-992x/ Optical Memory and Neural Networks], Vol. 2, No. 1, [http://www.mini.pw.edu.pl/~mandziuk/PRACE/omnn93.pdf pdf]
* [[Sebastian Thrun]], [[Tom Mitchell]] ('''1993'''). ''Integrating Inductive Neural Network Learning and Explanation-Based Learning''. Proceedings of the 13th IJCAI, pp. 930-936. Morgan Kaufmann, San Mateo, CA, [http://robots.stanford.edu/papers/thrun.EBNN_ijcai93.ps.gz zipped ps]
* [[Byoung-Tak Zhang]], [[Mathematician#HMuehlenbein|Heinz Mühlenbein]] ('''1993'''). ''Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor''. [https://en.wikipedia.org/wiki/Complex_Systems_(journal) Complex Systems], Vol. 7, [http://www.complex-systems.com/pdf/07-3-2.pdf pdf]
* [[Martin Riedmiller]], [[Heinrich Braun]] ('''1993'''). ''A direct adaptive method for faster backpropagation learning: The RPROP algorithm''. [http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=1059 IEEE International Conference On Neural Networks], [http://paginas.fe.up.pt/~ee02162/dissertacao/RPROP%20paper.pdf pdf]
'''1994'''
* [https://en.wikipedia.org/wiki/Paul_Werbos Paul Werbos] ('''1994'''). ''[http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471598976.html The Roots of Backpropagation. From Ordered Derivatives to Neural Networks and Political Forecasting]''. [https://en.wikipedia.org/wiki/John_Wiley_%26_Sons John Wiley & Sons]
* [[David E. Moriarty]], [[Risto Miikkulainen]] ('''1994'''). ''Evolving Neural Networks to focus Minimax Search''. [[AAAI|AAAI-94]], [http://www.cs.utexas.edu/~ai-lab/pubs/moriarty.focus.pdf pdf]
* [[Eric Postma]] ('''1994'''). ''SCAN: A Neural Model of Covert Attention''. Ph.D. thesis, [[Maastricht University]], advisor [[Jaap van den Herik]]
* [[Sebastian Thrun]] ('''1994'''). ''Neural Network Learning in the Domain of Chess''. Machines That Learn, [http://snowbird.djvuzone.org/ Snowbird], Extended abstract
* [[Christian Posthoff]], S. Schawelski, [[Michael Schlosser]] ('''1994'''). ''Neural Network Learning in a Chess Endgame Positions''. IEEE World Congress on Computational Intelligence
* [[Nicol N. Schraudolph]], [[Peter Dayan]], [[Terrence J. Sejnowski]] ('''1994'''). ''[http://nic.schraudolph.org/bib2html/b2hd-SchDaySej94.html Temporal Difference Learning of Position Evaluation in the Game of Go]''. [http://papers.nips.cc/book/advances-in-neural-information-processing-systems-6-1993 Advances in Neural Information Processing Systems 6] <ref>[http://satirist.org/learn-game/systems/go-net.html Nici Schraudolph’s go networks], review by [[Jay Scott]]</ref>
* [[Alois Heinz]] ('''1994'''). ''[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.55.3994 Efficient Neural Net α-β-Evaluators]''. [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.55.3994&rep=rep1&type=pdf pdf] <ref>[https://www.stmintz.com/ccc/index.php?id=11893 Re: Evaluation by neural network ?] by [[Jay Scott]], [[CCC]], November 10, 1997</ref>
* [[Alois Heinz]] ('''1994'''). ''Fast bounded smooth regression with lazy neural trees''. [http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=3013 ICNN 1994], DOI: [http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=374421 10.1109/ICNN.1994.374421]
* [[Martin Riedmiller]] ('''1994'''). ''Rprop - Description and Implementation Details''. Technical Report, [https://en.wikipedia.org/wiki/Karlsruhe_Institute_of_Technology University of Karlsruhe], [http://www.inf.fu-berlin.de/lehre/WS06/Musterererkennung/Paper/rprop.pdf pdf]
'''1995'''
* [https://peterbraspenning.wordpress.com/ Peter J. Braspenning], [[Frank Thuijsman]], [https://scholar.google.com/citations?user=Ba9L7CAAAAAJ Ton Weijters] (eds) ('''1995'''). ''[http://link.springer.com/book/10.1007%2FBFb0027019 Artificial neural networks: an introduction to ANN theory and practice]''. [https://de.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science LNCS] 931, [https://de.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Anton Leouski]] ('''1995'''). ''Learning of Position Evaluation in the Game of Othello''. Master's Project, [https://en.wikipedia.org/wiki/University_of_Massachusetts University of Massachusetts], [https://en.wikipedia.org/wiki/Amherst,_Massachusetts Amherst, Massachusetts], [http://people.ict.usc.edu/~leuski/publications/papers/UM-CS-1995-023.pdf pdf]
* [[Mathematician#SHochreiter|Sepp Hochreiter]], [[Jürgen Schmidhuber]] ('''1995'''). ''[http://www.idsia.ch/%7Ejuergen/nipsfm/ Simplifying Neural Nets by Discovering Flat Minima]''. In [[Gerald Tesauro]], [http://www.cs.cmu.edu/%7Edst/home.html David S. Touretzky] and [http://www.bme.ogi.edu/%7Etleen/ Todd K. Leen] (eds.), ''[http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=8420 Advances in Neural Information Processing Systems 7]'', NIPS'7, pages 529-536. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Sebastian Thrun]] ('''1995'''). ''[http://robots.stanford.edu/papers/thrun.nips7.neuro-chess.html Learning to Play the Game of Chess]''. in [[Gerald Tesauro]], [https://en.wikipedia.org/wiki/David_S._Touretzky David S. Touretzky], [http://mitpress.mit.edu/authors/todd-k-leen Todd K. Leen] (eds.) Advances in Neural Information Processing Systems 7, [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [http://www.bioinf.jku.at/people/hochreiter/ Sepp Hochreiter], [[Jürgen Schmidhuber]] ('''1995'''). ''[http://www.idsia.ch/%7Ejuergen/nipsfm/ Simplifying Neural Nets by Discovering Flat Minima]''. In [[Gerald Tesauro]], [http://www.cs.cmu.edu/%7Edst/home.html David S. Touretzky] and [http://www.bme.ogi.edu/%7Etleen/ Todd K. Leen] (eds.), ''[http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=8420 Advances in Neural Information Processing Systems 7]'', NIPS'7, pages 529-536. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Sebastian Thrun]] ('''1995'''). ''Explanation-Based Neural Network Learning - A Lifelong Learning Approach''. Ph.D. thesis, [https://en.wikipedia.org/wiki/University_of_Bonn University of Bonn], advisors [[Mathematician#ABCremers|Armin Cremers]] and [[Tom Mitchell]]
* [[Gerald Tesauro]] ('''1995'''). ''Temporal Difference Learning and TD-Gammon''. [[ACM#Communications|Communications of the ACM]] Vol. 38, No. 3
* [[Eric Postma]] ('''1995'''). ''Optimization Networks''. [http://www.informatik.uni-trier.de/~ley/db/conf/ann/ann1995.html#Postma95 Artificial Neural Networks]
* [http://www.informatik.uni-trier.de/~ley/pers/hd/j/Jelonek:Jacek.html Jacek Jelonek], [[Krzysztof Krawiec]], [http://www.informatik.uni-trier.de/~ley/pers/hd/s/Slowinski:Roman.html Roman Slowinski] ('''1995'''). ''[http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8640.1995.tb00036.x/abstract Rough Set Reduction of Attributes and their Domains for Neural Networks]''. [http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%291467-8640 Computational Intelligence], Vol. 11, No. 2
* [[Omar Syed]] ('''1995'''). ''[http://arimaa.com/arimaa/about/Thesis/ Applying Genetic Algorithms to Recurrent Neural Networks for Learning Network Parameters and Architecture]'', Masters Thesis, [https://en.wikipedia.org/wiki/Case_Western_Reserve_University Case Western Reserve University]
* [[Pascal Tang]] ('''1995'''). ''Forecasting with Neural networks''. [http://www.e-nns.org/index.php/ICANN/History/ ICANN 1995]
* [[Marco Wiering]] ('''1995'''). ''[https://scholar.google.com/citations?view_op=view_citation&hl=en&user=xVas0I8AAAAJ&cstart=20&citation_for_view=xVas0I8AAAAJ:roLk4NBRz8UC TD Learning of Game Evaluation Functions with Hierarchical Neural Architectures]''. Master's thesis, [https://en.wikipedia.org/wiki/University_of_Amsterdam University of Amsterdam], [http://webber.physik.uni-freiburg.de/~hon/vorlss02/Literatur/reinforcement/GameEvaluationWithNeuronal.pdf pdf]
* [[Michael A Arbib]] (ed.) ('''1995, 2002'''). ''[http://mitpress.mit.edu/books/handbook-brain-theory-and-neural-networks The Handbook of Brain Theory and Neural Networks]''. [https://en.wikipedia.org/wiki/MIT_Press The MIT Press]
* [[Nicol N. Schraudolph]] ('''1995'''). ''[http://nic.schraudolph.org/bib2html/b2hd-Schraudolph95 Optimization of Entropy with Neural Networks]''. Ph.D. thesis, [https://en.wikipedia.org/wiki/University_of_California,_San_Diego University of California, San Diego]
* [[Alois Heinz]] ('''1995'''). ''Pipelined Neural Tree Learning by Error Forward-Propagation''. [http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=3505 ICNN 1995], DOI: [http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=488132 10.1109/ICNN.1995.488132], [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.29.4154&rep=rep1&type=pdf pdf]
* [[Alois Heinz]], [[Christoph Hense]] ('''1995'''). ''[http://tr.informatik.uni-freiburg.de/1995/Report68/ Tools for Neural Trees]''. Technical Report No. 68
* [[Nicol N. Schraudolph]], [[Terrence J. Sejnowski]] ('''1995'''). ''Tempering Backpropagation Networks: Not All Weights are Created Equal''. [https://papers.nips.cc/book/advances-in-neural-information-processing-systems-8-1995 NIPS 1995], [https://papers.nips.cc/paper/1100-tempering-backpropagation-networks-not-all-weights-are-created-equal.pdf pdf]
'''1996'''
* [[Sebastian Thrun]] ('''1996'''). ''[http://robots.stanford.edu/papers/thrun.book.html Explanation-Based Neural Network Learning: A Lifelong Learning Approach]''. [https://en.wikipedia.org/wiki/Wolters_Kluwer Kluwer Academic Publishers]
* [[Wee Sun Lee]] ('''1996'''). ''Agnostic Learning and Single Hidden Layer Neural Networks.'' Ph.D. thesis, [[Australian National University]], [http://www.comp.nus.edu.sg/~leews/publications/thesis.ps ps]
* [[Markus Enzenberger]] ('''1996'''). ''[http://webdocs.cs.ualberta.ca/~emarkus/neurogo/neurogo1996.html The Integration of A Priori Knowledge into a Go Playing Neural Network]''.
* [[Pieter Spronck]] ('''1996'''). ''Elegance: Genetic Algorithms in Neural Reinforcement Control''. Master thesis, [[Delft University of Technology]], [http://ticc.uvt.nl/~pspronck/pubs/Elegance.pdf pdf]
* [[Raúl Rojas]] ('''1996'''). ''[http://www.inf.fu-berlin.de/inst/ag-ki/rojas_home/pmwiki/pmwiki.php?n=Books.NeuralNetworks Neural Networks - A Systematic Introduction]''. [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], available as [http://www.inf.fu-berlin.de/inst/ag-ki/rojas_home/documents/1996/NeuralNetworks/neuron.pdf pdf ebook]
'''1997'''
* [[Mathematician#SHochreiter|Sepp Hochreiter]], [[Jürgen Schmidhuber]] ('''1997'''). ''Long short-term memory''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 9, No. 8, [http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf pdf] <ref>[https://en.wikipedia.org/wiki/Long_short_term_memory Long short term memory from Wikipedia]</ref>
* [[Kieran Greer]], [[Piyush Ojha]], [[David A. Bell]] ('''1997'''). ''Learning Search Heuristics from Examples: A Study in Computer Chess''. Seventh Conference of the Spanish Association for Artificial Intelligence, CAEPIA’97, November, pp. 695-704.
* [[Don Beal]], [[Martin C. Smith]] ('''1997'''). ''Learning Piece Values Using Temporal Differences''. [[ICGA Journal#20_3|ICCA Journal, Vol. 20, No. 3]]
'''1998'''
* [[Kieran Greer]] ('''1998'''). ''A Neural Network Based Search Heuristic and its Application to Computer Chess''. D.Phil. Thesis, [https://en.wikipedia.org/wiki/University_of_Ulster University of Ulster]
* [[Nobusuke Sasaki]], [[Yasuji Sawada]], [[Jin Yoshimura]] ('''1998'''). ''[http://link.springer.com/chapter/10.1007%2F3-540-48957-6_10 A Neural Network Program of Tsume-Go]''. [[CG 1998]] <ref>[https://en.wikipedia.org/wiki/Tsumego Tsumego from Wikipedia]</ref>
* [[Krzysztof Krawiec]], [http://www.informatik.uni-trier.de/~ley/pers/hd/s/Slowinski:Roman.html Roman Slowinski], [http://www.informatik.uni-trier.de/~ley/pers/hd/s/Szczesniak:Irmina.html Irmina Szczesniak] ('''1998'''). ''[http://link.springer.com/chapter/10.1007%2F3-540-69115-4_60 Pedagogical Method for Extraction of Symbolic Knowledge from Neural Networks]''. [http://link.springer.com/book/10.1007%2F3-540-69115-4 Rough Sets and Current Trends in Computing 1998]
* [[Steven Walczak]] ('''1998'''). ''Neural network models for a resource allocation problem''. IEEE Transactions on Systems, Man, and Cybernetics, Part B 28(2)
* [[Jonathan Baxter]], [[Andrew Tridgell]], [[Lex Weaver]] ('''1998'''). ''Experiments in Parameter Learning Using Temporal Differences''. [[ICGA Journal#21_2|ICCA Journal, Volume 21 No. 2]], [http://cs.anu.edu.au/%7ELex.Weaver/pub_sem/publications/ICCA-98_equiv.pdf pdf]
* [[Guy Haworth]], [[Meel Velliste]] ('''1998'''). ''[http://centaur.reading.ac.uk/4569/ Chess Endgames and Neural Networks]''. [[ICGA Journal#21_4|ICCA Journal, Vol. 21, No. 4]]
* [[Don Beal]], [[Martin C. Smith]] ('''1998'''). ''First Results from Using Temporal Difference Learning in Shogi''. [[CG 1998]]
* [[Nicol N. Schraudolph]] ('''1998'''). ''[http://nic.schraudolph.org/bib2html/b2hd-Schraudolph98.html Centering Neural Network Gradient Factors]''. Neural Networks: Tricks of the Trade
* [[Toshinori Munakata]] ('''1998'''). ''[http://cis.csuohio.edu/~munakata/publs/book/sp.html Fundamentals of the New Artificial Intelligence: Beyond Traditional Paradigms]''. 1st edition, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [[Neural Networks#FundamentalsNAI2nd|2nd edition 2008]]
'''1999'''
* [[Kumar Chellapilla]], [[David B. Fogel]] ('''1999'''). ''[http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=784222 Evolution, Neural Networks, Games, and Intelligence]''. Proceedings of the IEEE, September, pp. 1471-1496. [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.99.979 CiteSeerX]
* [[Kumar Chellapilla]], [[David B. Fogel]] ('''1999'''). ''[http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=809083 Evolving Neural Networks to Play Checkers without Expert Knowledge]''. IEEE Transactions on Neural Networks, Vol. 10, No. 6, pp. 1382-1391.
* [[Kieran Greer]], [[Piyush Ojha]], [[David A. Bell]] ('''1999'''). ''A Pattern-Oriented Approach to Move Ordering: the Chessmaps Heuristic''. [[ICGA Journal#22_1|ICCA Journal, Vol. 22, No. 1]]
* Anna Górecka, [[Maciej Szmit]] ('''1999'''). ''Exchange rates prediction by ARIMA and Neural Networks Models''. 47th International Atlantic Economic Conerence (Abstract: International Advances of Economic Research Vol 5 Nr 4 Nov. 1999, St Louis, MO, USA 1999), [http://maciej.szmit.info/documents/annarima.pdf pdf]
* [[Don Beal]], [[Martin C. Smith]] ('''1999'''). ''Learning Piece-Square Values using Temporal Differences.'' [[ICGA Journal#22_4|ICCA Journal, Vol. 22, No. 4]]
* [https://en.wikipedia.org/wiki/Simon_Haykin Simon S. Haykin] ('''1999'''). ''[http://dl.acm.org/citation.cfm?id=521706 Neural Networks: A Comprehensive Foundation]''. 2nd Edition, [https://en.wikipedia.org/wiki/Prentice_Hall Prentice-Hall]
* [[Laurence F. Abbott]], [[Terrence J. Sejnowski]] (eds.) ('''1999'''). ''[https://mitpress.mit.edu/books/neural-codes-and-distributed-representations Neural Codes and Distributed Representations]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Mathematician#GEHinton|Geoffrey E. Hinton]], [[Terrence J. Sejnowski]] (eds.) ('''1999'''). ''[https://mitpress.mit.edu/books/unsupervised-learning Unsupervised Learning: Foundations of Neural Computation]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Peter Dayan]] ('''1999'''). ''Recurrent Sampling Models for the Helmholtz Machine''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 11, No. 3, [http://www.gatsby.ucl.ac.uk/~dayan/papers/rechelm99.pdf pdf] <ref>[https://en.wikipedia.org/wiki/Helmholtz_machine Helmholtz machine from Wikipedia]</ref>
==2000 ...==
* [[Levente Kocsis]], [[Jos Uiterwijk]], [[Jaap van den Herik]] ('''2000'''). ''[http://link.springer.com/chapter/10.1007%2F3-540-45579-5_11 Learning Time Allocation using Neural Networks]''. [[CG 2000]]
* [[Peter Auer]], [[Mathematician#SKwek|Stephen Kwek]], [[Mathematician#WMaass|Wolfgang Maass]], [[Mathematician#MKWarmuth|Manfred K. Warmuth]] ('''2000'''). ''[http://eccc.hpi-web.de/report/2000/055/ Learning of Depth Two Neural Networks with Constant Fan-in at the Hidden Nodes]''. [http://dblp.uni-trier.de/db/journals/eccc/eccc7.html#ECCC-TR00-055 Electronic Colloquium on Computational Complexity, Vol. 7]
* [[Jonathan Baxter]], [[Andrew Tridgell]], [[Lex Weaver]] ('''2000'''). ''Learning to Play Chess Using Temporal Differences''. [http://www.dblp.org/db/journals/ml/ml40.html#BaxterTW00 Machine Learning, Vol 40, No. 3], [http://www.cs.princeton.edu/courses/archive/fall06/cos402/papers/chess-RL.pdf pdf]
* [[Alois Heinz]] ('''2000'''). ''[http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=857884 Tree-Structured Neural Networks: Efficient Evaluation of Higher-Order Derivatives and Integrals]''. [http://dblp.uni-trier.de/db/conf/ijcnn/ijcnn2000-2.html#Heinz00 IJCNN 2000]
* [[Robert Levinson]], [[Ryan Weber]] ('''2000'''). ''[http://link.springer.com/chapter/10.1007/3-540-45579-5_9 Chess Neighborhoods, Function Combination, and Reinforcement Learning]''. [[CG 2000]], [https://users.soe.ucsc.edu/~levinson/Papers/CNFCRL.pdf pdf]
* [[Matthias Lüscher]] ('''2000'''). ''Automatic Generation of an Evaluation Function for Chess Endgames''. [[ETH Zurich]] Supervisors: [[Thomas Lincke]] and [[Christoph Wirth]], [http://www.datacomm.ch/m.luescher/evaluation_function_en.pdf pdf] » [[Endgame]]
* [[Miroslav Kubat]] ('''2000'''). ''[http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=58577 Designing neural network architectures for pattern recognition]''. [http://dblp2.uni-trier.de/db/journals/ker/ker15.html The Knowledge Engineering Review, Vol. 15], No. 2
* [https://scholar.google.com/citations?user=ZjfN_9AAAAAJ Igor Aizenberg], [http://dblp.uni-trier.de/pers/hd/a/Aizenberg:Naum_N= Naum N. Aizenberg], [https://scholar.google.com/citations?user=Swa4FrsAAAAJ Joos Vandewalle] ('''2000'''). ''[http://link.springer.com/book/10.1007%2F978-1-4757-3115-6 Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications]''. [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer] <ref>[https://plus.google.com/100849856540000067209/posts/7N6z251w2Wd?pid=6127540521703625346&oid=100849856540000067209 Who introduced the term “deep learning” to the field of Machine Learning] by [[Jürgen Schmidhuber]], [https://plus.google.com/100849856540000067209 Google+], March 18, 2015</ref>
'''2001'''
* [[Erik van der Werf]], [[Jaap van den Herik]] ('''2001'''). ''Visual Learning in Go''. [[6th Computer Olympiad#Workshop|6th Computer Olympiad Workshop]], [http://erikvanderwerf.tengen.nl/pubdown/visual_learning_in_go.pdf pdf]
* [[Levente Kocsis]], [[Jos Uiterwijk]], [[Jaap van den Herik]] ('''2001'''). ''Move Ordering using Neural Networks''. IEA/AIE 2001, [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science LNCS] 2070, [http://www.pradu.us/old/Nov27_2008/Buzz/research/parallel/fulltext.pdf pdf]
* [[Kee Siong Ng]] ('''2001'''). ''Neural Networks for Structured Data''. BSc-Thesis, [http://users.cecs.anu.edu.au/~kee/hon-thesis.ps.gz zipped ps]
* [[Jonathan Schaeffer]], [[Markian Hlynka]], [[Vili Jussila]] ('''2001'''). ''Temporal Difference Learning Applied to a High-Performance Game-Playing Program''. [http://www.informatik.uni-trier.de/~ley/db/conf/ijcai/ijcai2001.html#SchaefferHJ01 IJCAI 2001]
* [[Don Beal]], [[Martin C. Smith]] ('''2001'''). ''Temporal difference learning applied to game playing and the results of application to Shogi''. Theoretical Computer Science, Volume 252, Issues 1-2, pp. 105-119
* [[Nicol N. Schraudolph]], [[Peter Dayan]], [[Terrence J. Sejnowski]] ('''2001'''). ''[http://nic.schraudolph.org/bib2html/b2hd-SchDaySej01.html Learning to Evaluate Go Positions via Temporal Difference Methods]''. in [[Norio Baba]], [[Lakhmi C. Jain]] (eds.) ('''2001'''). ''[http://jasss.soc.surrey.ac.uk/7/1/reviews/takama.html Computational Intelligence in Games, Studies in Fuzziness and Soft Computing]''. [http://www.springer.com/economics?SGWID=1-165-6-73481-0 Physica-Verlag]
* [[Peter Dayan]], [[Laurence F. Abbott]] ('''2001, 2005'''). ''[http://www.gatsby.ucl.ac.uk/~dayan/book/index.html Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
'''2002'''
* [[Levente Kocsis]], [[Jos Uiterwijk]], [[Eric Postma]], [[Jaap van den Herik]] ('''2002'''). ''[http://link.springer.com/chapter/10.1007%2F978-3-540-40031-8_11 The Neural MoveMap Heuristic in Chess]''. [[CG 2002]]
* [[Erik van der Werf]], [[Jos Uiterwijk]], [[Eric Postma]], [[Jaap van den Herik]] ('''2002'''). ''[http://link.springer.com/chapter/10.1007%2F978-3-540-40031-8_26 Local Move Prediction in Go]''. [[CG 2002]]
* [[Gerald Tesauro]] ('''2002'''). ''Programming backgammon using self-teaching neural nets''. [http://www.informatik.uni-trier.de/~ley/db/journals/ai/ai134.html#Tesauro02 Artificial Intelligence Vol. 134 No. 1-2]
* [[Mark Winands]], [[Levente Kocsis]], [[Jos Uiterwijk]], [[Jaap van den Herik]] ('''2002'''). ''Temporal difference learning and the Neural MoveMap heuristic in the game of Lines of Action''. in GAME-ON 2002, [http://zaphod.aml.sztaki.hu/papers/winands-GAMEON02.pdf pdf]
* [[Jacek Mańdziuk]] ('''2002'''). ''Neural Networks for the N-Queens Problem: a Review''. [http://www.scimagojr.com/journalsearch.php?q=12928&tip=sid Control and Cybernetics], Vol. 31, No. 2, [http://www.mini.pw.edu.pl/~mandziuk/PRACE/nqp-rev.pdf pdf]
* [[Moshe Sipper]] ('''2002''') ''[http://books.google.com/books/about/Machine_Nature.html?id=fbFQAAAAMAAJ&redir_esc=y Machine Nature: The Coming Age of Bio-Inspired Computing]''. [https://en.wikipedia.org/wiki/McGraw-Hill_Financial McGraw-Hill, New York]
* [[Paul E. Utgoff]], [[David J. Stracuzzi]] ('''2002'''). ''Many-Layered Learning''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 14, No. 10, [http://people.cs.umass.edu/~utgoff/papers/neco-stl.pdf pdf]
* [[Michael I. Jordan]], [[Terrence J. Sejnowski]] (eds.) ('''2002'''). ''[https://mitpress.mit.edu/books/graphical-models Graphical Models: Foundations of Neural Computation]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
'''2003'''
* [[Levente Kocsis]] ('''2003'''). ''Learning Search Decisions''. Ph.D thesis, [[Maastricht University]], [https://project.dke.maastrichtuniversity.nl/games/files/phd/Kocsis_thesis.pdf pdf]
* [[Markus Enzenberger]] ('''2003'''). ''[http://webdocs.cs.ualberta.ca/~emarkus/neurogo/neurogo3/index.html Evaluation in Go by a Neural Network using Soft Segmentation]''. [[Advances in Computer Games 10]], [http://webdocs.cs.ualberta.ca/~emarkus/publications/neurogo3.pdf pdf]
* [[Alois Heinz]] ('''2003'''). ''[http://link.springer.com/chapter/10.1007%2F3-540-36477-3_13 Yes, Trees May Have Neurons]''. [http://dblp.uni-trier.de/db/conf/birthday/ottmann2003.html#Heinz03 Computer Science in Perspective 2003]
'''2004'''
* [http://dblp.uni-trier.de/pers/hd/p/Patist:Jan_Peter Jan Peter Patist], [[Marco Wiering]] ('''2004'''). ''[https://scholar.google.com/citations?view_op=view_citation&hl=en&user=xVas0I8AAAAJ&cstart=20&pagesize=80&citation_for_view=xVas0I8AAAAJ:Wp0gIr-vW9MC Learning to Play Draughts using Temporal Difference Learning with Neural Networks and Databases]''. [http://students.uu.nl/en/hum/cognitive-artificial-intelligence Cognitive Artiﬁcial Intelligence], [https://en.wikipedia.org/wiki/Utrecht_University Utrecht University], Benelearn’04
* [[Henk Mannen]], [[Marco Wiering]] ('''2004'''). ''[http://scholar.google.com/citations?view_op=view_citation&hl=en&user=xVas0I8AAAAJ&cstart=20&pagesize=80&citation_for_view=xVas0I8AAAAJ:7PzlFSSx8tAC Learning to play chess using TD(λ)-learning with database games]''. [http://students.uu.nl/en/hum/cognitive-artificial-intelligence Cognitive Artiﬁcial Intelligence], [https://en.wikipedia.org/wiki/Utrecht_University Utrecht University], Benelearn’04
* [[Mathieu Autonès]], [[Aryel Beck]], [[Phillippe Camacho]], [[Nicolas Lassabe]], [[Hervé Luga]], [[François Scharffe]] ('''2004'''). ''[http://link.springer.com/chapter/10.1007/978-3-540-24650-3_1 Evaluation of Chess Position by Modular Neural network Generated by Genetic Algorithm]''. [http://www.informatik.uni-trier.de/~ley/db/conf/eurogp/eurogp2004.html#AutonesBCLLS04 EuroGP 2004] <ref>[https://www.stmintz.com/ccc/index.php?id=358770 Presentation for a neural net learning chess program] by [[Dann Corbit]], [[CCC]], April 06, 2004</ref>
* [[Daniel Walker]], [[Robert Levinson]] ('''2004'''). ''The MORPH Project in 2004''. [[ICGA Journal#27_4|ICGA Journal, Vol. 27, No. 4]]
'''2006'''
* [[Holk Cruse]] ('''2006'''). ''[http://www.brains-minds-media.org/archive/615 Neural Networks as Cybernetic Systems]''. 2nd and revised edition, [http://www.uni-bielefeld.de/biologie/Kybernetik/ Department of Biological Cybernetics], [https://en.wikipedia.org/wiki/Bielefeld_University Bielefeld University]
* [[Mathematician#GEHinton|Geoffrey E. Hinton]], [https://www.linkedin.com/in/osindero Simon Osindero], [https://scholar.google.com/citations?user=y-nUzMwAAAAJ Yee Whye Teh] ('''2006'''). ''[http://www.mitpressjournals.org/doi/abs/10.1162/neco.2006.18.7.1527 A Fast Learning Algorithm for Deep Belief Nets]''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 18, No. 7, [https://www.cs.toronto.edu/~hinton/absps/fastnc.pdf pdf]
* [[Mathematician#GEHinton|Geoffrey E. Hinton]], [[Ruslan R. Salakhutdinov]] ('''2006'''). ''Reducing the Dimensionality of Data with Neural Networks''. [https://en.wikipedia.org/wiki/Science_(journal) Science], Vol. 313, [https://www.cs.toronto.edu/~hinton/science.pdf pdf]
'''2007'''
* [[Edward P. Manning]] ('''2007'''). ''[http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=4219046 Temporal Difference Learning of an Othello Evaluation Function for a Small Neural Network with Shared Weights]''. [[IEEE#CIG|IEEE Symposium on Computational Intelligence and AI in Games]]
* [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/d/Duan:Yong.html Yong Duan], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/c/Cui:Baoxia.html Baoxia Cui], [[Xinhe Xu]] ('''2007'''). ''State Space Partition for Reinforcement Learning Based on Fuzzy Min-Max Neural Network''. [http://www.informatik.uni-trier.de/~ley/db/conf/isnn/isnn2007-2.html#DuanCX07 ISNN 2007]
* [http://www.dkriesel.com/en/start David Kriesel] ('''2007'''). ''[http://www.dkriesel.com/en/science/neural_networks A Brief Introduction to Neural Networks]''. available at [http://www.dkriesel.com]
* [[Roland Stuckardt]] ('''2007'''). ''Applying Backpropagation Networks to Anaphor Resolution''. In: [http://www.di.fc.ul.pt/%7Eahb/ António Branco] (Ed.), ''[http://www.springer.com/computer/ai/book/978-3-540-71411-8 Anaphora: Analysis, Algorithms, and Applications]''. Selected Papers of the [http://daarc2007.di.fc.ul.pt/ 6th Discourse Anaphora and Anaphor Resolution Colloquium, DAARC 2007], [https://en.wikipedia.org/wiki/Lagos,_Portugal Lagos, Portugal]
'''2008'''
* [[Ilya Sutskever]], [[Vinod Nair]] ('''2008'''). ''Mimicking Go Experts with Convolutional Neural Networks''. [http://dblp.uni-trier.de/db/conf/icann/icann2008-2.html#SutskeverN08 ICANN 2008], [http://www.cs.utoronto.ca/~ilya/pubs/2008/go_paper.pdf pdf]
* [https://en.wikipedia.org/wiki/Simon_Haykin Simon S. Haykin] ('''2008'''). ''[http://www.amazon.com/Neural-Networks-Learning-Machines-Edition/dp/0131471392 Neural Networks: A Comprehensive Foundation]''. 3rd Edition, [https://en.wikipedia.org/wiki/Prentice_HallPrentice-Hall]
* [[Toshinori Munakata]] ('''2008'''). ''[http://link.springer.com/book/10.1007/978-1-84628-839-5 Fundamentals of the New Artificial Intelligence: Neural, Evolutionary, Fuzzy and More]''. 2nd edition, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [[Neural Networks#FundamentalsNAI1st|1st edition 1998]]
* [[Byoung-Tak Zhang]] ('''2008'''). ''Hypernetworks: A molecular evolutionary architecture for cognitive learning and memory''. [[IEEE|IEEE Computational Intelligence Magazine]], Vol. 3, No. 3, [https://bi.snu.ac.kr/Publications/Journals/International/IEEE_Comp_Int_3_Zhang.pdf pdf]
* [http://dblp.uni-trier.de/pers/hd/s/Song_0001:Qing Qing Song], [[James C. Spall]], [http://dblp.uni-trier.de/pers/hd/s/Soh:Yeng_Chai Yeng Chai Soh], [http://dblp.uni-trier.de/pers/hd/n/Ni:Jie Jie Ni] ('''2008'''). ''Robust Neural Network Tracking Controller Using Simultaneous Perturbation Stochastic Approximation''. [[IEEE#NN|IEEE Transactions on Neural Networks]], Vol. 19, No. 5, [https://pdfs.semanticscholar.org/3f2a/4d69ca8adbbc072d82af58b3c750621d36ab.pdf 2003 pdf] » [[SPSA]]
'''2009'''
* [[Daniel Shawul|Daniel Abdi]], Simon Levine, [http://www.eng.uwo.ca/civil/faculty/bitsuamlak_g/index.html Girma T. Bitsuamlak] ('''2009'''). ''Application of an Artificial Neural Network Model for Boundary Layer Wind Tunnel Profile Development''. 11th Americas conference on wind Engineering, [http://www.iawe.org/Proceedings/11ACWE/11ACWE-Abdi.pdf pdf]
==2010 ...==
* [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/s/Stewart:Ian.html Ian Stewart], [http://people.trentu.ca/wfeng/ Wenying Feng], [[Selim Akl]] ('''2010'''). ''Tuning Neural Networks by Both Connectivity and Size''. [http://www.informatik.uni-trier.de/~ley/db/conf/itng/itng2010.html#StewartFA10 ITNG 2010]
'''2011'''
* [[Jonathan K. Vis]] ('''2011'''). ''Discrete Tomography: A Neural Network Approach''. Master's thesis, [[Leiden University]], [http://www.liacs.nl/~jvis/thesis.pdf pdf]
* [[Jonathan K. Vis]], [[Walter Kosters]], [[Joost Batenburg]] ('''2011'''). ''Discrete Tomography: A Neural Network Approach''. [http://allserv.kahosl.be/bnaic2011/ BNAIC 2011] [http://www.liacs.nl/~jvis/bnaic2011.pdf pdf]
* [[Nikolaos Papahristou]], [[Ioannis Refanidis]] ('''2011'''). ''Training Neural Networks to Play Backgammon Variants Using Reinforcement Learning''. Proceedings of [http://conference.researchbib.com/?eventid=6400 Evogames 2011], Part I, LNCS 6624, [http://ai.uom.gr/nikpapa/publications/Training%20NN%20to%20Play%20Backgammon%20Variants%20Using%20RL.pdf pdf]
'''2012'''
* [http://dblp.uni-trier.de/pers/hd/d/Dries:Sjoerd_van_den Sjoerd van den Dries], [[Marco Wiering]] ('''2012'''). ''[https://scholar.google.com/citations?view_op=view_citation&hl=en&user=xVas0I8AAAAJ&cstart=40&citation_for_view=xVas0I8AAAAJ:P5F9QuxV20EC Neural-fitted TD-leaf learning for playing Othello with structured neural networks]''. [[IEEE#NN|IEEE Transactions on Neural Networks and Learning Systems]], Vol. 23, No. 11
* [[Jürgen Schmidhuber]], [http://www.idsia.ch/%7Etino/ Faustino Gomez], [http://www.idsia.ch/%7Esantiago/ Santiago Fernández], [http://www6.in.tum.de/Main/Graves Alex Graves], [http://www.bioinf.jku.at/people/hochreiter/ Sepp Hochreiter] ('''2012'''). ''[http://www.idsia.ch/%7Ejuergen/rnnbook.html Sequence Learning with Artificial Recurrent Neural Networks]''. (Aiming to become the definitive textbook on RNN.) Invited by [https://en.wikipedia.org/wiki/Cambridge_University_Press Cambridge University Press]
* [[Peter McLeod]], [http://content.cqu.edu.au/FCWViewer/staff.do?site=1829&sid=VERMAB Brijesh Verma] ('''2012'''). ''[http://ieeexplore.ieee.org/xpl/articleDetails.jsp;jsessionid=brTRQwvWb1pVpRlZ4K3pymTDhkp8FLyKpkZ1pT9XQTPr7GDpnG4C!58582823?arnumber=6252539&contentType=Conference+Publications Clustered ensemble neural network for breast mass classification in digital mammography]''. [http://www.informatik.uni-trier.de/~ley/db/conf/ijcnn/ijcnn2012.html#McLeodV12 IJCNN 2012]
* [[Mathematician#GMontavon|Grégoire Montavon]], [http://www.researchgate.net/profile/Genevieve_Orr Geneviève B. Orr], [[Mathematician#KRMueller|Klaus-Robert Müller]] (eds.) ('''2012'''). ''[http://link.springer.com/book/10.1007/978-3-642-35289-8 Neural Networks: Tricks of the Trade]''. (2nd Edition) [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science LNCS] 7700, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
: [[Nicol N. Schraudolph]] ('''2012'''). ''[http://link.springer.com/chapter/10.1007/978-3-642-35289-8_14 Centering Neural Network Gradient Factors]''.
: [https://en.wikipedia.org/wiki/L%C3%A9on_Bottou Léon Bottou] ('''2012'''). ''[http://link.springer.com/chapter/10.1007/978-3-642-35289-8_25 Stochastic Gradient Descent Tricks]''. [https://en.wikipedia.org/wiki/Microsoft_Research Microsoft Research], [http://cilvr.cs.nyu.edu/diglib/lsml/bottou-sgd-tricks-2012.pdf pdf]
: [[Ronan Collobert]], [[Koray Kavukcuoglu]], [[Clément Farabet]] ('''2012'''). ''[http://link.springer.com/chapter/10.1007%2F978-3-642-35289-8_28 Implementing Neural Networks Efficiently]''. <ref>[http://www.clement.farabet.net/code.html Clément Farabet | Code]</ref>
'''2013'''
* [[Mathematician#GMontavon|Grégoire Montavon]] ('''2013'''). ''[https://opus4.kobv.de/opus4-tuberlin/frontdoor/index/index/docId/4467 On Layer-Wise Representations in Deep Neural Networks]''. Ph.D. Thesis, [https://en.wikipedia.org/wiki/Technical_University_of_Berlin TU Berlin], advisor [[Mathematician#KRMueller|Klaus-Robert Müller]]
* [[Volodymyr Mnih]], [[Koray Kavukcuoglu]], [[David Silver]], [[Alex Graves]], [[Ioannis Antonoglou]], [[Daan Wierstra]], [[Martin Riedmiller]] ('''2013'''). ''Playing Atari with Deep Reinforcement Learning''. [http://arxiv.org/abs/1312.5602 arXiv:1312.5602] <ref>[http://www.nervanasys.com/demystifying-deep-reinforcement-learning/ Demystifying Deep Reinforcement Learning] by [http://www.nervanasys.com/author/tambet/ Tambet Matiisen], [http://www.nervanasys.com/ Nervana], December 21, 2015</ref>
'''2014'''
* [[Ian Goodfellow]], [[Jean Pouget-Abadie]], [[Mehdi Mirza]], [[Bing Xu]], [[David Warde-Farley]], [[Sherjil Ozair]], [[Aaron Courville]], [[Yoshua Bengio]] ('''2014'''). ''Generative Adversarial Networks''. [https://arxiv.org/abs/1406.2661v1 arXiv:1406.2661v1] <ref>[https://en.wikipedia.org/wiki/Generative_adversarial_networks Generative adversarial networks from Wikipedia]</ref>
* [[Christopher Clark]], [[Amos Storkey]] ('''2014'''). ''Teaching Deep Convolutional Neural Networks to Play Go''. [http://arxiv.org/abs/1412.3409 arXiv:1412.3409] <ref>[http://computer-go.org/pipermail/computer-go/2014-December/007010.html Teaching Deep Convolutional Neural Networks to Play Go] by [[Hiroshi Yamashita]], [http://computer-go.org/pipermail/computer-go/ The Computer-go Archives], December 14, 2014</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=54663 Teaching Deep Convolutional Neural Networks to Play Go] by [[Michel Van den Bergh]], [[CCC]], December 16, 2014</ref>
* [[Chris J. Maddison]], [[Shih-Chieh Huang|Aja Huang]], [[Ilya Sutskever]], [[David Silver]] ('''2014'''). ''Move Evaluation in Go Using Deep Convolutional Neural Networks''. [http://arxiv.org/abs/1412.6564v1 arXiv:1412.6564v1] » [[Go]]
* [[Ilya Sutskever]], [https://research.google.com/pubs/OriolVinyals.html Oriol Vinyals], [https://www.linkedin.com/in/quoc-v-le-319b5a8 Quoc V. Le] ('''2014'''). ''Sequence to Sequence Learning with Neural Networks''. [https://arxiv.org/abs/1409.3215 arXiv:1409.3215]
'''2015'''
* [http://michaelnielsen.org/ Michael Nielsen] ('''2015'''). ''[http://neuralnetworksanddeeplearning.com/ Neural networks and deep learning]''. Determination Press
* [[Mathematician#GEHinton|Geoffrey E. Hinton]], [https://research.google.com/pubs/OriolVinyals.html Oriol Vinyals], [https://en.wikipedia.org/wiki/Jeff_Dean_(computer_scientist) Jeff Dean] ('''2015'''). ''Distilling the Knowledge in a Neural Network''. [https://arxiv.org/abs/1503.02531 arXiv:1503.02531]
* [[James L. McClelland]] ('''2015'''). ''[https://web.stanford.edu/group/pdplab/pdphandbook/handbook3.html#handbookch10.html Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises]''. Second Edition, [https://web.stanford.edu/group/pdplab/pdphandbook/handbookli1.html Contents]
* [[Gábor Melis]] ('''2015'''). ''[http://jmlr.org/proceedings/papers/v42/meli14.html Dissecting the Winning Solution of the HiggsML Challenge]''. [https://nips.cc/Conferences/2014 NIPS 2014]
* [https://scholar.google.nl/citations?user=yyIoQu4AAAAJ Diederik P. Kingma], [https://scholar.google.ca/citations?user=ymzxRhAAAAAJ&hl=en Jimmy Lei Ba] ('''2015'''). ''Adam: A Method for Stochastic Optimization''. [https://arxiv.org/abs/1412.6980v8 arXiv:1412.6980v8], [http://www.iclr.cc/doku.php?id=iclr2015:main ICLR 2015] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=61948 Arasan 19.2] by [[Jon Dart]], [[CCC]], November 03, 2016 » [[Arasan#Tuning|Arasan's Tuning]]</ref>
* [[Volodymyr Mnih]], [[Koray Kavukcuoglu]], [[David Silver]], [[Andrei A. Rusu]], [[Joel Veness]], [[Marc G. Bellemare]], [[Alex Graves]], [[Martin Riedmiller]], [[Andreas K. Fidjeland]], [[Georg Ostrovski]], [[Stig Petersen]], [[Charles Beattie]], [[Amir Sadik]], [[Ioannis Antonoglou]], [[Helen King]], [[Dharshan Kumaran]], [[Daan Wierstra]], [[Shane Legg]], [[Demis Hassabis]] ('''2015'''). ''[http://www.nature.com/nature/journal/v518/n7540/abs/nature14236.html Human-level control through deep reinforcement learning]''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 518
* [[Jürgen Schmidhuber]] ('''2015'''). ''[http://people.idsia.ch/~juergen/deep-learning-overview.html Deep Learning in Neural Networks: An Overview]''. [https://en.wikipedia.org/wiki/Neural_Networks_(journal) Neural Networks], Vol. 61
* [https://scholar.google.fr/citations?user=MN9Kfg8AAAAJ&hl=en Zachary C. Lipton], [https://www.linkedin.com/in/john-berkowitz-92b24a7b John Berkowitz], [[Charles Elkan]] ('''2015'''). ''A Critical Review of Recurrent Neural Networks for Sequence Learning''. [https://arxiv.org/abs/1506.00019 arXiv:1506.00019v4]
* [[Guillaume Desjardins]], [[Karen Simonyan]], [[Razvan Pascanu]], [[Koray Kavukcuoglu]] ('''2015'''). ''Natural Neural Networks''. [https://arxiv.org/abs/1507.00210 arXiv:1507.00210]
* [[Barak Oshri]], [[Nishith Khandwala]] ('''2015'''). ''Predicting Moves in Chess using Convolutional Neural Networks''. [http://cs231n.stanford.edu/reports/ConvChess.pdf pdf] <ref>[https://github.com/BarakOshri/ConvChess GitHub - BarakOshri/ConvChess: Predicting Moves in Chess Using Convolutional Neural Networks]</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=63458 ConvChess CNN] by [[Brian Richardson]], [[CCC]], March 15, 2017</ref>
* [https://en.wikipedia.org/wiki/Yann_LeCun Yann LeCun], [[Yoshua Bengio]], [[Mathematician#GEHinton|Geoffrey E. Hinton]] ('''2015'''). ''[http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html Deep Learning]''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 521 <ref>[[Jürgen Schmidhuber]] ('''2015''') ''[http://people.idsia.ch/~juergen/deep-learning-conspiracy.html Critique of Paper by "Deep Learning Conspiracy" (Nature 521 p 436)]''.</ref>
* [[Matthew Lai]] ('''2015'''). ''Giraffe: Using Deep Reinforcement Learning to Play Chess''. M.Sc. thesis, [https://en.wikipedia.org/wiki/Imperial_College_London Imperial College London], [http://arxiv.org/abs/1509.01549v1 arXiv:1509.01549v1] » [[Giraffe]]
* [[Nikolai Yakovenko]], [[Liangliang Cao]], [[Colin Raffel]], [[James Fan]] ('''2015'''). ''Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games''. [https://arxiv.org/abs/1509.06731 arXiv:1509.06731]
* [[Yuandong Tian]], [[Yan Zhu]] ('''2015'''). ''Better Computer Go Player with Neural Network and Long-term Prediction''. [http://arxiv.org/abs/1511.06410 arXiv:1511.06410] <ref>[http://www.technologyreview.com/view/544181/how-facebooks-ai-researchers-built-a-game-changing-go-engine/?utm_campaign=socialsync&utm_medium=social-post&utm_source=facebook How Facebook’s AI Researchers Built a Game-Changing Go Engine | MIT Technology Review], December 04, 2015</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=58514 Combining Neural Networks and Search techniques (GO)] by Michael Babigian, [[CCC]], December 08, 2015</ref> » [[Go]]
* [[Peter H. Jin]], [[Kurt Keutzer]] ('''2015'''). ''Convolutional Monte Carlo Rollouts in Go''. [http://arxiv.org/abs/1512.03375 arXiv:1512.03375]
* [https://scholar.google.com/citations?user=DhtAFkwAAAAJ Kaiming He], [https://scholar.google.com/citations?user=yuB-cfoAAAAJ&hl=en Xiangyu Zhang], [http://shaoqingren.com/ Shaoqing Ren], [http://www.jiansun.org/ Jian Sun] ('''2015'''). ''Deep Residual Learning for Image Recognition''. [https://arxiv.org/abs/1512.03385 arXiv:1512.03385]
* [[Nicolas Heess]], [[Jonathan J. Hunt]], [[Timothy Lillicrap]], [[David Silver]] ('''2015'''). ''Memory-based control with recurrent neural networks''. [https://arxiv.org/abs/1512.04455 arXiv:1512.04455]
'''2016'''
* [[Dharshan Kumaran]], [[Demis Hassabis]], [[James L. McClelland]] ('''2016'''). ''What learning systems do intelligent agents need? Complementary Learning Systems Theory Updated''. [https://en.wikipedia.org/wiki/Trends_in_Cognitive_Sciences Trends in Cognitive Sciences], Vol. 20, No. 7, [https://drive.google.com/file/d/0B-Nvsz4idhaeVEZYMEVaWkFjLVU/view pdf]
* [[Ziyu Wang]], [[Nando de Freitas]], [[Marc Lanctot]] ('''2016'''). ''Dueling Network Architectures for Deep Reinforcement Learning''. [http://arxiv.org/abs/1511.06581 arXiv:1511.06581]
* [[David Silver]], [[Shih-Chieh Huang|Aja Huang]], [[Chris J. Maddison]], [[Arthur Guez]], [[Laurent Sifre]], [[George van den Driessche]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Veda Panneershelvam]], [[Marc Lanctot]], [[Sander Dieleman]], [[Dominik Grewe]], [[John Nham]], [[Nal Kalchbrenner]], [[Ilya Sutskever]], [[Timothy Lillicrap]], [[Madeleine Leach]], [[Koray Kavukcuoglu]], [[Thore Graepel]], [[Demis Hassabis]] ('''2016'''). ''[http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html Mastering the game of Go with deep neural networks and tree search]''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 529 » [[AlphaGo]]
* [[Tobias Graf]], [[Marco Platzner]] ('''2016'''). ''Using Deep Convolutional Neural Networks in Monte Carlo Tree Search''. [[CG 2016]]
* [[Shih-Chieh Huang|Aja Huang]] ('''2016'''). ''[[CG 2016#Keynote|AlphaGo: Combining Deep Neural Networks with Tree Search]]''. [[CG 2016]], Keynote Lecture
* [[Peter H. Jin]] , [[Kurt Keutzer]] ('''2016'''). ''Convolutional Monte Carlo Rollouts for Computer Go''. [[CG 2016]]
* [[Hung Guei]], [[Tinghan Wei]], [[Jin-Bo Huang]], [[I-Chen Wu]] ('''2016'''). ''An Empirical Study on Applying Deep Reinforcement Learning to the Game 2048''. [[CG 2016]]
* [[Omid David|Omid E. David]], [[Nathan S. Netanyahu]], [[Lior Wolf]] ('''2016'''). ''[http://link.springer.com/chapter/10.1007%2F978-3-319-44781-0_11 DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess]''. [http://icann2016.org/ ICAAN 2016], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science], Vol. 9887, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [http://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf pdf preprint] » [[DeepChess]] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=61748 DeepChess: Another deep-learning based chess program] by [[Matthew Lai]], [[CCC]], October 17, 2016</ref> <ref>[http://icann2016.org/index.php/conference-programme/recipients-of-the-best-paper-awards/ ICANN 2016 | Recipients of the best paper awards]</ref>
* [[Dror Sholomon]], [[Omid David|Omid E. David]], [[Nathan S. Netanyahu]] ('''2016'''). ''[http://link.springer.com/chapter/10.1007/978-3-319-44781-0_21 DNN-Buddies: A Deep Neural Network-Based Estimation Metric for the Jigsaw Puzzle Problem]''. [http://icann2016.org/ ICAAN 2016], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science], Vol. 9887, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer] <ref>[https://en.wikipedia.org/wiki/Jigsaw_puzzle Jigsaw puzzle from Wikipedia]</ref>
* [[Ian Goodfellow]], [[Yoshua Bengio]], [[Aaron Courville]] ('''2016'''). ''[http://www.deeplearningbook.org/ Deep Learning]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Volodymyr Mnih]], [[Adrià Puigdomènech Badia]], [[Mehdi Mirza]], [[Alex Graves]], [[Timothy Lillicrap]], [[Tim Harley]], [[David Silver]], [[Koray Kavukcuoglu]] ('''2016'''). ''Asynchronous Methods for Deep Reinforcement Learning''. [https://arxiv.org/abs/1602.01783 arXiv:1602.01783v2]
* [https://en.wikipedia.org/wiki/Patricia_Churchland Patricia Churchland], [[Terrence J. Sejnowski]] ('''2016'''). ''[https://mitpress.mit.edu/books/computational-brain-0 The Computational Brain, 25th Anniversary Edition]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Audrūnas Gruslys]], [[Rémi Munos]], [[Ivo Danihelka]], [[Marc Lanctot]], [[Alex Graves]] ('''2016'''). ''Memory-Efficient Backpropagation Through Time''. [https://arxiv.org/abs/1606.03401v1 arXiv:1606.03401]
* [[Andrei A. Rusu]], [[Neil C. Rabinowitz]], [[Guillaume Desjardins]], [[Hubert Soyer]], [[James Kirkpatrick]], [[Koray Kavukcuoglu]], [[Razvan Pascanu]], [[Raia Hadsell]] ('''2016'''). ''Progressive Neural Networks''. [https://arxiv.org/abs/1606.04671 arXiv:1606.04671]
* [[James Kirkpatrick]], [[Razvan Pascanu]], [[Neil C. Rabinowitz]], [[Joel Veness]], [[Guillaume Desjardins]], [[Andrei A. Rusu]], [[Kieran Milan]], [[John Quan]], [[Tiago Ramalho]], [[Agnieszka Grabska-Barwinska]], [[Demis Hassabis]], [[Claudia Clopath]], [[Dharshan Kumaran]], [[Raia Hadsell]] ('''2016'''). ''Overcoming catastrophic forgetting in neural networks''. [https://arxiv.org/abs/1612.00796 arXiv:1612.00796]
'''2017'''
* [[Yutian Chen]], [[Matthew W. Hoffman]], [[Sergio Gomez Colmenarejo]], [[Misha Denil]], [[Timothy Lillicrap]], [[Matthew Botvinick]], [[Nando de Freitas]] ('''2017'''). ''Learning to Learn without Gradient Descent by Gradient Descent''. [https://arxiv.org/abs/1611.03824v6 arXiv:1611.03824v6], [http://dblp.uni-trier.de/db/conf/icml/icml2017.html ICML 2017]
* [[Muthuraman Chidambaram]], [[Yanjun Qi]] ('''2017'''). ''Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently''. [https://arxiv.org/abs/1702.06762v1 arXiv:1702.06762v1] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=63252 Using GAN to play chess] by Evgeniy Zheltonozhskiy, [[CCC]], February 23, 2017</ref>
* [[Matej Moravčík]], [[Martin Schmid]], [[Neil Burch]], [[Viliam Lisý]], [[Dustin Morrill]], [[Nolan Bard]], [[Trevor Davis]], [[Kevin Waugh]], [[Michael Johanson]], [[Michael Bowling]] ('''2017'''). ''[http://science.sciencemag.org/content/356/6337/508 DeepStack: Expert-level artificial intelligence in heads-up no-limit poker]''. [https://en.wikipedia.org/wiki/Science_(journal) Science], Vol. 356, No. 6337
* [[David Silver]], [[Julian Schrittwieser]], [[Karen Simonyan]], [[Ioannis Antonoglou]], [[Shih-Chieh Huang|Aja Huang]], [[Arthur Guez]], [[Thomas Hubert]], [[Lucas Baker]], [[Matthew Lai]], [[Adrian Bolton]], [[Yutian Chen]], [[Timothy Lillicrap]], [[Fan Hui]], [[Laurent Sifre]], [[George van den Driessche]], [[Thore Graepel]], [[Demis Hassabis]] ('''2017'''). ''[https://www.nature.com/nature/journal/v550/n7676/full/nature24270.html Mastering the game of Go without human knowledge]''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 550 <ref>[https://deepmind.com/blog/alphago-zero-learning-scratch/ AlphaGo Zero: Learning from scratch] by [[Demis Hassabis]] and [[David Silver]], [[DeepMind]], October 18, 2017</ref>
* [[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2017'''). ''Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm''. [https://arxiv.org/abs/1712.01815 arXiv:1712.01815] » [[Neural Networks#AlphaZero|AlphaZero]] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=65909 Google's AlphaGo team has been working on chess] by [[Peter Kappler]], [[CCC]], December 06, 2017</ref>
* [[Tristan Cazenave]] ('''2017'''). ''[http://ieeexplore.ieee.org/document/7875402/ Residual Networks for Computer Go]''. [[IEEE#TOCIAIGAMES|IEEE Transactions on Computational Intelligence and AI in Games]], Vol. PP, No. 99, [http://www.lamsade.dauphine.fr/~cazenave/papers/resnet.pdf pdf] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=65923 Residual Networks for Computer Go] by Brahim Hamadicharef, [[CCC]], December 07, 2017</ref>

=Blog & Forum Posts=
==1996 ...==
* [https://groups.google.com/d/msg/rec.games.chess.computer/ZFTi2Bh8Qsk/NwZMSgRibS4J Q: Neural Nets/Genetic Algor. and Chess] by Jeff Hamm, [[Computer Chess Forums|rgcc]], March 01, 1996
* [https://groups.google.com/d/msg/rec.games.chess.computer/uVEujVXi7cw/My9fE3w9yzYJ neural networks for pawn-structures] by Romain Slootmaekers, [[Computer Chess Forums|rgcc]], March 29, 1996
* [https://groups.google.com/d/msg/rec.games.chess.computer/f2Nf5PjxrV0/JhSLvVzWPAgJ Neural Network based Chessprogram] by Michael Niemeck, [[Computer Chess Forums|rgcc]], April 16, 1996
* [https://groups.google.com/d/msg/rec.games.chess.computer/8CcXkJ5vOiw/bjxLqThd4tUJ Neural Nets in Chess? Question to experts] by George R. Barrett, [[Computer Chess Forums|rgcc]], February 02, 1997
* [https://groups.google.com/d/msg/rec.games.chess.computer/bd2LislJ_-8/Czr0M_HLwa0J Chess using Neural Networks/Fuzzy Logic] by [[Kumar Chellapilla]], [[Computer Chess Forums|rgcc]], February 12, 1997
* [https://www.stmintz.com/ccc/index.php?id=11870 Evaluation by neural network ?] by [[Mark Taylor]], [[CCC]], November 10, 1997
: [https://www.stmintz.com/ccc/index.php?id=11893 Re: Evaluation by neural network ?] by [[Jay Scott]], [[CCC]], November 10, 1997 <ref>[[Alois Heinz]] ('''1994'''). ''[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.55.3994 Efficient Neural Net α-β-Evaluators]''. [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.55.3994&rep=rep1&type=pdf pdf]</ref>
* [https://groups.google.com/d/msg/rec.games.chess.computer/cBoAi473cjg/nJctqxz-59EJ neural network and chess] by Yeeming Jih, [[Computer Chess Forums|rgcc]], April 23, 1998
* [https://www.stmintz.com/ccc/index.php?id=25139 Chess, Backgammon and Neural Nets (NN)] by [[Torsten Schoop]], [[CCC]], August 20, 1998
* [https://www.stmintz.com/ccc/index.php?id=41068 Chess and Neural Networks] by Frank Schubert, [[CCC]], January 27, 1999
* [https://groups.google.com/d/msg/rec.games.chess.computer/Ov0DhO-Opdo/VNtsNJt_eIEJ Neural networks] by Bill Keller, [[Computer Chess Forums|rgcc]], September 01, 1999
* [https://www.stmintz.com/ccc/index.php?id=83443 neural computing in eval function] by [[Tijs van Dam]], [[CCC]], December 20, 1999
==2000 ...==
* [https://groups.google.com/d/msg/rec.games.chess.computer/xthKCFRJkeM/BHukuAm1ne4J Whatever happened to Neural Network Chess programs?] by Ray Lopez, [[Computer Chess Forums|rgcc]], March 26, 2000
: [https://groups.google.com/d/msg/rec.games.chess.computer/xthKCFRJkeM/ZgORiY9-gF0J Re: Whatever happened to Neural Network Chess programs?] by [[Andy Walker]], [[Computer Chess Forums|rgcc]], March 28, 2000 » [[Advances in Computer Chess 1]], [[Ron Atkin]]
: [https://groups.google.com/d/msg/rec.games.chess.computer/xthKCFRJkeM/CwRxa1j7Q1IJ Combining Neural Networks and Alpha-Beta] by [[Matthias Lüscher]], [[Computer Chess Forums|rgcc]], April 01, 2000 » [[Chessterfield]]
* [https://www.stmintz.com/ccc/index.php?id=116293 Neural Networks in Chess] by [[Guy Haworth]], [[CCC]], June 23, 2000
* [https://groups.google.com/d/msg/rec.games.chess.computer/IgS1pwQYF3E/2jx2HTICOw0J Artificial Neural Networks for Chess] by Jet Nebula, [[Computer Chess Forums|rgcc]], April 02, 2002
* [https://www.stmintz.com/ccc/index.php?id=304075 chess and neural networks] by [[Ralph Stoesser]], [[CCC]], July 01, 2003
* [https://www.stmintz.com/ccc/index.php?id=358770 Presentation for a neural net learning chess program] by [[Dann Corbit]], [[CCC]], April 06, 2004 <ref> [[Mathieu Autonès]], [[Aryel Beck]], [[Phillippe Camacho]], [[Nicolas Lassabe]], [[Hervé Luga]], [[François Scharffe]] ('''2004'''). ''[http://link.springer.com/chapter/10.1007/978-3-540-24650-3_1 Evaluation of Chess Position by Modular Neural network Generated by Genetic Algorithm]''. [http://www.informatik.uni-trier.de/~ley/db/conf/eurogp/eurogp2004.html#AutonesBCLLS04 EuroGP 2004]</ref>
: [https://www.stmintz.com/ccc/index.php?id=359097 Neural nets in backgammon] by [[Albert Silver]], [[CCC]], April 07, 2004
* [https://groups.google.com/d/msg/rec.games.chess.computer/D8ug0bq02Cs/SBikPRjdhJEJ Chess Neural Network: ANOTHER VICTORY FOR OCTAVIUS!] by [[Luke Pellen]], [[Computer Chess Forums|rgcc]], May 04, 2004
==2005 ...==
* [http://www.talkchess.com/forum/viewtopic.php?t=16162 designing neural networks] by [[Gerd Isenberg]], [[CCC]], August 31, 2007
* [http://www.talkchess.com/forum/viewtopic.php?t=29056 naive bayes classifier] by [[Don Dailey]], [[CCC]], July 21, 2009 <ref>[https://en.wikipedia.org/wiki/Naive_Bayes_classifier Naive Bayes classifier from Wikipedia]</ref>
==2010 ...==
* [http://www.talkchess.com/forum/viewtopic.php?t=31545 Chess program with Artificial Neural Networks (ANN)?] by [[Stephan Vermeire]], [[CCC]], January 07, 2010
: [http://www.talkchess.com/forum/viewtopic.php?topic_view=threads&p=316511&t=31545 Re: Chess program with Artificial Neural Networks (ANN)?] by [[Gian-Carlo Pascutto]], [[CCC]], January 07, 2010 » [[Stoofvlees]]
: [http://www.talkchess.com/forum/viewtopic.php?topic_view=threads&p=316769&t=31545 Re: Chess program with Artificial Neural Networks (ANN)?] by [[Gian-Carlo Pascutto]], [[CCC]], January 08, 2010
: [http://www.talkchess.com/forum/viewtopic.php?topic_view=threads&p=316926&t=31545 Re: Chess program with Artificial Neural Networks (ANN)?] by [[Volker Annuss]], [[CCC]], January 08, 2010 » [[Hermann]]
* [http://www.talkchess.com/forum/viewtopic.php?t=41300 Is there place for neural networks in chess engines?] by E Diaz, [[CCC]], December 02, 2011
* [http://stats.stackexchange.com/questions/63152/what-does-the-hidden-layer-in-a-neural-network-compute What does the hidden layer in a neural network compute?] by GeorgeMcDowd, [http://stats.stackexchange.com/ Cross Validated], July 02, 2013
* [http://computer-go.org/pipermail/computer-go/2014-December/007046.html Move Evaluation in Go Using Deep Convolutional Neural Networks] by [[Shih-Chieh Huang|Aja Huang]], [http://computer-go.org/pipermail/computer-go/ The Computer-go Archives], December 19, 2014
==2015 ...==
* [http://www.talkchess.com/forum/viewtopic.php?t=55437 If you are interested in machine learning and Python !] by [http://www.talkchess.com/forum/profile.php?mode=viewprofile&u=881 Ruxy Sylwyka], [[CCC]], February 23, 2015
* [http://talkchess.com/forum/viewtopic.php?t=56913 *First release* Giraffe, a new engine based on deep learning] by [[Matthew Lai]], [[CCC]], July 08, 2015 » [[Giraffe]]
'''2016'''
* [https://www.game-ai-forum.org/viewtopic.php?f=2&t=75 Neural networks for Spanish checkers and beyond] by [[Alvaro Cardoso|Alvaro]], [[Computer Chess Forums|Game-AI Forum]], January 01, 2016
* [http://int8.io/chess-position-evaluation-with-convolutional-neural-networks-in-julia/ Chess position evaluation with convolutional neural network in Julia] by [[Kamil Czarnogorski]], [http://int8.io/ Machine learning with Julia and python], April 02, 2016 <ref>[https://github.com/pluskid/Mocha.jl GitHub - pluskid/Mocha.jl: Deep Learning framework for Julia]</ref>
* [http://www.talkchess.com/forum/viewtopic.php?t=60883 Deep Learning Chess Engine ?] by Eren Yavuz, [[CCC]], July 21, 2016
: [http://www.talkchess.com/forum/viewtopic.php?t=60883&start=4 Re: Deep Learning Chess Engine ?] by [[Alexandru Mosoi]], [[CCC]], July 21, 2016 » [[Zurichess]]
: [http://www.talkchess.com/forum/viewtopic.php?t=60883&start=7 Re: Deep Learning Chess Engine ?] by [[Matthew Lai]], [[CCC]], August 04, 2016 » [[Giraffe]] <ref>[https://en.wikipedia.org/wiki/Rectifier_(neural_networks) Rectifier (neural networks) from Wikipedia]</ref>
* [http://www.talkchess.com/forum/viewtopic.php?t=61312 Neuronet plus conventional approach combined?] by [[Rasmus Althoff]], [[CCC]], September 02, 2016
* [http://www.talkchess.com/forum/viewtopic.php?t=61748 DeepChess: Another deep-learning based chess program] by [[Matthew Lai]], [[CCC]], October 17, 2016 » [[DeepChess]]
* [http://www.talkchess.com/forum/viewtopic.php?t=61801 The scaling of Deep Learning MCTS Go engines] by [[Kai Laskos]], [[CCC]], October 23, 2016 » [[Neural Networks#Deep|Deep Learning]], [[Go]], [[Monte-Carlo Tree Search|MCTS]]
'''2017'''
* [http://www.talkchess.com/forum/viewtopic.php?t=63063 Deep Pink: a chess engine using deep learning] by [[Chao Ma]], [[CCC]], February 05, 2017 » [[Deep Pink]]
* [http://www.talkchess.com/forum/viewtopic.php?t=63252 Using GAN to play chess] by Evgeniy Zheltonozhskiy, [[CCC]], February 23, 2017 <ref>[[Muthuraman Chidambaram]], [[Yanjun Qi]] ('''2017'''). ''Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently''. [https://arxiv.org/abs/1702.06762v1 arXiv:1702.06762v1]</ref>
* [http://www.talkchess.com/forum/viewtopic.php?t=64096 Is AlphaGo approach unsuitable to chess?] by Mel Cooper, [[CCC]], May 27, 2017 » [[AlphaGo]], [[Deep Learning]], [[Giraffe]]
: [http://www.talkchess.com/forum/viewtopic.php?t=64096&start=12 Re: Is AlphaGo approach unsuitable to chess?] by [[Peter Österlund]], [[CCC]], May 31, 2017 » [[Texel]]
* [https://groups.google.com/d/msg/computer-go-archive/WImAk15gRN4/bhA7kSAnBgAJ Neural nets for Go - chain pooling?] by [[David Wu]], [https://groups.google.com/forum/#!forum/computer-go-archive Computer Go Archive], August 18, 2017
* [https://deepmind.com/blog/alphago-zero-learning-scratch/ AlphaGo Zero: Learning from scratch] by [[Demis Hassabis]] and [[David Silver]], [[DeepMind]], October 18, 2017
* [http://www.talkchess.com/forum/viewtopic.php?t=65481 We are doomed - AlphaGo Zero, learning only from basic rules] by [[Vincent Lejeune]], [[CCC]], October 18, 2017
* [http://www.talkchess.com/forum/viewtopic.php?t=65484 AlphaGo Zero] by [[Alberto Sanjuan]], [[CCC]], October 19, 2017
* [https://groups.google.com/d/msg/computer-go-archive/9DNayZWKXfk/Pk9yBx1lAgAJ Zero performance] by [[Gian-Carlo Pascutto]], [https://groups.google.com/forum/#!forum/computer-go-archive Computer Go Archive], October 20, 2017
* [https://groups.google.com/d/msg/computer-go-archive/ulsErJxW3jc/hMIhtiBYAgAJ Re: AlphaGo Zero] by [[Hendrik Baier]], [https://groups.google.com/forum/#!forum/computer-go-archive Computer Go Archive], October 20, 2017
* [http://www.talkchess.com/forum/viewtopic.php?t=65715 Neural networks for chess position evaluation- request] by [[Kamil Czarnogorski]], [[CCC]], November 13, 2017 » [[Evaluation]]
* [http://www.talkchess.com/forum/viewtopic.php?t=65829 AlphaGo's evaluation function] by Jens Kipper, [[CCC]], November 26, 2017
* [http://www.talkchess.com/forum/viewtopic.php?t=65831 Neural Nets can't explain what they do and this is a problem] by Myron Samsin, November 26, 2017
* [http://www.talkchess.com/forum/viewtopic.php?t=65909 Google's AlphaGo team has been working on chess] by [[Peter Kappler]], [[CCC]], December 06, 2017 » [[AlphaZero]]
* [http://www.talkchess.com/forum/viewtopic.php?t=65910 Historic Milestone: AlphaZero] by Miguel Castanuela, [[CCC]], December 06, 2017
* [http://www.talkchess.com/forum/viewtopic.php?t=66013 An AlphaZero inspired project] by [[Truls Edvard Stokke]], [[CCC]], December 14, 2017 » [[AlphaZero]]
'''2018'''
* [http://www.talkchess.com/forum/viewtopic.php?t=66280 Announcing lczero] by [[Gary Linscott|Gary]], [[CCC]], January 09, 2018 » [[LCZero]]
* [http://www.talkchess.com/forum/viewtopic.php?t=66443 Connect 4 AlphaZero implemented using Python...] by [[Steve Maughan]], [[CCC]], January 29, 2018 » [[AlphaZero]], [[Connect Four]], [[Python]]
* [http://www.talkchess.com/forum/viewtopic.php?t=66681 3 million games for training neural networks] by [[Álvaro Begué]], [[CCC]], February 24, 2018 » [[Automated Tuning]]
* [http://www.talkchess.com/forum/viewtopic.php?t=66791 Looking inside NNs] by [[J. Wesley Cleveland]], [[CCC]], March 09, 2018

=External Links=
* [https://en.wikipedia.org/wiki/Neural_network_%28disambiguation%29 Neural network (disambiguation) from Wikipedia]
* [https://www.scholarpedia.org/article/Category:Neural_networks Category:Neural networks - Scholarpedia]
* [http://psychology.wikia.com/wiki/Neural_networks Neural networks - Psychology Wiki]
'''Biological'''
* [https://en.wikipedia.org/wiki/Biological_neural_network Biological neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Biological_neuron_model Biological neuron model from Wikipedia]
* [https://en.wikipedia.org/wiki/Computational_neuroscience Computational neuroscience from Wikipedia]
* [https://en.wikipedia.org/wiki/Neuron Neuron from Wikipedia]
* [https://en.wikipedia.org/wiki/Neural_pathway Neural pathway from Wikipedia]
'''ANNs'''
* [https://en.wikipedia.org/wiki/Artificial_neural_network Artificial neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Types_of_artificial_neural_networks Types of artificial neural networks from Wikipedia]
* [https://en.wikibooks.org/wiki/Artificial_Neural_Networks Artificial Neural Networks - Wikibooks]
* [http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html Artificial Neural Networks] by Christos Stergiou and Dimitrios Siganos
* [https://www.dmoz.org/Computers/Artificial_Intelligence/Neural_Networks DMOZ - Computers: Artificial Intelligence: Neural Networks]
* [http://dms.irb.hr/tutorial/tut_nnets_short.php DMS Tutorial - Neural networks]
* [https://en.wikipedia.org/wiki/Helmholtz_machine Helmholtz machine from Wikipedia]
* [http://de.slideshare.net/piuprabhu/chess-end-games-using-neural-networks-presentation Chess end games using Neural Networks]
'''Topics'''
* [https://en.wikipedia.org/wiki/Artificial_neuron Artificial neuron from Wikipedia]
* [https://en.wikipedia.org/wiki/Connectionism Connectionism from Wikipedia]
* [https://en.wikipedia.org/wiki/Convolutional_neural_network Convolutional neural network from Wikipedia]
* [https://wiki.tum.de/display/lfdv/Convolutional+Neural+Networks+for+Image+and+Video+Processing Convolutional Neural Networks for Image and Video Processing], [https://wiki.tum.de/ TUM Wiki], [[Technical University of Munich]]
: [https://wiki.tum.de/display/lfdv/Convolutional+Neural+Networks#ConvolutionalNeuralNetworks-convolution Convolutional Neural Networks]
: [https://wiki.tum.de/display/lfdv/Deep+Residual+Networks Deep Residual Networks]
* [https://en.wikipedia.org/wiki/Deep_learning Deep Learning from Wikipeadia]
* [http://www.scholarpedia.org/article/Deep_Learning Deep Learning - Scholarpedia] by [[Jürgen Schmidhuber]]
* [https://en.wikipedia.org/wiki/Feedforward_neural_network Feedforward neural network from Wikipedia]
* [http://www.scholarpedia.org/article/Fuzzy_neural_network Fuzzy neural network - Scholarpedia]
* [https://en.wikipedia.org/wiki/Generative_adversarial_networks Generative adversarial networks from Wikipedia]
* [https://en.wikipedia.org/wiki/Grossberg_network Grossberg network from Wikipedia]
* [https://en.wikipedia.org/wiki/Modular_neural_network Modular neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Multilayer_perceptron Multilayer perceptron from Wikipedia]
* [https://en.wikipedia.org/wiki/Neocognitron Neocognitron from Wikipedia]
* [http://www.scholarpedia.org/article/Neocognitron Neocognitron - Scholarpedia] by [http://www.scholarpedia.org/article/User:Kunihiko_Fukushima Kunihiko Fukushima]
* [https://en.wikipedia.org/wiki/Neuromorphic_engineering Neuromorphic engineering from Wikipedia]
: [https://en.wikipedia.org/wiki/Neurogrid Neurogrid from Wikipedia]
* [https://en.wikipedia.org/wiki/Perceptron Perceptron from Wikipedia]
: [http://web.csulb.edu/~cwallis/artificialn/History.htm History of the Perceptron]
* [https://en.wikipedia.org/wiki/Physical_neural_network Physical neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Radial_basis_function_network Radial basis function network from Wikipedia]
* [https://en.wikipedia.org/wiki/Random_neural_network Random neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Recursive_neural_network Recursive neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Self-organizing_map Self-organizing map from Wikipedia]
* [https://en.wikipedia.org/wiki/Spiking_neural_network Spiking neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Time_delay_neural_network Time delay neural network from Wikipedia]
'''RNNs'''
* [https://en.wikipedia.org/wiki/Recurrent_neural_network Recurrent neural network from Wikipedia]
* [http://www.scholarpedia.org/article/Recurrent_neural_networks Recurrent neural networks - Scholarpedia]
* [http://people.idsia.ch/~juergen/rnn.html Recurrent Neural Networks] by [[Jürgen Schmidhuber]]
* [https://en.wikipedia.org/wiki/Bidirectional_associative_memory Bidirectional associative memory from Wikipedia]
* [https://en.wikipedia.org/wiki/Boltzmann_machine Boltzmann machine from Wikipedia]
: [https://en.wikipedia.org/wiki/Restricted_Boltzmann_machine Restricted Boltzmann machine from Wikipedia]
* [https://en.wikipedia.org/wiki/Echo_state_network Echo state network]
* [https://en.wikipedia.org/wiki/Hopfield_network Hopfield network from Wikipedia]
* [http://www.scholarpedia.org/article/Hopfield_network Hopfield network - Scholarpedia]
* [https://en.wikipedia.org/wiki/Long_short_term_memory Long short term memory from Wikipedia]
'''Activation Functions'''
* [https://en.wikipedia.org/wiki/Activation_function Activation function from Wikipedia]
* [https://en.wikipedia.org/wiki/Rectifier_(neural_networks) Rectifier (neural networks) from Wikipedia]
* [https://en.wikipedia.org/wiki/Sigmoid_function Sigmoid function from Wikipedia]
'''Backpropagation'''
* [https://en.wikipedia.org/wiki/Backpropagation Backpropagation from Wikipedia]
* [https://en.wikipedia.org/wiki/Backpropagation_through_structure Backpropagation through structure from Wikipedia]
* [https://en.wikipedia.org/wiki/Backpropagation_through_time Backpropagation through time from Wikipedia]
* [https://en.wikipedia.org/wiki/Rprop Rprop from Wikipedia]
* [http://people.idsia.ch/~juergen/who-invented-backpropagation.html Who Invented Backpropagation?] by [[Jürgen Schmidhuber]] (2014, 2015)
'''Gradient'''
* [https://en.wikipedia.org/wiki/Gradient Gradient from Wikipedia]
* [https://en.wikipedia.org/wiki/Del Del from Wikipedia]
* [https://en.wikipedia.org/wiki/Gradient_descent Gradient descent from Wikipedia]
* [https://en.wikipedia.org/wiki/Conjugate_gradient_method Conjugate gradient method from Wikipedia]
* [https://en.wikipedia.org/wiki/Stochastic_gradient_descent Stochastic gradient descent from Wikipedia]
* [https://blogs.princeton.edu/imabandit/2013/04/01/acceleratedgradientdescent/ ORF523: Nesterov’s Accelerated Gradient Descent] by [[Sébastien Bubeck]], [https://blogs.princeton.edu/imabandit/ I’m a bandit], April 1, 2013 » [[Mathematician#YNesterov|Yurii Nesterov]]
* [https://blogs.princeton.edu/imabandit/2014/03/06/nesterovs-accelerated-gradient-descent-for-smooth-and-strongly-convex-optimization/ Nesterov’s Accelerated Gradient Descent for Smooth and Strongly Convex Optimization] by [[Sébastien Bubeck]], [https://blogs.princeton.edu/imabandit/ I’m a bandit], March 6, 2014
* [https://blogs.princeton.edu/imabandit/2015/06/30/revisiting-nesterovs-acceleration/ Revisiting Nesterov’s Acceleration] by [[Sébastien Bubeck]], [https://blogs.princeton.edu/imabandit/ I’m a bandit], June 30, 2015
'''Blogs'''
* [https://theneural.wordpress.com/ Neural Networks Blog] by [[Ilya Sutskever]]
* [https://software.intel.com/en-us/articles/an-introduction-to-neural-networks-with-an-application-to-games An Introduction to Neural Networks with an Application to Games] by [https://www.linkedin.com/pub/dean-p-macri/a/762/68b Dean Macri], [https://en.wikipedia.org/wiki/Intel_Developer_Zone Intel Developer Zone], September 9, 2011
* [http://www.blogger.com/profile/07894297206547597169 John Wakefield's] [http://dynamicnotions.blogspot.com/ Dynamic Notions], a Blog about the evolution of neural networks with [[C sharp|C#]] samples
: [http://dynamicnotions.blogspot.com/2008/09/single-layer-perceptron.html The Single Layer Perceptron]
: [http://dynamicnotions.blogspot.de/2008/09/sigmoid-function-in-c.html The Sigmoid Function in C#]
: [http://dynamicnotions.blogspot.com/2008/09/hidden-neurons-and-feature-space.html Hidden Neurons and Feature Space]
: [http://dynamicnotions.blogspot.com/2008/09/training-neural-networks-using-back.html Training Neural Networks Using Back Propagation in C#]
: [http://dynamicnotions.blogspot.com/2008/09/data-mining-with-artificial-neural.html Data Mining with Artificial Neural Networks (ANN)]
* [http://www.millermattson.com/dave/?tag=neural-net Dave Miller Blog - Tag: Neural Net]
: [https://vimeo.com/19569529 Neural Net in C++ Tutorial] on [https://en.wikipedia.org/wiki/Vimeo Vimeo] (also on [https://www.youtube.com/watch?v=KkwX7FkLfug YouTube])
* [https://theclevermachine.wordpress.com/2014/09/11/a-gentle-introduction-to-artificial-neural-networks/ A Gentle Introduction to Artificial Neural Networks] by [https://scholar.google.com/citations?user=oY7AJUgAAAAJ&hl=en Dustin Stansbury], [https://theclevermachine.wordpress.com/about-theclevermachine/ The Clever Machine], September 11, 2014
* [http://erikbern.com/2014/11/29/deep-learning-for-chess/ Deep learning for… chess] by [[Erik Bernhardsson]], November 29, 2014 <ref>[https://github.com/erikbern/deep-pink erikbern/deep-pink · GitHub]</ref>
* [http://blog.dominodatalab.com/gpu-computing-and-deep-learning/ Faster deep learning with GPUs and Theano] by [https://www.linkedin.com/pub/manojit-nandi/35/688/384 Manojit Nandi], August 05, 2015 » [[GPU]], [[Python]]
* [https://deepmind.com/blog/enabling-continual-learning-in-neural-networks/ Enabling Continual Learning in Neural Networks] by [[James Kirkpatrick]], [[Joel Veness]] et al., [[DeepMind]], March 13, 2017
* [https://blog.waya.ai/deep-residual-learning-9610bb62c355 Understand Deep Residual Networks — a simple, modular learning framework that has redefined state-of-the-art] by [https://blog.waya.ai/@waya.ai Michael Dietz], [https://blog.waya.ai/ Waya.ai], May 02, 2017
* [https://medium.com/applied-data-science/how-to-build-your-own-alphazero-ai-using-python-and-keras-7f664945c188 How to build your own AlphaZero AI using Python and Keras] by [https://www.linkedin.com/in/davidtfoster/ David Foster], January 26, 2018 » [[AlphaZero]], [[Connect Four]], [[Python]]
'''Libraries'''
* [https://en.wikipedia.org/wiki/Eigen_%28C%2B%2B_library%29 Eigen (C++ library) from Wikipedia]
* [http://leenissen.dk/fann/wp/ Fast Artificial Neural Network Library (FANN)]
* [https://wiki.python.org/moin/PythonForArtificialIntelligence PythonForArtificialIntelligence - Python Wiki] [[Python]]
* [https://en.wikipedia.org/wiki/TensorFlow TensorFlow from Wikipedia]
'''Software'''
* [https://en.wikipedia.org/wiki/Neural_network_software Neural network software from Wikipedia]
: [https://en.wikipedia.org/wiki/Neural_Lab Neural Lab from Wikipedia]
: [https://en.wikipedia.org/wiki/SNNS SNNS from Wikipedia]
* [https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software Comparison of deep learning software from Wikipedia]
'''Courses'''
* [http://www.cedar.buffalo.edu/~srihari/CSE574/index.html Machine Learning and Probabilistic Graphical Models: Course Materials - 5. Neural Networks] by [https://en.wikipedia.org/wiki/Sargur_Srihari Sargur Srihari], [https://en.wikipedia.org/wiki/University_at_Buffalo University at Buffalo]
* [http://www.holehouse.org/mlclass/08_Neural_Networks_Representation.html Neural Networks - Representation] from [http://www.holehouse.org/mlclass/index.html Stanford Machine Learning] by [[Andrew Ng]]
* [http://www.holehouse.org/mlclass/09_Neural_Networks_Learning.html Neural Networks - Learning] from [http://www.holehouse.org/mlclass/index.html Stanford Machine Learning] by [[Andrew Ng]]
* [http://nbviewer.ipython.org/github/stephencwelch/Neural-Networks-Demysitifed/tree/master/ Neural Networks Demystified] by [https://twitter.com/stephencwelch Stephen Welch], [http://www.welchlabs.com/ Welch Labs]
: [https://www.youtube.com/watch?v=bxe2T-V8XRs Part 1: Data and Architecture], [https://en.wikipedia.org/wiki/YouTube YouTube] Videos
: [https://www.youtube.com/watch?v=UJwK6jAStmg Part 2: Forward Propagation]
: [https://www.youtube.com/watch?v=5u0jaA3qAGk Part 3: Gradient Descent]
: [https://www.youtube.com/watch?v=GlcnxUlrtek Part 4: Backpropagation]
: [https://www.youtube.com/watch?v=pHMzNW8Agq4 Part 5: Numerical Gradient Checking]
: [https://www.youtube.com/watch?v=9KM9Td6RVgQ Part 6: Training]
: [https://www.youtube.com/watch?v=S4ZUwgesjS8 Part 7: Overfitting, Testing, and Regularization]
* [https://www.youtube.com/watch?v=UdSK7nnJKHU Deep Learning Master Class] by [[Ilya Sutskever]], [https://en.wikipedia.org/wiki/YouTube YouTube] Video
* [https://www.youtube.com/watch?v=Ih5Mr93E-2c&hd=1 Lecture 10 - Neural Networks] from [http://work.caltech.edu/telecourse.html Learning From Data - Online Course (MOOC)] by [https://en.wikipedia.org/wiki/Yaser_Abu-Mostafa Yaser Abu-Mostafa], [https://en.wikipedia.org/wiki/California_Institute_of_Technology Caltech], [https://en.wikipedia.org/wiki/YouTube YouTube] Video
* [https://www.youtube.com/watch?v=q0pm3BrIUFo Lecture 12 - Learning: Neural Nets, Back Propagation] by [[Patrick Winston]], [[Massachusetts Institute of Technology|MIT]], [[Patrick Winston#AI_Lectures|AI Lectures - Fall 2010]] [https://en.wikipedia.org/wiki/YouTube YouTube] Videos
* [http://www.3blue1brown.com/videos/2017/10/9/neural-network Neural Networks] by [https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw 3Blue1Brown], October 9, 2017, [https://en.wikipedia.org/wiki/YouTube YouTube] Videos <ref>[http://www.talkchess.com/forum/viewtopic.php?t=66076 Neural networks (NN) explained] by [[Erin Dame]], [[CCC]], December 20, 2017</ref>
: [https://youtu.be/aircAruvnKk But what *is* a Neural Network? | Chapter 1]
: [https://youtu.be/IHZwWFHWa-w Gradient descent, how neural networks learn | Chapter 2]
: [https://youtu.be/Ilg3gGewQ5U What is backpropagation really doing? | Chapter 3]
: [https://youtu.be/tIeHLnjs5U8 Backpropagation calculus | Appendix to Chapter 3]
'''Music'''
* [https://en.wikipedia.org/wiki/John_Zorn#The_Dreamers The Dreamers] & [[Videos#JohnZorn|John Zorn]] - Gormenghast, [https://en.wikipedia.org/wiki/Pellucidar:_A_Dreamers_Fantabula Pellucidar: A Dreamers Fantabula] (2015), [https://en.wikipedia.org/wiki/YouTube YouTube] Video
: [[Videos#MarcRibot|Marc Ribot]], [https://en.wikipedia.org/wiki/Kenny_Wollesen Kenny Wollesen], [https://en.wikipedia.org/wiki/Joey_Baron Joey Baron], [https://en.wikipedia.org/wiki/Jamie_Saft Jamie Saft], [https://en.wikipedia.org/wiki/Trevor_Dunn Trevor Dunn], [https://en.wikipedia.org/wiki/Cyro_Baptista Cyro Baptista], John Zorn
: {{#evu:https://www.youtube.com/watch?v=97MsK88rjy8|alignment=left|valignment=top}}

=References=
<references />

'''[[Learning|Up one Level]]'''

GerdIsenberg

Bureaucrats, Administrators

25,161

edits

Changes

Neural Networks

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Tools