Changes

Jump to: navigation, search

Neural Networks

1,791 bytes added, 22:13, 6 January 2020
no edit summary
The [https://en.wikipedia.org/wiki/Perceptron perceptron] is an algorithm for [[Supervised Learning|supervised learning]] of [https://en.wikipedia.org/wiki/Binary_classification binary classifiers]. It was the first artificial neural network, introduced in 1957 by [https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] <ref>[https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] ('''1957'''). ''The Perceptron - a Perceiving and Recognizing Automaton''. Report 85-460-1, [https://en.wikipedia.org/wiki/Calspan#History Cornell Aeronautical Laboratory]</ref>, implemented in custom hardware. In its basic form it consists of a single neuron with multiple inputs and associated weights.
[[Supervised Learning|Supervised learning]] is applied using a set D of labeled [https://en.wikipedia.org/wiki/Test_set training data] with pairs of [https://en.wikipedia.org/wiki/Feature_vector feature vectors] (x) and given results as desired output (d), usually started with cleared or randomly initialized weight vector w. The output is calculated by all inputs of a sample, multiplied by its corresponding weights, passing the sum to the activation function f. The difference of desired and actual value is then immediately used modify the weights for all features using a learning rate 0.0 < α <= 1.0:
<pre>
for (j=0, Σ = 0.0; j < nSamples; ++j) {
===Alpha Zero===
In December 2017, the [[Google]] [[DeepMind]] team along with former [[Giraffe]] author [[Matthew Lai]] reported on their generalized [[AlphaZero]] algorithm, combining [[Deep Learning|Deep learning]] with [[Monte-Carlo Tree Search]]. AlphaZero can achieve, tabula rasa, superhuman performance in many challenging domains with some training effort. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved a superhuman level of play in the games of chess and [[Shogi]] as well as Go, and convincingly defeated a world-champion program in each case <ref>[[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2017'''). ''Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm''. [https://arxiv.org/abs/1712.01815 arXiv:1712.01815]</ref>.
<span id="engines"></span>
===NN Chess Programs===
* [[:Category:NN]]
=See also=
* [[Pattern Recognition]]
* [[Temporal Difference Learning]]
<span id="engines"></span>
=NN Chess Programs=
* [[Alexs]]
* [[AlphaZero]]
* [[Arminius]]
* [[Blondie25]]
* [[ChessMaps]]
* [[Chessterfield]]
* [[Deep Pink]]
* [[Giraffe]]
* [[Golch]]
* [[Hermann]]
* [[Leela Chess Zero]]
* [[Morph]]
* [[Nebiyu]]
* [[NeuroChess]]
* [[Octavius]]
* [[RamJet]]
* [[SAL]]
* [[Scorpio]]
* [[Spawkfish]]
* [[Stoofvlees]]
* [[Tempo (engine)|Tempo]]
* [[Zurichess]]
=Selected Publications=
* [[Eric B. Baum]] ('''1989'''). ''[http://papers.nips.cc/paper/226-the-perceptron-algorithm-is-fast-for-non-malicious-distributions The Perceptron Algorithm Is Fast for Non-Malicious Distributions]''. [http://papers.nips.cc/book/advances-in-neural-information-processing-systems-2-1989 NIPS 1989]
* [[Eric B. Baum]] ('''1989'''). ''[http://www.mitpressjournals.org/doi/abs/10.1162/neco.1989.1.2.201#.VfGX0JdpluM A Proposal for More Powerful Learning Algorithms]''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 1, No. 2
* [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/i/Irani:E=_A=.html Erach A. Irani], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/m/Matts:John_P=.html John P. Matts], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/l/Long:John_M=.html John M. Long], [[James R. Slagle]], POSCH group ('''1989'''). ''Using Artificial Neural Nets for Statistical Discovery: Observations after Using Backpropogation, Expert Systems, and Multiple-Linear Regression on Clinical Trial Data''. [[University of Minnesota]], Minneapolis, MN 55455, USA, Complex Systems 3, [http://www.complex-systems.com/pdf/03-3-5.pdf pdf]
* [[Gerald Tesauro]], [[Terrence J. Sejnowski]] ('''1989'''). ''A Parallel Network that Learns to Play Backgammon''. [https://en.wikipedia.org/wiki/Artificial_Intelligence_%28journal%29 Artificial Intelligence], Vol. 39, No. 3
* [[Mathematician#EGelenbe|Erol Gelenbe]] ('''1989'''). ''[http://cognet.mit.edu/journal/10.1162/neco.1989.1.4.502 Random Neural Networks with Negative and Positive Signals and Product Form Solution]''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 1, No. 4
* [[Mathematician#XZhang|Xiru Zhang]], [https://dblp.uni-trier.de/pers/hd/m/McKenna:Michael Michael McKenna], [[Mathematician#JPMesirov|Jill P. Mesirov]], [[David Waltz]] ('''1989'''). ''[http://papers.neurips.cc/paper/281-an-efficient-implementation-of-the-back-propagation-algorithm-on-the-connection-machine-cm-2 An Efficient Implementation of the Back-propagation Algorithm on the Connection Machine CM-2]''. [https://dblp.uni-trier.de/db/conf/nips/nips1989.html NIPS 1989]
==1990 ...==
* [[Mathematician#PWerbos|Paul Werbos]] ('''1990'''). ''Backpropagation Through Time: What It Does and How to Do It''. Proceedings of the [[IEEE]], Vol. 78, No. 10, [http://deeplearning.cs.cmu.edu/pdfs/Werbos.backprop.pdf pdf]
* [https://dblp.uni-trier.de/pers/hd/r/Ruck:Dennis_W= Dennis W. Ruck], [http://spie.org/profile/Steven.Rogers-5480?SSO=1 Steven K. Rogers], [https://dblp.uni-trier.de/pers/hd/k/Kabrisky:Matthew Matthew Kabrisky], [[Mathematician#MEOxley|Mark E. Oxley]], [[Bruce W. Suter]] ('''1990'''). ''[https://ieeexplore.ieee.org/document/80266 The multilayer perceptron as an approximation to a Bayes optimal discriminant function]''. [[IEEE#NN|IEEE Transactions on Neural Networks]], Vol. 1, No. 4
* [https://dblp.uni-trier.de/pers/hd/h/Hellstrom:Benjamin_J= Benjamin J. Hellstrom], [[Laveen Kanal|Laveen N. Kanal]] ('''1990'''). ''[https://ieeexplore.ieee.org/document/5726889 The definition of necessary hidden units in neural networks for combinatorial optimization]''. [https://dblp.uni-trier.de/db/conf/ijcnn/ijcnn1990.html IJCNN 1990]
* [[Mathematician#XZhang|Xiru Zhang]], [https://dblp.uni-trier.de/pers/hd/m/McKenna:Michael Michael McKenna], [[Mathematician#JPMesirov|Jill P. Mesirov]], [[David Waltz]] ('''1990'''). ''[https://www.sciencedirect.com/science/article/pii/016781919090084M The backpropagation algorithm on grid and hypercube architectures]''. [https://www.journals.elsevier.com/parallel-computing Parallel Computing], Vol. 14, No. 3
'''1991'''
* [[Mathematician#SHochreiter|Sepp Hochreiter]] ('''1991'''). ''Untersuchungen zu dynamischen neuronalen Netzen''. Diploma thesis, [[Technical University of Munich|TU Munich]], advisor [[Jürgen Schmidhuber]], [http://people.idsia.ch/~juergen/SeppHochreiter1991ThesisAdvisorSchmidhuber.pdf pdf] (German) <ref>[http://people.idsia.ch/~juergen/fundamentaldeeplearningproblem.html Sepp Hochreiter's Fundamental Deep Learning Problem (1991)] by [[Jürgen Schmidhuber]], 2013</ref>
* [[Pieter Spronck]] ('''1996'''). ''Elegance: Genetic Algorithms in Neural Reinforcement Control''. Master thesis, [[Delft University of Technology]], [http://ticc.uvt.nl/~pspronck/pubs/Elegance.pdf pdf]
* [[Raúl Rojas]] ('''1996'''). ''Neural Networks - A Systematic Introduction''. Springer, available as [http://www.inf.fu-berlin.de/inst/ag-ki/rojas_home/documents/1996/NeuralNetworks/neuron.pdf pdf ebook]
* [[Ida Sprinkhuizen-Kuyper]], [https://dblp.org/pers/hd/b/Boers:Egbert_J=_W= Egbert J. W. Boers] ('''1996'''). ''[https://ieeexplore.ieee.org/abstract/document/6796246 The Error Surface of the Simplest XOR Network Has Only Global Minima]''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 8, No. 6, [http://www.socsci.ru.nl/idak/publications/papers/NeuralComputation.pdf pdf]
'''1997'''
* [[Mathematician#SHochreiter|Sepp Hochreiter]], [[Jürgen Schmidhuber]] ('''1997'''). ''Long short-term memory''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 9, No. 8, [http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf pdf] <ref>[https://en.wikipedia.org/wiki/Long_short_term_memory Long short term memory from Wikipedia]</ref>
* [[Mathematician#GEHinton|Geoffrey E. Hinton]], [[Terrence J. Sejnowski]] (eds.) ('''1999'''). ''[https://mitpress.mit.edu/books/unsupervised-learning Unsupervised Learning: Foundations of Neural Computation]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Peter Dayan]] ('''1999'''). ''Recurrent Sampling Models for the Helmholtz Machine''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 11, No. 3, [http://www.gatsby.ucl.ac.uk/~dayan/papers/rechelm99.pdf pdf] <ref>[https://en.wikipedia.org/wiki/Helmholtz_machine Helmholtz machine from Wikipedia]</ref>
* [[Ida Sprinkhuizen-Kuyper]], [https://dblp.org/pers/hd/b/Boers:Egbert_J=_W= Egbert J. W. Boers] ('''1999'''). ''[https://ieeexplore.ieee.org/document/774274 A local minimum for the 2-3-1 XOR network]''. [[IEEE#NN|IEEE Transactions on Neural Networks]], Vol. 10, No. 4
==2000 ...==
* [[Levente Kocsis]], [[Jos Uiterwijk]], [[Jaap van den Herik]] ('''2000'''). ''[http://link.springer.com/chapter/10.1007%2F3-540-45579-5_11 Learning Time Allocation using Neural Networks]''. [[CG 2000]]
* [[Moshe Sipper]] ('''2002''') ''[http://books.google.com/books/about/Machine_Nature.html?id=fbFQAAAAMAAJ&redir_esc=y Machine Nature: The Coming Age of Bio-Inspired Computing]''. [https://en.wikipedia.org/wiki/McGraw-Hill_Financial McGraw-Hill, New York]
* [[Paul E. Utgoff]], [[David J. Stracuzzi]] ('''2002'''). ''Many-Layered Learning''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 14, No. 10, [http://people.cs.umass.edu/~utgoff/papers/neco-stl.pdf pdf]
* [[Mathematician#MIJordan|Michael I. Jordan]], [[Terrence J. Sejnowski]] (eds.) ('''2002'''). ''[https://mitpress.mit.edu/books/graphical-models Graphical Models: Foundations of Neural Computation]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
'''2003'''
* [[Levente Kocsis]] ('''2003'''). ''Learning Search Decisions''. Ph.D thesis, [[Maastricht University]], [https://project.dke.maastrichtuniversity.nl/games/files/phd/Kocsis_thesis.pdf pdf]
* [[George Rajna]] ('''2016'''). ''Deep Neural Networks''. [http://vixra.org/abs/1609.0126 viXra:1609.0126]
* [[James Kirkpatrick]], [[Razvan Pascanu]], [[Neil C. Rabinowitz]], [[Joel Veness]], [[Guillaume Desjardins]], [[Andrei A. Rusu]], [[Kieran Milan]], [[John Quan]], [[Tiago Ramalho]], [[Agnieszka Grabska-Barwinska]], [[Demis Hassabis]], [[Claudia Clopath]], [[Dharshan Kumaran]], [[Raia Hadsell]] ('''2016'''). ''Overcoming catastrophic forgetting in neural networks''. [https://arxiv.org/abs/1612.00796 arXiv:1612.00796] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70704 catastrophic forgetting] by [[Daniel Shawul]], [[CCC]], May 09, 2019</ref>
* [https://dblp.uni-trier.de/pers/hd/n/Niu:Zhenxing Zhenxing Niu], [https://dblp.uni-trier.de/pers/hd/z/Zhou:Mo Mo Zhou], [https://dblp.uni-trier.de/pers/hd/w/Wang_0003:Le Le Wang], [[Xinbo Gao]], [https://dblp.uni-trier.de/pers/hd/h/Hua_0001:Gang Gang Hua] ('''2016'''). ''Ordinal Regression with Multiple Output CNN for Age Estimation''. [https://dblp.uni-trier.de/db/conf/cvpr/cvpr2016.html CVPR 2016], [https://www.cv-foundation.org/openaccess/content_cvpr_2016/app/S21-20.pdf pdf]
'''2017'''
* [[Yutian Chen]], [[Matthew W. Hoffman]], [[Sergio Gomez Colmenarejo]], [[Misha Denil]], [[Timothy Lillicrap]], [[Matthew Botvinick]], [[Nando de Freitas]] ('''2017'''). ''Learning to Learn without Gradient Descent by Gradient Descent''. [https://arxiv.org/abs/1611.03824v6 arXiv:1611.03824v6], [http://dblp.uni-trier.de/db/conf/icml/icml2017.html ICML 2017]

Navigation menu