25,161
edits
Changes
no edit summary
=ANNs=
[https://en.wikipedia.org/wiki/Artificial_neural_network Artificial Neural Networks] ('''ANNs''') are a family of [https://en.wikipedia.org/wiki/Machine_learning statistical learning] devices or algorithms used in [https://en.wikipedia.org/wiki/Regression_analysis regression], and [https://en.wikipedia.org/wiki/Binary_classification binary] or [[https://en.wikipedia.org/wiki/Multiclass_classification multiclass classification|multiclass classification]], implemented in [[Hardware|hardware]] or [[Software|software]] inspired by their biological counterparts. The [https://en.wikipedia.org/wiki/Artificial_neuron artificial neurons] of one or more layers receive one or more inputs (representing dendrites), and after being weighted, sum them to produce an output (representing a neuron's axon). The sum is passed through a [https://en.wikipedia.org/wiki/Nonlinear_system nonlinear] function known as an [https://en.wikipedia.org/wiki/Activation_function activation function] or transfer function. The transfer functions usually have a [https://en.wikipedia.org/wiki/Sigmoid_function sigmoid shape], but they may also take the form of other non-linear functions, [https://en.wikipedia.org/wiki/Piecewise piecewise] linear functions, or [https://en.wikipedia.org/wiki/Artificial_neuron#Step_function step functions] <ref>[https://en.wikipedia.org/wiki/Artificial_neuron Artificial neuron from Wikipedia]</ref>. The weights of the inputs of each layer are tuned to minimize a [https://en.wikipedia.org/wiki/Loss_function cost or loss function], which is a task in [https://en.wikipedia.org/wiki/Mathematical_optimization mathematical optimization] and machine learning.
==Perceptron==
The [https://en.wikipedia.org/wiki/Perceptron perceptron] is an algorithm for [[Supervised Learning|supervised learning]] of [https://en.wikipedia.org/wiki/Binary_classification binary classifiers]. It was the first artificial neural network, introduced in 1957 by [https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] <ref>[https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] ('''1957'''). ''The Perceptron - a Perceiving and Recognizing Automaton''. Report 85-460-1, [https://en.wikipedia.org/wiki/Calspan#History Cornell Aeronautical Laboratory]</ref>, implemented in custom hardware. In its basic form it consists of a single neuron with multiple inputs and associated weights.
[[Supervised Learning|Supervised learning]] is applied using a set D of labeled [https://en.wikipedia.org/wiki/Test_set training data] with pairs of [https://en.wikipedia.org/wiki/Feature_vector feature vectors] (x) and given results as desired output (d), usually started with cleared or randomly initialized weight vector w. The output is calculated by all inputs of a sample, multiplied by its corresponding weights, passing the sum to the activation function f. The difference of desired and actual value is then immediately used modify the weights for all features using a learning rate 0.0 < α <= 1.0:
<pre>
for (j=0, Σ = 0.0; j < nSamples; ++j) {
Typical CNN <ref>Typical [https://en.wikipedia.org/wiki/Convolutional_neural_network CNN] architecture, Image by Aphex34, December 16, 2015, [https://creativecommons.org/licenses/by-sa/4.0/deed.en CC BY-SA 4.0], [https://en.wikipedia.org/wiki/Wikimedia_Commons Wikimedia Commons]</ref>
<span id="Residual"></span>
==Residual NetsNet==
[[FILE:ResiDualBlock.png|border|right|thumb|link=https://arxiv.org/abs/1512.03385| A residual block <ref>The fundamental building block of residual networks. Figure 2 in [https://scholar.google.com/citations?user=DhtAFkwAAAAJ Kaiming He], [https://scholar.google.com/citations?user=yuB-cfoAAAAJ&hl=en Xiangyu Zhang], [http://shaoqingren.com/ Shaoqing Ren], [http://www.jiansun.org/ Jian Sun] ('''2015'''). ''Deep Residual Learning for Image Recognition''. [https://arxiv.org/abs/1512.03385 arXiv:1512.03385]</ref> <ref>[https://blog.waya.ai/deep-residual-learning-9610bb62c355 Understand Deep Residual Networks — a simple, modular learning framework that has redefined state-of-the-art] by [https://blog.waya.ai/@waya.ai Michael Dietz], [https://blog.waya.ai/ Waya.ai], May 02, 2017</ref> ]]
A '''Residual netsnet''' add (ResNet) adds the input of a layer, typically composed of a convolutional layer and of a [https://en.wikipedia.org/wiki/Rectifier_(neural_networks) ReLU] layer, to its output. This modification, like convolutional nets inspired from image classification, enables faster training and deeper networks <ref>[[Tristan Cazenave]] ('''2017'''). ''[http://ieeexplore.ieee.org/document/7875402/ Residual Networks for Computer Go]''. [[IEEE#TOCIAIGAMES|IEEE Transactions on Computational Intelligence and AI in Games]], Vol. PP, No. 99, [http://www.lamsade.dauphine.fr/~cazenave/papers/resnet.pdf pdf]</ref> <ref>[https://wiki.tum.de/display/lfdv/Deep+Residual+Networks Deep Residual Networks] from [https://wiki.tum.de/ TUM Wiki], [[Technical University of Munich]]</ref> <ref>[https://towardsdatascience.com/understanding-and-visualizing-resnets-442284831be8 Understanding and visualizing ResNets] by Pablo Ruiz, October 8, 2018</ref>.
=ANNs in Games=
<span id="AlphaZero"></span>
===Alpha Zero===
In December 2017, the [[Google]] [[DeepMind]] team along with former [[Giraffe]] author [[Matthew Lai]] reported on their generalized [[AlphaZero]] algorithm, combining [[Deep Learning|Deep learning]] with [[Monte-Carlo Tree Search]]. AlphaZero can achieve, tabula rasa, superhuman performance in many challenging domains with some training effort. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved a superhuman level of play in the games of chess and [[Shogi]] as well as Go, and convincingly defeated a world-champion program in each case <ref>[[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2017'''). ''Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm''. [https://arxiv.org/abs/1712.01815 arXiv:1712.01815]</ref>. The open souece projects [[Leela Zero]] (Go) and its chess adaptation [[Leela Chess Zero]] successfully re-implemented the ideas of DeepMind.===NNUE===[[NNUE]] reverse of ƎUИИ - Efficiently Updatable Neural Networks, is an NN architecture intended to replace the [[Evaluation|evaluation]] of [[Shogi]], [[Chess|chess]] and other board game playing [[Alpha-Beta|alpha-beta]] searchers. NNUE was introduced in 2018 by [[Yu Nasu]] <ref>[[Yu Nasu]] ('''2018'''). ''ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''. Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract)</ref>,and was used in Shogi adaptations of [[Stockfish]] such as [[YaneuraOu]] <ref>[https://github.com/yaneurao/YaneuraOu GitHub - yaneurao/YaneuraOu: YaneuraOu is the World's Strongest Shogi engine(AI player), WCSC29 1st winner, educational and USI compliant engine]</ref> ,and [[Kristallweizen]] <ref>[https://github.com/Tama4649/Kristallweizen/ GitHub - Tama4649/Kristallweizen: 第29回世界コンピュータ将棋選手権 準優勝のKristallweizenです。]</ref>, apparently with [[AlphaZero]] strength <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72754 The Stockfish of shogi] by [[Larry Kaufman]], [[CCC]], January 07, 2020</ref>. [[Hisayori Noda|Nodchip]] incorporated NNUE into the chess playing Stockfish 10 as a proof of concept <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74059 Stockfish NN release (NNUE)] by [[Henk Drost]], [[CCC]], May 31, 2020</ref>, yielding in the hype about [[Stockfish NNUE]] in summer 2020 <ref>[http://yaneuraou.yaneu.com/2020/06/19/stockfish-nnue-the-complete-guide/ Stockfish NNUE – The Complete Guide], June 19, 2020 (Japanese and English)</ref>.Its heavily over parametrized computational most expensive input layer is efficiently [[Incremental Updates|incremental updated]] in [[Make Move|make]] and [[Unmake Move|unmake move]].<span id="engines"></span>===NN Chess Programs===* [[:Category:NN]]
=See also=
* [[Memory]]
* [[Neural MoveMap Heuristic]]
* [[NNUE]]
* [[Pattern Recognition]]
* [[Temporal Difference Learning]]
=Selected Publications=
* [[John von Neumann]] ('''1956'''). ''Probabilistic Logic and the Synthesis of Reliable Organisms From Unreliable Components''. in
: [[Claude Shannon]], [[John McCarthy]] (eds.) ('''1956'''). ''Automata Studies''. [http://press.princeton.edu/math/series/amh.html Annals of Mathematics Studies], No. 34, [http://www.dna.caltech.edu/courses/cs191/paperscs191/VonNeumann56.pdf pdf]
* [[Nathaniel Rochester]], [[Mathematician#Holland|John H. Holland]], [httphttps://dblp.uni-trier.de/pers/hd/h/Haibt:L=_H= L. H. Haibt], [httphttps://dblp.uni-trier.de/pers/hd/d/Duda:WWilliam_L=_L= W. William L. Duda] ('''1956'''). ''[https://www.semanticscholar.org/paper/Tests-on-a-cell-assembly-theory-of-the-action-of-a-Rochester-Holland/878d615b84cf779e162f62c4a9192d6bddeefbf9 Tests on a Cell Assembly Theory of the Action of the Brain, Using a Large Digital Computer]''. [httphttps://dblp.uni-trier.de/db/journals/tit/tit2n.html#RochesterHHD56 IRE Transactions on Information Theory, Vol. 2], No. 3
* [https://en.wikipedia.org/wiki/Frank_Rosenblatt Frank Rosenblatt] ('''1957'''). ''The Perceptron - a Perceiving and Recognizing Automaton''. Report 85-460-1, [https://en.wikipedia.org/wiki/Calspan#History Cornell Aeronautical Laboratory] <ref>[http://csis.pace.edu/~ctappert/srd2011/rosenblatt-contributions.htm Rosenblatt's Contributions]</ref>
==1960 ...==
* [[Eric B. Baum]] ('''1989'''). ''[http://papers.nips.cc/paper/226-the-perceptron-algorithm-is-fast-for-non-malicious-distributions The Perceptron Algorithm Is Fast for Non-Malicious Distributions]''. [http://papers.nips.cc/book/advances-in-neural-information-processing-systems-2-1989 NIPS 1989]
* [[Eric B. Baum]] ('''1989'''). ''[http://www.mitpressjournals.org/doi/abs/10.1162/neco.1989.1.2.201#.VfGX0JdpluM A Proposal for More Powerful Learning Algorithms]''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 1, No. 2
* [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/i/Irani:E=_A=.html Erach A. Irani], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/m/Matts:John_P=.html John P. Matts], [http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/l/Long:John_M=.html John M. Long], [[James R. Slagle]], POSCH group ('''1989'''). ''Using Artificial Neural Nets for Statistical Discovery: Observations after Using Backpropogation, Expert Systems, and Multiple-Linear Regression on Clinical Trial Data''. [[University of Minnesota]], Minneapolis, MN 55455, USA, Complex Systems 3, [http://www.complex-systems.com/pdf/03-3-5.pdf pdf]
* [[Gerald Tesauro]], [[Terrence J. Sejnowski]] ('''1989'''). ''A Parallel Network that Learns to Play Backgammon''. [https://en.wikipedia.org/wiki/Artificial_Intelligence_%28journal%29 Artificial Intelligence], Vol. 39, No. 3
* [[Mathematician#EGelenbe|Erol Gelenbe]] ('''1989'''). ''[http://cognet.mit.edu/journal/10.1162/neco.1989.1.4.502 Random Neural Networks with Negative and Positive Signals and Product Form Solution]''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 1, No. 4
* [[Mathematician#XZhang|Xiru Zhang]], [https://dblp.uni-trier.de/pers/hd/m/McKenna:Michael Michael McKenna], [[Mathematician#JPMesirov|Jill P. Mesirov]], [[David Waltz]] ('''1989'''). ''[http://papers.neurips.cc/paper/281-an-efficient-implementation-of-the-back-propagation-algorithm-on-the-connection-machine-cm-2 An Efficient Implementation of the Back-propagation Algorithm on the Connection Machine CM-2]''. [https://dblp.uni-trier.de/db/conf/nips/nips1989.html NIPS 1989]
==1990 ...==
* [[Mathematician#PWerbos|Paul Werbos]] ('''1990'''). ''Backpropagation Through Time: What It Does and How to Do It''. Proceedings of the [[IEEE]], Vol. 78, No. 10, [http://deeplearning.cs.cmu.edu/pdfs/Werbos.backprop.pdf pdf]
* [https://dblp.uni-trier.de/pers/hd/r/Ruck:Dennis_W= Dennis W. Ruck], [http://spie.org/profile/Steven.Rogers-5480?SSO=1 Steven K. Rogers], [https://dblp.uni-trier.de/pers/hd/k/Kabrisky:Matthew Matthew Kabrisky], [[Mathematician#MEOxley|Mark E. Oxley]], [[Bruce W. Suter]] ('''1990'''). ''[https://ieeexplore.ieee.org/document/80266 The multilayer perceptron as an approximation to a Bayes optimal discriminant function]''. [[IEEE#NN|IEEE Transactions on Neural Networks]], Vol. 1, No. 4
* [https://dblp.uni-trier.de/pers/hd/h/Hellstrom:Benjamin_J= Benjamin J. Hellstrom], [[Laveen Kanal|Laveen N. Kanal]] ('''1990'''). ''[https://ieeexplore.ieee.org/document/5726889 The definition of necessary hidden units in neural networks for combinatorial optimization]''. [https://dblp.uni-trier.de/db/conf/ijcnn/ijcnn1990.html IJCNN 1990]
* [[Mathematician#XZhang|Xiru Zhang]], [https://dblp.uni-trier.de/pers/hd/m/McKenna:Michael Michael McKenna], [[Mathematician#JPMesirov|Jill P. Mesirov]], [[David Waltz]] ('''1990'''). ''[https://www.sciencedirect.com/science/article/pii/016781919090084M The backpropagation algorithm on grid and hypercube architectures]''. [https://www.journals.elsevier.com/parallel-computing Parallel Computing], Vol. 14, No. 3
* [[Simon Lucas]], [https://dblp.uni-trier.de/pers/hd/d/Damper:Robert_I= Robert I. Damper] ('''1990'''). ''[https://www.tandfonline.com/doi/abs/10.1080/09540099008915669 Syntactic Neural Networks]''. [https://www.tandfonline.com/toc/ccos20/current Connection Science], Vol. 2, No. 3
'''1991'''
* [[Mathematician#SHochreiter|Sepp Hochreiter]] ('''1991'''). ''Untersuchungen zu dynamischen neuronalen Netzen''. Diploma thesis, [[Technical University of Munich|TU Munich]], advisor [[Jürgen Schmidhuber]], [http://people.idsia.ch/~juergen/SeppHochreiter1991ThesisAdvisorSchmidhuber.pdf pdf] (German) <ref>[http://people.idsia.ch/~juergen/fundamentaldeeplearningproblem.html Sepp Hochreiter's Fundamental Deep Learning Problem (1991)] by [[Jürgen Schmidhuber]], 2013</ref>
* [[Alex van Tiggelen]] ('''1991'''). ''Neural Networks as a Guide to Optimization - The Chess Middle Game Explored''. [[ICGA Journal#14_3|ICCA Journal, Vol. 14, No. 3]]
* [[Mathematician#TMartinetz|Thomas Martinetz]], [[Mathematician#KSchulten|Klaus Schulten]] ('''1991'''). ''A "Neural-Gas" Network Learns Topologies''. In [[Mathematician#TKohonen|Teuvo Kohonen]], [https://dblp.uni-trier.de/pers/hd/m/Makisara:Kai Kai Mäkisara], [http://users.ics.tkk.fi/ollis/ Olli Simula], [http://cis.legacy.ics.tkk.fi/jari/ Jari Kangas] (eds.) ('''1991'''). ''[https://www.elsevier.com/books/artificial-neural-networks/makisara/978-0-444-89178-5 Artificial Neural Networks]''. [https://en.wikipedia.org/wiki/Elsevier Elsevier], [http://www.ks.uiuc.edu/Publications/Papers/PDF/MART91B/MART91B.pdf pdf]* [[Jürgen Schmidhuber]], [[Rudolf Huber]] ('''1991'''). ''[https://www.researchgate.net/publication/2290900_Using_Adaptive_Sequential_Neurocontrol_For_Efficient_Learning_Of_Translation_And_Rotation_Invariance Using sequential adaptive Neuro-control for efficient Learning of Rotation and Translation Invariance]''. In [https://en.wikipedia.org/wiki/Teuvo_Kohonen [Mathematician#TKohonen|Teuvo Kohonen]], [https://dblp.uni-trier.de/pers/hd/m/Makisara:Kai Kai Mäkisara], [http://users.ics.tkk.fi/ollis/ Olli Simula], [http://cis.legacy.ics.tkk.fi/jari/ Jari Kangas] (eds.) ('''1991'''). ''[https://www.sciencedirect.com/book/9780444891785/artificial-neural-networks#book-description Artificial Neural Networks]''. [https://en.wikipedia.org/wiki/Elsevier Elsevier]
* [[Jürgen Schmidhuber]] ('''1991'''). ''[http://www.idsia.ch/%7Ejuergen/promotion/ Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem]'' (Dynamic Neural Nets and the Fundamental Spatio-Temporal Credit Assignment Problem). Ph.D. thesis
* [[Yoav Freund]], [[Mathematician#DHHaussler|David Haussler]] ('''1991'''). ''Unsupervised Learning of Distributions of Binary Vectors Using 2-Layer Networks''. [http://dblp.uni-trier.de/db/conf/nips/nips1991.html#FreundH91 NIPS 1991]
* [[Byoung-Tak Zhang]], [[Gerd Veenker]] ('''1991'''). ''[http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=170480&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D170480 Neural networks that teach themselves through genetic discovery of novel examples]''. [http://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000500 IEEE IJCNN'91], [https://bi.snu.ac.kr/Publications/Conferences/International/IJCNN91.pdf pdf]
* [[Simon Lucas]], [https://dblp.uni-trier.de/pers/hd/d/Damper:Robert_I= Robert I. Damper] ('''1991'''). ''[https://link.springer.com/chapter/10.1007/978-1-4615-3752-6_30 Syntactic neural networks in VLSI]''. [https://link.springer.com/book/10.1007/978-1-4615-3752-6 VLSI for Artificial Intelligence and Neural Networks]
* [[Simon Lucas]] ('''1991'''). ''[https://eprints.soton.ac.uk/256263/ Connectionist architectures for syntactic pattern recognition]''. Ph.D. thesis, [https://en.wikipedia.org/wiki/University_of_Southampton University of Southampton]
'''1992'''
* [[Michael Reiss]] ('''1992'''). ''Temporal Sequence Processing in Neural Networks''. Ph.D. thesis, [https://en.wikipedia.org/wiki/King%27s_College_London King's College London], advisor [[Mathematician#JGTaylor|John G. Taylor]], [http://www.reiss.demon.co.uk/misc/m_reiss_phd.pdf pdf]
'''1994'''
* [[Mathematician#PWerbos|Paul Werbos]] ('''1994'''). ''The Roots of Backpropagation. From Ordered Derivatives to Neural Networks and Political Forecasting''. [https://en.wikipedia.org/wiki/John_Wiley_%26_Sons John Wiley & Sons]
* [[David E. Moriarty]], [[Risto Miikkulainen]] ('''1994'''). ''[http://nn.cs.utexas.edu/?moriarty:aaai94 Evolving Neural Networks to focus Minimax Search]''. [[Conferences#AAAI-94|AAAI-94]], » [[http://www.cs.utexas.edu/~ai-lab/pubs/moriarty.focus.pdf pdfOthello]]
* [[Eric Postma]] ('''1994'''). ''SCAN: A Neural Model of Covert Attention''. Ph.D. thesis, [[Maastricht University]], advisor [[Jaap van den Herik]]
* [[Sebastian Thrun]] ('''1994'''). ''Neural Network Learning in the Domain of Chess''. Machines That Learn, [http://snowbird.djvuzone.org/ Snowbird], Extended abstract
'''1995'''
* [https://peterbraspenning.wordpress.com/ Peter J. Braspenning], [[Frank Thuijsman]], [https://scholar.google.com/citations?user=Ba9L7CAAAAAJ Ton Weijters] (eds) ('''1995'''). ''[http://link.springer.com/book/10.1007%2FBFb0027019 Artificial neural networks: an introduction to ANN theory and practice]''. [https://de.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science LNCS] 931, [https://de.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[David E. Moriarty]], [[Risto Miikkulainen]] ('''1995'''). ''[http://nn.cs.utexas.edu/?moriarty:connsci95 Discovering Complex Othello Strategies Through Evolutionary Neural Networks]''. [https://www.scimagojr.com/journalsearch.php?q=24173&tip=sid Connection Science], Vol. 7
* [[Anton Leouski]] ('''1995'''). ''Learning of Position Evaluation in the Game of Othello''. Master's Project, [https://en.wikipedia.org/wiki/University_of_Massachusetts University of Massachusetts], [https://en.wikipedia.org/wiki/Amherst,_Massachusetts Amherst, Massachusetts], [http://people.ict.usc.edu/~leuski/publications/papers/UM-CS-1995-023.pdf pdf]
* [[Mathematician#SHochreiter|Sepp Hochreiter]], [[Jürgen Schmidhuber]] ('''1995'''). ''[http://www.idsia.ch/%7Ejuergen/nipsfm/ Simplifying Neural Nets by Discovering Flat Minima]''. In [[Gerald Tesauro]], [http://www.cs.cmu.edu/%7Edst/home.html David S. Touretzky] and [http://www.bme.ogi.edu/%7Etleen/ Todd K. Leen] (eds.), ''[http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=8420 Advances in Neural Information Processing Systems 7]'', NIPS'7, pages 529-536. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Pieter Spronck]] ('''1996'''). ''Elegance: Genetic Algorithms in Neural Reinforcement Control''. Master thesis, [[Delft University of Technology]], [http://ticc.uvt.nl/~pspronck/pubs/Elegance.pdf pdf]
* [[Raúl Rojas]] ('''1996'''). ''Neural Networks - A Systematic Introduction''. Springer, available as [http://www.inf.fu-berlin.de/inst/ag-ki/rojas_home/documents/1996/NeuralNetworks/neuron.pdf pdf ebook]
* [[Ida Sprinkhuizen-Kuyper]], [https://dblp.org/pers/hd/b/Boers:Egbert_J=_W= Egbert J. W. Boers] ('''1996'''). ''[https://ieeexplore.ieee.org/abstract/document/6796246 The Error Surface of the Simplest XOR Network Has Only Global Minima]''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 8, No. 6, [http://www.socsci.ru.nl/idak/publications/papers/NeuralComputation.pdf pdf]
'''1997'''
* [[Mathematician#SHochreiter|Sepp Hochreiter]], [[Jürgen Schmidhuber]] ('''1997'''). ''Long short-term memory''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 9, No. 8, [http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf pdf] <ref>[https://en.wikipedia.org/wiki/Long_short_term_memory Long short term memory from Wikipedia]</ref>
* [[Don Beal]], [[Martin C. Smith]] ('''1997'''). ''Learning Piece Values Using Temporal Differences''. [[ICGA Journal#20_3|ICCA Journal, Vol. 20, No. 3]]
* [https://dblp.uni-trier.de/pers/hd/t/Thiesing:Frank_M= Frank M. Thiesing], [[Oliver Vornberger]] ('''1997'''). ''Forecasting Sales Using Neural Networks''. [https://dblp.uni-trier.de/db/conf/fuzzy/fuzzy1997.html Fuzzy Days 1997], [http://www2.inf.uos.de/papers_pdf/fuzzydays_97.pdf pdf]
* [[Simon Lucas]] ('''1997'''). ''[https://link.springer.com/chapter/10.1007/BFb0032531 Forward-Backward Building Blocks for Evolving Neural Networks with Intrinsic Learning Behaviors]''. [https://dblp.uni-trier.de/db/conf/iwann/iwann1997.html IWANN 1997]
'''1998'''
* [[Kieran Greer]] ('''1998'''). ''A Neural Network Based Search Heuristic and its Application to Computer Chess''. D.Phil. Thesis, [https://en.wikipedia.org/wiki/University_of_Ulster University of Ulster]
* <span id="FundamentalsNAI1st"></span>[[Toshinori Munakata]] ('''1998'''). ''[http://cis.csuohio.edu/~munakata/publs/book/sp.html Fundamentals of the New Artificial Intelligence: Beyond Traditional Paradigms]''. 1st edition, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [[Neural Networks#FundamentalsNAI2nd|2nd edition 2008]]
* [[Lex Weaver]], [https://bjbs.csu.edu.au/schools/computing-and-mathematics/staff/profiles/professorial-staff/terry-bossomaier Terry Bossomaier] ('''1998'''). ''Evolution of Neural Networks to Play the Game of Dots-and-Boxes''. [https://arxiv.org/abs/cs/9809111 arXiv:cs/9809111]
* [[Norman Richards]], [[David E. Moriarty]], [[Risto Miikkulainen]] ('''1998'''). ''[http://nn.cs.utexas.edu/?richards:apin98 Evolving Neural Networks to Play Go]''. [https://www.springer.com/journal/10489 Applied Intelligence], Vol. 8, No. 1
'''1999'''
* [[Kumar Chellapilla]], [[David B. Fogel]] ('''1999'''). ''[http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=784222 Evolution, Neural Networks, Games, and Intelligence]''. Proceedings of the IEEE, September, pp. 1471-1496. [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.99.979 CiteSeerX]
* [[Mathematician#GEHinton|Geoffrey E. Hinton]], [[Terrence J. Sejnowski]] (eds.) ('''1999'''). ''[https://mitpress.mit.edu/books/unsupervised-learning Unsupervised Learning: Foundations of Neural Computation]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Peter Dayan]] ('''1999'''). ''Recurrent Sampling Models for the Helmholtz Machine''. [https://en.wikipedia.org/wiki/Neural_Computation_(journal) Neural Computation], Vol. 11, No. 3, [http://www.gatsby.ucl.ac.uk/~dayan/papers/rechelm99.pdf pdf] <ref>[https://en.wikipedia.org/wiki/Helmholtz_machine Helmholtz machine from Wikipedia]</ref>
* [[Ida Sprinkhuizen-Kuyper]], [https://dblp.org/pers/hd/b/Boers:Egbert_J=_W= Egbert J. W. Boers] ('''1999'''). ''[https://ieeexplore.ieee.org/document/774274 A local minimum for the 2-3-1 XOR network]''. [[IEEE#NN|IEEE Transactions on Neural Networks]], Vol. 10, No. 4
==2000 ...==
* [[Levente Kocsis]], [[Jos Uiterwijk]], [[Jaap van den Herik]] ('''2000'''). ''[http://link.springer.com/chapter/10.1007%2F3-540-45579-5_11 Learning Time Allocation using Neural Networks]''. [[CG 2000]]
* [[Levente Kocsis]], [[Jos Uiterwijk]], [[Jaap van den Herik]] ('''2001'''). ''Move Ordering using Neural Networks''. IEA/AIE 2001, [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science LNCS] 2070, [http://www.pradu.us/old/Nov27_2008/Buzz/research/parallel/fulltext.pdf pdf]
* [[Kee Siong Ng]] ('''2001'''). ''Neural Networks for Structured Data''. BSc-Thesis, [http://users.cecs.anu.edu.au/~kee/hon-thesis.ps.gz zipped ps]
* [[Peter Dayan]], [https://en.wikipedia.org/wiki/Larry_Abbott Laurence F. Abbott] ('''2001, 2005'''). ''[http://www.gatsby.ucl.ac.uk/~dayan/book/index.html Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
'''2002'''
* [[Moshe Sipper]] ('''2002''') ''[http://books.google.com/books/about/Machine_Nature.html?id=fbFQAAAAMAAJ&redir_esc=y Machine Nature: The Coming Age of Bio-Inspired Computing]''. [https://en.wikipedia.org/wiki/McGraw-Hill_Financial McGraw-Hill, New York]
* [[Paul E. Utgoff]], [[David J. Stracuzzi]] ('''2002'''). ''Many-Layered Learning''. [https://en.wikipedia.org/wiki/Neural_Computation_%28journal%29 Neural Computation], Vol. 14, No. 10, [http://people.cs.umass.edu/~utgoff/papers/neco-stl.pdf pdf]
* [[Mathematician#MIJordan|Michael I. Jordan]], [[Terrence J. Sejnowski]] (eds.) ('''2002'''). ''[https://mitpress.mit.edu/books/graphical-models Graphical Models: Foundations of Neural Computation]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]* [[Kenneth O. Stanley]], [[Risto Miikkulainen]] ('''2002'''). ''[http://nn.cs.utexas.edu/?stanley:ec02 Evolving Neural Networks Through Augmenting Topologies]''. [https://en.wikipedia.org/wiki/Evolutionary_Computation_(journal) Evolutionary Computation], Vol. 10, No. 2
'''2003'''
* [[Levente Kocsis]] ('''2003'''). ''Learning Search Decisions''. Ph.D thesis, [[Maastricht University]], [https://project.dke.maastrichtuniversity.nl/games/files/phd/Kocsis_thesis.pdf pdf]
* [[Mathematician#GMontavon|Grégoire Montavon]] ('''2013'''). ''[https://opus4.kobv.de/opus4-tuberlin/frontdoor/index/index/docId/4467 On Layer-Wise Representations in Deep Neural Networks]''. Ph.D. Thesis, [https://en.wikipedia.org/wiki/Technical_University_of_Berlin TU Berlin], advisor [[Mathematician#KRMueller|Klaus-Robert Müller]]
* [[Volodymyr Mnih]], [[Koray Kavukcuoglu]], [[David Silver]], [[Alex Graves]], [[Ioannis Antonoglou]], [[Daan Wierstra]], [[Martin Riedmiller]] ('''2013'''). ''Playing Atari with Deep Reinforcement Learning''. [http://arxiv.org/abs/1312.5602 arXiv:1312.5602] <ref>[http://www.nervanasys.com/demystifying-deep-reinforcement-learning/ Demystifying Deep Reinforcement Learning] by [http://www.nervanasys.com/author/tambet/ Tambet Matiisen], [http://www.nervanasys.com/ Nervana], December 21, 2015</ref>
* [[Risto Miikkulainen]] ('''2013'''). ''Evolving Neural Networks''. [https://dblp.org/db/conf/ijcnn/ijcnn2013 IJCNN 2013], [http://nn.cs.utexas.edu/downloads/slides/miikkulainen.ijcnn13.pdf pdf]
'''2014'''
* [[Mathematician#YDauphin|Yann Dauphin]], [[Mathematician#RPascanu|Razvan Pascanu]], [[Mathematician#CGulcehre|Caglar Gulcehre]], [[Mathematician#KCho|Kyunghyun Cho]], [[Mathematician#SGanguli|Surya Ganguli]], [[Mathematician#YBengio|Yoshua Bengio]] ('''2014'''). ''Identifying and attacking the saddle point problem in high-dimensional non-convex optimization''. [https://arxiv.org/abs/1406.2572 arXiv:1406.2572] <ref>[https://groups.google.com/d/msg/fishcooking/wOfRuzTSi_8/VgjN8MmSBQAJ high dimensional optimization] by [[Warren D. Smith]], [[Computer Chess Forums|FishCooking]], December 27, 2019</ref>* [[Mathematician#IGoodfellow|Ian Goodfellow]], [[Jean Pouget-Abadie]], [[Mehdi Mirza]], [[Bing Xu]], [[David Warde-Farley]], [[Sherjil Ozair]], [[Mathematician#ACourville|Aaron Courville]], [[Mathematician#YBengio|Yoshua Bengio]] ('''2014'''). ''Generative Adversarial Networks''. [https://arxiv.org/abs/1406.2661v1 arXiv:1406.2661v1] <ref>[https://en.wikipedia.org/wiki/Generative_adversarial_networks Generative adversarial networks from Wikipedia]</ref>
* [[Christopher Clark]], [[Amos Storkey]] ('''2014'''). ''Teaching Deep Convolutional Neural Networks to Play Go''. [http://arxiv.org/abs/1412.3409 arXiv:1412.3409] <ref>[http://computer-go.org/pipermail/computer-go/2014-December/007010.html Teaching Deep Convolutional Neural Networks to Play Go] by [[Hiroshi Yamashita]], [http://computer-go.org/pipermail/computer-go/ The Computer-go Archives], December 14, 2014</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=54663 Teaching Deep Convolutional Neural Networks to Play Go] by [[Michel Van den Bergh]], [[CCC]], December 16, 2014</ref>
* [[Chris J. Maddison]], [[Shih-Chieh Huang|Aja Huang]], [[Ilya Sutskever]], [[David Silver]] ('''2014'''). ''Move Evaluation in Go Using Deep Convolutional Neural Networks''. [http://arxiv.org/abs/1412.6564v1 arXiv:1412.6564v1] » [[Go]]
* [[James L. McClelland]] ('''2015'''). ''[https://web.stanford.edu/group/pdplab/pdphandbook/handbook3.html#handbookch10.html Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises]''. Second Edition, [https://web.stanford.edu/group/pdplab/pdphandbook/handbookli1.html Contents]
* [[Gábor Melis]] ('''2015'''). ''[http://jmlr.org/proceedings/papers/v42/meli14.html Dissecting the Winning Solution of the HiggsML Challenge]''. [https://nips.cc/Conferences/2014 NIPS 2014]
* [[Volodymyr Mnih]], [[Koray Kavukcuoglu]], [[David Silver]], [[Mathematician#AARusu|Andrei A. Rusu]], [[Joel Veness]], [[Marc G. Bellemare]], [[Alex Graves]], [[Martin Riedmiller]], [[Andreas K. Fidjeland]], [[Georg Ostrovski]], [[Stig Petersen]], [[Charles Beattie]], [[Amir Sadik]], [[Ioannis Antonoglou]], [[Helen King]], [[Dharshan Kumaran]], [[Daan Wierstra]], [[Shane Legg]], [[Demis Hassabis]] ('''2015'''). ''[http://www.nature.com/nature/journal/v518/n7540/abs/nature14236.html Human-level control through deep reinforcement learning]''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 518
* [[Jürgen Schmidhuber]] ('''2015'''). ''[http://people.idsia.ch/~juergen/deep-learning-overview.html Deep Learning in Neural Networks: An Overview]''. [https://en.wikipedia.org/wiki/Neural_Networks_(journal) Neural Networks], Vol. 61
* [https://scholar.google.fr/citations?user=MN9Kfg8AAAAJ&hl=en Zachary C. Lipton], [https://www.linkedin.com/in/john-berkowitz-92b24a7b John Berkowitz], [[Charles Elkan]] ('''2015'''). ''A Critical Review of Recurrent Neural Networks for Sequence Learning''. [https://arxiv.org/abs/1506.00019 arXiv:1506.00019v4]
* [[Guillaume Desjardins]], [[Karen Simonyan]], [[Mathematician#RPascanu|Razvan Pascanu]], [[Koray Kavukcuoglu]] ('''2015'''). ''Natural Neural Networks''. [https://arxiv.org/abs/1507.00210 arXiv:1507.00210]
* [[Barak Oshri]], [[Nishith Khandwala]] ('''2015'''). ''Predicting Moves in Chess using Convolutional Neural Networks''. [http://cs231n.stanford.edu/reports/ConvChess.pdf pdf] <ref>[https://github.com/BarakOshri/ConvChess GitHub - BarakOshri/ConvChess: Predicting Moves in Chess Using Convolutional Neural Networks]</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=63458 ConvChess CNN] by [[Brian Richardson]], [[CCC]], March 15, 2017</ref>
* [https://en.wikipedia.org/wiki/Yann_LeCun [Mathematician#YLeCun|Yann LeCun]], [[Mathematician#YBengio|Yoshua Bengio]], [[Mathematician#GEHinton|Geoffrey E. Hinton]] ('''2015'''). ''[http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html Deep Learning]''. [https://en.wikipedia.org/wiki/Nature_%28journal%29 Nature], Vol. 521 <ref>[[Jürgen Schmidhuber]] ('''2015''') ''[http://people.idsia.ch/~juergen/deep-learning-conspiracy.html Critique of Paper by "Deep Learning Conspiracy" (Nature 521 p 436)]''.</ref>
* [[Matthew Lai]] ('''2015'''). ''Giraffe: Using Deep Reinforcement Learning to Play Chess''. M.Sc. thesis, [https://en.wikipedia.org/wiki/Imperial_College_London Imperial College London], [http://arxiv.org/abs/1509.01549v1 arXiv:1509.01549v1] » [[Giraffe]]
* [[Nikolai Yakovenko]], [[Liangliang Cao]], [[Colin Raffel]], [[James Fan]] ('''2015'''). ''Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games''. [https://arxiv.org/abs/1509.06731 arXiv:1509.06731]
* [[Ilya Loshchilov]], [[Frank Hutter]] ('''2015'''). ''Online Batch Selection for Faster Training of Neural Networks''. [https://arxiv.org/abs/1511.06343 arXiv:1511.06343]
* [[Yuandong Tian]], [[Yan Zhu]] ('''2015'''). ''Better Computer Go Player with Neural Network and Long-term Prediction''. [http://arxiv.org/abs/1511.06410 arXiv:1511.06410] <ref>[http://www.technologyreview.com/view/544181/how-facebooks-ai-researchers-built-a-game-changing-go-engine/?utm_campaign=socialsync&utm_medium=social-post&utm_source=facebook How Facebook’s AI Researchers Built a Game-Changing Go Engine | MIT Technology Review], December 04, 2015</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=58514 Combining Neural Networks and Search techniques (GO)] by Michael Babigian, [[CCC]], December 08, 2015</ref> » [[Go]]
* [[Peter H. Jin]], [[Kurt Keutzer]] ('''2015'''). ''Convolutional Monte Carlo Rollouts in Go''. [http://arxiv.org/abs/1512.03375 arXiv:1512.03375]
* [[Eli David|Omid E. David]], [[Nathan S. Netanyahu]], [[Lior Wolf]] ('''2016'''). ''[http://link.springer.com/chapter/10.1007%2F978-3-319-44781-0_11 DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess]''. [http://icann2016.org/ ICAAN 2016], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science], Vol. 9887, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [http://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf pdf preprint] » [[DeepChess]] <ref>[http://www.talkchess.com/forum/viewtopic.php?t=61748 DeepChess: Another deep-learning based chess program] by [[Matthew Lai]], [[CCC]], October 17, 2016</ref> <ref>[http://icann2016.org/index.php/conference-programme/recipients-of-the-best-paper-awards/ ICANN 2016 | Recipients of the best paper awards]</ref>
* [[Dror Sholomon]], [[Eli David|Omid E. David]], [[Nathan S. Netanyahu]] ('''2016'''). ''[http://link.springer.com/chapter/10.1007/978-3-319-44781-0_21 DNN-Buddies: A Deep Neural Network-Based Estimation Metric for the Jigsaw Puzzle Problem]''. [http://icann2016.org/ ICAAN 2016], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science], Vol. 9887, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer] <ref>[https://en.wikipedia.org/wiki/Jigsaw_puzzle Jigsaw puzzle from Wikipedia]</ref>
* [[Mathematician#IGoodfellow|Ian Goodfellow]], [[Mathematician#YBengio|Yoshua Bengio]], [[Mathematician#ACourville|Aaron Courville]] ('''2016'''). ''[http://www.deeplearningbook.org/ Deep Learning]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Volodymyr Mnih]], [[Adrià Puigdomènech Badia]], [[Mehdi Mirza]], [[Alex Graves]], [[Timothy Lillicrap]], [[Tim Harley]], [[David Silver]], [[Koray Kavukcuoglu]] ('''2016'''). ''Asynchronous Methods for Deep Reinforcement Learning''. [https://arxiv.org/abs/1602.01783 arXiv:1602.01783v2]
* [https://scholar.google.ca/citations?user=mZfgLA4AAAAJ&hl=en Vincent Dumoulin], [https://scholar.google.it/citations?user=kaAnZw0AAAAJ&hl=en Francesco Visin] ('''2016'''). ''A guide to convolution arithmetic for deep learning''. [https://arxiv.org/abs/1603.07285 arXiv:1603.07285]
* [https://en.wikipedia.org/wiki/Patricia_Churchland Patricia Churchland], [[Terrence J. Sejnowski]] ('''2016'''). ''[https://mitpress.mit.edu/books/computational-brain-0 The Computational Brain, 25th Anniversary Edition]''. [https://en.wikipedia.org/wiki/MIT_Press MIT Press]
* [[Ilya Loshchilov]], [[Frank Hutter]] ('''2016'''). ''CMA-ES for Hyperparameter Optimization of Deep Neural Networks''. [https://arxiv.org/abs/1604.07269 arXiv:1604.07269] <ref>[https://en.wikipedia.org/wiki/CMA-ES CMA-ES from Wikipedia]</ref>
* [[Audrūnas Gruslys]], [[Rémi Munos]], [[Ivo Danihelka]], [[Marc Lanctot]], [[Alex Graves]] ('''2016'''). ''Memory-Efficient Backpropagation Through Time''. [https://arxiv.org/abs/1606.03401v1 arXiv:1606.03401]
* [[Mathematician#AARusu|Andrei A. Rusu]], [[Neil C. Rabinowitz]], [[Guillaume Desjardins]], [[Hubert Soyer]], [[James Kirkpatrick]], [[Koray Kavukcuoglu]], [[Mathematician#RPascanu|Razvan Pascanu]], [[Mathematician#RHadsell|Raia Hadsell]] ('''2016'''). ''Progressive Neural Networks''. [https://arxiv.org/abs/1606.04671 arXiv:1606.04671]
* [[George Rajna]] ('''2016'''). ''Deep Neural Networks''. [http://vixra.org/abs/1609.0126 viXra:1609.0126]
* [[James Kirkpatrick]], [[Mathematician#RPascanu|Razvan Pascanu]], [[Neil C. Rabinowitz]], [[Joel Veness]], [[Guillaume Desjardins]], [[Mathematician#AARusu|Andrei A. Rusu]], [[Kieran Milan]], [[John Quan]], [[Tiago Ramalho]], [[Agnieszka Grabska-Barwinska]], [[Demis Hassabis]], [[Claudia Clopath]], [[Dharshan Kumaran]], [[Mathematician#RHadsell|Raia Hadsell]] ('''2016'''). ''Overcoming catastrophic forgetting in neural networks''. [https://arxiv.org/abs/1612.00796 arXiv:1612.00796] <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=70704 catastrophic forgetting] by [[Daniel Shawul]], [[CCC]], May 09, 2019</ref>* [https://dblp.uni-trier.de/pers/hd/n/Niu:Zhenxing Zhenxing Niu], [https://dblp.uni-trier.de/pers/hd/z/Zhou:Mo Mo Zhou], [https://dblp.uni-trier.de/pers/hd/w/Wang_0003:Le Le Wang], [[Xinbo Gao]], [https://dblp.uni-trier.de/pers/hd/h/Hua_0001:Gang Gang Hua] ('''2016'''). ''Ordinal Regression with Multiple Output CNN for Age Estimation''. [https://dblp.uni-trier.de/db/conf/cvpr/cvpr2016.html CVPR 2016], [https://www.cv-foundation.org/openaccess/content_cvpr_2016/app/S21-20.pdf pdf]* [[Li Jing]], [[Yichen Shen]], [[Tena Dubček]], [[John Peurifoy]], [[Scott Skirlo]], [[Mathematician#YLeCun|Yann LeCun]], [[Max Tegmark]], [[Marin Soljačić]] ('''2016'''). ''Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs''. [https://arxiv.org/abs/1612.05231 arXiv:1612.05231]
'''2017'''
* [[Yutian Chen]], [[Matthew W. Hoffman]], [[Sergio Gomez Colmenarejo]], [[Misha Denil]], [[Timothy Lillicrap]], [[Matthew Botvinick]], [[Nando de Freitas]] ('''2017'''). ''Learning to Learn without Gradient Descent by Gradient Descent''. [https://arxiv.org/abs/1611.03824v6 arXiv:1611.03824v6], [http://dblp.uni-trier.de/db/conf/icml/icml2017.html ICML 2017]
* [[Kei Takada]], [[Hiroyuki Iizuka]], [[Masahito Yamamoto]] ('''2017'''). ''Reinforcement Learning for Creating Evaluation Function Using Convolutional Neural Network in Hex''. TAAI 2017 » [[Hex]]
* [[Chao Gao]], [[Martin Müller]], [[Ryan Hayward]] ('''2017'''). ''Focused Depth-first Proof Number Search using Convolutional Neural Networks for the Game of Hex''. [[Conferences#IJCAI2017|IJCAI 2017]]
* [[Thomas Elsken]], [[Jan Hendrik Metzen]], [[Frank Hutter]] ('''2017'''). ''Simple And Efficient Architecture Search for Convolutional Neural Networks''. [https://arxiv.org/abs/1711.04528 arXiv:1711.04528]
* [[Joel Veness]], [[Tor Lattimore]], [https://github.com/avishkar58 Avishkar Bhoopchand], [https://scholar.google.co.uk/citations?user=mB4yebIAAAAJ&hl=en Agnieszka Grabska-Barwinska], [https://dblp.org/pers/hd/m/Mattern:Christopher Christopher Mattern], [https://dblp.org/pers/hd/t/Toth:Peter Peter Toth] ('''2017'''). ''Online Learning with Gated Linear Networks''. [https://arxiv.org/abs/1712.01897 arXiv:1712.01897]
* [https://dblp.uni-trier.de/pers/hd/c/Chen:Qiming Qiming Chen], [[Ren Wu]] ('''2017'''). ''CNN Is All You Need''. [https://arxiv.org/abs/1712.09662 arXiv:1712.09662]
* [https://dblp.org/pers/hd/s/Serb:Alexander Alexantrou Serb], [[Edoardo Manino]], [https://dblp.org/pers/hd/m/Messaris:Ioannis Ioannis Messaris], [https://dblp.org/pers/hd/t/Tran=Thanh:Long Long Tran-Thanh], [https://www.orc.soton.ac.uk/people/tp1f12 Themis Prodromakis] ('''2017'''). ''[https://eprints.soton.ac.uk/425616/ Hardware-level Bayesian inference]''. [https://nips.cc/Conferences/2017 NIPS 2017] » [[Analog Evaluation]]
'''2018'''
* [[Yu Nasu]] ('''2018'''). ''ƎUИИ Efficiently Updatable Neural-Network based Evaluation Functions for Computer Shogi''. Ziosoft Computer Shogi Club, [https://github.com/ynasu87/nnue/blob/master/docs/nnue.pdf pdf] (Japanese with English abstract) » [[NNUE]]
* [[Kei Takada]], [[Hiroyuki Iizuka]], [[Masahito Yamamoto]] ('''2018'''). ''[https://link.springer.com/chapter/10.1007%2F978-3-319-75931-9_2 Computer Hex Algorithm Using a Move Evaluation Method Based on a Convolutional Neural Network]''. [https://link.springer.com/bookseries/7899 Communications in Computer and Information Science] » [[Hex]]
* [[Matthia Sabatelli]], [[Francesco Bidoia]], [[Valeriu Codreanu]], [[Marco Wiering]] ('''2018'''). ''Learning to Evaluate Chess Positions with Deep Neural Networks and Limited Lookahead''. ICPRAM 2018, [http://www.ai.rug.nl/~mwiering/GROUP/ARTICLES/ICPRAM_CHESS_DNN_2018.pdf pdf]
* [[Ashwin Srinivasan]], [[Lovekesh Vig]], [[Michael Bain]] ('''2018'''). ''Logical Explanations for Deep Relational Machines Using Relevance Information''. [https://arxiv.org/abs/1807.00595 arXiv:1807.00595]
* [[Thomas Elsken]], [[Jan Hendrik Metzen]], [[Frank Hutter]] ('''2018'''). ''Neural Architecture Search: A Survey''. [https://arxiv.org/abs/1808.05377 arXiv:1808.05377]
* [[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]], [[Ioannis Antonoglou]], [[Matthew Lai]], [[Arthur Guez]], [[Marc Lanctot]], [[Laurent Sifre]], [[Dharshan Kumaran]], [[Thore Graepel]], [[Timothy Lillicrap]], [[Karen Simonyan]], [[Demis Hassabis]] ('''2018'''). ''[http://science.sciencemag.org/content/362/6419/1140 A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play]''. [https://en.wikipedia.org/wiki/Science_(journal) Science], Vol. 362, No. 6419 <ref>[https://deepmind.com/blog/alphazero-shedding-new-light-grand-games-chess-shogi-and-go/ AlphaZero: Shedding new light on the grand games of chess, shogi and Go] by [[David Silver]], [[Thomas Hubert]], [[Julian Schrittwieser]] and [[Demis Hassabis]], [[DeepMind]], December 03, 2018</ref>
* [[Chao Gao]], [[Siqi Yan]], [[Ryan Hayward]], [[Martin Müller]] ('''2018'''). ''A transferable neural network for Hex''. [[ICGA Journal#40_3|ICGA Journal, Vol. 40, No. 3]]
'''2019'''
* [[Marius Lindauer]], [[Frank Hutter]] ('''2019'''). ''Best Practices for Scientific Research on Neural Architecture Search''. [https://arxiv.org/abs/1909.02453 arXiv:1909.02453]
* [[Guy Haworth]] ('''2019'''). ''Chess endgame news: an endgame challenge for neural nets''. [[ICGA Journal#41_3|ICGA Journal, Vol. 41, No. 3]] » [[Endgame]]
==2020 ...==
* [[Oisín Carroll]], [[Joeran Beel]] ('''2020'''). ''Finite Group Equivariant Neural Networks for Games''. [https://arxiv.org/abs/2009.05027 arXiv:2009.05027]
=Blog & Forum Posts=
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=68119 Instruction for running Scorpio with neural network on linux] by [[Daniel Shawul]], [[CCC]], August 01, 2018 » [[Scorpio]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=69069 Are draws hard to predict?] by [[Daniel Shawul]], [[CCC]], November 27, 2018 » [[Draw]]
* [https://groups.google.com/d/msg/lczero/EGcJSrZYLiw/netJ4S38CgAJ use multiple neural nets?] by [[Warren D. Smith]], [[Computer Chess Forums|LCZero Forum]], December 25, 2018 » [[Leela Chess Zero]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=69393 neural network architecture] by jackd, [[CCC]], December 26, 2018
'''2019'''
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=71301 A question to MCTS + NN experts] by [[Maksim Korzh]], [[CCC]], July 17, 2019 » [[Monte-Carlo Tree Search]]
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=71301&start=3 Re: A question to MCTS + NN experts] by [[Daniel Shawul]], [[CCC]], July 17, 2019
* [https://groups.google.com/d/msg/fishcooking/wOfRuzTSi_8/VgjN8MmSBQAJ high dimensional optimization] by [[Warren D. Smith]], [[Computer Chess Forums|FishCooking]], December 27, 2019 <ref>[[Mathematician#YDauphin|Yann Dauphin]], [[Mathematician#RPascanu|Razvan Pascanu]], [[Mathematician#CGulcehre|Caglar Gulcehre]], [[Mathematician#KCho|Kyunghyun Cho]], [[Mathematician#SGanguli|Surya Ganguli]], [[Mathematician#YBengio|Yoshua Bengio]] ('''2014'''). ''Identifying and attacking the saddle point problem in high-dimensional non-convex optimization''. [https://arxiv.org/abs/1406.2572 arXiv:1406.2572]</ref>
==2020 ...==
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74077 How to work with batch size in neural network] by Gertjan Brouwer, [[CCC]], June 02, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531 NNUE accessible explanation] by [[Martin Fierz]], [[CCC]], July 21, 2020 [[NNUE]]
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=1 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], July 23, 2020
: [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74531&start=5 Re: NNUE accessible explanation] by [[Jonathan Rosenthal]], [[CCC]], July 24, 2020
* [http://www.talkchess.com/forum3/viewtopic.php?f=2&t=74607 LC0 vs. NNUE - some tech details...] by [[Srdja Matovic]], [[CCC]], July 29, 2020 » [[Leela Chess Zero#Lc0|Lc0]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74771 AB search with NN on GPU...] by [[Srdja Matovic]], [[CCC]], August 13, 2020 » [[GPU]] <ref>[https://forums.developer.nvidia.com/t/kernel-launch-latency/62455 kernel launch latency - CUDA / CUDA Programming and Performance - NVIDIA Developer Forums] by LukeCuda, June 18, 2018</ref>
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74777 Neural Networks weights type] by [[Fabio Gobbato]], [[CCC]], August 13, 2020 » [[Stockfish NNUE]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=74955 Train a neural network evaluation] by [[Fabio Gobbato]], [[CCC]], September 01, 2020 » [[Automated Tuning]], [[NNUE]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75042 Neural network quantization] by [[Fabio Gobbato]], [[CCC]], September 08, 2020 » [[NNUE]]
* [http://www.talkchess.com/forum3/viewtopic.php?f=7&t=75190 First success with neural nets] by [[Jonathan Kreuzer]], [[CCC]], September 23, 2020
=External Links=
* [https://en.wikipedia.org/wiki/Neocognitron Neocognitron from Wikipedia]
* [http://www.scholarpedia.org/article/Neocognitron Neocognitron - Scholarpedia] by [http://www.scholarpedia.org/article/User:Kunihiko_Fukushima Kunihiko Fukushima]
* [https://en.wikipedia.org/wiki/Neural_architecture_search Neural architecture search from Wikipedia]
* [https://en.wikipedia.org/wiki/Neuromorphic_engineering Neuromorphic engineering from Wikipedia]
: [https://en.wikipedia.org/wiki/Neurogrid Neurogrid from Wikipedia]
: [https://towardsdatascience.com/types-of-convolutions-in-deep-learning-717013397f4d An Introduction to different Types of Convolutions in Deep Learning] by [http://plpp.de/ Paul-Louis Pröve], July 22, 2017
: [https://towardsdatascience.com/squeeze-and-excitation-networks-9ef5e71eacd7 Squeeze-and-Excitation Networks] by [http://plpp.de/ Paul-Louis Pröve], October 17, 2017
* [https://towardsdatascience.com/deep-convolutional-neural-networks-ccf96f830178 Deep Convolutional Neural Networks] by Pablo Ruiz, October 11, 2018
===ResNet===
* [https://en.wikipedia.org/wiki/Residual_neural_network Residual neural network from Wikipedia]
* [https://wiki.tum.de/display/lfdv/Deep+Residual+Networks Deep Residual Networks] from [https://wiki.tum.de/ TUM Wiki], [[Technical University of Munich]]
* [https://towardsdatascience.com/understanding-and-visualizing-resnets-442284831be8 Understanding and visualizing ResNets] by Pablo Ruiz, October 8, 2018
===RNNs===
* [https://en.wikipedia.org/wiki/Recurrent_neural_network Recurrent neural network from Wikipedia]
* [https://en.wikipedia.org/wiki/Rectifier_(neural_networks) Rectifier (neural networks) from Wikipedia]
* [https://en.wikipedia.org/wiki/Sigmoid_function Sigmoid function from Wikipedia]
* [https://en.wikipedia.org/wiki/Softmax_function Softmax function from Wikipedia]
==Backpropagation==
* [https://en.wikipedia.org/wiki/Backpropagation Backpropagation from Wikipedia]
: [https://www.youtube.com/watch?v=9KM9Td6RVgQ Part 6: Training]
: [https://www.youtube.com/watch?v=S4ZUwgesjS8 Part 7: Overfitting, Testing, and Regularization]
* [https://www.youtube.com/playlist?list=PLgomWLYGNl1dL1Qsmgumhcg4HOcWZMd3k NN - Fully Connected Tutorial], [https://en.wikipedia.org/wiki/YouTube YouTube] Videos by [[Finn Eggers]]
* [https://www.youtube.com/watch?v=UdSK7nnJKHU Deep Learning Master Class] by [[Ilya Sutskever]], [https://en.wikipedia.org/wiki/YouTube YouTube] Video
* [https://www.youtube.com/watch?v=Ih5Mr93E-2c&hd=1 Lecture 10 - Neural Networks] from [http://work.caltech.edu/telecourse.html Learning From Data - Online Course (MOOC)] by [https://en.wikipedia.org/wiki/Yaser_Abu-Mostafa Yaser Abu-Mostafa], [https://en.wikipedia.org/wiki/California_Institute_of_Technology Caltech], [https://en.wikipedia.org/wiki/YouTube YouTube] Video
