85
edits
Changes
GPU
,→See also
GPUs are much more suited than CPUs to implement and train [[Neural Networks#Convolutional|Convolutional Neural Networks]] (CNN), and were therefore also responsible for the [[Deep Learning|deep learning]] boom,
also affecting game playing programs combining CNN with [[Monte-Carlo Tree Search|MCTS]], as pioneered by [[Google]] [[DeepMind|DeepMind's]] [[AlphaGo]] and [[AlphaZero]] entities in [[Go]], [[Shogi]] and [[Chess]] using [https://en.wikipedia.org/wiki/Tensor_processing_unit TPUs], and the open source projects [[Leela Zero]] headed by [[Gian-Carlo Pascutto]] for [[Go]] and its [[Leela Chess Zero]] adaption.
= History =
Although GPUs were invented in the late 1990s to accelerate 3d simulations, they drew upon a rich history of vector-compute and SIMD-compute from 1980s and 1970s supercomputers. As such, many publications relating to Cray-vector supercomputers or the Connection Machine supercomputer easily apply to modern GPUs. For example, all the algorithms described in the 1986 publication "[https://dl.acm.org/citation.cfm?id=7903 Data Parallel Algorithms]" can be efficiently executed on a modern GPU workgroup (roughly ~256x GPU threads). The Data Parallel Algorithms paper is a beginner-level algorithms paper, demonstrating simple and efficient parallel-prefix sum, parallel-linked list traversal, parallel RegEx matching on the 4096x parallel Connection Machine-2 supercomputer.
Modern papers on GPUs, such as NVidia's excellent [Chapter 39. Parallel Prefix Sum (Scan) with CUDA (GPU Gems 3) https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch39.html], are built on top of these papers from the 1980s or 1990s. As such, the beginner will find it far easier to read the papers from the 1980s or 90s before attempting to read a modern piece like GPU Gems 3.
=See also=
