422
edits
Changes
GPU
,update on instruction throughput section
* 64 bit Integer Performance
: Current GPU [https://en.wikipedia.org/wiki/Processor_register registers] and Vector-[https://en.wikipedia.org/wiki/Arithmetic_logic_unit ALUs] are 32 bit wide and have to emulate 64 bit integer operations.<ref>[https://en.wikichip.org/w/images/a/a1/vega-whitepaper.pdf |AMD Vega White Paper]</ref> <ref>[https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf Nvidia Turing White Paper]</ref>
* TensorCores
: With Nvidia [https://en.wikipedia.org/wiki/Volta_(microarchitecture) Volta] series TensorCores were introduced. They offer fp16*fp16+fp32, matrix-multiplication-accumulate-units, used to accelerate neural networks.<ref>[https://on-demand.gputechconf.com/gtc/2017/presentation/s7798-luke-durant-inside-volta.pdf INSIDE VOLTA]</ref>
* Mixed Precision Support
: Newer architectures like Nvidia [https://en.wikipedia.org/wiki/Turing_(microarchitecture) Turing] and AMD [https://en.wikipedia.org/wiki/AMD_RX_Vega_series Vega] have mixed precision support, which can double . Vega doubles the [https://en.wikipedia.org/w/index.php?title=FP16&redirect=no FP16] and quadruples the [https://en.wikipedia.org/wiki/Half-precision_floating-point_format fp16Integer_(computer_science)#Common_integral_data_types INT8] throughput resp. quadruple <ref>[https://en.wikipedia.org/wiki/Graphics_Core_Next#fifth Vega (GCN 5th generation)]</ref>Turing doubles the int8 FP16 throughput and boost neural networks significantly. * TensorCores: With Nvidia of its [https://en.wikipedia.org/wiki/Volta_(microarchitecture) VoltaFloating-point_unit FPUs] series and its 2nd gen TensorCores were introduced. They offer fp16*fp16+fp32support now FP16, matrix-matrix-multiplication unitsINT8, used to accelerate neural networksINT4 optimized computation.<ref>[https://on-demandwww.gputechconfanandtech.com/gtc/2017show/presentation13282/s7798nvidia-luketuring-durantarchitecture-insidedeep-volta.pdf INSIDE VOLTAdive/6 AnandTech - Nvidia Turing Deep Dive]</ref>
==Throughput Examples==
