25,161
edits
Changes
NUMA
,Created page with "'''Home * Hardware * Memory * NUMA''' FILE:NUMA.svg|border|right|thumb| Possible NUMA system <ref>One possible architecture of a NUMA system. Original..."
'''[[Main Page|Home]] * [[Hardware]] * [[Memory]] * NUMA'''
[[FILE:NUMA.svg|border|right|thumb| Possible NUMA system <ref>One possible architecture of a NUMA system. Originally created in [https://en.wikipedia.org/wiki/Microsoft_Visio Visio 2010], cleaned up with [https://en.wikipedia.org/wiki/Inkscape Inkscape], by Moop2000, October 4, 2010, [https://en.wikipedia.org/wiki/Wikimedia_Commons Wikimedia Commons]</ref> ]]
'''NUMA''', (Non-uniform memory access)<br/>
a [https://en.wikipedia.org/wiki/Multiprocessing multiprocessing] memory design where the main memory is partitioned between processors. Opposed to [[SMP]], where all processors compete for access to the centralized shared [https://en.wikipedia.org/wiki/Memory_bus memory bus], making it difficult to scale well bejoind 8 to 12 CPUs <ref>[http://lse.sourceforge.net/numa/faq/ NUMA Frequently Asked Questions - 9. Why should I use NUMA? What are the benefits of NUMA?]</ref>, NUMA splits the main memory into so called nodes with separate memory busses for subsets of processors, and high speed interconnection between nodes, either directly in so called 1-hop distance, or indirectly in 2-hop distance. Despite the high speed interconnection, NUMA memory access time varies considerably between faster local memory and remote memory of other nodes. Maintaining [https://en.wikipedia.org/wiki/Cache_coherence cache coherence] of processor caches adds significant overhead to NUMA Systems, addressed by [https://en.wikipedia.org/wiki/Non-uniform_memory_access#Cache_coherent_NUMA_.28ccNUMA.29 ccNUMA] which is mostly used synonymous for current NUMA implementations <ref>[http://lse.sourceforge.net/numa/faq/ NUMA Frequently Asked Questions - 4. What is the difference between NUMA and ccNUMA?]</ref>.
=x86=
[[AMD]] implemented NUMA with its [[x86-64|Opteron]] processor in 2003, using [https://en.wikipedia.org/wiki/HyperTransport HyperTransport]. [[Intel]] announced NUMA compatibility for their [[x86]] servers in late 2007 with [https://en.wikipedia.org/wiki/Nehalem_(microarchitecture) Nehalem CPUs] using [https://en.wikipedia.org/wiki/Intel_QuickPath_Interconnect QuickPath Interconnect] <ref>[https://en.wikipedia.org/wiki/Non-uniform_memory_access Non-Uniform Memory Access (NUMA) from Wikipedia]</ref>.
=Considerations=
Scheduling of [[Thread|threads]] across nodes and cores of a system is a complicated topic due to access of independent or shared data. There are several considerations in ccNUMA aware [https://en.wikipedia.org/wiki/Operating_system operating systems] and software, such as keeping data local by virtue of first touch <ref>[https://doc.xdevs.com/doc/AMD/_Performance/Performance%20Guidelines%20for%20AMD%20Athlon%2064%20and%20AMD%20Opteron%20ccNUMA%20Multiprocessor%20Systems.%20rev.3.00%5D.%5B2006-06%5D.pdf Performance Guidelines for AMD Athlon™ 64 and AMD Opteron™ ccNUMA Multiprocessor Systems] (pdf) - 3.2.1 Keeping Data Local by Virtue of first Touch, pp. 22</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=56858&start=3 Re: thread affinity] by [[Robert Hyatt]], [[CCC]], July 03, 2015</ref>. NUMA and [https://en.wikipedia.org/wiki/Processor_affinity processor affinity] [https://en.wikipedia.org/wiki/Application_programming_interface APIs] help application programmers to bind threads or processes to NUMA nodes or to allocate memory from a certain node.
=See also=
* [[Optimization]]
* [[Parallel Search]]
* [[SMP]]
* [[Thread]]
=Selected Publications=
* [http://www.halobates.de/ Andi Kleen] ('''2004'''). ''An NUMA API for Linux''. SUSE Labs, [http://halobates.de/numaapi3.pdf pdf]
* [http://de.wikipedia.org/wiki/Ulrich_Drepper Ulrich Drepper] ('''2007'''). ''What Every Programmer Should Know About Memory''. [http://www.akkadia.org/drepper/cpumemory.pdf pdf], also hosted by [https://en.wikipedia.org/wiki/LWN.net LWN.net]
: [http://lwn.net/Articles/250967/ Memory part 1]
: [http://lwn.net/Articles/252125/ Memory part 2: CPU caches]
: [http://lwn.net/Articles/253361/ Memory part 3: Virtual Memory]
* [http://lwn.net/Articles/254445/ Memory part 4: NUMA support]
: [http://lwn.net/Articles/255364/ Memory part 5: What programmers can do]
* [https://www.linkedin.com/in/nakulmanchanda Nakul Manchanda], [https://www.linkedin.com/in/anandkaran Karan Anand] ('''2010'''). ''Non-Uniform Memory Access (NUMA)''. [https://en.wikipedia.org/wiki/New_York_University New York University], [http://cs.nyu.edu/~lerner/spring10/projects/NUMA.pdf pdf]
* [https://scholar.google.de/citations?user=wx0D5q0AAAAJ&hl=en Stefan Lankes], [https://www.researchgate.net/profile/Thomas_Roehl Thomas Roehl], [https://terboven.com/ Christian Terboven], [https://www.linkedin.com/in/thomas-bemmerl-b3657547 Thomas Bemmerl] ('''2012'''). ''[http://publications.rwth-aachen.de/record/207349 Node-Based Memory Management for Scalable NUMA Architectures]''. [https://en.wikipedia.org/wiki/RWTH_Aachen_University RWTH Aachen], [http://www.mcs.anl.gov/events/workshops/ross/2012/ ROSS 2012], [http://htor.inf.ethz.ch/ross2012/slides/ross2012-lankes.pdf slides as pdf]
* [https://www.rrze.fau.de/wir-ueber-uns/organigramm/mitarbeiter/index.shtml/georg-hager.shtml Georg Hager] <ref>[https://blogs.fau.de/hager/ Georg Hager's Blog | Random thoughts on High Performance Computing]</ref>, [http://dblp.uni-trier.de/pers/hd/t/Treibig:Jan Jan Treibig], [http://dblp.uni-trier.de/pers/hd/w/Wellein:Gerhard Gerhard Wellein] ('''2013'''). ''The Practitioner's Cookbook for Good Parallel Performance on Multi- and Many-Core Systems''. [https://de.wikipedia.org/wiki/Regionales_Rechenzentrum_Erlangen RRZE], [http://sc13.supercomputing.org/ SC13], [https://blogs.fau.de/hager/files/2013/11/sc13_tutorial_134.pdf slides as pdf]
* [https://www.linkedin.com/in/rikvanriel Rik van Riel], [https://www.linkedin.com/in/chegu Vinod Chegu] ('''2014'''). ''Automatic NUMA Balancing''. [https://www.redhat.com/en/about/press-releases/red-hat-announces-dates-for-red-hat-summit-2014-in-san-francisco Red Hat Summit 2014], [http://events.linuxfoundation.org/sites/events/files/slides/summit2014_riel_chegu_w_0340_automatic_numa_balancing_0.pdf slides as pdf], [https://youtu.be/mjVw_oe1hEA video lecture] by Rik van Riel
=Forum Posts=
==2000 ...==
* [https://www.stmintz.com/ccc/index.php?id=249651 DTS NUMA] by [[Vincent Diepeveen]], [[CCC]], September 03, 2002 » [[Dynamic Tree Splitting]]
* [https://www.stmintz.com/ccc/index.php?id=360103 What's the difference between NUMA, SMP and MPI for chess?] by [[Joachim Rang]], [[CCC]], April 15, 2004 » [[SMP]]
* [https://www.stmintz.com/ccc/index.php?id=410357 Opteron NUMA/SMP question] by Matthew Hull, [[CCC]], February 09, 2005
==2010 ...==
* [http://www.talkchess.com/forum/viewtopic.php?t=47367 optimizing performance on dual Xeon systems (NUMA)] by [[Jon Dart]], [[CCC]], February 28, 2013
* [http://www.talkchess.com/forum/viewtopic.php?t=52503 Smp concepts] by [[Michael Hoffmann]], [[CCC]], June 01, 2014 » [[SMP]]
==2015 ...==
* [http://www.talkchess.com/forum/viewtopic.php?t=55467 NUMA-awareness] by [[Louis Zulli]], [[CCC]], February 25, 2015
* [http://www.talkchess.com/forum/viewtopic.php?t=56858 thread affinity] by [[Martin Sedlak]], [[CCC]], July 03, 2015 » [[Thread]]
: [http://www.talkchess.com/forum/viewtopic.php?t=56858&start=3 Re: thread affinity] by [[Robert Hyatt]], [[CCC]], July 03, 2015
* [http://www.talkchess.com/forum/viewtopic.php?t=56937 Actual speedups from YBWC and ABDADA on 8+ core machines?] by [[Tom Kerrigan]], [[CCC]], July 10, 2015 » [[Young Brothers Wait Concept]], [[ABDADA]]
* [http://www.talkchess.com/forum/viewtopic.php?t=58830 NUMA 101] by [[Robert Hyatt]], [[CCC]], January 07, 2016 » [[Crafty]]
* [http://www.talkchess.com/forum/viewtopic.php?t=60875 NUMA in a YBWC implementation] by [[Edsel Apostol]], [[CCC]], July 20, 2016 » [[Young Brothers Wait Concept]]
* [https://groups.google.com/d/msg/fishcooking/ezt6MrAuXqs/qIR2HEciEgAJ lets get the ball moving down the field on numa awareness] by [[Mohammed Li]], [[Computer Chess Forums|FishCooking]], August 30, 2016
* [https://groups.google.com/d/msg/fishcooking/bcFKVr85fIQ/Z6-hoGcKFAAJ search thread memory allocation (NUMA)] by [[Ronald de Man]], [[Computer Chess Forums|FishCooking]], September 06, 2016
* [http://www.talkchess.com/forum/viewtopic.php?t=61472 What do you do with NUMA?] by [[Matthew Lai]], [[CCC]], September 19, 2016
* [https://groups.google.com/d/msg/fishcooking/7hHC075ZnnM/IaITCiLaBwAJ NUMA test compilation] by Joachim Müller, [[Computer Chess Forums|FishCooking]], November 05, 2016 » [[Stockfish]]
* [http://www.talkchess.com/forum/viewtopic.php?t=63581 What Linux compatible Numa aware engines are available?] by [[Dann Corbit]], [[CCC]], March 29, 2017 » [[Linux]]
=External Links=
* [https://en.wikipedia.org/wiki/Non-uniform_memory_access Non-Uniform Memory Access (NUMA) from Wikipedia]
* [http://lse.sourceforge.net/numa/faq/ NUMA Frequently Asked Questions]
* [http://wiki.osdev.org/Multiprocessing Multiprocessing - OSDev Wiki]
* [http://netlib.org/utk/papers/advanced-computers/ccnuma.html ccNUMA machines] in [[Aad J. van der Steen]], [http://www.netlib.org/utk/people/JackDongarra/ Jack J. Dongarra] ('''2004'''). ''[http://netlib.org/utk/papers/advanced-computers/overview.html Overview of Recent Supercomputers]''.
==[[Linux]]==
* [http://man7.org/linux/man-pages/man7/numa.7.html numa(7) - Linux manual page]
* [http://developer.amd.com/wordpress/media/2012/10/LibNUMA-WP-fv1.pdf A NUMA API for Linux] (pdf, April 2015)
==[[Windows]]==
* [http://msdn.microsoft.com/en-us/library/aa965223%28v=VS.85%29.aspx Allocating Memory from a NUMA Node, MSDN]
* [https://msdn.microsoft.com/en-us/library/windows/desktop/aa363804(v=vs.85).aspx NUMA Support (Windows), MSDN]
* [https://msdn.microsoft.com/en-us/library/windows/desktop/dd405503 Processor Groups (Windows), MSDN]
==[[x86]]==
* [http://software.intel.com/en-us/articles/optimizing-applications-for-numa Optimizing Applications for NUMA | Intel® Developer Zone]
* [https://doc.xdevs.com/doc/AMD/_Performance/Performance%20Guidelines%20for%20AMD%20Athlon%2064%20and%20AMD%20Opteron%20ccNUMA%20Multiprocessor%20Systems.%20rev.3.00%5D.%5B2006-06%5D.pdf Performance Guidelines for AMD Athlon™ 64 and AMD Opteron™ ccNUMA Multiprocessor Systems] (pdf)
=References=
<references />
'''[[Memory|Up one Level]]'''
[[FILE:NUMA.svg|border|right|thumb| Possible NUMA system <ref>One possible architecture of a NUMA system. Originally created in [https://en.wikipedia.org/wiki/Microsoft_Visio Visio 2010], cleaned up with [https://en.wikipedia.org/wiki/Inkscape Inkscape], by Moop2000, October 4, 2010, [https://en.wikipedia.org/wiki/Wikimedia_Commons Wikimedia Commons]</ref> ]]
'''NUMA''', (Non-uniform memory access)<br/>
a [https://en.wikipedia.org/wiki/Multiprocessing multiprocessing] memory design where the main memory is partitioned between processors. Opposed to [[SMP]], where all processors compete for access to the centralized shared [https://en.wikipedia.org/wiki/Memory_bus memory bus], making it difficult to scale well bejoind 8 to 12 CPUs <ref>[http://lse.sourceforge.net/numa/faq/ NUMA Frequently Asked Questions - 9. Why should I use NUMA? What are the benefits of NUMA?]</ref>, NUMA splits the main memory into so called nodes with separate memory busses for subsets of processors, and high speed interconnection between nodes, either directly in so called 1-hop distance, or indirectly in 2-hop distance. Despite the high speed interconnection, NUMA memory access time varies considerably between faster local memory and remote memory of other nodes. Maintaining [https://en.wikipedia.org/wiki/Cache_coherence cache coherence] of processor caches adds significant overhead to NUMA Systems, addressed by [https://en.wikipedia.org/wiki/Non-uniform_memory_access#Cache_coherent_NUMA_.28ccNUMA.29 ccNUMA] which is mostly used synonymous for current NUMA implementations <ref>[http://lse.sourceforge.net/numa/faq/ NUMA Frequently Asked Questions - 4. What is the difference between NUMA and ccNUMA?]</ref>.
=x86=
[[AMD]] implemented NUMA with its [[x86-64|Opteron]] processor in 2003, using [https://en.wikipedia.org/wiki/HyperTransport HyperTransport]. [[Intel]] announced NUMA compatibility for their [[x86]] servers in late 2007 with [https://en.wikipedia.org/wiki/Nehalem_(microarchitecture) Nehalem CPUs] using [https://en.wikipedia.org/wiki/Intel_QuickPath_Interconnect QuickPath Interconnect] <ref>[https://en.wikipedia.org/wiki/Non-uniform_memory_access Non-Uniform Memory Access (NUMA) from Wikipedia]</ref>.
=Considerations=
Scheduling of [[Thread|threads]] across nodes and cores of a system is a complicated topic due to access of independent or shared data. There are several considerations in ccNUMA aware [https://en.wikipedia.org/wiki/Operating_system operating systems] and software, such as keeping data local by virtue of first touch <ref>[https://doc.xdevs.com/doc/AMD/_Performance/Performance%20Guidelines%20for%20AMD%20Athlon%2064%20and%20AMD%20Opteron%20ccNUMA%20Multiprocessor%20Systems.%20rev.3.00%5D.%5B2006-06%5D.pdf Performance Guidelines for AMD Athlon™ 64 and AMD Opteron™ ccNUMA Multiprocessor Systems] (pdf) - 3.2.1 Keeping Data Local by Virtue of first Touch, pp. 22</ref> <ref>[http://www.talkchess.com/forum/viewtopic.php?t=56858&start=3 Re: thread affinity] by [[Robert Hyatt]], [[CCC]], July 03, 2015</ref>. NUMA and [https://en.wikipedia.org/wiki/Processor_affinity processor affinity] [https://en.wikipedia.org/wiki/Application_programming_interface APIs] help application programmers to bind threads or processes to NUMA nodes or to allocate memory from a certain node.
=See also=
* [[Optimization]]
* [[Parallel Search]]
* [[SMP]]
* [[Thread]]
=Selected Publications=
* [http://www.halobates.de/ Andi Kleen] ('''2004'''). ''An NUMA API for Linux''. SUSE Labs, [http://halobates.de/numaapi3.pdf pdf]
* [http://de.wikipedia.org/wiki/Ulrich_Drepper Ulrich Drepper] ('''2007'''). ''What Every Programmer Should Know About Memory''. [http://www.akkadia.org/drepper/cpumemory.pdf pdf], also hosted by [https://en.wikipedia.org/wiki/LWN.net LWN.net]
: [http://lwn.net/Articles/250967/ Memory part 1]
: [http://lwn.net/Articles/252125/ Memory part 2: CPU caches]
: [http://lwn.net/Articles/253361/ Memory part 3: Virtual Memory]
* [http://lwn.net/Articles/254445/ Memory part 4: NUMA support]
: [http://lwn.net/Articles/255364/ Memory part 5: What programmers can do]
* [https://www.linkedin.com/in/nakulmanchanda Nakul Manchanda], [https://www.linkedin.com/in/anandkaran Karan Anand] ('''2010'''). ''Non-Uniform Memory Access (NUMA)''. [https://en.wikipedia.org/wiki/New_York_University New York University], [http://cs.nyu.edu/~lerner/spring10/projects/NUMA.pdf pdf]
* [https://scholar.google.de/citations?user=wx0D5q0AAAAJ&hl=en Stefan Lankes], [https://www.researchgate.net/profile/Thomas_Roehl Thomas Roehl], [https://terboven.com/ Christian Terboven], [https://www.linkedin.com/in/thomas-bemmerl-b3657547 Thomas Bemmerl] ('''2012'''). ''[http://publications.rwth-aachen.de/record/207349 Node-Based Memory Management for Scalable NUMA Architectures]''. [https://en.wikipedia.org/wiki/RWTH_Aachen_University RWTH Aachen], [http://www.mcs.anl.gov/events/workshops/ross/2012/ ROSS 2012], [http://htor.inf.ethz.ch/ross2012/slides/ross2012-lankes.pdf slides as pdf]
* [https://www.rrze.fau.de/wir-ueber-uns/organigramm/mitarbeiter/index.shtml/georg-hager.shtml Georg Hager] <ref>[https://blogs.fau.de/hager/ Georg Hager's Blog | Random thoughts on High Performance Computing]</ref>, [http://dblp.uni-trier.de/pers/hd/t/Treibig:Jan Jan Treibig], [http://dblp.uni-trier.de/pers/hd/w/Wellein:Gerhard Gerhard Wellein] ('''2013'''). ''The Practitioner's Cookbook for Good Parallel Performance on Multi- and Many-Core Systems''. [https://de.wikipedia.org/wiki/Regionales_Rechenzentrum_Erlangen RRZE], [http://sc13.supercomputing.org/ SC13], [https://blogs.fau.de/hager/files/2013/11/sc13_tutorial_134.pdf slides as pdf]
* [https://www.linkedin.com/in/rikvanriel Rik van Riel], [https://www.linkedin.com/in/chegu Vinod Chegu] ('''2014'''). ''Automatic NUMA Balancing''. [https://www.redhat.com/en/about/press-releases/red-hat-announces-dates-for-red-hat-summit-2014-in-san-francisco Red Hat Summit 2014], [http://events.linuxfoundation.org/sites/events/files/slides/summit2014_riel_chegu_w_0340_automatic_numa_balancing_0.pdf slides as pdf], [https://youtu.be/mjVw_oe1hEA video lecture] by Rik van Riel
=Forum Posts=
==2000 ...==
* [https://www.stmintz.com/ccc/index.php?id=249651 DTS NUMA] by [[Vincent Diepeveen]], [[CCC]], September 03, 2002 » [[Dynamic Tree Splitting]]
* [https://www.stmintz.com/ccc/index.php?id=360103 What's the difference between NUMA, SMP and MPI for chess?] by [[Joachim Rang]], [[CCC]], April 15, 2004 » [[SMP]]
* [https://www.stmintz.com/ccc/index.php?id=410357 Opteron NUMA/SMP question] by Matthew Hull, [[CCC]], February 09, 2005
==2010 ...==
* [http://www.talkchess.com/forum/viewtopic.php?t=47367 optimizing performance on dual Xeon systems (NUMA)] by [[Jon Dart]], [[CCC]], February 28, 2013
* [http://www.talkchess.com/forum/viewtopic.php?t=52503 Smp concepts] by [[Michael Hoffmann]], [[CCC]], June 01, 2014 » [[SMP]]
==2015 ...==
* [http://www.talkchess.com/forum/viewtopic.php?t=55467 NUMA-awareness] by [[Louis Zulli]], [[CCC]], February 25, 2015
* [http://www.talkchess.com/forum/viewtopic.php?t=56858 thread affinity] by [[Martin Sedlak]], [[CCC]], July 03, 2015 » [[Thread]]
: [http://www.talkchess.com/forum/viewtopic.php?t=56858&start=3 Re: thread affinity] by [[Robert Hyatt]], [[CCC]], July 03, 2015
* [http://www.talkchess.com/forum/viewtopic.php?t=56937 Actual speedups from YBWC and ABDADA on 8+ core machines?] by [[Tom Kerrigan]], [[CCC]], July 10, 2015 » [[Young Brothers Wait Concept]], [[ABDADA]]
* [http://www.talkchess.com/forum/viewtopic.php?t=58830 NUMA 101] by [[Robert Hyatt]], [[CCC]], January 07, 2016 » [[Crafty]]
* [http://www.talkchess.com/forum/viewtopic.php?t=60875 NUMA in a YBWC implementation] by [[Edsel Apostol]], [[CCC]], July 20, 2016 » [[Young Brothers Wait Concept]]
* [https://groups.google.com/d/msg/fishcooking/ezt6MrAuXqs/qIR2HEciEgAJ lets get the ball moving down the field on numa awareness] by [[Mohammed Li]], [[Computer Chess Forums|FishCooking]], August 30, 2016
* [https://groups.google.com/d/msg/fishcooking/bcFKVr85fIQ/Z6-hoGcKFAAJ search thread memory allocation (NUMA)] by [[Ronald de Man]], [[Computer Chess Forums|FishCooking]], September 06, 2016
* [http://www.talkchess.com/forum/viewtopic.php?t=61472 What do you do with NUMA?] by [[Matthew Lai]], [[CCC]], September 19, 2016
* [https://groups.google.com/d/msg/fishcooking/7hHC075ZnnM/IaITCiLaBwAJ NUMA test compilation] by Joachim Müller, [[Computer Chess Forums|FishCooking]], November 05, 2016 » [[Stockfish]]
* [http://www.talkchess.com/forum/viewtopic.php?t=63581 What Linux compatible Numa aware engines are available?] by [[Dann Corbit]], [[CCC]], March 29, 2017 » [[Linux]]
=External Links=
* [https://en.wikipedia.org/wiki/Non-uniform_memory_access Non-Uniform Memory Access (NUMA) from Wikipedia]
* [http://lse.sourceforge.net/numa/faq/ NUMA Frequently Asked Questions]
* [http://wiki.osdev.org/Multiprocessing Multiprocessing - OSDev Wiki]
* [http://netlib.org/utk/papers/advanced-computers/ccnuma.html ccNUMA machines] in [[Aad J. van der Steen]], [http://www.netlib.org/utk/people/JackDongarra/ Jack J. Dongarra] ('''2004'''). ''[http://netlib.org/utk/papers/advanced-computers/overview.html Overview of Recent Supercomputers]''.
==[[Linux]]==
* [http://man7.org/linux/man-pages/man7/numa.7.html numa(7) - Linux manual page]
* [http://developer.amd.com/wordpress/media/2012/10/LibNUMA-WP-fv1.pdf A NUMA API for Linux] (pdf, April 2015)
==[[Windows]]==
* [http://msdn.microsoft.com/en-us/library/aa965223%28v=VS.85%29.aspx Allocating Memory from a NUMA Node, MSDN]
* [https://msdn.microsoft.com/en-us/library/windows/desktop/aa363804(v=vs.85).aspx NUMA Support (Windows), MSDN]
* [https://msdn.microsoft.com/en-us/library/windows/desktop/dd405503 Processor Groups (Windows), MSDN]
==[[x86]]==
* [http://software.intel.com/en-us/articles/optimizing-applications-for-numa Optimizing Applications for NUMA | Intel® Developer Zone]
* [https://doc.xdevs.com/doc/AMD/_Performance/Performance%20Guidelines%20for%20AMD%20Athlon%2064%20and%20AMD%20Opteron%20ccNUMA%20Multiprocessor%20Systems.%20rev.3.00%5D.%5B2006-06%5D.pdf Performance Guidelines for AMD Athlon™ 64 and AMD Opteron™ ccNUMA Multiprocessor Systems] (pdf)
=References=
<references />
'''[[Memory|Up one Level]]'''
