NUMA

NUMA

1. nOn-UniFOrM MeMOry aCCess (NUMA) is the phenomenon that memory at various points in the address space of a processor have different performance characteristics. At current processor speeds, the signal path length from the processor to memory plays a significant role. increased signal path length not only increases latency to memory

   • how operating systems handle nUma memory
     • There are several broad categories inwhich modern production operatingsystems allow for the management of NUMA: accepting the performance mismatch, hardware memory striping, heuristic memory placement,a static NUMA configurations, and application-controlled NUMA placement.

2. how Does Linux handle nUma?

   • Linux manages memory in zones. In a non-NUMA Linux system, zones are used to describe memory ranges required to support devices that are not able to perform DMA (direct memory access) to all memory locations. Zones are also used to mark memory for other special needs such as movable memory or memory that requires explicit mappings for access by the kernel (HIGHMEM), but that is not relevant to the discussion here. When NUMA is enabled, then more memory zones are created and they are also associated with NUMA nodes. A NUMA node can have multiple zones since it may be able to serve multiple DMA areas.
     • Memory policies.
       • The most important memory policies are: NODE LOCAL. The allocation occurs from the memory node local to where the code of the process is currently executing. INTERLEAVE. Allocation occurs round-robin. First a page will be allocated from node 0, then from node 1, then again from node 0, and so on. Interleaving is used to distribute memory accesses for structures that may be accessed from multiple pro- cessors in the system in order to have an even load on the interconnect and the memory of each node.
     • Basic operations on process startup.
       • Processes inherit the memory policy from their parent. Most of the time the policy is left at the default, which means NODE LOCAL. When a process is started on a processor, then memory is allocated for that process from the local NUMA node. All other allocations of the process (through growing the heap, page faults, mmap, and so on) will also be satisfied from the local NUMA node.
     • NUMA information
       • There are numerous ways to view information about the NUMA characteristics of the system and of various processes currently running. The hardware NUMA configuration of a system can be viewed through the use of numactl --hardware.