AutoNUMA

kernel-level optimization designed to improve performance on NUMA architecture

Task | running process or thread that executes on a CPU core

Page belonging to the same task can reside on multiple NUMA nodes.

Default memory allocation policy in Linux is first-touch, memory pages are allocated on the NUMA nodes where the task first accesses them. if a multi-threaded application runs across multiple NUMA nodes, different threads may allocate memory on their local NUMA nodes, leading to a situation where a single task has memory in multiple locations.

If applications are using share memory e.g. database, different process/threads running on different NUMA nodes might share memory regions, causing memory to be spread across multiple nodes.

If a system has limited free memory on a single NUMA node, allocation may be spread across multiple NUMA nodes.

Periodically each task’s memory is unmapped

PTE for a task’s memory gets cleared.
Why? AutoNUMA needs to detect which memory pages are accessed remotely.

Page fault → Tack whether the faulting CPU is on the same node or it’s a remote access: log it → Task is migrated to the node where most of the memory is or the memory pages are migrated to the node where the task is running.

Migration in two ways

Migrate the process to the NUMA node where most of the memory is

Migrate the memory pages to the NUMA node where the process is running

Virtual Memory | Memory space assigned to each program

Each program has its own virtual memory that doesn’t overlap with others

Page | smallest fixed-size block of memory that the CPU and OS manage in a virtual memory system

A small chunk of memory managed by the OS

typical page size: 4 KB, 2 MB (huge page)

Memory is divided into pages and page can be mapped to:

• Physical RAM
• Swap space
• Shared memory regions
• Another process’s memory

Page Table | Data Structure to manage virtual to physical address translation

Page Table Entry | Stores information about a single page

A Page Table Entry (PTE) is an individual entry in the page table that stores information about a single page.
Each PTE typically contains:

• Physical Page Frame Address → The location of the page in RAM.
• Present Bit (P) → Whether the page is in RAM (1) or swapped to disk (0).
• Accessed Bit (A) → Whether the page has been accessed recently.
• Dirty Bit (D) → Whether the page has been modified (important for write-back caching).
• Read/Write (R/W) Bit → Indicates whether the page is read-only or writable.
• User/Supervisor (U/S) Bit → Determines if user-mode code can access the page.
• NX (No-Execute) Bit → Prevents execution of code from certain pages (used for security, e.g., preventing buffer overflow attacks).

Downsides of Artificial Page Faults

• Temporary slowdown-During the unmapping phase, the task suffers from extra page faults, hurting performance in the short term.
• Not ideal for short-lived tasks — if a process runs for a very short time, it mightn’t benefit from NUMA balancing
• Overhead — the constant monitoring & migration introduces some overhead

Why AutoNUMA creates artificial page faults

• Improve memory access speeds: Accessing memory from a local NUMA node is much faster than accessing from a remote node
• Balance load across NUMA nodes: Balance the utilization of NUMA nodes
• Detect access patterns: OS can track which threads access which pages

AutoNUMA Tiering

Optimizes memory placement across heterogeneous memory tiers

• Page access monitoring: marks pages using PROT_NONE
• Memory Tier decision: Hot pages stay in DRAM, Cold pages migrate to PMEM
• Automatic Page Migration: If a cold page becomes hot, it gets promoted back to DRAM & if a hot page becomes cold, it’s demoted to a slower memory

Benefits

Transparent memory boost: No need for application modification
Efficient use of expensive DRAM: Reduces memory pressure
Dynamic adaption: Adjusts automatically based on workload patterns