LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.
Low-latency NUMA-aware fork-join thread-pool with zero allocations, syscalls, CAS, or false-sharing on the hot path for C, C++, Rust, and Zig 🍴
LLM infrastructure cost reduction via NUMA-aware weight banking: 147 t/s (8.8x stock llama.cpp) on refurbished enterprise POWER8. Self-hosted inference, no cloud APIs. Part of the Proof of Physical AI stack.
Python Multi-Process Execution Pool: concurrent asynchronous execution pool with custom resource constraints (memory, timeouts, affinity, CPU cores and caching), load balancing and profiling capabilities of the external apps on NUMA architecture
A community-oriented list of useful NUMA-related libraries, tools, and other resources
Multi-core Window-Based Stream Processing Engine
NUMAPROF is a NUMA memory profliler based on Pintool to track your remote memory accesses.
Rust bindings to Open MPI Portable Hardware Locality "hwloc" library, covering version 2.0 and above.
Data Plane Development Kit (DPDK) integration into OpenWrt
Hardware-accelerated Linux kernel block driver and multi-tier memory cascade (ZRAM ➔ PCIe VRAM ➔ NVMe). Opportunistically leases idle GPU memory as a zero-allocation, revocable swap cache with sub-millisecond latency, zero SSD wear, and crash-safety on Linux & WSL2.
Local-affinity first NUMA-aware allocator with optional fallback.
NUMA-aware multi-CPU multi-GPU data transfer benchmarks
Non-unix, custom-API hybrid OS kernel written in C++ which can be thought of as an emulated microkernel. The native API is almost fully asynchronous and the kernel is aimed at high-scaling, high-throughput-requiring multiprocessor workloads, with working support for SMP and NUMA already implemented. Join the IRC channel, #zbz-dev on freenode!
cgroups-based cpuset isolator and resource estimator modules for mesos
To associate your repository with the numa topic, visit your repo's landing page and select "manage topics."