NUMA
A 16-core CPU can load a quantized local LLM, show activity across many cores, and still generate text much more slowly than expected. Adding threads may help at first.
Local AI, Local LLMs, GPU hardware, inference, RAG and AI server infrastructure explained clearly.