Servers
Why Memory Bandwidth Matters More Than CPU Core Count for Local LLMs.
1225
A 16-core CPU can load a quantized local LLM, show activity across many cores, and still generate text much more slowly than expected. Adding threads may help at first.
Local LLMs
Which GGUF Quantization Should You Use? Q4 vs Q5 vs Q8 Explained
0347
Q4_K_M, Q5_K_M and Q8_0 can run the same model with very different memory requirements. Here is how to choose without wasting VRAM.
Hardware
Context Windows and VRAM: Why Longer Conversations Need More Memory
2262
A local LLM can fit comfortably into your GPU at an 8,000-token context and run out of memory when you increase that context several times over.