Local llm

Which open-weight LLMs actually fit in 8GB of VRAM or RAM in 2026, with measured file sizes, KV cache math from published configs, and Ollama commands for Qwen3.5, Gemma 4, Ministral 3, Granite 4.1, Nemotron 3 Nano, and Phi-4-mini.

A buying guide for running coding agents on your own hardware. Why prompt caching makes generation speed the number that matters, where a cold cache costs you minutes instead, a comparison table of Mac Studio M5 Ultra, RTX 5090, Radeon AI PRO R9700, Strix Halo and DGX Spark with September 2026 prices, and the memory every open-weight coding model actually needs.

Apple's M5 Ultra Mac Studio tops out at 512GB of unified memory at 1.2TB/s. Here is the memory math, which large open-weight LLMs actually fit, measured tokens per second, and three ways to run a model bigger than 4-bit allows: Unsloth GGUF quants, streaming weights off SSD, or a Thunderbolt 5 cluster.

oMLX turns a Mac into an OpenAI-compatible inference server with continuous batching and SSD-backed KV caching. Here's how it works, its new multi-Mac clustering, and how to reach it from your phone or a coding agent with Pinggy.

Fine-tune LLMs locally with Unsloth: QLoRA on a single GPU, Unsloth Studio, Dynamic GGUF quants, and sharing your model over a Pinggy tunnel.

Bonsai 27B compresses a 27B-parameter Qwen3.6 model to 3.9GB using native 1-bit weights and runs on an iPhone at 11 tok/s. Here's what it gives up.

A single-file C engine called Colibrì streams GLM-5.2's 744B mixture-of-experts weights off an NVMe drive to run the full model on 25 GB of RAM at 0.05-2 tokens/second. Here's how it works, what Hacker News made of it, and how to check on a queued run from your phone with Pinggy.

whichllm auto-detects your GPU, CPU, and RAM, then ranks local models by real benchmarks rather than parameter count. Here's how to use it, run your pick with Ollama, and share it remotely via Pinggy.

A practical hardware guide for self-hosting LLMs in 2026. Compare consumer GPUs, Apple Silicon, enterprise cards, and pre-built AI workstations. Find the right setup for 7B to 405B models at every budget.

Discover the best open source LLMs for coding and development that you can self-host. Compare Kimi K3, Qwen3.8-Max, GLM-5.3, GLM-5.3-Flash, DeepSeek-V4-Pro, MiniMax M3, Qwen3.8-27B, Muse Glimmer 30B, Nemotron 3.5 Lightning and more with benchmarks, hardware requirements, and deployment guides.