Bonsai 27B: A 27B-Parameter LLM That Fits on an iPhone

3.9GB. That’s the entire footprint of a 27-billion-parameter language model that now runs on an iPhone 17 Pro Max, fully offline, at 11 tokens a second. No API key, no network call, no server bill. The same weights that would normally need 54GB in standard 16-bit precision.

PrismML, a Caltech spinout backed by Khosla Ventures, shipped this as Bonsai 27B on July 14, 2026. It’s a compressed build of Alibaba’s open Qwen3.6 27B, and within a day it was sitting near the top of Hacker News with well over 500 points, mostly because the top comments were busy poking holes in the benchmark chart rather than just cheering the file size.

What actually shipped

Bonsai 27B is a multimodal model built on Qwen3.6 27B’s hybrid-attention architecture (roughly 75% linear attention layers, 25% full attention). The ~27.32B language weights get the 1-bit or ternary treatment; a 461M-parameter vision tower rides along in 4-bit so the model can still read screenshots and documents, not just text. It ships in two compressed forms:

• 1-bit build - 3.9GB, 1.125 effective bits per weight, tuned to fit inside the memory budget of an iPhone 17 Pro.
• Ternary build - 5.9GB, 1.71 effective bits per weight, aimed at laptop-class hardware where you can afford a bit more memory for better quality.

Both are on Hugging Face under the prism-ml org, in GGUF and MLX formats, licensed Apache 2.0. If you’ve got llama.cpp or mlx-lm set up already, this is a normal model pull, not a special SDK.

The trick: train at 1-bit, don’t quantize down to it

Most “1-bit” or “2-bit” LLMs you’ve seen are a full-precision model that gets quantized after training, with a handful of sensitive layers (usually the embeddings and the LM head) left at higher precision because they break too badly otherwise. PrismML didn’t do that here. Bonsai was trained natively at 1-bit from the start, with the low-precision constraint applied end-to-end across embeddings, attention, and the MLP layers. Nothing gets a high-precision escape hatch.

The ternary build uses what PrismML calls a “g128” format: weights are grouped in blocks of 128, and each group shares a single FP16 scale factor across three possible weight values (-1, 0, +1). That’s the real trick behind the density number - you’re not storing 128 individual scales, just one, so the per-weight overhead stays close to the theoretical 1.58 bits of a ternary value. Smaller groups would preserve more precision per group but need more scale factors; PrismML’s numbers land at 1.71 effective bits per weight as the tradeoff point.

That’s roughly a 9x reduction for the ternary build and close to 14x for the 1-bit build, off the same 27B parameter count. No pruning, no smaller model swapped in - the same architecture, just represented with far fewer bits per weight.

How fast it actually is

Numbers, not vibes: on an iPhone 17 Pro Max, the 1-bit build runs at about 11 tokens per second. On an Apple M5 Max, PrismML reports up to 87 tok/s for the 1-bit build and 58 tok/s for the heavier ternary build. Both builds carry a 262K-token context window, which is unusually long for anything you’d expect to run on a phone.

Part of that speed comes from DSpark, a speculative-decoding drafter layer trained specifically against Bonsai as its target model, alongside a standard multi-token-prediction (MTP) path. Speculative decoding lets a small, cheap draft model guess several tokens ahead and the big model verify them in one pass instead of generating token-by-token - and because it’s a lossless verification scheme, you get the speedup (PrismML reports 1.37x on the CUDA serving path) without changing the output distribution. It’s a good example of how “make the model smaller” and “make the model faster” are increasingly two different engineering problems solved with two different tricks stacked together.

The benchmark that gives away the tradeoff

Here’s where the marketing copy and the Hacker News thread start to diverge. PrismML’s own release notes break results out across 15 benchmarks in thinking mode: the ternary build averages 80.49 against an uncompressed baseline of 85.07 (about 95% retention), and the 1-bit build averages 76.11 (about 90% retention).

That’s a genuinely good result, but it’s an average, and averages hide the interesting part. Math and code hold up close to parity, while tool-calling and vision take the real hit - and the 1-bit build’s tool-calling score falls a lot further than the ternary build’s does. Math stays solidly in the 90s (93.4 ternary, 91.7 1-bit, versus a 95.3 baseline); coding trails a bit behind in the mid-to-high 80s (86.0 and 81.9, versus 88.7). Tool-calling and agentic tasks are the outlier: the ternary build drops to 74.0, and the 1-bit build falls further still, to 66.0, a much steeper cut off an 80.0 baseline. Zoom into a single agentic benchmark and it gets starker - on Tau2-Bench, a multi-step tool-use test, the 1-bit build scores 61.3 against a baseline of 82.9, a 26% relative drop. Quantizing a model this aggressively doesn’t cost you evenly across skills. Reasoning that stays inside the model’s own head (chain-of-thought math, writing code) degrades gracefully. Anything that depends on precisely structured, multi-step output - calling a tool with the right arguments in the right format, in the exact schema an agent harness expects - is more fragile, and it shows.

If you’re picturing Bonsai as the backbone for an on-device agent that calls tools, reads your calendar, and books things for you, that’s exactly the workload where the compression bites hardest. It’s a fine model for a phone-based chat assistant or a private coding helper. It is not yet a drop-in replacement for a full-precision model in an agentic pipeline - the steepest drops in PrismML’s own numbers are concentrated in exactly the multi-step, tool-calling benchmarks a real agent harness would lean on.

Try it without installing anything

You don’t need a phone or a beefy GPU to see this working. PrismML worked with the WebML community to build custom WebGPU kernels for Bonsai, and there’s a live demo running the 1-bit build entirely in-browser at the Hugging Face Space. WebGPU is the successor to WebGL - it gives the browser near-native access to the GPU’s compute shaders, which is what makes running a 3.9GB model’s matrix multiplications in a browser tab even remotely practical.

Expect 8 to 30 tokens per second depending on your GPU, and a one-time multi-gigabyte download that gets cached for later runs. Chrome and Edge support WebGPU well; Safari’s support is newer and more experimental, including on iOS.

Why Apple is suddenly in this story

The same day Bonsai 27B shipped, CNBC reported that Apple is evaluating PrismML’s compression technology, with PrismML CEO Babak Hassibi saying Apple and other companies have been benchmarking the startup’s models for speed, energy efficiency, and on-device performance. Hassibi described the talks as early - “things are progressing nicely” is about as much specificity as he gave - but it’s not hard to see why Apple would be interested. Apple has spent years pitching on-device, privacy-preserving AI as a differentiator, and a technique that claims up to 15x memory reduction with 90%+ benchmark retention is exactly the kind of thing that turns “on-device AI” from a marketing slide into something that actually fits in a phone’s RAM.

Worth separating fact from spin here: this is “evaluating,” not an acquisition or a signed deal. Companies benchmark outside vendors’ technology constantly without it going anywhere. But the timing (Apple’s own on-device models have been criticized for lagging the frontier) makes it a genuinely interesting thread to watch rather than routine startup PR.

What this means if you’re building something

The headline trick here is that Bonsai needs no server at all - it’s designed to run entirely on the device in your pocket. But most people prototyping with a 27B-class model day to day are still going to run it on a workstation or a home server first, where you have more RAM to spare and don’t have to fight a phone’s power budget. If you’re serving the ternary build from a Mac Studio or a Linux box with something like llama.cpp or Ollama and want to hit that API from your phone, or hand a teammate a URL without deploying anything, you don’t need to open a router port. One tunnel command does it:

ssh -p 443 -R0:localhost:8000 free.pinggy.io

That forwards whatever’s listening on port 8000 - your local model server - to a public HTTPS URL you can hit from anywhere, torn down the moment you close the terminal. It’s the same approach covered in our guide to exposing an Ollama server online, and it works identically for any OpenAI-compatible endpoint a Bonsai-based server exposes.

The bigger picture

Bonsai 27B is another data point in a trend that’s been building all year: the interesting frontier in local AI isn’t only “can we make a bigger model,” it’s “how much of a big model’s capability can we keep after we’ve made it small enough to fit where people actually want to run it.” PrismML’s answer - train at low precision from scratch instead of quantizing afterward - is a genuinely different approach from KV-cache compression tricks or mixture-of-experts sparsity, and it’s aggressive enough to draw attention from Apple.

The honest version of the pitch isn’t “a 27B model that’s exactly as good as the original, now on your phone.” It’s “a 27B model that’s still surprisingly good at reasoning and code, meaningfully worse at precise tool use, and small enough that none of that matters if all you needed was a private assistant that never leaves your device.” For a lot of use cases, that’s already enough.