PrismML Open-Sources Bonsai 27B: A 27-Billion-Parameter Model That Fits on a Phone at Just 3.9GB
A 27-billion-parameter large language model now genuinely runs on a phone — not a demo video, but an open-source model you can download and use offline.
Three Key Takeaways
3.9GB on a phone. Released on July 14, PrismML's Bonsai 27B applies 1-bit binary quantization to Qwen3.6 27B, compressing it to 3.9GB. On an iPhone 17 Pro it runs at roughly 11 tokens per second — smooth enough for real-time streaming output. A higher-fidelity 1.58-bit ternary variant weighs 5.9GB and hits 87 tokens/s on an M5 Max laptop.
90%+ benchmark retention. The 1-bit version retains about 90% of the full-precision model's performance across 15 benchmarks; the ternary version keeps over 95%. The model supports a 262K-token context window and is multimodal — a 4-bit vision tower ships alongside it, handling screenshots, documents, and camera input.
Fully open under Apache 2.0. Both variants are released under Apache 2.0 with support for tool calling and speculative decoding. According to TechTimes, Apple is evaluating the technology for potential integration into its on-device AI capabilities.
WangDou's Take
For the past two years, "on-device LLM" has mostly been a marketing concept — either the parameter count was too small to be useful, or you needed hardware heavier than the phone itself. PrismML used 1-bit quantization to cram 27B parameters into 3.9GB, and the real story isn't "AI runs on a phone now" — it's that extreme compression plus 90% performance retention might be more practical than endlessly scaling cloud models. Eleven tokens per second isn't fast, but it's fast enough to use a model that actually reasons on an airplane, on a subway, anywhere without a network connection. Cloud AI isn't going anywhere, but "you must be online to think" is no longer the only option starting today.