AMD and Cerebras Split the Inference Pipeline: Prefill on GPUs, Decode on Wafer-Scale Chips
AMD and Cerebras Systems announced on July 23 a "disaggregated inference" architecture that splits large-model inference into its two natural stages and assigns each to the hardware best suited for it, claiming up to 5x improvement in throughput per watt.
Three Key Takeaways
Prefill goes to AMD, decode goes to Cerebras. In LLM inference, the prefill stage — processing long user-input contexts — is compute-intensive and suits GPU parallelism. The decode stage — generating tokens one by one — is memory-bandwidth-intensive, which is exactly what the Cerebras Wafer-Scale Engine (WSE) excels at. The joint system routes prefill to AMD Helios racks running Instinct GPUs and decode to the WSE, with the two connected over the network.
The 5x claim comes with caveats. According to joint modeling by AMD Performance Labs and Cerebras in July 2026, the Helios + WSE combination delivers roughly 5x more tokens per second per kilowatt than a WSE-only setup, measured on the Kimi 2.6 1T model. Crucially, the baseline is Cerebras-only — not Nvidia GPUs. The claim proves that splitting is better than not splitting; it does not claim superiority over Nvidia's inference stack.
Launching on Cerebras Cloud in H2 2026. The joint solution is expected to go live on Cerebras Cloud in the second half of 2026, targeting coding copilots, real-time agents, robotics control, and scientific workloads. AMD EPYC processors supply the CPU layer, Instinct GPUs handle prefill, and the WSE runs decode — three chip types, each doing what it does best.
WangDou's Take
The most interesting thing about this partnership is not the technology — it is the anxiety it reveals. AMD's GPUs have been getting crushed by Nvidia in the inference market, and Cerebras' wafer-scale chip is fast at decode but weak at prefill. Two players with complementary blind spots decided to pool their bets. The logic is sound: LLM inference really is two completely different workloads, and cramming both into the same chip wastes silicon either way. But that 5x number needs the fine print: it is a modeling estimate, self-vs-self comparison, run on Moonshot AI's model. The real exam is a head-to-head price-performance shootout against Nvidia's H200/B200, and they have not turned in that scorecard yet. Still, the inference market is enormous — big enough for more than one architecture. If this setup genuinely drives cost per watt low enough, cloud providers will give it a spin.
Source: AMD Newsroom, Tom's Hardware, TECHi