Cerebras Inference Review (2026): Features, Pricing & Verdict
Cerebras Inference is the fastest AI inference service in the world, and it achieves that title through the most audacious hardware bet in the industry: instead of cutting a silicon wafer into dozens of chips, Cerebras keeps the entire wafer as a single chip. That chip — the Wafer-Scale Engine 3 (WSE-3) — is the largest ever built, a 300mm wafer packing 4 trillion transistors, 900,000 AI-optimised cores and 44 GB of on-chip SRAM delivering 21 petabytes per second of internal memory bandwidth (roughly 7,000x an NVIDIA H100). The reason this matters is that LLM inference is fundamentally bottlenecked by memory: to generate each token, a GPU must read the entire model out of external HBM into its compute cores, and that shuttling — the “memory wall” — is what makes GPU token generation slow. Cerebras dissolves the bottleneck by keeping the whole model in SRAM right next to the compute, and the results are staggering. It runs Llama 3.3 70B at over 2,100 tokens per second, GPT-OSS 120B at around 3,000, and even Llama 3.1 405B at 969 tokens per second — versus the 10–80 tokens per second typical of GPU clouds, commonly 10–20x faster and, on the largest models, up to 75x. Crucially, unlike rivals that trade accuracy for speed through aggressive quantisation, Cerebras stays in the 16-bit domain for the entire run, so it’s fast and full-quality. It’s delivered through a clean OpenAI-compatible API at competitive per-token prices, with a free tier for developers. And 2026 turned Cerebras from a niche marvel into cloud infrastructure: it struck a partnership to bring wafer-scale inference to Amazon Bedrock and began powering OpenAI’s Codex-Spark coding model. The trade-offs are architectural — the model has to fit in on-chip SRAM — plus a curated catalogue and some structural business risk, all covered below.
- Best for
- Throughput-bound and latency-critical workloads on open-source models — real-time chat and voice, coding assistants, long-form generation, reasoning chains and multi-step agent loops that produce many tokens — where raw tokens-per-second is the primary constraint and full-precision quality matters
- Platform
- Inference API on the Cerebras CS-3 / WSE-3 wafer-scale chip — open-source LLMs (Llama, GPT-OSS, Qwen, GLM, DeepSeek) served at thousands of tokens/second in full 16-bit precision, via an OpenAI-compatible API; free developer tier, pay-as-you-go and enterprise dedicated endpoints
- Key differentiator
- Wafer-scale silicon that holds the entire model in on-chip SRAM, eliminating the GPU memory bottleneck to deliver the world’s highest inference throughput — while uniquely maintaining full 16-bit accuracy rather than quantising for speed
- Pricing
- Free developer tier (waitlist-gated, no card). Pay-as-you-go per-token, roughly $0.10–$6/M depending on model, flat-rated across prompt and completion; reserved capacity from ~$0.25–$0.60/M with volume commitments. Enterprise custom. Broadly competitive with GPU clusters
- Vendor
- Cerebras Systems (US, founded 2016 by Andrew Feldman) — builds the world’s largest chip. 2026 partnerships with AWS (Bedrock) and OpenAI (powering Codex-Spark); S-1 filed for IPO. Early users include DeepLearning.AI and GSK
What Is Cerebras Inference?
Cerebras Inference is an AI inference service that runs large language models on Cerebras Systems’ proprietary wafer-scale hardware, offered to developers as a simple, pay-per-token API. To understand it, you have to understand the chip, because the chip is the entire value proposition. Cerebras, founded in 2016 by Andrew Feldman, set out to solve the slowdown of Moore’s law and the hard physical limit that a single conventional chip can be no larger than one reticle (about 858mm²) by doing something the industry considered impractical: making the whole silicon wafer into one chip. The result is the Wafer-Scale Engine 3 (WSE-3), the largest chip ever manufactured — an entire 300mm wafer, roughly 46,000mm² of silicon, containing 4 trillion transistors, 900,000 AI-optimised cores and 44 GB of on-chip SRAM, delivering 125 petaflops of compute. For comparison, that’s around 50x the transistors of an NVIDIA H100. The engineering feat that made wafer-scale economical was defect tolerance: because a single flaw on a wafer-sized device could disable a huge region, Cerebras designed for defects rather than against them, with redundant cores, redundant routing and a fail-in-place architecture that isolates flaws and routes around them. Why does this produce fast inference? Because LLM decoding is memory-bandwidth-bound: generating each token requires reading the model’s weights, and on a GPU those weights live in external HBM, so every token demands a slow round-trip between memory and compute — the memory wall. Cerebras keeps the entire model in on-chip SRAM, immediately adjacent to the 900,000 cores, so weights never leave the chip and the bottleneck largely vanishes. The service exposes this through an OpenAI-compatible Chat Completions API — migrating an existing OpenAI app is usually just a base-URL and key change — serving a curated set of popular open-source models. For models larger than a single wafer’s SRAM (like Llama 3.1 405B), Cerebras uses a disaggregated architecture that separates prompt processing (prefill) from token generation (decode), and can cluster wafers together. Within this site’s Machine Learning & MLOps category, Cerebras Inference sits in the model-inference tier as a custom-silicon speed specialist — the wafer-scale counterpart to Groq’s LPU, and a fundamentally different approach from the GPU-based hosts (Together AI, Fireworks, Baseten) it dramatically outruns on throughput.
Core Features
Wafer-scale speed: the world’s fastest inference
Cerebras Inference’s defining and singular strength is throughput, and by this measure it is, quite simply, the fastest inference available. The benchmark figures are consistently extraordinary. On Llama 3.3 70B, Cerebras sustains over 2,100 tokens per second (some measurements as high as 2,314); on GPT-OSS 120B it reaches roughly 2,700–3,000 tokens per second — against about 900 on NVIDIA’s Blackwell B200; and most strikingly, on the enormous Llama 3.1 405B it delivers 969 tokens per second where typical GPU clusters manage just 10–15, a roughly 75x advantage. Across supported models it runs commonly 10–20x faster than GPU-based solutions, and on some exotic workloads the company has measured over 1,000x. To put the human meaning on these numbers: at typical GPU cloud speeds of around 50 tokens per second, a 500-word response takes six to seven seconds to stream and the user feels the delay; at 2,100 tokens per second, that same response completes in under 0.2 seconds — before the reader finishes the first line. It is, as one analysis put it, the difference between an AI product that feels like autocomplete and one that feels like instantaneous thought. Two technical points make this more than a raw-speed stunt. First, the source of the advantage is architectural and hard to replicate: the WSE-3’s 21 petabytes/second of on-chip memory bandwidth (roughly 7,000x an H100) directly attacks the memory-bandwidth wall that throttles GPU inference, and because the whole model computation stays on a single device, there’s no multi-GPU cross-chip communication overhead — end-to-end latency drops to around 6 milliseconds per token, below the 10–12ms floor GPU clusters struggle beneath at the same throughput. Second, and importantly, Cerebras achieves this without sacrificing quality: unlike approaches that quantise models to lower precision to gain speed, Cerebras stays in the 16-bit domain for the entire inference run, so you get state-of-the-art accuracy at record speed rather than a fast-but-degraded output. This matters enormously for the workloads where speed is most valuable — agentic pipelines that call an LLM repeatedly, real-time coding assistants, reasoning chains, and any interactive experience — which is exactly why figures like Andrew Ng have highlighted Cerebras’ speed as transformative for multi-step agentic workflows. For throughput-bound applications on supported models, this wafer-scale speed leadership is unmatched.
The API, model catalogue and developer experience
Cerebras wraps its exotic hardware in a deliberately conventional, developer-friendly interface, which is essential to making wafer-scale accessible rather than a bespoke supercomputer engagement. The centrepiece is a fully OpenAI-compatible API: Cerebras Inference implements the OpenAI Chat Completions interface, so a developer already using the OpenAI Python SDK can switch to Cerebras by changing the base URL and API key, with no other code changes required — the same drop-in migration story that has become table stakes among modern inference providers, and one that makes trialling Cerebras genuinely low-friction. The model catalogue is curated rather than exhaustive, focused on popular, high-value open-source models that benefit most from the speed: the published line-up includes GPT-OSS 120B (~3,000 tokens/s), Llama 3.1 8B (~2,200 tokens/s), Llama 3.3 70B (~2,100 tokens/s), Qwen 3 235B Instruct (~1,400 tokens/s), Z.ai GLM 4.7 (~1,000 tokens/s), and DeepSeek R1-70B, with additional model families available to enterprise customers through dedicated endpoints. This is a smaller selection than GPU-based marketplaces offer, and it reflects both the platform’s focus and the architectural reality that models must fit the wafer’s memory profile — a genuine trade-off discussed in the weaknesses. Access starts with a free developer tier that provides all supported models with daily rate limits and requires no credit card, which is excellent for prototyping — though, notably, access currently requires waitlist approval (community reports suggest around a week’s wait) even for the free tier, adding a step of friction that instant-signup competitors like Groq don’t impose. Beyond the free tier, a pay-as-you-go plan handles production token billing, and enterprise agreements provide dedicated throughput, SLA guarantees, volume discounts and access to a broader model set. One pricing nuance that shapes the developer experience: Cerebras charges a flat per-token rate that doesn’t differentiate between prompt (input) and completion (output) tokens, unlike OpenAI or Anthropic — which makes cost more predictable and is advantageous for completion-heavy workloads like summarisation and code generation, where competitors’ higher output-token pricing would otherwise bite. The overall developer story, then, is strong: a familiar API, a focused catalogue of the models most teams want served fast, and a free tier to start — with the main friction being the waitlist and the deliberately narrower model selection.
From niche marvel to cloud infrastructure: AWS, OpenAI and scale
The most consequential development in Cerebras’ 2026 story is that wafer-scale inference stopped being an interesting experiment and became mainstream cloud infrastructure, validated by two of the most significant partnerships an inference company could land. The first is with Amazon Web Services: in March 2026 Cerebras announced a multi-year partnership to bring its fast inference to Amazon Bedrock, giving startups, AI-native companies and enterprises easy, globally distributed access to Cerebras throughput through AWS’s platform — a powerful signal that hyperscale clouds now view wafer-scale as production infrastructure rather than a niche curiosity, and a major distribution channel that removes adoption friction for the enormous population of teams already on AWS. The second, and arguably more validating, is with OpenAI: Cerebras signed an agreement with OpenAI in late December 2025, began delivering capacity in January 2026, and by February 2026 OpenAI’s Codex-Spark coding model was running on Cerebras infrastructure. That a frontier lab chose Cerebras to power a latency-sensitive coding product is strong external confirmation that the speed is real and production-grade at scale — coding assistants live or die on responsiveness, and Codex-Spark’s placement on Cerebras is a direct endorsement. These deals also reflect a broader industry thesis that fast inference is becoming the highest-value tier of AI compute: NVIDIA’s own leadership has publicly argued that the most valuable inference tier warrants dramatically higher performance, effectively validating the total market Cerebras is competing for. Strategically, Cerebras is positioning itself as a “neocloud” — one of the first semiconductor companies to run its own cloud business — and is scaling capacity aggressively to serve commitments like OpenAI’s, with plans to secure hundreds of megawatts of power over the coming years. For a buyer, the practical implication is reassuring: rather than a fragile startup, Cerebras in 2026 is an inference provider embedded in AWS’s marketplace and trusted by OpenAI, which materially de-risks building on it. The honest counterweight — a heavy reliance on a small number of very large customers and a pending IPO — is a real structural consideration addressed in the verdict, but the trajectory from research novelty to cloud infrastructure is unmistakable and strengthens the case for taking Cerebras seriously as a production inference option.
Scored Categories
Pricing
| Tier | Price | Notes |
|---|---|---|
| Free (Developer) | $0 (no card) | Access to all supported models with daily rate limits — excellent for prototyping. Requires waitlist approval (community reports ~1 week) even for the free tier |
| Pay-as-you-go | ~$0.10–$6/M tokens | Per-token production billing, varying by model (e.g. Llama 3.1 8B ~$0.10/M, Llama 3.3 70B ~$0.60/M, larger models more). Flat-rated across prompt and completion tokens — predictable, favourable for output-heavy work |
| Reserved capacity | ~$0.25–$0.60/M | Discounted rates for high-volume/batch with a minimum commitment (around 10M tokens/month). Roughly 40–70% cheaper than GPU self-hosting for supported workloads |
| Enterprise | Custom (contact sales) | Dedicated throughput, SLA guarantees, volume discounts, and additional model families via dedicated endpoints. Also accessible through Amazon Bedrock |
Strengths
- The fastest inference in the world — thousands of tokens/second, 10–20x (up to 75x) faster than GPUs
- Wafer-scale WSE-3 — the largest chip ever built (4T transistors, 44GB on-chip SRAM, 21 PB/s bandwidth)
- Eliminates the GPU memory-bandwidth wall by keeping the whole model in on-chip SRAM
- Maintains full 16-bit accuracy — fast and full quality, no quantisation compromise
- Transformative for agentic and real-time workloads — reasoning chains, coding assistants, voice
- Flat per-token pricing (prompt = completion) — predictable and favourable for output-heavy work
- Competitive cost-per-token with GPU clusters; reserved rates ~40–70% cheaper than self-hosting
- OpenAI-compatible API — drop-in migration by changing a base URL
- Free developer tier for prototyping
- 2026 validation — AWS Bedrock partnership and powering OpenAI’s Codex-Spark
Weaknesses
- Model-size ceiling — models must fit in on-chip SRAM; very large (400B+) models need clustering/disaggregation
- Curated, narrower catalogue than GPU hosts; some current frontier open models cheaper elsewhere
- Free tier and access are waitlist-gated (~1 week) — friction versus instant-signup rivals
- No self-serve fine-tuning or custom-model hosting on the public API
- Long-context/large KV-cache workloads strain the SRAM architecture
- Pay-as-you-go per-token rates historically less transparently published
- Structural business risk — heavy customer concentration and a pending IPO
- Open-source models only — no proprietary GPT, Claude or Gemini
Verdict: 8.3 / 10 — The World’s Fastest Inference
Cerebras Inference earns a strong 8.3 as the fastest inference platform on the planet and one of the most genuinely differentiated technologies in the entire category. Its advantage is real, physical and extremely hard to replicate: by turning an entire silicon wafer into a single chip with 44 GB of on-chip SRAM and 21 petabytes per second of memory bandwidth, Cerebras dissolves the memory-bandwidth wall that throttles GPU inference and delivers throughput measured in thousands of tokens per second — 10–20x faster than GPUs on supported models and up to 75x on the very largest — while, crucially, preserving full 16-bit accuracy rather than trading quality for speed. For the workloads where speed is most valuable — real-time coding assistants, agentic pipelines that call models repeatedly, reasoning chains, interactive chat and voice — this is transformative, and 2026’s validation removed most doubt about whether it works at scale: partnerships bringing Cerebras to Amazon Bedrock and powering OpenAI’s Codex-Spark took it from research marvel to production cloud infrastructure. It’s all wrapped in a familiar OpenAI-compatible API at pricing that competes with GPU clusters, which makes the price-performance genuinely compelling. Two sets of considerations keep it alongside Groq at the top of the speed tier rather than above the full-lifecycle platforms. The first is architectural: because the model must fit in on-chip SRAM, Cerebras has a model-size ceiling — very large frontier models require clustering or disaggregated inference, long-context KV-caches strain the memory profile, and the catalogue is curated rather than comprehensive, so it’s a specialist for supported models rather than an everything-platform. It also offers no self-serve fine-tuning, and access is gated behind a waitlist. The second is structural: Cerebras carries meaningful business risk in the form of heavy customer concentration and a pending IPO, which is worth weighing for a long-term dependency even as the AWS and OpenAI relationships de-risk it considerably. The clean verdict: if your workload is throughput-bound or latency-critical, runs on the open models Cerebras supports, and benefits from full-precision quality — and especially if it’s agentic or real-time — Cerebras Inference is very likely the fastest option you can buy, and now a credibly production-grade one thanks to AWS and OpenAI. Reach for it when speed at full accuracy is the priority; pair it with a broader GPU-based host for models it doesn’t serve or for fine-tuning; and keep your OpenAI-compatible code portable given the platform’s youth and concentration risk.
Frequently Asked Questions
How does wafer-scale hardware make Cerebras so much faster than GPUs?
The speed advantage comes down to solving the single biggest bottleneck in LLM inference — memory bandwidth — through a radically different chip design. Here’s the underlying problem. When a language model generates text, it produces one token at a time, and to produce each token it must multiply the current state by the model’s weights, which means reading the entire set of model weights from memory. On a GPU, those weights are stored in external high-bandwidth memory (HBM) sitting beside the chip, so every single token requires shuttling gigabytes of weights from HBM into the compute cores. Because the actual math per byte moved is small, this makes inference “memory-bandwidth bound” — the GPU spends most of its time waiting for data to travel, not computing — and it’s why GPU token generation tops out at tens of tokens per second on large models. Cerebras attacks this directly with its Wafer-Scale Engine. Instead of using external HBM, the WSE-3 places 44 GB of SRAM directly on the chip, immediately next to its 900,000 compute cores, providing an astonishing 21 petabytes per second of memory bandwidth — roughly 7,000 times an H100’s. Because the model’s weights live in that on-chip SRAM rather than external memory, they don’t have to make the slow round-trip for each token; the memory wall largely disappears, and token generation accelerates by an order of magnitude or more. There’s a second benefit from the wafer being a single giant chip: a model that fits on one wafer runs entirely on one device, so there’s no inter-chip communication overhead. On GPU clusters, large models are split across many cards that must constantly synchronise over network links, and that communication overhead kills throughput; Cerebras eliminates it for on-wafer models, which is why it achieves not just high throughput but very low per-token latency (around 6 milliseconds). Making a wafer-sized chip work at all required solving manufacturing defects — a single flaw could ruin a huge area — which Cerebras did with redundant cores and routing plus a fail-in-place design that isolates and routes around defects. The net result is that Cerebras runs Llama-class models at thousands of tokens per second versus tens on GPUs. The catch, which follows directly from the architecture, is that on-chip SRAM has limited capacity (44 GB), so the fastest path is for the whole model to fit on the wafer; models larger than that require splitting across clustered wafers or a disaggregated prefill/decode approach, which is why very large models are more constrained on Cerebras than on GPU clusters that can pool huge external memory across many cards.
How does Cerebras Inference compare to Groq?
Cerebras and Groq are the two leading custom-silicon inference companies, and they’re natural rivals because both reject the GPU approach in favour of purpose-built chips that keep model weights in fast on-chip SRAM to eliminate the memory bottleneck. They arrive at similar goals — dramatically faster inference than GPUs — through related but distinct architectures, and the practical differences are worth understanding. On raw throughput, Cerebras generally posts the higher numbers: its wafer-scale WSE-3, with 44 GB of on-chip SRAM on a single enormous device, tends to deliver the highest sustained tokens-per-second figures in the industry (thousands of tokens per second across supported models, and it can hold larger models like a 70B on effectively one wafer). Groq’s LPU takes a different form factor — many smaller chips networked together — and while it’s also far faster than GPUs, serving a large model on Groq requires orchestrating many LPUs. A commonly drawn distinction is that Cerebras is optimised for maximum sustained throughput (making it excellent for throughput-bound work: batch generation, long-form output, reasoning chains, agent loops that emit many tokens), while Groq is often highlighted for its extremely low and deterministic first-token latency (making it a strong fit where time-to-first-token is the critical metric, like conversational voice). In practice both are exceptional at real-time work and the gap is workload-dependent, so if latency is your priority it’s worth benchmarking the two directly on your specific model and traffic. A notable quality point in Cerebras’ favour is that it emphasises staying in full 16-bit precision for the entire inference run, marketing accuracy preservation as a differentiator. Beyond the chips, their 2026 situations differ: Cerebras has leaned into becoming a cloud provider with major AWS (Bedrock) and OpenAI partnerships and a pending IPO, whereas Groq’s most significant recent event was NVIDIA licensing its LPU architecture in a large deal that moved much of Groq’s engineering talent to NVIDIA, creating some uncertainty about the standalone platform’s long-term trajectory. Both share the same fundamental trade-offs versus GPU hosts: curated (not exhaustive) model catalogues, open-source models only, no self-serve fine-tuning, and architectures where very large models and long contexts are more constrained than on GPU clusters. The bottom line: choose either when you need speed far beyond what GPUs offer on supported open models; lean Cerebras for maximum throughput and full-precision quality on larger supported models, and lean Groq where first-token latency and instant self-serve access matter most — and, given both are specialists, keep a GPU-based host in reserve for models and capabilities they don’t cover.
What are the main limitations of Cerebras Inference?
Despite being the fastest inference available, Cerebras has real limitations that flow from its architecture and its stage as a company, and they define where it’s the right tool versus the wrong one. The most fundamental is the model-size ceiling. Because Cerebras’ speed depends on keeping the entire model in the wafer’s on-chip SRAM (44 GB), models that fit comfortably run blisteringly fast, but very large models push against that capacity — Llama 3.1 405B, for instance, requires a disaggregated architecture that separates prefill from decode, and running the largest frontier models needs clustering many wafers together, which is more costly and complex. GPU clusters, by contrast, can pool enormous external memory across many cards and scale to trillion-parameter models via pipeline parallelism, so for the very biggest models the GPU approach is more flexible. A related architectural constraint is long context: agentic, multi-turn workloads generate large KV-caches that can exceed on-chip SRAM, so long-context use cases strain the design more than they do memory-rich GPU systems. Second, the model catalogue is curated rather than exhaustive — a focused set of popular open models — so if you need a specific niche model, one of the newest frontier open releases, or the breadth of a marketplace, GPU-based providers like Together, Fireworks or DeepInfra offer wider selection (and sometimes lower per-token cost on those specific models). Third, Cerebras only serves open-source models — no proprietary GPT, Claude or Gemini — and offers no self-serve fine-tuning or custom-model hosting on its public API, so teams needing custom fine-tuned models must look elsewhere or negotiate an enterprise arrangement. Fourth, there’s access friction: even the free developer tier is gated behind a waitlist with a roughly one-week wait, unlike instant-signup competitors, and historically the detailed pay-as-you-go pricing has been less transparently published, which complicates upfront cost modelling. Finally, there’s a structural business consideration rather than a product flaw: Cerebras has had heavy revenue concentration among a small number of very large customers and has an IPO pending, which introduces some vendor-stability risk to weigh for a long-term dependency — though its 2026 AWS and OpenAI partnerships meaningfully diversify and de-risk that picture. None of these undercut the core proposition for its target use case — throughput-bound, latency-critical inference on supported open models at full precision — but they mean Cerebras is best deployed as a specialist for the workloads it excels at, complemented by a broader GPU-based host for model breadth, fine-tuning, the largest models, and long-context needs.