Cerebras Wafer vs GPU for LLM Inference: A GPT-OSS Benchmark

Does a Cerebras wafer actually beat a GPU for LLM inference? The answer depends on the kind of request, so the test below goes in in two stages. First comes a hands-on benchmark with GPT-OSS, OpenAI's open-weight reasoning model, served from Cerebras Cloud on one side and from Fireworks AI's ordinary GPU infrastructure on the other. Second, the same hardware shows up behind one of OpenAI's own commercial frontier models, GPT-5.6 Sol, and the question becomes how much faster it gets there.

Three terms to get straight first

• Inference means running an already-trained LLM to produce an answer for a given input.
• Nearly all of that work today happens on GPUs, chips packed with cores that handle thousands of calculations in parallel.
• Cerebras goes in a different direction altogether. Its Wafer-Scale Engine (WSE) puts an entire silicon wafer to work as one processor.

The model under test

GPT-OSS comes in 20B and 120B sizes and is built for reasoning and tool use, with deployment possible outside OpenAI's infrastructure. The 120B version suits this experiment. It's large enough that the hardware underneath really matters, yet several providers host it, so the same weights can be compared across vendors.

The exact model ID is openai/gpt-oss-120b. Its specs:

• 36 transformer layers using a Mixture-of-Experts design
• 116.8 billion parameters in total, with roughly 5.1 billion active for each token
• 128 experts, four of them active per token
• a 128K-token context window
• MoE weights quantized to MXFP4

OpenAI sized it to fit on one 80 GB GPU, which keeps the GPU side of the comparison simple and fair against the WSE.

Why one prompt isn't enough

Sending a single prompt would prove very little. Different inference jobs burn their time in different places. A short reply is mostly startup latency. A long reply reveals how fast the system can keep generating tokens. A huge prompt loads the input-processing stage before a single output token appears.

So the benchmark covers five workload types. Each runs 10 times on the WSE and 10 times on the GPU setup, and each run logs how long the first token took, when the first real answer text arrived, total latency, throughput, tokens consumed and the bill.

The whole thing lives in a Google Colab notebook that includes the five workload tiers, paired repeated runs, token and latency tracking, cost math and result aggregation. Running it takes accounts on both Cerebras Cloud and Fireworks AI with enough credits, and the API keys go into Colab Secrets.

What the numbers showed

Across the 10 paired runs per workload, Cerebras pulled furthest ahead when the model had a lot of reasoning or generation to do. The more tokens it had to emit, whether as thinking or as the final answer, the bigger the end-to-end speedup. A large input by itself didn't produce the same effect.

Cost moved the other way. Each WSE request was pricier. For long generations, though, the trade looked good: median cost rose about 26% while end-to-end latency fell about 87%.

The same idea on a commercial model

Cerebras isn't confined to open-weight models. OpenAI itself runs GPT-5.6 Sol on Cerebras hardware through an Ultrafast service tier. The model stays gpt-5.6-sol, and only the way the request is served changes. OpenAI claims Ultrafast can reach up to 750 output tokens per second, which is up to 14x faster than Standard processing. It's a serving tier and not a separate GPT-5.6 model.

API users with Ultrafast access just set a request option to take the WSE-backed route. At the time of writing, ChatGPT doesn't offer this mode. Don't confuse it with the "Ultra" reasoning-effort setting in ChatGPT Work, which is a different thing.

In the snippet below, the ordinary gpt-5.6-sol model is chosen, and service_tier="ultrafast" asks OpenAI to handle that call through the Ultrafast tier.

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-5.6-sol",
    service_tier="ultrafast",
    input="Explain why inference latency matters for multi-step AI agents."
)

print(response.output_text)

Training, too

Faster answers aren't the only thing the hardware offers. Cerebras also rents out wafer-scale capacity for training and fine-tuning. Its Training Cloud covers jobs from fine-tuning an existing model to pretraining a new one, across dense, mixture-of-experts, multimodal, reasoning and custom architectures. The selling point is simplicity: the same training code is said to scale up to very large models without hand-sharding the model or building model-parallel infrastructure yourself.