LLM Scaling Laws, GPT-4o and Reinforcement Learning: What the Curves Predict

Almost every big jump in language model quality over the last several years has come from doing the same thing at a larger size: more parameters, more training tokens, more GPUs. Scaling laws are the equations that turn that habit into something you can plan around, because they predict how a model's test loss will fall as you raise a resource such as training compute. They are also widely misread, which is why headlines alternate between "scaling is guaranteed" and "scaling has hit a wall." After working through this piece you will be able to read a log-log scaling plot correctly, explain what the Kaplan and Chinchilla results actually say, reason about the "slowdown" debate without taking sides on vibes, and see why agents and reasoning models are the next places where compute is being scaled.

The structure is bottom-up. We start with the math of a power law, move to the research that established scaling laws for language models, follow how those laws guided the GPT family and later models, look at the arguments that scaling is running out of steam, and finish with the directions that keep progress moving even if pretraining gains flatten.

The building block: power laws

Two quantities, one predictable relationship

A power law is simply a rule linking two measurements so that a proportional change in one produces a proportional change in the other. In the LLM setting, one of the two quantities is almost always test loss (sometimes a proxy such as accuracy on a downstream task), and the other is the knob being turned: parameter count, dataset size, or total training compute.

A typical claim from the GPT-3 paper reads roughly like this: provided there is enough data, validation loss should follow an approximately smooth power law in model size. The practical meaning is that parameter count and test loss are tied together in a measurable way. Grow the model, keep the other conditions healthy (enough data, enough compute), and loss drops by a factor you can predict in advance.

A useful property here is scale invariance. If you multiply the input by some constant factor, the output is multiplied by a fixed factor too, regardless of where on the curve you started. That is exactly what makes power laws attractive for forecasting: the relationship you measure between a 10M and a 100M parameter model tells you something concrete about the step from 10B to 100B.

Standard and inverse forms

The basic form is y = a * x^p, where x and y are the two quantities and a and p are constants fitted from data. With a positive exponent, y grows as x grows. On ordinary axes the curve bends upward; on log-log axes it becomes a straight line, because taking logarithms turns log y = log a + p * log x into a linear equation with slope p. Scaling papers almost always plot on log scales for this reason.

The curves you see in LLM papers slope the other way, though: loss goes down as resources go up. That is an inverse power law, y = a * x^(-p), which is the same equation with a negative exponent. Flipping the sign of the exponent flips the plot, and on log-log axes you get the characteristic straight line heading down and to the right. When a paper shows loss against compute, data, or parameters as a clean descending line, this is the relationship being depicted. Keep that in mind, because the straightness of the line is an artifact of the logarithmic axes, and a lot of confusion about scaling comes from forgetting it.

The first systematic study: loss scales with parameters, data, and compute

The open questions before 2020

When the original GPT appeared, it was not obvious how much scale mattered. Language models were promising but limited, and there was no clear recipe for improving them. Several questions had no settled answers. Does the shape of the network, meaning how many layers and how wide each one is, matter much? Does a bigger model reliably perform better? How much data does a bigger model need?

The 2020 OpenAI paper Scaling Laws for Neural Language Models set out to answer these by measuring how test loss responds to model size, model shape, dataset size, training compute, and batch size. Its headline finding was that loss follows a power law in each of three quantities, with some trends holding across more than seven orders of magnitude:

1. Number of model parameters.
2. Dataset size.
3. Training compute.

The important qualifier is that each relationship holds only while the other two quantities are not the limiting factor. A huge model trained on a tiny dataset does not follow the parameter curve, because data has become the bottleneck.

How the experiments were run

The researchers pretrained models of up to 1.5B parameters on subsets of the WebText2 corpus ranging from 22M to 23B tokens. Every model used a 1,024-token context and the usual next-token prediction objective, which is a cross-entropy loss over the vocabulary. The same cross-entropy measured on held-out text served as the main performance metric. None of this is exotic; it is essentially the standard pretraining setup used for most LLMs, which is why the results generalized so well.

What the fitted laws look like

Test loss on WebText2 improved steadily as parameters, data, and compute increased. The trends covered eight orders of magnitude of compute, six of model size, and two of dataset size. The fitted equations resemble the inverse power law above, but written slightly differently: the leading constant is set to one, and a scale constant sits inside the parentheses. For model size, for instance, the form is L(N) = (N_c / N)^alpha_N, where N is the parameter count, N_c is a fitted constant, and alpha_N is the fitted exponent. Analogous expressions exist for dataset size D and compute C.

One detail was needed to get clean fits: parameters in the positional and token embedding tables are excluded from the count. Including them makes the trends noisier, while counting only the non-embedding parameters produces tidy straight lines. If you ever try to reproduce a scaling plot, this is the sort of accounting choice that can quietly change your results.

Because each law assumes the other factors are not bottlenecks, the practical rule is to scale all three together. Push only one of them and you run into diminishing returns quickly.

Reading the curves without the log scale

The log-log plots look wonderfully linear, which invites an overly optimistic reading. Replot the same fitted laws on ordinary axes and the shape changes completely: the curve drops steeply at first and then flattens, looking much like exponential decay.

This matters because a lot of popular commentary implies the opposite intuition, as if model quality rose exponentially while compute grew only logarithmically. The math says something much more sober. Each further reduction in loss costs a multiplicative increase in resources, so squeezing out improvements becomes progressively, and quite dramatically, more expensive as you go. A straight line on a log-log chart means "ten times the compute buys a fixed fractional improvement," not "a little more compute buys a lot more quality."

Shape barely matters, scale matters a lot

Beyond the three power laws, the paper found that architectural choices such as depth versus width had only a small effect on loss compared with raw scale. More parameters, more data, and more compute produced smooth, reliable improvements; fiddling with the network's proportions did not move the needle nearly as much.

The researchers also observed that bigger models are more sample-efficient. A larger model reaches a given loss after seeing fewer tokens than a smaller one. From a pure training-compute perspective, this suggests an unusual strategy: train a very large model on a relatively modest amount of data and stop long before it converges. That really is compute-optimal for training, but it ignores what happens after training. A large model is expensive to serve for every request it answers, so in practice teams usually do the reverse and train smaller models on more data, because the smaller model is much cheaper to host.

The paper also looked at how dataset size should grow alongside model size and concluded that data can grow more slowly than parameters. Roughly, making the model about eight times larger required only about five times more data to avoid overfitting. (That ratio follows from the fitted relationship, in which the data requirement grows like the model size raised to a power a bit below one.) Keep this recommendation in mind, because the Chinchilla work later overturned it.

Finally, the same laws held on several other evaluation datasets once a fixed offset was added to the loss to account for the different data distribution. Taken together, these results made a strong case that language modeling improves smoothly and predictably when model size, data, and compute are scaled appropriately, and that larger models would be both better and more sample-efficient. That case is what justified the enormous pretraining runs that followed.

How labs actually use scaling laws

The upside of large-scale pretraining comes with an uncomfortable catch. The best results require enormous models and datasets, and those runs cost a great deal of money. Imagine spending on the order of ten million dollars on a single run only to discover the model is disappointing. At that price there is no room for trial-and-error tuning on the full-size model; you need confidence before you start that the configuration will work.

Scaling laws are the tool that provides that confidence. Beyond proving that the relationships exist, they give researchers a concrete workflow:

• Train a family of much smaller models under a range of settings.
• Fit scaling laws to how those small models perform.
• Extrapolate the fitted laws to predict how a far larger model will do.

This process is not foolproof. Extrapolating from small runs to a large one can be inaccurate, and some behaviors change with scale in ways a small model will not reveal. Techniques such as hyperparameter transfer schemes have been developed to make small-to-large extrapolation more reliable, and fitting scaling laws on cheap runs is now routine practice. Besides reducing technical risk, a credible forecast also makes it much easier to justify the budget for a large run to whoever is paying for it.

There is a sensible rule of thumb for when not to trust an extrapolation: if your small runs do not span at least a couple of orders of magnitude, or if you change the data mixture, tokenizer, or architecture between the small runs and the big one, the fitted constants may no longer apply. The value of a scaling law comes entirely from holding everything else fixed.

Scaling in the age of pretraining, from GPT-3 to GPT-4

Once scaling laws were established, the recipe for progress became almost embarrassingly simple: train larger models on larger and cleaner datasets. OpenAI followed it through the GPT series, and most other notable models followed it too. Ilya Sutskever later described this period as the "age of pretraining," characterized by very large networks trained on very large datasets.

The GPT family as a scaling experiment

The GPT line is the best-known application of these ideas. The discussion here concentrates on the models up to GPT-3, for two reasons: their details were published openly, and the later models owe much of their improvement to post-training advances rather than to pretraining scale alone. If you want a generation-by-generation view of what each release changed for application developers, see our walkthrough from GPT-1 to reasoning models; here the focus stays on what each step says about scaling.

GPT was small by today's standards: 12 layers and 117M parameters. It was first pretrained on BooksCorpus, the raw text of about 7,000 books, and then finetuned separately on each downstream task with a supervised objective and a dedicated classification head per task. It was one of the first demonstrations of large-scale self-supervised pretraining of a decoder-only transformer, and it surfaced a few lessons:

• Self-supervised pretraining on plain text works remarkably well.
• Training on long, contiguous stretches of text matters.
• A model pretrained this way can be finetuned to reach state-of-the-art accuracy on a wide range of tasks.

The model itself was not remarkable, but it established the two ingredients, a decoder-only transformer and self-supervised pretraining, that later work would push to much larger scale.

GPT-2 followed soon after as a set of models up to 1.5B parameters. The architecture and training objective were the same as GPT's, but two things changed:

• Pretraining used WebText, a dataset scraped from the internet and much larger than BooksCorpus.
• There was no task-specific finetuning. Tasks were solved zero-shot, by prompting the pretrained model directly.

GPT-2 did not reach state-of-the-art results on most benchmarks, but its scores improved consistently as the model got bigger, which was clear evidence that adding parameters paid off. Just as important, even the largest GPT-2 still appeared to underfit WebText. That implied there was headroom left: scaling both model and data further should keep helping. The GPT-2 paper also argued that a language model with enough capacity will start to infer and carry out the tasks it sees demonstrated in natural language, simply because doing so helps it predict the text better. That argument, plus the measured trends, gave researchers the confidence to keep going.

GPT-3 was the turning point that settled the question of whether large-scale pretraining pays off. At 175B parameters it was more than 100 times larger than the biggest GPT-2. The architecture was again a decoder-only transformer, close to its predecessors, but the training data was much larger: a corpus built on CommonCrawl, around ten times the size of WebText, blended with several other sources into a data mixture.

GPT-3 was evaluated mostly through few-shot prompting, which is worth distinguishing from the earlier approaches:

• GPT used finetuning, updating the weights for each task.
• GPT-2 used zero-shot prompting, describing the task with no examples.
• GPT-3 used few-shot prompting, placing a handful of worked examples in the prompt.

The GPT-3 paper called this in-context learning. The name is slightly misleading, because nothing is learned in the usual sense: the weights do not change at all. The examples in the prompt simply act as context that steers the model toward a better output. GPT-3 turned out to be very good at this, which suggested in-context learning was a capability that emerges with scale.

Across language understanding benchmarks, few-shot accuracy climbed with model size. Larger models used the information in their context window better and more efficiently than smaller ones, and on several tasks GPT-3's few-shot results beat the previous state of the art, again improving smoothly with scale. That one model could do this across so many tasks, with nothing but a change of prompt, was striking at the time. GPT-3 became one of the first real foundation models and introduced prompting as the main way people interact with LLMs.

After GPT-3: post-training takes a bigger share

GPT-3 set off a wave of interest in LLMs focused mainly on bigger pretraining runs. OpenAI's next releases, InstructGPT, ChatGPT, and GPT-4, combined large-scale pretraining with new post-training methods, namely supervised finetuning and reinforcement learning from human feedback (RLHF), and the resulting quality jump brought AI to mainstream attention. The GPT-4 technical report describes it as a transformer pretrained on next-token prediction whose post-training alignment improves factuality and adherence to desired behavior.

Around this time OpenAI began publishing far less. Models shipped through the API without detailed papers. Some information is still available, though. InstructGPT, the direct predecessor of ChatGPT, has a paper describing its post-training pipeline in depth, and that paper states that GPT-3 is its base model. It is reasonable to conclude that InstructGPT's gains came mostly from post-training rather than from a bigger pretraining run.

GPT-4 was clearly more capable than ChatGPT, but its technical report disclosed very little. It confirms only that:

1. The model is a transformer.
2. It is pretrained with next-token prediction.
3. The data includes both public and licensed third-party sources.
4. It is finetuned with RLHF.

Scaling nonetheless sits at the center of that report. A major engineering goal was a training stack that behaves predictably across scales, so that results from small runs could be extrapolated to the huge and very expensive final run. The report states that the final loss of properly trained large models is well approximated by power laws in training compute.

This is the practical workflow from earlier, applied at the frontier. Because a full pretraining run is so expensive, there is effectively one attempt and no room for model-specific tuning. Instead, the team trained models with 1,000 to 10,000 times less compute, fitted power laws to them, and used those laws to forecast the big model. The GPT-4 report shows exactly this, a compute-versus-loss power law used to predict GPT-4's final loss.

The fitted equation has one addition compared with what we saw before: an irreducible loss term, giving a form like L(C) = a * C^b + c. The constant c captures the fact that loss will never reach zero, because natural text contains genuine randomness that no model can predict. With the law fitted on small runs, the prediction of GPT-4's final loss turned out to be very accurate. Notice also that this particular plot uses linear axes, and the diminishing returns are plain to see: the improvement per unit of compute visibly shrinks as compute grows.

The report also acknowledged that raw loss is hard to interpret and tried to forecast more meaningful metrics. One example is the pass rate on the HumanEval coding benchmark. The problems were grouped into buckets by difficulty, and a scaling law was fitted to predict pass rate per bucket. Using runs with about 1,000 times less compute, the team predicted GPT-4's HumanEval pass rate accurately.

The lesson from this lineage is consistent: larger pretraining runs are worth it but costly, and scaling laws make them predictable enough that you avoid wasting compute on runs that will not deliver.

Chinchilla: models were starved of data

The 2020 study recommended growing parameters faster than data. For a while after GPT-3, the field mostly followed that advice, and it did not go well. Models far larger than GPT-3, such as the 530B-parameter MT-NLG, were trained on datasets of roughly GPT-3's size and failed to beat it. Meanwhile, models that increased both parameters and data, such as DeepMind's Gopher, did noticeably better.

Those observations motivated Training Compute-Optimal Large Language Models, the Chinchilla paper, which revisited scaling from scratch with much larger experiments: models from 70M to 17B parameters trained on datasets exceeding a trillion tokens. By training many combinations of model size and data size, the researchers fitted a law predicting loss from both factors at once. From that law you can work out the best allocation for any fixed compute budget.

Their conclusion was that compute-optimal training should grow model size and data size in equal proportion. By that standard, most existing LLMs were undertrained for their size; they would benefit from far more data. The fitted laws suggested, for example, that Gopher should have been trained on a dataset about 20 times larger. The paper also noted, pointedly, that the amount of data this implies is far beyond what was then being used.

To test the claim, the team trained Chinchilla, a 70B-parameter model trained on 1.4 trillion tokens. It used the same data and evaluation setup as Gopher but was four times smaller and trained on much more data, and it outperformed Gopher consistently.

Chinchilla's scaling laws became the reference point for years. "Chinchilla-optimal" entered the everyday vocabulary of the field, and the paper is still cited constantly even though plenty of scaling research has come out since.

It helps to be precise about what "Chinchilla-optimal" means, because it is often misused. It describes the best loss for a fixed training budget. It says nothing about inference cost. A team that expects to serve a model billions of times may deliberately train a smaller model well past the Chinchilla ratio, accepting a less efficient training run in exchange for a cheaper model in production. That trade-off comes up again below.

The debate about scaling hitting a wall after GPT-4o

As model releases and visible improvements slowed through the second half of 2024, scaling became a heated topic. After years in which scale drove almost every advance, a stream of reports suggested it might be stalling:

• Reuters reported that OpenAI was adjusting its product strategy after hitting a plateau with current methods.
• The Information reported that the pace of improvement between GPT models was slowing.
• Bloomberg described several frontier labs struggling to build meaningfully more capable systems.
• TechCrunch argued that scaling was delivering diminishing returns.
• Time published a more measured essay on the various factors behind the narrative that progress was slowing.
• Ilya Sutskever, in his test-of-time award talk at NeurIPS 2024, said that pretraining as we know it will end.

Other prominent voices disagreed. Anthropic's CEO Dario Amodei said scaling would probably continue, and OpenAI's Sam Altman insisted there was no wall. Rather than picking a side, the more useful exercise is to understand what the research actually supports, because both camps are partly right.

Is scaling slowing down? It depends on what you measure

Nathan Lambert put the situation neatly: scaling can still be working technically while the rate of improvement that users feel is slowing. The honest answer to "is scaling slowing?" depends on the definition, and both yes and no can be defended. So instead of forcing a verdict, it is more productive to look at a few separate issues.

Diminishing returns are built into the math

Recall what a scaling law actually states: a power-law relationship between training compute (or model or data size) and test loss. It does not promise exponential gains from modest extra compute. Plotted on normal axes, the curve looks like exponential decay, and each additional improvement costs more than the last.

In other words, scaling laws flatten by design. The slowdown people are noticing is, to a large extent, exactly what the equations predict. If you plan with a scaling law, you should expect each new generation to require a multiplicative increase in resources for a smaller absolute gain.

Loss is not the thing anyone cares about

There is also a measurement mismatch. Scaling laws are fitted to pretraining test loss, but what lower loss means for capability is unclear. Does it translate into higher accuracy on downstream tasks? Does it produce new abilities? Research on over-training notes that practitioners generally judge models by downstream benchmark accuracy, not by perplexity on held-out text. So there is a gap between the two statements in play:

• Scaling laws say that larger pretraining runs will lower test loss smoothly.
• What people want is a "better" model, however they define it.

That definition varies enormously. Everyday users mostly judge chat quality. Practitioners care about accuracy on the specific tasks they deploy. Researchers at frontier labs often hold very high and very specific bars, such as producing a PhD thesis or solving research-level mathematics problems like those in the FrontierMath benchmark. LLMs are so broad that there are many valid lenses for evaluating them, and no single one captures everything.

Given that spread, any claim that scaling is or is not "working" will always be contested. A clearer definition of success helps. If the science predicts lower loss for larger models, that does not mean every new model will satisfy every audience. Failing to reach AGI, or failing to outperform award-winning mathematicians, does not show that scaling has stopped working in the technical sense. A good part of the perceived slowdown is arguably a problem of expectations and perception rather than a failure of the underlying laws.

The data supply is finite

The strongest argument against continued pretraining scale concerns data. Scaling pretraining requires growing both the model and the dataset. The early 2020 work suggested data mattered less than parameters, but Chinchilla showed it matters just as much. On top of that, later research on over-training points out that it has become standard to train smaller models well beyond the Chinchilla-optimal dataset size, precisely because larger models are costly at inference time. That practice demands even more data per parameter.

All of this leads to one conclusion: further pretraining scale needs larger datasets. And most current pretraining data comes from crawling the web. There is only one internet, and finding entirely new sources of high-quality text at that scale is hard.

Sutskever has made precisely this argument: compute keeps growing quickly, but data does not, because it depends on web scraping. On that view pretraining cannot be scaled indefinitely, and the field needs new routes to progress. His phrase for the situation was that we have reached "peak data."

Ways pretraining can still scale

Diminishing returns are real, and the data argument is sound. Even so, several research directions can still improve pretraining.

Synthetic data and smarter data use

Growing pretraining by several more orders of magnitude will probably require synthetic data. There are legitimate concerns that leaning too heavily on generated text reduces diversity, but synthetic data is already being used more and more for LLMs, with results that appear positive. Alongside it, curriculum-style approaches and continued pretraining have produced solid gains by changing the data rather than the model, for example by adjusting the data mixture or adding instruction-style data near the end of pretraining.

Scaling laws for metrics people understand

Loss-based laws are hard to act on, so recent work fits laws to more interpretable targets. The over-training study mentioned earlier fits laws whose targets are scores on the evaluation suite that ships with LLM Foundry, rather than raw loss. That is far easier to reason about. Few people know what a 5% lower test loss means in practice, but going from 85% to 90% accuracy on a benchmark you care about is immediately understandable. Other work has extended the idea to predict performance after post-training and quantization, and to model how performance evolves during the pretraining run itself.

DeepSeek-V3 as evidence that scale still pays

Despite the pessimism, scaling pretraining still produces periodic jumps. DeepSeek-V3 is a 671B-parameter mixture-of-experts (MoE) model trained on 14.8T tokens and released with open weights. Its technical report shows it outperforming GPT-4o and Claude 3.5 Sonnet on the benchmarks it covers. For comparison, Meta's Llama 3 models were trained on more than 15T tokens.

Beating GPT-4o was a significant milestone for open-weight models, one that even the largest Llama models had not reached. DeepSeek-V3 combines several techniques:

• An optimized MoE architecture carried over from DeepSeek-V2.
• A load-balancing strategy for the experts that does not rely on an auxiliary loss.
• A multi-token prediction training objective.
• Distillation of reasoning ability from a long chain-of-thought model, in the spirit of OpenAI's o1.

It also goes through post-training, including supervised finetuning and RLHF, to align it with human preferences. The report notes that pretraining on 14.8T tokens was remarkably stable, with no unrecoverable loss spikes and no rollbacks needed.

Still, the main reason for its strength is sheer scale: a very large model trained on a correspondingly large dataset. Runs of this size are hard for many reasons, including hardware failures and loss spikes, so a stable run at a cost that is reasonable by LLM standards is notable in itself. It suggests that very large pretraining jobs are becoming easier to manage and more efficient over time. Treat the specific benchmark comparisons as a snapshot, since the leaderboard has moved on since then.

Another order of magnitude is an engineering project

Genuinely testing the scaling laws further means training models several orders of magnitude beyond today's. Whatever you think about the value of that, there are hard practical constraints. It requires:

• Bigger compute clusters.
• More and better hardware.
• Very large amounts of electrical power.
• New algorithms, for example for distributed training at larger scale, possibly across several data centers.

The next generation is not just a matter of buying more GPUs. It is a multidisciplinary engineering effort, and those take time. As a reference point, GPT-4 arrived in March 2023, nearly three years after GPT-3. Expecting a similar or longer gap for each further 10 to 100 times increase in scale is reasonable. Ege Erdil of Epoch AI has summarized the dynamic well: each order-of-magnitude increase requires its own new innovations.

Where progress comes from if pretraining stalls: reinforcement learning and test-time compute

Suppose, purely for the sake of argument, that pretraining gains stopped tomorrow. AI research would still have plenty of room to move quickly. Synthetic data is one avenue already covered. Two others are especially active right now:

1. LLM systems and agents.
2. Reasoning models.

Getting more out of existing models with LLM systems

Beyond the single-call paradigm

Most LLM applications today follow a single-model pattern: send the task to one model and use its output directly as the answer. If you want that system to handle harder tasks more accurately, the obvious lever is a better underlying model, but that makes you wait for the next generation. The alternative is to build a system that combines several LLM calls, or other components, to solve the task.

Decomposition and chaining

The core idea of an LLM system is to split a complex task into pieces that are easier for a model, or another module, to handle. There are two main strategies:

1. Task decomposition: divide the task into sub-tasks, solve each independently, and combine the results into a final answer.
2. Chaining: solve a task or sub-task with a sequence of LLM calls rather than one.

They can be used separately or together. Take a system that summarizes books. You can decompose the job by summarizing each chapter first. From there you can either:

• Decompose further, splitting chapters into smaller chunks of text, which is the idea behind recursive or hierarchical summarization.
• Chain calls, for example having one call extract the key facts from a chapter and a second call write the chapter summary from those facts.

Finally, one more call summarizes the concatenated chapter summaries into a summary of the whole book. Most complex tasks can be broken into simple parts like this, which is what makes LLM systems so powerful. As decomposition and chaining get more elaborate, these systems become sophisticated pieces of software, and designing them well is a substantial field of applied AI in its own right.

Building products people actually use

For all the attention LLMs get, the number of practical, widely adopted use cases is still small. The biggest today are code generation and chat, both fairly obvious fits for the technology.

There are many domains where LLMs could be useful but are not yet, so building genuinely helpful LLM-powered products is itself an important research and engineering problem. Capable models already exist; turning them into something people want to use is a different challenge, and it comes down to building reliable, capable systems around those models.

Agents: tools and actions

The boundary between an LLM system and an agent is fuzzy, partly because "agent" has become an overloaded term. The important point is that LLM systems can be extended in many useful directions. A model can be taught to call tools, such as a calculator or a search engine, while solving a problem. It can run code it writes, or take actions on a user's behalf, like booking a hotel or sending an email. The range of tools and modules you can attach to a model gives almost unlimited room for building more capable systems. If you want a concrete picture of how the agent loop is structured, our guide to goals, tools, memory, and the agent loop covers it in detail.

Robustness is the real bottleneck

Reliability is one of the biggest obstacles to more capable systems, and the arithmetic explains why. Suppose a system makes ten LLM calls, each succeeds 95% of the time, and all ten must succeed for the final result to be correct. The system as a whole succeeds only about 60% of the time, since 0.95^10 is roughly 0.60. Every component looks dependable on its own, yet the pipeline fails four times in ten.

Adding more steps makes this worse multiplicatively, which caps how complex a system can get before it becomes unusable. Building bigger systems therefore requires making each component much more reliable. Some evidence suggests that scale itself improves robustness. Another lever is better meta-generation algorithms. Instead of taking a single sample from the model, these methods use techniques such as parallel decoding, step-level verification, and critiques to arrive at a more refined and accurate output. A thorough survey of this area is the paper "From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models."

This line of work is advancing quickly and is likely to be a major driver of progress. As meta-generation improves, models become more reliable, and more complex LLM and agent systems become practical.

A practical implication for anyone building such systems: before adding another step to a pipeline, estimate the end-to-end success rate by multiplying the per-step rates, and consider whether a verification or retry step would raise the weakest link more cheaply than adding capability elsewhere.

Reasoning models and a new scaling axis

Early LLMs were often dismissed as pattern matchers that memorized data and could not really reason. That criticism has largely been refuted in recent years. The emerging picture is that these models probably had latent reasoning ability all along, and the right prompting or training method is what draws it out.

Chain of thought

Chain-of-thought (CoT) prompting was one of the first techniques to show this. It is purely a prompting method: you ask the model to explain its reasoning before giving its final answer. When the model writes out a step-by-step rationale before answering, its reasoning performance improves substantially. A side benefit is that the rationale is human-readable, which makes the output easier to interpret.

The idea is simple and general, and it has become central to improving LLM reasoning. It has been reused in many forms:

• LLM-as-a-judge evaluators usually write a scoring rationale before producing their final score.
• Supervised finetuning and instruction-tuning methods have been developed to teach smaller or open models to produce better chains of thought.
• Models are often asked to reflect on, critique, or verify their own output and then revise it based on that feedback.

Complex reasoning remains a fast-moving research area. Training methods that teach models to include step-level verification in their reasoning have shown promising results, and further gains are likely as better training strategies appear.

o1: thinking before answering

OpenAI's o1 was a major step in LLM reasoning, and its approach is built on chains of thought. Much as a person thinks before answering a hard question, o1 spends time "thinking" before it responds. Concretely, its thoughts are long chains of thought in which the model works through the problem, breaks it into simpler steps, tries different approaches, and corrects its own errors. OpenAI describes it as a model trained with reinforcement learning to reason, one that can produce a long internal chain of thought before replying.

The exact training recipe has not been published. OpenAI has said that o1 learns to reason through a "large-scale reinforcement learning" algorithm that is "highly data-efficient" and focused on improving the model's ability to produce useful chains of thought. Public comments from OpenAI researchers suggest the model is trained with pure reinforcement learning, which runs against earlier speculation that o1 performs some kind of tree search at inference time.

On reasoning-heavy tasks, o1 beats GPT-4o almost across the board. Among OpenAI's reported results, the model:

• Scores in the 89th percentile on competitive programming problems from Codeforces.
• Performs at the level of the top 500 US students on AIME, a qualifying exam for the USA Math Olympiad.
• Exceeds the accuracy of human PhD-level experts on graduate-level physics, biology, and chemistry questions from the GPQA benchmark.

From o1 to o3: scaling training and inference compute

One of o1's most interesting properties is that you can buy more reasoning ability with more inference compute. To handle harder problems, the model simply writes longer chains of thought, and spending more compute at inference time on those longer chains produces a smooth improvement in reasoning performance. According to OpenAI, o1 keeps getting better along two separate dials: the amount of reinforcement learning applied during training (train-time compute) and the length of deliberation allowed per question (test-time compute).

The same holds for training: o1 gets steadily better as more compute goes into reinforcement learning. That is apparently the route that led to o3. OpenAI previewed o3's evaluation results in late 2024 and shared few details, but given that it appeared only about three months after o1, it is most likely a scaled-up o1 with more compute invested in reinforcement learning.

At the time of that preview, o3 had not been released, and the preview figures were striking:

• 87.5% on the ARC-AGI benchmark, where GPT-4o scored only 5%. That made o3 the first model to exceed the 85% human-level threshold on ARC-AGI, a benchmark described as a "North Star" for AGI that had gone unbeaten for more than five years. That top score came from a high-compute configuration.
• 71.7% on SWE-Bench Verified, and a Codeforces Elo rating of 2727, which would place it among the top 200 competitive programmers in the world.
• 25.2% on Epoch AI's FrontierMath benchmark, up from a previous state of the art of 2.0%. Terence Tao had described this benchmark as extremely difficult and expected it to resist AI systems for several years at least.

A distilled variant, o3-mini, was previewed at the same time and showed strong results with much better compute efficiency. These were preview numbers from a single announcement; check current documentation and independent evaluations for how the released models compare.

Why this is a new scaling paradigm

If the o1 and o3 plots look familiar by now, that is because they are the same kind of chart this whole piece has been about: log-scale compute on the x-axis and a smooth, straight rise in performance. In other words, these reasoning models show a clear power-law relationship between performance and two quantities:

• Training-time compute spent on reinforcement learning.
• Inference-time compute spent on thinking.

This differs from classic scaling laws. Rather than scaling pretraining, you scale the compute spent on post-training and on inference. It is a genuinely new scaling axis, and the results so far are strong. It shows that there are ways to scale beyond pretraining, and that the field has found its next hill to climb. The form may change, but scaling is still what drives progress.

One engineering consequence is worth stating plainly. When quality scales with inference compute, cost and latency per request become tunable quality knobs rather than fixed properties of a model. That changes how you budget and design applications: harder queries can be routed to longer reasoning, and simpler ones kept cheap.

What to remember when you read a scaling plot

• A scaling law is a power law between a resource (parameters, data, or compute) and test loss. The straight line on a log-log plot hides steep diminishing returns, so each improvement costs multiplicatively more.
• The laws hold only when the other factors are not the bottleneck, which is why data, parameters, and compute must grow together.
• Chinchilla reversed the earlier advice: for a fixed training budget, model size and data should grow in proportion, and many earlier models were badly undertrained. Serving costs often push teams to train smaller models on even more data.
• The main practical use of scaling laws is risk reduction: fit them on cheap runs and forecast the expensive one, as the GPT-4 report did for both loss and HumanEval pass rate.
• The perceived slowdown mixes three things: diminishing returns that the math predicts, a gap between loss and what users value, and the long lead time of engineering the next order of magnitude. The finite supply of web data is the most serious constraint on pretraining.
• Progress does not depend on pretraining alone. Well-designed LLM systems need robustness above all, since per-step error rates compound, and reasoning models show that training-time reinforcement learning and inference-time thinking follow their own scaling curves.

The useful question is no longer whether scaling continues, but which resource is being scaled and whether the gains show up in the metric you actually care about.