Recurrent Depth Explained: Looped Transformers and Hidden Reasoning

Most of today's reasoning models can only carry a serial thought forward by writing it down as tokens, which is why chain-of-thought transcripts became the main window for monitoring what a model is doing. Recurrent depth changes that: the model re-runs part of its own layer stack on its hidden state before emitting anything, so some of the "thinking" never becomes text. This article explains how the mechanism works, why it is so attractive economically, which published research already demonstrates it, and what it is likely to mean for open models running on your own hardware and for anyone who relies on reading a model's reasoning.

A familiar pattern: hidden mechanism, open replication

The reasoning-model story has played out once already. When OpenAI released o1 in 2024, the model deliberated before answering, but its reasoning was hidden and truncated, and the company said openly that this concealment was intentional. Many observers assumed the underlying technique would remain proprietary for years. Instead, DeepSeek published R1 with open weights and visible reasoning traces a little over a year later, and the open community reproduced the approach quickly once the recipe was public.

The recurring shape is simple: a closed lab ships a capability, keeps the mechanism opaque, and open research later reproduces it and releases it freely. Recurrent depth looks like the same pattern, with one difference that matters: open implementations with published weights already exist before the closed version has even been confirmed.

What is confirmed about Astra, and what is not

OpenAI launched Astra on September 2, 2026. Keeping the claims in separate buckets is essential here, because the interesting part is exactly the part that is not official.

Confirmed by OpenAI

• Astra is the first model the company rated "Critical" for cybersecurity.
• It scored 100% on ExploitBench, a benchmark in which models find and exploit real software vulnerabilities.
• It shipped with chain-of-thought monitors observing its written reasoning.

Reported but unconfirmed

On September 1, The Information reported that Astra uses recurrent depth, meaning internal reasoning loops. OpenAI has neither confirmed nor denied this. It is the most credible account available, sourced reporting rather than rumor, but you should not repeat it as established fact.

Claimed but not checkable

Chief scientist Jakub Pachocki stated publicly that the depth of Astra's computation graph is "within a factor of two of GPT-4". If accurate, that would mean the loops are deliberately shallow and the model's reasoning is only modestly more opaque than a conventional single-pass model. The actual figure has not been published, so outsiders cannot verify the bound; it has to be taken on trust.

Pure speculation

Suggestions that Astra is secretly the rumored "GPT-6" are unsupported. Nobody outside the company knows.

Why the monitoring question is so pointed

In July 2025, forty-one researchers from OpenAI, DeepMind, Anthropic and the UK AI Security Institute co-signed a position paper on chain-of-thought monitoring. It warned specifically that latent-reasoning architectures, which reason in vectors rather than words, could close the monitoring window that much of current safety practice depends on. Pachocki was among the signatories. If the reporting is right, a lab has now shipped the kind of architecture its own chief scientist had flagged as a risk to oversight. For a broader look at how Astra compares with other frontier agents, see our comparison of Astra, Flash, Fable and Mythos.

How recurrent depth works

A standard transformer sends each token through its stack of layers exactly once. If the model needs more serial computation to solve a problem, the only lever it has is to generate more tokens: every visible reasoning step is another forward pass, and each step is readable text.

A recurrent-depth model splits the stack into three parts. A prelude embeds the input into a latent state, a core block is applied repeatedly to that state, and a coda maps the final state to output. The diagram below contrasts the two shapes; notice that the looped core reuses the same weights on each pass and that all repetitions finish before any word is produced.

Standard:  token → L1 → L2 → ... → L96 → out
           (one pass; "extra thinking" = more visible words)

Astra:     token → PRELUDE → [ CORE × R ] → CODA → out
                                    ↑
                  the same block of layers, re-run on the
                  model's own hidden state, R times -
                  before a single word is written

Written as a formula, one iteration is h_next = f(h_current), and f stands for the same shared group of layers every time. Passes two through R introduce no new parameters; they simply apply the same weights again to a state that keeps evolving. A useful mental model is a draft that is revised several times in private, with only the final version ever handed in.

The crucial consequence is this: every iteration after the first happens entirely in vector space, and vectors leave no transcript. Nothing about those intermediate passes is written anywhere a monitor could read.

The loop count R is also a runtime setting rather than a different model. The same checkpoint can run with R=1 for a cheap tier and R=20 for a premium one. From the outside, you see a prompt and an answer; how many internal passes happened in between is invisible unless the provider tells you.

Why labs want it: the cost of thinking out loud

Set safety aside for a moment, because the economic case is strong enough on its own.

Chain-of-thought reasoning behaves like a telegraph that bills by the word. Each reasoning token is generated serially, every later token attends to it, and it occupies space in the KV cache for the rest of the request. Providers end up charging customers, at token prices, for the model's own scratch work.

Latent loops deliver serial computation without emitting any extra tokens. There is no per-thought generation cost, no growth in context length and no additional cache pressure. At the scale of billions of reasoning requests per day, making the reasoning internal is a margin decision as much as an architectural one. It also fits tiered pricing neatly: the same checkpoint, looped more times, answers harder questions at a higher price, and the label "thinking" covers the difference.

Two ways to buy capability: wide or deep

If you run inference locally, the trade-off looks different. There are now two distinct strategies, and they spend different resources.

The approach most open models have taken is to go wide with mixture-of-experts. Qwen's Flash-Next, for instance, keeps 512 experts available and activates only around 2% of them per token. That is parallel, memory-bound compute; on a 64GB MacBook it has been reported to sustain roughly 30 tokens per second.

Going wide costs memory, and memory has become a scarce and expensive component. At the time of writing, reported prices for a 64GB DDR5 kit went from an average of about $191 in August 2025 to about $1,118 a year later, an increase of roughly 485% and steeper than most other PC parts, GPUs included. The driver is AI infrastructure: memory manufacturers are shifting production toward high-bandwidth memory for data-center accelerators, hyperscalers are reserving capacity years ahead, and the shortage is expected to last into 2027. Treat these figures as a snapshot and check current market data before relying on them.

Recurrent depth goes deep instead: the same layers, run R times. It spends time. On local hardware, each extra loop divides your decode speed; at R=4, a model that decoded at 30 tokens per second would drop to around 7. A looped model on a laptop does not fail visibly, it simply slows as depth increases.

This explains the appeal. Loops extract more computation from every byte of weights that has to be fetched from memory, and memory bandwidth is the constrained resource. The arithmetic units that execute the extra passes are frequently idle between memory fetches anyway. Wide models spend the resource everyone is competing for; deep models spend one that data-center customers never see itemized. On your own machine, though, you see it directly as a lower tokens-per-second number.

The research trail: eight years of open papers

None of the underlying ideas are secret. The core mechanism has been in the open literature since 2018, and reading these papers will give you a better understanding of looped models than any product announcement.

Weight-shared depth (2018–2019)

• Universal Transformer (Dehghani et al., 2018). The original loop: apply the same layers repeatedly with shared parameters. It outperformed same-size transformers on GLUE while using roughly a quarter of the FLOPs, and it became heavily cited, yet no one deployed it at large scale.
• ALBERT (2019). Used layer sharing as a compression technique, showing that reusing depth is cheaper than paying for additional unique layers.

Reasoning in latent space (2024–2025)

• Coconut (Meta, December 2024). Its title, "Training LLMs to reason in a continuous latent space", describes the idea: intermediate reasoning stays in hidden vectors, and only the conclusion is turned into text. On planning tasks it beat chain-of-thought while producing fewer visible steps, a direct challenge to the assumption that more written reasoning means better reasoning.
• Huginn (Geiping et al., February 2025), "Scaling up Test-Time Compute with Latent Reasoning." A 3.5B-parameter model trained on 800B tokens. Holding weights and data fixed and changing only the loop count from r=1 to r=32 moved ARC-E from 34.9 to 69.9, with reasoning gains up to a compute budget equivalent to a 50B-parameter model.
• Mixture-of-Recursions (July 2025). Instead of looping every token equally, each token gets its own recursion depth. Easy tokens exit after the first pass; difficult ones keep iterating. The result is per-token adaptive computation, with KV-cache savings as a side benefit.

Open weights and scaling (2025–2026)

• Ouro / LoopLM (ByteDance, October 2025). The most practically relevant entry, because the weights are published on Hugging Face. The family includes 1.4B and 2.6B looped models trained on 7.7 trillion tokens, reported to match state-of-the-art models of up to 12B parameters. Controlled experiments attributed the gains to knowledge manipulation rather than knowledge capacity: the loops do not store more facts, they make better use of them. The team also found that LoopLM's reasoning traces agreed with its final answers more closely than chain-of-thought traces did.
• "Quantifying the Necessity of Chain of Thought through Opaque Serial Depth" (March 2026). A theory paper that supplies the vocabulary for this debate. Opaque serial depth is the length of computation a network can perform without interpretable intermediate steps, and the authors released a tool that computes an upper bound on it for any network.
• "Sparse Layers are Critical to Scaling Looped Language Models" (May 2026). Dense looped models do not scale well, but looped mixture-of-experts models do and beat standard baselines, because different experts activate on each pass through the shared layers. That recovers expressivity without adding parameters.
• Looped models for tool calling (August 2026). Multi-step, dependency-aware agentic tasks improve with recurrent depth, which suggests loops help with planning, not just puzzles.

Taken together, the mechanism, the benchmark evidence, a scaling recipe and a measurement tool have all been public for some time. What is new is a frontier deployment, not the idea.

What this likely means for open models and local inference

The following are forecasts, not facts, but each follows from the research above.

An open looped release is probably close

The scaling recipe that works is looped mixture-of-experts, and the leading open model families, such as Qwen and DeepSeek, are already mixture-of-experts architectures. Adding a recurrent core to an existing MoE model is an incremental change rather than a redesign, so it is reasonable to expect an open release with a recurrent core within one or two model generations, likely marketed around the idea of thinking longer rather than being bigger.

Smaller files for the same capability

Local inference has been dominated by file size: huge weight files, wired-memory limits and manual offloading of expert layers just to make a model fit. If a 1.4B looped model can genuinely compete with 12B-class dense models, that is roughly a 3GB download replacing a 24GB one. Such a model is fine-tunable with LoRA on consumer hardware and leaves memory free for longer context. Capability per byte would shift in favor of people with a single GPU or a laptop. If you are weighing local models against hosted APIs for resilience reasons, our piece on AI access as a dependency risk covers the broader trade-off.

Edge hardware suits loops well

Loops consume compute rather than memory bandwidth, and devices such as Apple Silicon machines have ample compute relative to their bandwidth. That balance was a poor fit for the last generation of inference workloads and a good fit for this one. Looped models are also friendly to kernel optimization: fewer distinct operations, and weights that stay hot in cache across passes. The missing piece is software. Runtimes such as llama.cpp and MLX will need explicit support for recurrent depth and adaptive early exit before local users can benefit.

Depth could become a user-facing setting

With per-token recursion, a small model can spend large-model compute only where a token warrants it: quick for routine text, deep for hard steps. Just as you tune sampling temperature today (see our guide to temperature, top-k and top-p), you may end up choosing a depth budget per request. The tool-calling results point the same way: agents that plan, call tools and track state gain the most from extra depth, which is exactly where a local coding assistant needs it.

Open models could reverse the transparency trade-off

The opaque-serial-depth tool is public. A closed provider can ship loops with an undisclosed R, but anyone holding open weights can measure a model's hidden reasoning depth directly. Open adoption therefore does not have to repeat the opacity problem; the number that closed labs withhold can be a measured, published property of an open model. The Ouro finding also cautions against assuming that latent reasoning is inherently less trustworthy: its traces tracked final answers more faithfully than conventional chain-of-thought.

The real open question is not whether open models will adopt loops, since Ouro and Huginn weights are already available. It is whether the community will pair loops with the one practice closed labs have declined to adopt: publishing the loop count.

When recurrent depth is not the right tool

• If you need auditable reasoning, for example in regulated workflows, visible chain-of-thought remains easier to inspect than any latent process.
• If latency matters more than accuracy on a local device, each additional loop directly reduces throughput.
• If your inference runtime lacks support for recurrent cores or adaptive exit, you will not realize the efficiency gains yet.

Wrapping up

For roughly a decade of language modeling, extra reasoning had to be expressed in text, and oversight grew around reading that text. Recurrent depth removes that coupling: a model can do meaningful serial computation without producing a word. The monitoring window still exists, but it no longer necessarily shows everything.

The research record is reassuring in one respect. The techniques behind looped models, along with open weights and a tool to measure hidden depth, are already public. Closed labs may keep the first move, but the open ecosystem can set the norm.

• Recurrent depth reuses a core block of layers R times on the hidden state, adding serial compute without extra tokens or parameters.
• It trades memory for time, which makes it attractive for data centers and for compute-rich, bandwidth-limited local hardware.
• Universal Transformers, Coconut, Huginn, Mixture-of-Recursions, Ouro and looped MoE together document the mechanism and its scaling.
• Reports that Astra uses it remain unconfirmed; treat the loop count as a question to ask any provider.
• For open models, measuring and publishing depth with the opaque-serial-depth tool is the most useful safeguard available.