LOTUS: Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers

TL;DR

Looped Transformers are more than parameter-efficient recurrent-depth models: they provide a practical architecture for iterative reasoning over a parallel workspace and can be trained with surprisingly simple supervision, just plain cross-entropy loss. We instantiate this as LOTUS (Looped Transformers with parallel supervision on latents). It is the first latent method to match explicit CoT at the 3B scale on GSM8K while thinking 2.5×–6.9× faster, with latents that decode back into readable reasoning.

LOTUS matches explicit chain-of-thought on GSM8K while reasoning in latent space, and the gap holds as models scale from GPT-2 to Llama-3.2-3B. — **LOTUS bridges the latent–explicit CoT accuracy gap across scale on GSM8K.** **(a)** Math-expression CoT: LOTUS tracks explicit CoT across scales, where prior latent methods fall behind, and thinks **2.5×** faster than CoT at 3B. **(b)** Natural-language CoT (3B): LOTUS matches explicit CoT and beats latent baselines, thinking **6.9×** faster than CoT.

Training

LOTUS inserts $K$ latent blocks (each with $c$ tokens) between the question and answer, then loops the base model over them $R$ times to get the post-loop latents. Every block is refined in parallel within each iteration, so the thought phase no longer scales with the number of decoded reasoning tokens. After the loop, every latent block is supervised in parallel, each aligned to its matching CoT step. The answer is then supervised with standard next-token prediction conditioned on the post-loop latents.

The LOTUS objective: a looped forward pass supervises K parallel latent blocks through the base LM head, then a final forward pass decodes the answer.

Inference

At inference, we run the same looped forward, now without any losses, and then decode the answer. The question's KV cache is computed once, the loop iterates $R$ times to fill the $K$ latent blocks, and the answer is decoded autoregressively from those post-loop latents through the base LM head. All the reasoning is carried by the parallel latent blocks, so the only sequential decoding is the short answer suffix, the source of LOTUS's latency gains.

Results

On Llama-3.2-3B-Instruct, LOTUS is the only latent method that stays with explicit chain-of-thought, on compact math-expression reasoning and longer natural-language reasoning, while thinking 2.5×–6.9× faster than CoT.

Math-expression CoT: accuracy

Accuracy (%) on GSM8k-Aug (in-domain GSM8K) and OOD GSM-Hard / SVAMP, vs. latent-reasoning methods (Llama-3.2-3B). LOTUS stays within ~1.5 points of explicit CoT and leads every latent baseline on all three. **Bold** = best latent method.
Method	GSM8K	GSM-Hard	SVAMP
Explicit CoT	71.5	17.0	71.0
PCCoT	54.7	13.5	69.5
CODI	60.8	14.3	73.3
CODI + SIM-CoT	62.3	14.6	74.9
KaVa	65.7	15.2	72.7
LOTUS	70.0	16.0	75.7

Math-expression CoT: thought latency

Thought-phase latency (ms/example) on GSM8K (Llama-3.2-3B, batch size 1, greedy decoding, NVIDIA H100 NVL). LOTUS computes its latent thought in parallel, **2.5×** faster than explicit CoT.
Method	Thought (ms/example)
Explicit CoT	338.8 (1.0×)
LOTUS	133.0 (2.5×)

Natural-language CoT: accuracy

Accuracy (%) on the natural-language GSM8K stress test (Llama-3.2-3B), where each step is a full sentence. LOTUS matches explicit CoT and beats every latent baseline, even topping explicit CoT on SVAMP. **Bold** = best latent method.
Method	GSM8K	GSM-Hard	SVAMP
Explicit CoT	68.41±0.59	18.27±0.85	71.93±1.62
PCCoT	47.6	11.0	65.2
CODI	55.9	13.6	70.1
KaVa	60.0	14.8	66.1
LOTUS	68.13±0.77	16.27±0.19	73.40±0.35

Natural-language CoT: thought latency

Thought-phase latency (ms/example) on the natural-language GSM8K test (Llama-3.2-3B). Longer chains make explicit CoT much slower, so LOTUS's parallel latent reasoning is **6.9×** faster.
Method	Thought (ms/example)
Explicit CoT	963.6 (1.0×)
LOTUS	140.8 (6.9×)

Ablations

LOTUS needs enough of both axes of its latent budget, sequential loop depth ($R$) and parallel width per block ($c$), and parallel supervision through the base LM head is equally essential. Accuracy climbs with each axis and saturates around $R=6$, $c=25$. We keep the number of latent blocks $K$ fixed.

GSM8K accuracy rises with both the number of looped iterations R and the number of latent tokens per block c, on Llama-3.2-3B. — **(a)** Looped iterations $R$ (sequential depth) and **(b)** latent tokens per block $c$ (parallel width) on Llama-3.2-3B: each lifts GSM8K accuracy, saturating around $R=6$ and $c=25$.

The same budget can also be dialed at inference without retraining: accuracy degrades gracefully below the trained setting and plateaus above it, so test-time compute can be traded for accuracy.

At inference, GSM8K accuracy varies smoothly with the number of looped iterations R and latent tokens per block c, without retraining. — **Inference-time** sweeps (no retraining) on Llama-3.2-3B. **(a)** Looped iterations $R$: accuracy climbs to the trained $R=6$ and holds beyond it. **(b)** Latent tokens per block $c$: accuracy rises to the trained $c=25$ and plateaus, while thought latency grows only modestly.

Interpretable latents

LOTUS's latent space isn't a black box. Projecting the post-loop hidden states through the base model's own LM head decodes each latent block into human-readable reasoning, recovering the gold CoT steps, and even surfacing alternative valid intermediate computations never seen at training time. This is direct evidence that the latent representation is interpretable and CoT-aligned.

Decoding LOTUS's latent blocks through the LM head recovers human-readable reasoning steps that reach the correct answer. — Reading out LOTUS's latent blocks on a GSM8K example: each post-loop block $h^{(R)}$, projected through $f_{\text{head}}$, decodes to a concrete reasoning step, converging to the correct answer (`### 540`) after 6 iterations.

Models

Pretrained LOTUS checkpoints (Llama-3.2-3B) are available on the Hugging Face collection:

gsm-lotus-llama3b · LOTUS on math-expression CoT (GSM8k-Aug), GSM8K 70.05%
gsm-lotus-llama3b-codi · LOTUS + CODI, GSM8K 70.58%
gsm-lotus-llama3b-nl · LOTUS on natural-language CoT, GSM8K 68.54%

BibTeX

For more details and additional interesting results, check out the paper. If you find it useful, please cite:

@article{fan2026bridging,
  title={Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers},
  author={Fan, Ying and Svete, Anej and Lee, Kangwook},
  journal={arXiv preprint arXiv:2606.31779},
  year={2026}
}