Skip to main content
Watch: Grok Bot Built My AI Video Arena While I Tested 8 Models

Reasoning / Training / Inference

Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It

Zehao Jin, Ruixuan Deng, Junran Wang · Georgia Institute of Technology

arXiv:2609.3658562 upvotes

Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It

Authors: Zehao Jin, Ruixuan Deng, Junran Wang (Georgia Institute of Technology)

arXiv ID: 2609.36585

Problem: Pretrained transformers use only a small part of their depth for in-context reference following. Give a model a chain like K = apple, B = K, D = B and ask for print(D), and it stops following after a few lines and guesses. Whether that is a capacity limit of the weights or a default computation path that training never turns on was not known.

Key Methodology:

  • Chain-following probes across 13 base models from 0.6B to 32B parameters, plus looped models, measuring how many reference lines a model follows before accuracy collapses.
  • A task-trained rank-8 LoRA added to the residual stream at the input of a single early layer, trained with all model weights frozen (65,537 trainable parameters, under 0.01% of Qwen3-8B), with a penalty preserving ordinary-language behavior (WikiText perplexity 10.14 to 10.148).
  • Layer sweeps and attention cuts to locate the mechanism, including cutting each line's attention to its parent line and removing the heads that read the parent line.
  • A preregistered frozen-model cutoff measurement, tested on held-out models, to predict where the LoRA still works.

Key Results:

  • The failure is broad at the tested scale: 13 base models follow only 1.4 to 3.6 lines reliably; depth does not help (OLMo-3-32B and OLMo-3-7B both reach about 2.6 lines); looped models plateau after one or two extra lines.
  • Qwen3-8B improves from 15.5% to 99% exact accuracy on 24-line chains in a single pass; a LoRA trained on longer programs answers 98% of 40-line and 88% of 48-line chains; Ouro-1.4B follows 60 lines after four loops and at least 160 after eight; MuSiQue gains are +11.4, +9.4, +17.9 and +11.7 exact-match points on Qwen3-8B, OLMo-3-7B, Llama-3.1-8B and Ouro-1.4B.
  • Mechanism: the LoRA starts a relay in which program lines pass their chain identity through layers 16 to 22; cutting parent-line attention in those layers returns 6-, 8- and 12-line chains to chance (53, 48 and 55%), and removing ten parent-reading heads drops 16-line accuracy from 89% to 53% while ten random heads leave 83%.
  • Placement dominates form: four different interventions at layer 14 all extend the chain and all four stay at chance at layer 26; moving the LoRA from layer 20 to layer 21 drops reach from 20.5 to 5.2 lines; the frozen-model cutoff measurement predicted useful placement in three of four held-out models.

Applied Context: Default answers understate the computation accessible through a tiny, well-placed edit, which matters both for adapting small open models to fixed multi-hop or reference-following tasks and for reading capability evals at face value. The practical recipe is a task-specific rank-8 LoRA at the earliest effective layer, with the cutoff measurement used to pick the layer instead of a sweep. The effect is shown on reference chains and multi-hop QA and the LoRA is task-trained, so the paper does not claim general reasoning gains.

Paper: arXiv:2609.36585