GEAR: Guided End-to-End AutoRegression for Image Synthesis

Authors: Bin Lin, Zheyuan Liu, Chenguo Lin, Sixiang Chen, Yunyang Ge, Yunlong Lin, Jianwei Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Li Yuan

arXiv ID: 2606.32039

Problem: Standard two-stage visual generative models (tokenizer → frozen generator) decouple training, leaving the tokenizer unaware of what the generator finds easy or hard to model.

Key Methodology:

Trains a VQ tokenizer and an autoregressive (AR) generator jointly end-to-end via representation alignment, overcoming the non-differentiable VQ index bottleneck.
Uses a dual read-out of the codebook assignment: a hard one-hot branch for next-token prediction loss (AR), and a differentiable soft branch carrying a representation-alignment loss that flows back to guide only the tokenizer.
The AR model steers its tokenizer toward an index distribution it can predict more easily - shifting the alignment burden from the tokenizer to the AR (the opposite of diffusion-side recipes).

Key Results:

Speeds up ImageNet gFID convergence by up to 10× relative to the LlamaGen-REPA baseline.
Learns markedly better patch-level and spatially-coherent features.
Generalizes across quantizers (VQVAE, LFQ, IBQ) and to text-to-image generation.

Applied Context: This means builders can train image generation models that converge an order of magnitude faster with better feature quality, without needing separate tokenizer pre-training - useful for any application relying on autoregressive visual generation (e.g., text-to-image, image editing, or video synthesis pipelines).