Diffusion / Code / Training
GEAR: Guided End-to-End AutoRegression for Image Synthesis
** Bin Lin, Zheyuan Liu, Chenguo Lin, Sixiang Chen, Yunyang Ge, Yunlong Lin, Jianwei Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Li Yuan
GEAR: Guided End-to-End AutoRegression for Image Synthesis
Authors: Bin Lin, Zheyuan Liu, Chenguo Lin, Sixiang Chen, Yunyang Ge, Yunlong Lin, Jianwei Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Li Yuan
arXiv ID: 2606.32039
Problem: Standard two-stage visual generative models (tokenizer → frozen generator) decouple training, leaving the tokenizer unaware of what the generator finds easy or hard to model.
Key Methodology:
- Trains a VQ tokenizer and an autoregressive (AR) generator jointly end-to-end via representation alignment, overcoming the non-differentiable VQ index bottleneck.
- Uses a dual read-out of the codebook assignment: a hard one-hot branch for next-token prediction loss (AR), and a differentiable soft branch carrying a representation-alignment loss that flows back to guide only the tokenizer.
- The AR model steers its tokenizer toward an index distribution it can predict more easily - shifting the alignment burden from the tokenizer to the AR (the opposite of diffusion-side recipes).
Key Results:
- Speeds up ImageNet gFID convergence by up to 10× relative to the LlamaGen-REPA baseline.
- Learns markedly better patch-level and spatially-coherent features.
- Generalizes across quantizers (VQVAE, LFQ, IBQ) and to text-to-image generation.
Applied Context: This means builders can train image generation models that converge an order of magnitude faster with better feature quality, without needing separate tokenizer pre-training - useful for any application relying on autoregressive visual generation (e.g., text-to-image, image editing, or video synthesis pipelines).