iREPA
⚠️ arxiv_id 待确认，论文名可能有误，请手动核实
📌 核心贡献
这篇论文探讨了表征对齐（REPA）在生成模型训练中的关键因素，即目标表征的全局语义信息与空间结构哪个对生成效果更重要。该研究通过将预训练视觉编码器的表征蒸馏到中间扩散特征，创新性地分析了两者在指导生成训练中的不同作用。它挑战了传统观念，即仅依靠更强的全局语义信息来优化生成模型。
📖 摘要
Representation alignment (REPA) guides generative training by distilling representations from a strong, pretrained vision encoder to intermediate diffusion features. We investigate a fundamental question: what aspect of the target representation matters for generation, its \textit{global} \revision{semantic} information (e.g., measured by ImageNet-1K accuracy) or its spatial structure (i.e. pairwise cosine similarity between patch tokens)? Prevalent wisdom holds that stronger global semantic performance leads to better generation as a target representation. To study this, we first perform a large-scale empirical analysis across 27 different vision encoders and different model scales. The results are surprising; spatial structure, rather than global performance, drives the generation performance of a target representation. To further study this, we introduce two straightforward modifications, which specifically accentuate the transfer of \emph{spatial} information. We replace the standard MLP projection layer in REPA with a simple convolution layer and introduce a spatial normalization layer for the external representation. Surprisingly, our simple method (implemented in 4 lines of code), termed iREPA, consistently improves convergence speed of REPA, across a diverse set of vision encoders, model sizes, and training variants (such as REPA, REPA-E, Meanflow, JiT etc). %, etc. Our work motivates revisiting the fundamental working mechanism of representational alignment and how it can be leveraged for improved training of generative models. The code and project page are available at https://end2end-diffusion.github.io/irepa
🔍 关键信息
| 字段 | 内容 |
|---|---|
| 机构 | |
| 发表 | 2025-12-11 |
| 分类 | cs.CV, cs.AI, cs.GR |
| 链接 | arXiv |