JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation
📌 核心贡献
JanusFlow提出了一个强大的框架，通过将自回归语言模型与修正流（Rectified Flow）集成，在一个模型中统一了图像理解和生成。该研究发现修正流可以简单地在大型语言模型框架内进行训练，无需复杂的架构修改，并通过解耦理解和生成编码器进一步提升了统一模型的性能。
📖 摘要
We present JanusFlow, a powerful framework that unifies image understanding and generation in a single model. JanusFlow introduces a minimalist architecture that integrates autoregressive language models with rectified flow, a state-of-the-art method in generative modeling. Our key finding demonstrates that rectified flow can be straightforwardly trained within the large language model framework, eliminating the need for complex architectural modifications. To further improve the performance of our unified model, we adopt two key strategies: (i) decoupling the understanding and generation encoders, and (ii) aligning their representations during unified training. Extensive experiments show that JanusFlow achieves comparable or superior performance to specialized models in their respective domains, while significantly outperforming existing unified approaches across standard benchmarks. This work represents a step toward more efficient and versatile vision-language models.
🔍 关键信息
| 字段 | 内容 |
|---|---|
| 发表 | 2024-11-12 |
| 分类 | cs.CV, cs.AI |
| 链接 | arXiv |