Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model
📌 核心贡献
Muddit提出了一个统一的离散扩散Transformer，旨在解决自回归和非自回归统一模型在跨模态任务中推理速度慢或泛化能力弱的问题。该模型实现了文本和图像模态的快速并行生成，是第一个从头开始训练的统一扩散模型，打破了传统文本到图像生成的限制。
📖 摘要
Unified generation models aim to handle diverse tasks across modalities — such as text generation, image generation, and vision-language reasoning — within a single architecture and decoding paradigm. Autoregressive unified models suffer from slow inference due to sequential decoding, and non-autoregressive unified models suffer from weak generalization due to limited pretrained backbones. We introduce Muddit, a unified discrete diffusion transformer that enables fast and parallel generation across both text and image modalities. Unlike prior unified diffusion models trained from scratch, Muddit integrates strong visual priors from a pretrained text-to-image backbone with a lightweight text decoder, enabling flexible and high-quality multimodal generation under a unified architecture. Empirical results show that Muddit achieves competitive or superior performance compared to significantly larger autoregressive models in both quality and efficiency. The work highlights the potential of purely discrete diffusion, when equipped with strong visual priors, as a scalable and effective backbone for unified generation.
🔍 关键信息
| 字段 | 内容 |
|---|---|
| 发表 | 2025-05-29 |
| 分类 | cs.LG, cs.CV |
| 链接 | arXiv |