Ming-Lite-Uni: Advancements in Unified Architecture for Natural Multimodal Interaction
📌 核心贡献
Ming-Lite-Uni引入了一个开源多模态框架，包含新设计的统一视觉生成器和原生多模态自回归模型，专为统一视觉和语言而定制。该项目提供了集成MetaQueries和M2-omni框架的开源实现，并引入了新型多尺度可学习令牌和多尺度表示对齐策略，使得原生多模态自回归模型能够执行文本到图像生成和基于指令的理解。
📖 摘要
We introduce Ming-Lite-Uni, an open-source multimodal framework featuring a newly designed unified visual generator and a native multimodal autoregressive model tailored for unifying vision and language. Specifically, this project provides an open-source implementation of the integrated MetaQueries and M2-omni framework, while introducing the novel multi-scale learnable tokens and multi-scale representation alignment strategy. By leveraging a fixed MLLM and a learnable diffusion model, Ming-Lite-Uni enables native multimodal AR models to perform both text-to-image generation and instruction based image editing tasks, expanding their capabilities beyond pure visual understanding. Our experimental results demonstrate the strong performance of Ming-Lite-Uni and illustrate the impressive fluid nature of its interactive process. All code and model weights are open-sourced to foster further exploration within the community. Notably, this work aligns with concurrent multimodal AI milestones - such as ChatGPT-4o with native image generation updated in March 25, 2025 - underscoring the broader significance of unified models like Ming-Lite-Uni on the path toward AGI. Ming-Lite-Uni is in alpha stage and will soon be further refined.
🔍 关键信息
| 字段 | 内容 |
|---|---|
| 发表 | 2025-05-05 |
| 分类 | cs.CV |
| 链接 | arXiv |