UniM: A Unified Any-to-Any Interleaved Multimodal Benchmark
📌 核心贡献
这篇论文搞了个「万能多模态」基准测试【UniM】，就好比搭了个超级擂台。AI选手不仅要能同时理解和生成文本、图像、音频、视频、文档、代码甚至3D模型，而且这些模态还能像拼乐高一样随便组合、任意交错。更酷的是，他们还推出了一个叫 UniMA 的新模型，直接在这个擂台上刷新了多项纪录！目标是让多模态大语言模型（MLLMs）能像人一样，看到什么、听到什么，都能理解并做出反应，还能以任何形式输出。
📖 摘要
In real-world multimodal applications, systems usually need to comprehend arbitrarily combined and interleaved multimodal inputs from users, while also generating outputs in any interleaved multimedia form. This capability defines the goal of any-to-any interleaved multimodal learning under a unified paradigm of understanding and generation, posing new challenges and opportunities for advancing Multimodal Large Language Models (MLLMs). To foster and benchmark this capability, this paper introduces the UniM benchmark, the first Unified Any-to-Any Interleaved Multimodal dataset. UniM contains 31K high-quality instances across 30 domains and 7 representative modalities: text, image, audio, video, document, code, and 3D, each requiring multiple intertwined reasoning and generation capabilities. We further introduce the UniM Evaluation Suite, which assesses models along three dimensions: Semantic Correctness & Generation Quality, Response Structure Integrity, and Interleaved Coherence. In addition, we propose UniMA, an agentic baseline model equipped with traceable reasoning for structured interleaved generation. Comprehensive experiments demonstrate the difficulty of UniM and highlight key challenges and directions for advancing unified any-to-any multimodal intelligence. The project page is https://any2any-mllm.github.io/unim.
🔍 关键信息
| 字段 | 内容 |
|---|---|
| 发表 | 2026-03-05 |
| 机构 | |
| 分类 | cs.CV |
| 链接 | arXiv |