UniData: Universal Multimodal Instruction Generation Pipeline

# UniData: Universal Multimodal Instruction Generation Pipeline
Source: [https://arxiv.org/html/2610.11363](https://arxiv.org/html/2610.11363)
Jiaqi Tang††thanks:Equal contribution\.Yi\-Feng Wu11footnotemark:1Affiliation:ATH, Alibaba GroupYuting Zhang11footnotemark:1Affiliation:The Hong Kong University of Science and Technology \(Guangzhou\)Hao LuAffiliation:The Hong Kong University of Science and Technology \(Guangzhou\)Bowen FuAffiliation:Northwestern Polytechnical UniversityQing\-Guo ChenAffiliation:ATH, Alibaba GroupXiaogang XuAffiliation:Zhejiang University\{jtang092, cqf\}@ust\.hk,yingcongchen@hkust\-gz\.edu\.cnYuwei HuAffiliation:ATH, Alibaba GroupShiyin LuAffiliation:ATH, Alibaba GroupWei WeiAffiliation:Northwestern Polytechnical UniversityLei ZhangAffiliation:Northwestern Polytechnical UniversityZhao XuAffiliation:ATH, Alibaba GroupWeihua LuoAffiliation:ATH, Alibaba GroupQifeng Chen††thanks:Corresponding authors\.Affiliation:The Hong Kong University of Science and TechnologyYing\-Cong Chen22footnotemark:2Affiliation:The Hong Kong University of Science and Technology \(Guangzhou\)

###### Abstract

Multimodal Large Language Models \(MLLMs\) are increasingly being applied in a wider range of real\-world scenarios\. However, due to the substantial labor cost, creating high\-quality multimodal instruction datasets for MLLMs remains a significant challenge\. Although some methods propose to generate instruction data, they often face limitations in modality support and struggle with generating multi\-round instructions\. To address these problems, we introduce UniData, a universal instruction generation pipeline, to transform simple user requirements into multi\-round, multimodal instructions\. Specifically, UniData first expands user requirements into multiple diverse events\. Using these events, UniData then integrates an any\-to\-any large model for multimodal instruction generation\. Finally, UniData enhances data quality by correcting irrelevant and redundant flows during inference, leveraging correlations between instruction rounds\. To train this pipeline, we also build UniDataset, a dataset comprising20,00020,000entries across nine modalities for improved multimodal generation\. Our experiments demonstrate that UniData achieves SOTA performance in data quality and can also enhance the understanding and generation capabilities of other multimodal models\.

## 1Introduction

> “In God we trust; all others must bring data\.”

— W\. Edwards Deming

![Refer to caption](https://arxiv.org/html/2610.11363v1/Method.png)Figure 1:Usage of UniData\.\(I\)Users only input one simple requirement, combined with random emotions and occasions to create diverse events\.\(II\)Each diverse event will generate one multimodal instruction\.\(III\)The generated multimodal instructions include various modalities\.Recently, Multimodal Large Language Models \(MLLMs\) have been extensively developed across various fields[Cui et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib40);[Li et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib43)\. However, to propel MLLMs toward fulfilling real\-world requirements, data scarcity has gradually become a significant issue[Liu et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib39);[Müller and Unay \(2017\)](https://arxiv.org/html/2610.11363#bib.bib16);[Yin et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib32);[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34);[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35)\.

To acquire high\-quality data, most previous methods have relied on manual collection and annotation[Li et al\. \(2017\)](https://arxiv.org/html/2610.11363#bib.bib15), which consequently demands substantial human resources[Tang et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib41);[Liu et al\. \(2024b\)](https://arxiv.org/html/2610.11363#bib.bib14);[Fang et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib13)\. While some approaches[Li et al\. \(2023b\)](https://arxiv.org/html/2610.11363#bib.bib42);[Liu et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib39)have started utilizing GPT[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28)for automated or semi\-automated data generation to alleviate labor costs, most of them are designed for one specific scenario and cannot generalize to broader applications\.

In this paper, we propose UniData, a universal multimodal instruction generation pipeline\. As shown in Figure[1](https://arxiv.org/html/2610.11363#S1.F1), by simply inputting basic requirements, users can finally obtain large\-scale multimodal instruction datasets\.

While some solutions have attempted to generate instructions in a universal setting, such as Self\-Instruct[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35)and VIGC[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34), these methods are primarily restricted to language modality \(as shown in Figure[2](https://arxiv.org/html/2610.11363#S1.F2)\(A\)\(B\)\)\. Multimodal Self\-Instruct[Zhang et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib33)attempts to generate instructions with corresponding abstract images, but its capabilities are limited to this \(as shown in Figure[2](https://arxiv.org/html/2610.11363#S1.F2)\(C\)\)\. Furthermore, all of these methods provide limited support for multi\-round instruction generation\.

To address the aforementioned problems, we first focus on enhancing data diversity by expanding user keyword requirements into a series of potential instructional events, as described in Section[4\.1](https://arxiv.org/html/2610.11363#S4.SS1)\. Next, to improve multimodal support, we propose an any\-to\-any large model that assists users in generating multi\-round, multimodal instructions iteratively, as described in Section[4\.2](https://arxiv.org/html/2610.11363#S4.SS2)\. Finally, to ensure data quality in a multi\-round context, we correct the instruction flow by assessing the correlation between instructions from previous and current rounds\. This process effectively minimizes redundancy and irrelevance during inference \(Section[4\.3](https://arxiv.org/html/2610.11363#S4.SS3)\)\.

To address the limitations in quantity, quality, and copyright issues of existing data \(in Section[3](https://arxiv.org/html/2610.11363#S3)\), we also construct UniDataset to support the training of UniData\. This dataset comprises 20,000 entries and incorporates nine distinct modalities on both the input and output sides\. These modalities include<language\>,<image\>,<music\>,<emoji\>,<code\>,<map\>,<link\>,<math\>, and<QR code\>\(see Section[3\.1](https://arxiv.org/html/2610.11363#S3.SS1)\)\. The inclusion of these diverse modalities is intended to enhance our data generation pipeline, enabling the creation of more comprehensive multimodal instructions and thereby improving overall versatility\.

Experimental results demonstrate that the quality of our generated data surpasses that of other generation pipelines on all four dimensions we evaluate: reasonableness, clarity, detail, and relevance\. Furthermore, our generated data can also enhance the understanding and generation capabilities of other multimodal large language models\. Our contribution is shown as follows:

- •We propose UniData, a universal multimodal instruction generation pipeline that can create multi\-round multimodal instructions from one simple requirement\.
- •We propose UniDataset, a multimodal instruction dataset comprising20,00020,000instructions with nine distinct modalities in both input and output\. UniDataset addresses existing limitations in data quantity, quality, and copyright status, and provides a robust data foundation for training UniData or other any\-to\-any models\.
- •Through comprehensive experiments, we demonstrate that UniData surpasses existing pipelines in data quality, achieving SOTA performance in reasonableness, clarity, detail, and relevance\. Additionally, the generated data can improve the understanding and generation capabilities of other multimodal models\.

Figure 2:Comparison of different instruction generation frameworks\.\(A\)Self\-Instruct[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35)can only support language modality\.\(B\)VIGC[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34)integrates visual modality into its generation\.\(C\)Multimodal Self\-Instruct[Zhang et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib33)can output abstract images\.\(D\)Ours \(UniData\) can freely understand and generate different modalities in instructions\.
## 2Related Work

#### Multimodal Large Language Models

Multimodal Large Language Models can understand multimodal information[Liang et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib44)by integrating various modalities, including but not limited to text, images, audio, and video\. Due to their versatility, they have been widely applied in diverse fields, such as embodied intelligence[Li et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib43), video understanding[Tang et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib41), and autonomous driving[Cui et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib40)\.

Most previous multimodal models[Liu et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib39);[Li et al\. \(2023b\)](https://arxiv.org/html/2610.11363#bib.bib42);[Lu et al\. \(2024b\)](https://arxiv.org/html/2610.11363#bib.bib1)were designed to understand multimodal information, yet their output is limited to language only\. Recently, SEED\-X[Ge et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib38)and Show\-o[Xie et al\. \(2025\)](https://arxiv.org/html/2610.11363#bib.bib37)have begun to explore the integrated capabilities of image understanding and generation\. Additionally, NExT\-GPT[Wu et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib36)has advanced by integrating more modalities, including images, audio, and video, for unified understanding and generation\.

Nonetheless, training these models encounters substantial challenges, particularly regarding data availability\. For example, NExT\-GPT[Wu et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib36)is unable to provide training data due to copyright constraints\. Furthermore, Show\-o[Xie et al\. \(2025\)](https://arxiv.org/html/2610.11363#bib.bib37)is limited to training in an end\-to\-end way due to the lack of an appropriate dataset\. Motivated by these challenges, we propose UniData, which is designed to generate multimodal instructions to facilitate the model training process and potentially support further applications\.

#### Instruction Generation

The scarcity of training data impacts the performance of Multimodal Large Language Models, leading to increased interest in how to generate high\-quality instruction data\. Self\-Instruct[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35)pioneered the generation of diverse instructions using a small set of seed tasks, but it focuses only on the language modality \(Figure[2](https://arxiv.org/html/2610.11363#S1.F2)\(A\)\)\. To overcome these limitations, VIGC[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34)integrates visual understanding into the instruction generation process \(Figure[2](https://arxiv.org/html/2610.11363#S1.F2)\(B\)\)\. Besides, Multimodal Self\-Instruct[Zhang et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib33)addresses the problem of abstract image synthesis in instruction generation, utilizing synthetic tools to create both diagrams and corresponding text \(Figure[2](https://arxiv.org/html/2610.11363#S1.F2)\(C\)\)\. However, these methods still support limited multimodal capabilities\.

To address these constraints, we propose UniData, a universal multimodal instruction generation pipeline designed to both understand and generate nine types of multimodal information \(Figure[2](https://arxiv.org/html/2610.11363#S1.F2)\(D\)\)\. This approach broadens the applicability of multimodal instruction generation compared to previous research\.

![Refer to caption](https://arxiv.org/html/2610.11363v1/datageneration.png)Figure 3:Procedure of data construction\.Stage \(I\)prepares diverse events by random topic pairs\.Stage \(II\)leverages these events to produce multi\-round instructions\.Stage \(III\)forecasts eight types of multimodal indicator tokens and integrates these into instructions\.Stage \(IV\)uses a toolkit to generate multimodal content\.

## 3Dataset Construction

To train UniData, a dataset containing both multimodal inputs and outputs is essential\. However, datasets meeting these criteria are scarce\. As summarised in Table[10](https://arxiv.org/html/2610.11363#A1.T10)\(Appendix[A\.3](https://arxiv.org/html/2610.11363#A1.SS3)\), the majority of existing datasets are confined to a single modality, especially for the output\. While NExT\-GPT \(MosIT\)[Wu et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib36)and GenHowTo[Souček et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib29)have developed some initial datasets encompassing multimodal inputs and outputs, they are hindered by a restricted range of modalities, limited instruction rounds, and a small data size\. Furthermore, copyright concerns limit their availability\.

To enrich our data resources, we have also developed a multimodal dataset named UniDataset \(theOursrow in Table[10](https://arxiv.org/html/2610.11363#A1.T10)\)\. This dataset supports inputs and outputs across nine modalities, thereby facilitating the training process of UniData\.

### 3\.1Procedure of Data Construction

#### Stage \(I\): Preparing Diverse Events

To accommodate data diversity, it is crucial to generate varied<events\>\(tasks\) for each multi\-round instruction\. Drawing inspiration from Self\-Instruct[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35), we first create aTopic Poolwith9090<topics\>spanning multiple domains, such as e\-commerce, lifestyle, and sports\. Then, we randomly pair two topics and input them into GPT\-4o[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28)to generate diverse<events\>, as shown in Figure[3](https://arxiv.org/html/2610.11363#S2.F3)\(I\)\. Compared to a single<topic\>, this approach introduces greater randomness \(\(902\)=4,005\\binom\{90\}\{2\}=4\{,\}005unique combinations\), thereby facilitating data diversity\.

#### Stage \(II\): Generating Multi\-Round Instructions

After obtaining<events\>\(tasks\), we utilize them to generate multi\-round instructions\. Considering other attributes of instructions, we also randomly choose different<rounds\>\(ranging from55to3030\),<emotion\>\(random proportions of seven distinct emotions\), and<occasion\>\(formal or informal\) to collectively guide GPT\-4o[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28)in generating diverse instructions, as shown in Figure[3](https://arxiv.org/html/2610.11363#S2.F3)\(II\)\. Ultimately, we generate20,00020,000multi\-round instructions\.

#### Stage \(III\): Forecasting Multimodal Integration

Our current multi\-round instructions are restricted to a single modality \(<language\>\)\. To generate multimodal content, we utilize GPT\-4o[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28)to predict potential multimodal content and incorporate the relevant indicators\.

Beyond the base<language\>modality, we forecast indicators for eight modalities:<image\>,<music\>,<emoji\>,<code\>,<map\>,<link\>,<math\>, and<QR code\>\. Additionally, we generate prompts that describe the corresponding multimodal content, as shown in Figure[3](https://arxiv.org/html/2610.11363#S2.F3)\(III\)\. For instance, we generate prompts for GPT\-4o[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28)for modalities such as<image\>,<code\>, and<math\>\. For<map\>and<link\>, we develop search queries intended for use with Google Search and Google Maps\.

#### Stage \(IV\): Generating Multimodal Content

In the final stage, we develop an advanced Toolkit for Generating Multimodal Content\. This toolkit includes GPT\-4o[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28), Google Search, Google Maps, DALL·E 3, MusicGen[Copet et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib19), and the Apple Emoji Archive to assist in producing multimodal content\. We leverage the generated prompts/content/queries to drive this toolkit, enabling the creation of multimodal content\.

### 3\.2Data Format and Statistics

Each UniDataset sample is generated by mapping a contextual triple to a multimodalinstructionthat contains nine typed slots, one per modality \(Eq\. \([1](https://arxiv.org/html/2610.11363#S3.E1)\)\)\. Slots are filled by dedicated generators \(Appendix[A\.2](https://arxiv.org/html/2610.11363#A1.SS2)\) conditioned on a modality\-specific cue \(a<prompt\>,<label\>,<query\>, or<content\>string\), while the<language\>slot carries the dialogue text itself\.

\(<event\>,<emotion\>,<occasion\>\)\\displaystyle\\bigl\(\\texttt\{<event\>\},\\,\\texttt\{<emotion\>\},\\,\\texttt\{<occasion\>\}\\bigr\)\(1\)→Generateinstruction​\{→<language\><prompt\>→<image\><prompt\>→<music\><label\>→<emoji\><prompt\>→<code\><query\>→<map\><query\>→<link\><prompt\>→<math\><content\>→<QR code\>\.\\displaystyle\\xrightarrow\{\\;\\text\{Generate\}\\;\}\\texttt\{\{instruction\}\}\\left\\\{\\begin\{array\}\[\]\{r@\{\\;\\to\\;\}l\}&\\texttt\{<language\>\}\\\\ \\texttt\{<prompt\>\}&\\texttt\{<image\>\}\\\\ \\texttt\{<prompt\>\}&\\texttt\{<music\>\}\\\\ \\texttt\{<label\>\}&\\texttt\{<emoji\>\}\\\\ \\texttt\{<prompt\>\}&\\texttt\{<code\>\}\\\\ \\texttt\{<query\>\}&\\texttt\{<map\>\}\\\\ \\texttt\{<query\>\}&\\texttt\{<link\>\}\\\\ \\texttt\{<prompt\>\}&\\texttt\{<math\>\}\\\\ \\texttt\{<content\>\}&\\texttt\{<QR\\,code\>\}\.\\end\{array\}\\right\.

## 4Methodology

Previous works, such as Multimodal Self\-Instruct[Zhang et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib33)and VIGC[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34), typically support only limited multimodal generation or understanding capabilities, which do not satisfy users’ practical needs\. In real\-world applications, users often prefer to provide only one or a few simple requirements to generate large volumes of high\-quality data\.

#### Overview

To address the aforementioned issues, we propose UniData, a pipeline for generating multimodal instructions\.Initially, based on user\-provided requirements, we develop an event generator that expands on the limited inputs to create diverse events in Section[4\.1](https://arxiv.org/html/2610.11363#S4.SS1)\.Next, to generate multimodal instructions, we construct a large model for multimodal understanding and generation in Section[4\.2](https://arxiv.org/html/2610.11363#S4.SS2)\.Finally, to mitigate potential biases during long\-term multi\-round instruction generation, we correct the instruction flow to filter out unreasonable data in Section[4\.3](https://arxiv.org/html/2610.11363#S4.SS3)\.

### 4\.1Expanding Diverse Events

Typically, users seek to generate one high\-quality instruction dataset by providing only a few keyword requirements \(K\), such that \(K→∑n=1NIn\\text\{K\}\\rightarrow\\sum\_\{n=1\}^\{N\}\\text\{I\}\_\{n\}\)\. However, if we treat oneKas a single<event\>\(T\), it becomes challenging to produce a diverse set of instructions\.

Therefore, we first need to expand the variety ofT, so we build the Diverse Event Generator \(𝐃𝐄𝐆\\mathbf\{DEG\}\) to expand diverse events, as shown in Figure[4](https://arxiv.org/html/2610.11363#S4.F4)and Eq\. \([2](https://arxiv.org/html/2610.11363#S4.E2)\), as

∑n=1NIn←\{T1,T2,…,TN\}=𝐃𝐄𝐆⁡\(K⊕S\),\\sum\_\{n=1\}^\{N\}\\text\{I\}\_\{n\}\\leftarrow\\\{\\text\{T\}\_\{1\},\\text\{T\}\_\{2\},\\ldots,\\text\{T\}\_\{N\}\\\}=\\mathbf\{DEG\}\(\\text\{K\}\\oplus\\text\{S\}\),\(2\)where⊕\\oplusis the concatenation operator\. The set\{T1,T2,…,TN\}\\\{\\text\{T\}\_\{1\},\\text\{T\}\_\{2\},\\ldots,\\text\{T\}\_\{N\}\\\}denotes the collection of<events\>, each corresponding to a distinctinstruction\(In\\text\{I\}\_\{n\}\)\. The termSrepresents stochastic factors, such as random emotion proportions, random occasions, and other random seeds, which collectively enhance the capability of𝐃𝐄𝐆⁡\(⋅\)\\mathbf\{DEG\}\(\\cdot\)to generate a broader spectrum of diverse events\.

### 4\.2Generating Multimodal Instructions

After obtaining diverse<events\>, the next step is to generate multimodal instructions\. Inspired by previous any\-to\-any models[Wu et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib36);[Xie et al\. \(2025\)](https://arxiv.org/html/2610.11363#bib.bib37), we develop the Multimodal Instruction Generator \(𝐌𝐈𝐆\\mathbf\{MIG\}\), a text\-centred generator coupled with modality\-specific realisation tools that can both understand and generate multimodal content, as shown in Figure[5](https://arxiv.org/html/2610.11363#S4.F5)and Eq\. \([3](https://arxiv.org/html/2610.11363#S4.E3)\),

InM=𝐌𝐈𝐆⁡\(Tn⊕∑m=1M−1Inm\),\\text\{I\}\_\{n\}^\{M\}=\\mathbf\{MIG\}\(\\text\{T\}\_\{n\}\\oplus\\sum\_\{m=1\}^\{M\-1\}\\text\{I\}\_\{n\}^\{m\}\),\(3\)where𝐌𝐈𝐆⁡\(⋅\)\\mathbf\{MIG\}\(\\cdot\)takes as input one eventTn\\text\{T\}\_\{n\}and previously generated multimodal instructions \(∑m=1M−1Inm\\sum\_\{m=1\}^\{M\-1\}\\text\{I\}\_\{n\}^\{m\}\), and predicts the next\-round multimodal instruction \(InM\\text\{I\}\_\{n\}^\{M\}\)\.

![Refer to caption](https://arxiv.org/html/2610.11363v1/M_Part1.png)Figure 4:Pipeline of Diverse Event Generator \(𝐃𝐄𝐆\\mathbf\{DEG\}\)\. Users can generate diverse events by inputting their needs\.Figure 5:Pipeline of Multimodal Instruction Generator \(𝐌𝐈𝐆\\mathbf\{MIG\}\)\. With justoneevent provided by the user,𝐌𝐈𝐆\\mathbf\{MIG\}can iteratively generatemulti\-round multimodalinstructions based on the context\.#### Multimodal Tokenization

To generate the next\-round instructions \(InM\\text\{I\}\_\{n\}^\{M\}\),𝐌𝐈𝐆\\mathbf\{MIG\}must understand the multimodal instruction from previous rounds \(∑m=1M−1Inm\\sum\_\{m=1\}^\{M\-1\}\\text\{I\}\_\{n\}^\{m\}\)\. Thus, it is necessary to convert the previous multimodal instructions into token embeddings within a unified feature space\.

Given that our training dataset, UniDataset, comprises nine distinct modalities, our tokenizing strategy \(𝐅⁡\(⋅\)\\mathbf\{F\}\(\\cdot\)\) is shown as Eq\. \([4](https://arxiv.org/html/2610.11363#S4.E4)\),

𝐅⁡\(I\)=\[𝐟text​\[IL,𝐂⁡\(IE⊕IC⊕IMP⊕ILK⊕IMT⊕IQ\)\]𝐟img​\(II\)𝐟mus​\(IM\)\],\\mathbf\{F\}\(\\text\{I\}\)=\\begin\{bmatrix\}\\mathbf\{f\}\_\{\\text\{text\}\}\[\\text\{I\}\_\{\\text\{L\}\},\\mathbf\{C\}\(\\text\{I\}\_\{\\text\{E\}\}\\oplus\\text\{I\}\_\{\\text\{C\}\}\\oplus\\text\{I\}\_\{\\text\{MP\}\}\\oplus\\text\{I\}\_\{\\text\{LK\}\}\\oplus\\text\{I\}\_\{\\text\{MT\}\}\\oplus\\text\{I\}\_\{\\text\{Q\}\}\)\]\\\\ \\mathbf\{f\}\_\{\\text\{img\}\}\(\\text\{I\}\_\{\\text\{I\}\}\)\\\\ \\mathbf\{f\}\_\{\\text\{mus\}\}\(\\text\{I\}\_\{\\text\{M\}\}\)\\end\{bmatrix\},\(4\)where𝐟x​\(⋅\),x∈\{text,img,mus\}\\mathbf\{f\}\_\{x\}\(\\cdot\),x\\in\\\{\\text\{text\},\\text\{img\},\\text\{mus\}\\\}represent the tokenizer and embedding layer/projection for different modalities \(Ix,x∈\{L,I,M,E,C,MP,LK,MT,Q\}\\text\{I\}\_\{x\},x\\in\\\{\\text\{L\},\\text\{I\},\\text\{M\},\\text\{E\},\\text\{C\},\\text\{MP\},\\text\{LK\},\\text\{MT\},\\text\{Q\}\\\}\)\.

For modalities that are not inherently linguistic but can be represented as text \(Ix,x∈\{E,C,MP,LK,MT,Q\}\\text\{I\}\_\{x\},x\\in\\\{\\text\{E\},\\text\{C\},\\text\{MP\},\\text\{LK\},\\text\{MT\},\\text\{Q\}\\\}\), we can easily convert these into textual representations \(prompt/label/query/content\) by the formatter \(𝐂⁡\(⋅\)\\mathbf\{C\}\(\\cdot\)\)\. Then, these modalities can be combined with the language modality \(IL\\text\{I\}\_\{\\text\{L\}\}\) and tokenized directly by using the default LLaMA Tokenizer[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib22)\(𝐟text​\(⋅\)\\mathbf\{f\}\_\{\\text\{text\}\}\(\\cdot\)\)\. The tokenized outputs are subsequently passed through a pre\-trained embedding layer to generate token embeddings\.

For<image\>\(II\\text\{I\}\_\{\\text\{I\}\}\), we utilize the visual tokenizer and projection \(𝐟img​\(⋅\)\\mathbf\{f\}\_\{\\text\{img\}\}\(\\cdot\)\) from Ovis\-1\.5[Lu et al\. \(2024b\)](https://arxiv.org/html/2610.11363#bib.bib1)to convert images into visual embeddings\. For<music\>\(IM\\text\{I\}\_\{\\text\{M\}\}\), drawing on previous research[Gong et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib27);[Chu et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib26);[Du et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib25);[Verma \(2024\)](https://arxiv.org/html/2610.11363#bib.bib24), we integrate a pre\-trained Whisper[Radford et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib23)as the music tokenizer, followed by a Multi\-Layer Perceptron \(MLP\) that projects into the desired embedding space\.

#### Multimodal Indicator

To distinguish different modalities, we introduce new special tokens to indicate the beginning and end of each modality\. As shown in Figure[5](https://arxiv.org/html/2610.11363#S4.F5), the start of a modality is indicated by<\|modality\|\>, where

modality∈\{image,music,emoji,code,\\displaystyle\\texttt\{modality\}\\in\\\{\\texttt\{image\},\\texttt\{music\},\\texttt\{emoji\},\\texttt\{code\},\(5\)map,link,math,qrcode\},\\displaystyle\\texttt\{map\},\\texttt\{link\},\\texttt\{math\},\\texttt\{qrcode\}\\\},and the end of the modality is indicated by<\|modality\_end\|\>\. This strategy allows the model to identify the position of multimodal insertion\.

#### Architecture of Large Language Model

After aligning the multimodal signals into a unified embedding space, we select LLaMA\-3[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib22)as the main backbone of our pipeline\. We also use the causal attention mechanism to sequentially predict the next token\.

#### Multimodal Generation

To generate multimodal instructions, we first need to establish multimodal token labels during training\. Although most modalities can be represented as text during tokenization, the image and music modalities lack corresponding text representations\. However, we find that during data construction \(as described in Eq\. \([1](https://arxiv.org/html/2610.11363#S3.E1)\)\), we obtain<prompts\>\(PI\\text\{P\}\_\{\\text\{I\}\}andPM\\text\{P\}\_\{\\text\{M\}\}\) for generating images and music\. Therefore, we can use these prompts as tokens to represent these two modalities discretely\. Eq\. \([6](https://arxiv.org/html/2610.11363#S4.E6)\) shows the ground truth label \(𝐕\\mathbf\{V\}\),

𝐕=\\displaystyle\\mathbf\{V\}=\{VL,\(VE,VC,VMP,VLK,VMT,VQ\),VI,VM\},\\displaystyle\\\{\\text\{V\}\_\{\\text\{L\}\},\(\\text\{V\}\_\{\\text\{E\}\},\\text\{V\}\_\{\\text\{C\}\},\\text\{V\}\_\{\\text\{MP\}\},\\text\{V\}\_\{\\text\{LK\}\},\\text\{V\}\_\{\\text\{MT\}\},\\text\{V\}\_\{\\text\{Q\}\}\),\\text\{V\}\_\{\\text\{I\}\},\\text\{V\}\_\{\\text\{M\}\}\\\},\\ \\\(6\)whereVI=𝐟text​\(PI\),VM=𝐟text​\(PM\),\\displaystyle\\text\{where\}\\quad\\text\{V\}\_\{\\text\{I\}\}=\\mathbf\{f\}\_\{\\text\{text\}\}\(\\text\{P\}\_\{\\text\{I\}\}\),\\quad\\text\{V\}\_\{\\text\{M\}\}=\\mathbf\{f\}\_\{\\text\{text\}\}\(\\text\{P\}\_\{\\text\{M\}\}\),and𝐟text​\(⋅\)\\mathbf\{f\}\_\{\\text\{text\}\}\(\\cdot\)is the tokenizer of LLaMA\-3[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib22)andVx,x∈\{L,I,M,E,C,MP,LK,MT,Q\}\\text\{V\}\_\{x\},x\\in\\\{\\text\{L\},\\text\{I\},\\text\{M\},\\text\{E\},\\text\{C\},\\text\{MP\},\\text\{LK\},\\text\{MT\},\\text\{Q\}\\\}are the labels of different modalities\. Our goal is to predict the next token \(𝐕k\\mathbf\{V\}^\{k\}\) given the previous tokens \(\{𝐕i\}i=1k−1\\\{\\mathbf\{V\}^\{i\}\\\}\_\{i=1\}^\{k\-1\}\) and the model parameters \(Θ\\Theta\), as Eq\. \([7](https://arxiv.org/html/2610.11363#S4.E7)\),

ℒ=−∑klogp\(𝐕k\|\{𝐕i\}i=1k−1;Θ\)\.\\mathcal\{L\}=\-\\sum\_\{k\}\\log p\(\\mathbf\{V\}^\{k\}\|\\\{\\mathbf\{V\}^\{i\}\\\}\_\{i=1\}^\{k\-1\};\\Theta\)\.\(7\)

#### Multimodal Detokenization

We use the default LLaMA\-3 text detokenizer[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib22)to convert all modalities into their text representations\. For<code\>,<map\>,<link\>, and<math\>, they can be directly detokenized into their original information\. For<image\>, we use Stable Diffusion 3[Esser et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib20)to transform the text into the corresponding image\. For<music\>, we employ MusicGen[Copet et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib19)to convert the text into the corresponding music\. For<emoji\>, we perform a similarity search of the output text to retrieve the corresponding emoji from the emoji archive\. For<QR code\>, we can directly convert its content into a QR code by qrcode111[https://pypi\.org/project/qrcode/](https://pypi.org/project/qrcode/)\.

### 4\.3Correcting Instruction Flow

Although now we can generate multimodal instructions by𝐌𝐈𝐆\\mathbf\{MIG\}, during inference, it may also encounter common issues like repetitive generation or hallucinations[Xu et al\. \(2022\)](https://arxiv.org/html/2610.11363#bib.bib18);[Huang et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib17)\. Therefore, cleaning and filtering data during long\-term inference is crucial for maintaining data quality\.

Figure 6:Inference chain for correcting instruction flow\.We design an inference chain in Figure[6](https://arxiv.org/html/2610.11363#S4.F6)\. We compare the newly generated one\-round instruction \(InM\\text\{I\}\_\{n\}^\{M\}\) against the previouskk\-round instructions \(∑m=M−kM−1Inm\\sum\_\{m=M\-k\}^\{M\-1\}\\text\{I\}\_\{n\}^\{m\}\) and apply the following three rules:

1. 1\.Repetitive Generation: If the new instruction is close to the previous ones, increase theRepetition Penalty\(↑\\uparrow\) to reduce repetition\.
2. 2\.Irrelevant Generation: If the new instruction is unrelated to the previous ones, raise theTemperature\(↑\\uparrow\) to introduce randomness, thereby sampling more alternatives\.
3. 3\.High\-Quality Generation: If the instruction is neither repetitive nor irrelevant, we regard it as high\-quality and proceed to the generation of the next round\.

To assess deviation, we use cosine similarity, as Eq\. \([8](https://arxiv.org/html/2610.11363#S4.E8)\),

𝐑=1k​∑m=M−kM−1cos\_s​\(InM,Inm\)=1k​∑m=M−kM−1InM⋅Inm\|InM\|​\|Inm\|\.\\mathbf\{R\}=\\frac\{1\}\{k\}\\sum\_\{m=M\-k\}^\{M\-1\}\\texttt\{cos\\\_s\}\(\\text\{I\}\_\{n\}^\{M\},\\text\{I\}\_\{n\}^\{m\}\)=\\frac\{1\}\{k\}\\sum\_\{m=M\-k\}^\{M\-1\}\\frac\{\\text\{I\}\_\{n\}^\{M\}\\cdot\\text\{I\}\_\{n\}^\{m\}\}\{\|\\text\{I\}\_\{n\}^\{M\}\|\\left\|\\text\{I\}\_\{n\}^\{m\}\\right\|\}\.\(8\)If𝐑<α1\\mathbf\{R\}<\\alpha\_\{1\}, it indicates “Irrelevant Generation”\. If𝐑\>α2\\mathbf\{R\}\>\\alpha\_\{2\}, it indicates “Repetitive Generation”\. Ifα2\>𝐑\>α1\\alpha\_\{2\}\>\\mathbf\{R\}\>\\alpha\_\{1\}, it indicates “High\-Quality Generation”\. Empirically, we currently setα1=0\.05\\alpha\_\{1\}=0\.05andα2=0\.4\\alpha\_\{2\}=0\.4to minimize errors\. Users can also adjust these values dynamically to maintain instruction quality and relevance based on their needs\.

Table 1:Quantitative performance in data quality\.Redindicates the best performance\.\(\#Others includes all other modalities \(\#Emoji, \#Code, \#Math, \#Map, \#Link, \#QR Code\)\)\.MethodBackbonesGPT\-Guided Metrics\(↑\)\(\\mathbf\{\\uparrow\}\)[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28)Avg\. \#Multimodalities\(↑\)\(\\mathbf\{\\uparrow\}\)Avg\. \#RoundsReasonablenessClarityDetailRelevance\#Image\#Music\#OthersSelf\-Instruct[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35)GPT\-4[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28)0\.6240\.4440\.5940\.520\-\-\-≈\\approx3VIGC[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34)Vicuna 7B[Chiang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib11)0\.4600\.3590\.5340\.569\-\-\-≈\\approx1Ours \(UniData\)LLaMA\-3 7B[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib22)0\.6610\.5270\.6670\.6821\.920\.747\.37≈\\approx17\.5

## 5Experiments

In this section, we compare our instruction generation pipeline with others to evaluate instruction quality \(see Section[5\.1](https://arxiv.org/html/2610.11363#S5.SS1)\)\. Besides, we assess whether the generated instructions can improve multimodal generation and understanding performance after fine\-tuning \(see Section[5\.2](https://arxiv.org/html/2610.11363#S5.SS2)\)\. Finally, we demonstrate the effectiveness of expanding events and correcting instruction flow \(see Section[5\.3](https://arxiv.org/html/2610.11363#S5.SS3)\)\.

Furthermore, we conduct a user study to assess the quality of the generated instructions, demonstrate the adaptability and balance of multimodal content to support different needs, validate performance on out\-of\-domain data, and provide additional qualitative results in the Appendix\.

#### Training & Testing

During the pre\-training phase, we adhere to the training strategy and dataset outlined in Ovis\-1\.5[Lu et al\. \(2024b\)](https://arxiv.org/html/2610.11363#bib.bib1), enabling the model to acquire initial multimodal understanding ability\. In the fine\-tuning phase, we keep the parameters of the visual and music tokenizers fixed and randomly select90%90\\%of UniDataset for model fine\-tuning, reserving the remaining10%10\\%for testing\.

### 5\.1Instruction Quality Evaluation

#### Baselines

We select two representative instruction generation frameworks, Self\-Instruct[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35)and VIGC[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34), for comparison on our test data\. In Self\-Instruct[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35), due to the unavailability of GPT\-3, we replace it with the latest GPT\-4[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28)\. A discussion of more recent instruction\-data generation methods and how they compare in scope is provided in Appendix[C\.7](https://arxiv.org/html/2610.11363#A3.SS7)\.

#### Evaluation Metrics

Following previous work[Li et al\. \(2023b\)](https://arxiv.org/html/2610.11363#bib.bib42);[Tang et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib41), we use GPT\-Guided metrics to evaluate the quality differences between generated results and references\. Our goal is not to precisely match the ground truth but to produce high\-quality instructions\. We focus on four key metrics:1Reasonableness\- assesses if the instruction is logical and appropriate;2Clarity\- ensures the instructions are clear and unambiguous;3Detail\- evaluates whether the instructions include more detailed information;4Relevance\- checks that the content is related to the desired outcome\.

![Refer to caption](https://arxiv.org/html/2610.11363v1/QualityE.png)Figure 7:Qualitative comparison in data quality\.
#### Performance Analysis

Table[1](https://arxiv.org/html/2610.11363#S4.T1)presents the quantitative results of our data quality\. UniData surpasses both baselines on all four dimensions, with the largest margin on Clarity and Relevance\. Furthermore, UniData supports more rounds of instruction generation and can produce rich multimodal content within instructions\. Figure[7](https://arxiv.org/html/2610.11363#S5.F7)shows the qualitative performance in data quality\. Compared to other baselines that are limited to generating textual content, UniData can generate high\-quality multimodal instructions\.

### 5\.2Post Fine\-Tuning Assessment

Referring to previous research[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34);[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35), we also evaluatethe effectiveness of the generated multimodal instructions\(\+ℐUniData\\mathcal\{I\}\_\{\\texttt\{UniData\}\}\) by fine\-tuning other multimodal models\.

Table 2:Improvement of music understanding\.BLEU \(↑\\uparrow\)BLEU\-4 \(↑\\uparrow\)METEOR \(↑\\uparrow\)ROUGE \(↑\\uparrow\)BERT \(↑\\uparrow\)Baseline0\.26750\.20190\.32320\.33760\.8932Baseline\+ℐUniData\\mathcal\{I\}\_\{\\texttt\{UniData\}\}0\.26810\.20760\.33300\.34960\.8936

Table 3:Music generation\.FAD \(↓\\downarrow\)CLAP \(↑\\uparrow\)KL \(↓\\downarrow\)Baseline16\.690\.31382\.1358Baseline\+ℐUniData\\mathcal\{I\}\_\{\\texttt\{UniData\}\}16\.510\.31922\.1038

Table 4:Improvement on different modalities\.ModalityCode\(↑\)\(\\mathbf\{\\uparrow\}\)Math\(↑\)\(\\mathbf\{\\uparrow\}\)Text\-based\(↑\)\(\\mathbf\{\\uparrow\}\)Reasoning\(↑\)\(\\mathbf\{\\uparrow\}\)BenchmarkMBPPMathVistaMMLU\-ProMathVisionBaseline20\.859\.420\.312\.8Baseline \+ℐUniData\\mathcal\{I\}\_\{\\texttt\{UniData\}\}21\.360\.121\.913\.0

Table 5:Image understanding/generation\.Image UnderstandingImageGeneration\(↑\\uparrow\)Single \(↑\\uparrow\)Multi \(↑\\uparrow\)Interleaved \(↑\\uparrow\)Baseline26\.19%37\.73%19\.42%24\.43%Baseline \+ℐUniData\\mathcal\{I\}\_\{\\texttt\{UniData\}\}27\.15%39\.09%35\.25%25\.90%

Table 6:Improvement on visual understanding\.BenchmarkMMMU\(↑\)\(\\uparrow\)ChartQA\(↑\)\(\\uparrow\)TextVQA\(↑\)\(\\uparrow\)Baseline35\.817\.458\.9Baseline \+ℐUniData\\mathcal\{I\}\_\{\\texttt\{UniData\}\}36\.417\.960\.1

Table 7:Quantitative performance in ablation study\.Reasonableness\(↑\)\(\\mathbf\{\\uparrow\}\)Clarity\(↑\)\(\\mathbf\{\\uparrow\}\)Detail\(↑\)\(\\mathbf\{\\uparrow\}\)Relevance\(↑\)\(\\mathbf\{\\uparrow\}\)\#Ins\.w/o Expansion0\.5570\.5180\.6070\.550kkw/o Correction0\.6480\.4890\.6100\.654N×kN\\times kOurs0\.6610\.5270\.6670\.682N×kN\\times k

#### Visual Understanding

We utilize a suite of benchmarks \(MMMU[Yue et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib4), ChartQA[Masry et al\. \(2022\)](https://arxiv.org/html/2610.11363#bib.bib3), and TextVQA[Singh et al\. \(2019\)](https://arxiv.org/html/2610.11363#bib.bib2)\) to evaluate the effectiveness of our generated data in visual reasoning tasks\. As detailed in Table[7](https://arxiv.org/html/2610.11363#S5.T7), the integration of our data improves the performance of Ovis2\-1B[Lu et al\. \(2024b\)](https://arxiv.org/html/2610.11363#bib.bib1)in these benchmarks\.

#### Image Understanding and Generation

We evaluate image understanding and generation capabilities using the SEED\-X model[Ge et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib38)and the SEED\-Bench\-2 benchmark[Li et al\. \(2023a\)](https://arxiv.org/html/2610.11363#bib.bib10)\. Our data improves both Image\-to\-Text and Text\-to\-Image tasks\. As shown in Table[7](https://arxiv.org/html/2610.11363#S5.T7), the gain is largest on the interleaved setting \(19\.42%→35\.25%19\.42\\%\\\!\\rightarrow\\\!35\.25\\%\), with smaller gains on the single\- and multi\-image settings\. Additionally, in the domain of image generation, our dataset helps the model produce relevant images based on textual inputs\.

#### Music Understanding and Generation

We utilize the MuMu\-LLaMA model[Liu et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib12)to assess advancements in music understanding and generation\. As illustrated in Tables[7](https://arxiv.org/html/2610.11363#S5.T7)and[7](https://arxiv.org/html/2610.11363#S5.T7), our generated instructions improve both domains, although the margins are small\. In the realm of music understanding \(Table[7](https://arxiv.org/html/2610.11363#S5.T7)\), all five metrics move in the expected direction, with the clearest gains on METEOR and ROUGE\. In music generation \(Table[7](https://arxiv.org/html/2610.11363#S5.T7)\), the fine\-tuned model exhibits decreased FAD and KL scores alongside an elevated CLAP score, reflecting more coherent and high\-quality music synthesis from textual prompts\.

#### Code Generation

We evaluate the impact of our generated data on code modality by enhancing the code generation capabilities of a large language model, LLaMA 2[Touvron et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib21), using the Mostly Basic Python Programming \(MBPP\) Benchmark[Austin et al\. \(2021\)](https://arxiv.org/html/2610.11363#bib.bib8)\. As presented in Table[7](https://arxiv.org/html/2610.11363#S5.T7), our instructions improve performance\.

#### Mathematical Problem

To evaluate the efficacy of our generated instructions in enhancing complex reasoning and mathematical problem\-solving capabilities, we benchmark the Ovis2\-1B[Lu et al\. \(2024b\)](https://arxiv.org/html/2610.11363#bib.bib1)on the MathVista[Lu et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib7)and MathVision[Wang et al\. \(2024b\)](https://arxiv.org/html/2610.11363#bib.bib6)datasets\. In Table[7](https://arxiv.org/html/2610.11363#S5.T7), our instructions improve performance on both mathematical benchmarks\. The gains are consistent in direction across the two, though modest in magnitude\.

#### Other Text\-based Modalities

To confirm the versatility of our data across various text\-based modalities, including language, QR code, link, and map, we assessed their performance using a general language\-based benchmark, MMLU\-Pro[Wang et al\. \(2024c\)](https://arxiv.org/html/2610.11363#bib.bib5)\. As shown in Table[7](https://arxiv.org/html/2610.11363#S5.T7), adding our data raises MMLU\-Pro accuracy from20\.320\.3to21\.921\.9, indicating that the text\-based modalities also transfer to a general language benchmark\.

![Refer to caption](https://arxiv.org/html/2610.11363v1/Ab1.png)Figure 8:\(left\) Difference distribution of the events in three random generations\. \(right\) Example of diverse events based on the same keywords\.

### 5\.3Ablation Study

#### Effectiveness of Expanding Diverse Events

To demonstrate the effectiveness of expanding events, we randomly expand three events for each keyword on the test data to observe whether these events differ from one another \(diversity\)\. As shown in Figure[8](https://arxiv.org/html/2610.11363#S5.F8)\(left\), the TF\-IDF[Ramos and others \(2003\)](https://arxiv.org/html/2610.11363#bib.bib9)feature distributions across three generation runs \(on a random subset\) show considerable diversity\. In Figure[8](https://arxiv.org/html/2610.11363#S5.F8)\(right\), we also provide an example showing that different events exhibit distinct content \(in highlighted text\)\. In thew/o Expansionsetting of Table[7](https://arxiv.org/html/2610.11363#S5.T7), the generation of instructions relies solely on the few keywords provided by the user\. Thus, the generation pipeline lacks the necessary enriched prompt information, which leads to a decline in the quality of generated instructions\. Additionally, the number of instructions that can be generated is substantially reduced\.

![Refer to caption](https://arxiv.org/html/2610.11363v1/Ab2.png)Figure 9:Example of flow error correction\.
#### Effectiveness of Correcting Instruction Flow

To demonstrate the effectiveness of correcting instruction flow, we run inference twice,withandwithoutthe correction, as shown in Table[7](https://arxiv.org/html/2610.11363#S5.T7)and Figure[9](https://arxiv.org/html/2610.11363#S5.F9)\. Table[7](https://arxiv.org/html/2610.11363#S5.T7)shows that after correction, our data quality improves on all four dimensions\. Figure[9](https://arxiv.org/html/2610.11363#S5.F9)provides an example of the repetitive generation problem \(same<emojis\>\) and irrelevant generation problem \(<code\>or “null”\) in multimodal content\. After correction, instruction generation can effectively avoid these errors\.

## 6Conclusion

This paper presents UniData, a universal pipeline for multimodal instruction generation\. As a fundamental contribution, UniData establishes a framework for scalable data acquisition critical to MLLMs\. By delivering high\-quality, diverse, and cost\-effective instruction sets, UniData accelerates progress in multimodal learning paradigms and their downstream applications\.

## Limitations

UniData is a text\-centered orchestration pipeline rather than a native any\-to\-any model, so per\-modality quality is bounded by the external tools that realize each modality\. Long\-horizon structured reasoning is handled only by simple correction rules, and the nine\-modality coverage still omits video, 3D, motion, tables, biological data, and spoken input\. Finally, UniData inherits biases from its LLM backbones, search APIs, and modality generators; these may enter UniDataset and be amplified downstream, and systematic bias audits remain future work\.

## References

- Austinet al\.\(2021\)J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le,et al\.Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px4.p1.1)\.
- Chianget al\.\(2023\)W\. Chiang, Z\. Li, Z\. Lin, Y\. Sheng, Z\. Wu, H\. Zhang, L\. Zheng, S\. Zhuang, Y\. Zhuang, J\. E\. Gonzalez, I\. Stoica, and E\. P\. XingVicuna: an open\-source chatbot impressing GPT\-4 with 90%\* ChatGPT quality\.Note:[https://lmsys\.org/blog/2023\-03\-30\-vicuna/](https://lmsys.org/blog/2023-03-30-vicuna/)Cited by:[Table 12](https://arxiv.org/html/2610.11363#A3.T12.6.1.4.2),[Table 1](https://arxiv.org/html/2610.11363#S4.T1.6.1.4.2)\.
- Chuet al\.\(2023\)Y\. Chu, J\. Xu, X\. Zhou, Q\. Yang, S\. Zhang, Z\. Yan, C\. Zhou, and J\. ZhouQwen\-audio: advancing universal audio understanding via unified large\-scale audio\-language models\.External Links:2311\.07919,[Link](https://arxiv.org/abs/2311.07919)Cited by:[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px1.p4.1)\.
- Copetet al\.\(2023\)J\. Copet, F\. Kreuk, I\. Gat, T\. Remez, D\. Kant, G\. Synnaeve, Y\. Adi, and A\. DéfossezSimple and controllable music generation\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=jtiQ26sCJi)Cited by:[Table 9](https://arxiv.org/html/2610.11363#A1.T9.2.6.1),[§3\.1](https://arxiv.org/html/2610.11363#S3.SS1.SSS0.Px4.p1.1),[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px5.p1.1)\.
- Cuiet al\.\(2024\)C\. Cui, Y\. Ma, X\. Cao, W\. Ye, Y\. Zhou, K\. Liang, J\. Chen, J\. Lu, Z\. Yang, K\. Liao,et al\.A survey on multimodal large language models for autonomous driving\.InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision,pp\. 958–979\.Cited by:[§1](https://arxiv.org/html/2610.11363#S1.p2.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p1.1)\.
- Dinget al\.\(2025\)S\. Ding, S\. Wu, X\. Zhao, Y\. Zang, H\. Duan, X\. Dong, P\. Zhang, Y\. Cao, D\. Lin, and J\. WangMM\-IFEngine: towards multimodal instruction following\.InProceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 1099–1109\.Cited by:[3rd item](https://arxiv.org/html/2610.11363#A3.I2.i3.p1.1)\.
- Duet al\.\(2023\)Z\. Du, J\. Wang, Q\. Chen, Y\. Chu, Z\. Gao, Z\. Li, K\. Hu, X\. Zhou, J\. Xu, Z\. Ma, W\. Wang, S\. Zheng, C\. Zhou, Z\. Yan, and S\. ZhangLauraGPT: listen, attend, understand, and regenerate audio with gpt\.External Links:2310\.04673,[Link](https://arxiv.org/abs/2310.04673)Cited by:[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px1.p4.1)\.
- Esseret al\.\(2024\)P\. Esser, S\. Kulal, A\. Blattmann, R\. Entezari, J\. Müller, H\. Saini, Y\. Levi, D\. Lorenz, A\. Sauer, F\. Boesel, D\. Podell, T\. Dockhorn, Z\. English, and R\. RombachScaling rectified flow transformers for high\-resolution image synthesis\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 12606–12633\.Cited by:[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px5.p1.1)\.
- Fanget al\.\(2024\)X\. Fang, K\. Mao, H\. Duan, X\. Zhao, Y\. Li, D\. Lin, and K\. ChenMMBench\-Video: a long\-form multi\-shot benchmark for holistic video understanding\.InAdvances in Neural Information Processing Systems \(NeurIPS\) Datasets and Benchmarks Track,Cited by:[§1](https://arxiv.org/html/2610.11363#S1.p3.1)\.
- Fenget al\.\(2026\)Y\. Feng, J\. Sun, C\. Li, Z\. Li, J\. Ai, F\. Zhang, Y\. Chang, S\. Zhou, S\. Zhang, Y\. Dai, and K\. ZhangA high quality dataset and reliable evaluation for interleaved image\-text generation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[4th item](https://arxiv.org/html/2610.11363#A3.I2.i4.p1.1),[§C\.7](https://arxiv.org/html/2610.11363#A3.SS7.p3.1)\.
- Geet al\.\(2024\)Y\. Ge, S\. Zhao, J\. Zhu, Y\. Ge, K\. Yi, L\. Song, C\. Li, X\. Ding, and Y\. ShanSEED\-x: multimodal models with unified multi\-granularity comprehension and generation\.arXiv preprint arXiv:2404\.14396\.Cited by:[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p2.1),[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px2.p1.1)\.
- Gonget al\.\(2023\)Y\. Gong, A\. H\. Liu, H\. Luo, L\. Karlinsky, and J\. GlassJoint audio and speech understanding\.In2023 IEEE Automatic Speech Recognition and Understanding Workshop \(ASRU\),Cited by:[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px1.p4.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The llama 3 herd of models\.External Links:2407\.21783Cited by:[Table 12](https://arxiv.org/html/2610.11363#A3.T12.6.1.5.2),[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px1.p3.1),[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px3.p1.1),[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px4.p1.3),[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px5.p1.1),[Table 1](https://arxiv.org/html/2610.11363#S4.T1.6.1.5.2)\.
- Hansenet al\.\(2025\)J\. Hansen, W\. Lin, J\. Kang, M\. J\. Mirza, H\. Luo, R\. Feris, A\. Ritter, J\. Glass, and L\. KarlinskyInstructify: demystifying metadata to visual instruction tuning data conversion\.arXiv preprint arXiv:2505\.18115\.Cited by:[2nd item](https://arxiv.org/html/2610.11363#A3.I2.i2.p1.1)\.
- Huanget al\.\(2024\)L\. Huang, W\. Yu, W\. Ma, W\. Zhong, Z\. Feng, H\. Wang, Q\. Chen, W\. Peng, X\. Feng, B\. Qin, and T\. LiuA survey on hallucination in large language models: principles, taxonomy, challenges, and open questions\.ACM Transactions on Information Systems\.External Links:ISSN 1558\-2868,[Link](http://dx.doi.org/10.1145/3703155),[Document](https://dx.doi.org/10.1145/3703155)Cited by:[§4\.3](https://arxiv.org/html/2610.11363#S4.SS3.p1.1)\.
- Liet al\.\(2023a\)B\. Li, Y\. Ge, Y\. Ge, G\. Wang, R\. Wang, R\. Zhang, and Y\. ShanSEED\-bench\-2: benchmarking multimodal large language models\.arXiv preprint arXiv:2311\.17092\.Cited by:[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px2.p1.1)\.
- Liet al\.\(2017\)G\. Li, Y\. Zheng, J\. Fan, J\. Wang, and R\. ChengCrowdsourced data management: overview and challenges\.InProceedings of the 2017 ACM international conference on Management of Data,pp\. 1711–1716\.Cited by:[§1](https://arxiv.org/html/2610.11363#S1.p3.1)\.
- Liet al\.\(2023b\)K\. Li, Y\. He, Y\. Wang, Y\. Li, W\. Wang, P\. Luo, Y\. Wang, L\. Wang, and Y\. QiaoVideochat: chat\-centric video understanding\.arXiv preprint arXiv:2305\.06355\.Cited by:[Table 10](https://arxiv.org/html/2610.11363#A1.T10.30.1.4.1),[§1](https://arxiv.org/html/2610.11363#S1.p3.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p2.1),[§5\.1](https://arxiv.org/html/2610.11363#S5.SS1.SSS0.Px2.p1.1)\.
- Liet al\.\(2024a\)L\. Li, Y\. Li, X\. Zhang, Y\. He, J\. Yang, B\. Tian, Y\. Ai, L\. Li, A\. Nüchter, and Z\. XuanyuanEmbodied intelligence in mining: leveraging multi\-modal large language model for autonomous driving in mines\.IEEE Transactions on Intelligent Vehicles\.Cited by:[§1](https://arxiv.org/html/2610.11363#S1.p2.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2024b\)P\. Li, Z\. Gao, B\. Zhang, T\. Yuan, Y\. Wu, M\. Harandi, Y\. Jia, S\. Zhu, and Q\. LiFIRE: a dataset for feedback integration and refinement evaluation of multimodal models\.InThe Thirty\-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=DFb1gwnhQS)Cited by:[Table 10](https://arxiv.org/html/2610.11363#A1.T10.30.1.5.1)\.
- Lianget al\.\(2024\)Z\. Liang, Y\. Xu, Y\. Hong, P\. Shang, Q\. Wang, Q\. Fu, and K\. LiuA survey of multimodel large language models\.InProceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineering,CAICE ’24,New York, NY, USA,pp\. 405–409\.External Links:ISBN 9798400716942,[Link](https://doi.org/10.1145/3672758.3672824),[Document](https://dx.doi.org/10.1145/3672758.3672824)Cited by:[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2023\)H\. Liu, C\. Li, Q\. Wu, and Y\. J\. LeeVisual instruction tuning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.36\.Cited by:[Table 10](https://arxiv.org/html/2610.11363#A1.T10.30.1.3.1),[§1](https://arxiv.org/html/2610.11363#S1.p2.1),[§1](https://arxiv.org/html/2610.11363#S1.p3.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p2.1)\.
- Liuet al\.\(2024a\)S\. Liu, A\. S\. Hussain, Q\. Wu, C\. Sun, and Y\. ShanMuMu\-llama: multi\-modal music understanding and generation via large language models\.arXiv preprint arXiv:2412\.06660\.Cited by:[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px3.p1.1)\.
- Liuet al\.\(2024b\)Y\. Liu, H\. Duan, Y\. Zhang, B\. Li, S\. Zhang, W\. Zhao, Y\. Yuan, J\. Wang, C\. He, Z\. Liu, K\. Chen, and D\. LinMMBench: is your multi\-modal model an all\-around player?\.InComputer Vision – ECCV 2024,pp\. 216–233\.Cited by:[§1](https://arxiv.org/html/2610.11363#S1.p3.1)\.
- Luet al\.\(2024a\)P\. Lu, H\. Bansal, T\. Xia, J\. Liu, C\. Li, H\. Hajishirzi, H\. Cheng, K\. Chang, M\. Galley, and J\. GaoMathVista: evaluating mathematical reasoning of foundation models in visual contexts\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px5.p1.1)\.
- Luet al\.\(2024b\)S\. Lu, Y\. Li, Q\. Chen, Z\. Xu, W\. Luo, K\. Zhang, and H\. YeOvis: structural embedding alignment for multimodal large language model\.arXiv:2405\.20797\.Cited by:[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p2.1),[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px1.p4.1),[§5](https://arxiv.org/html/2610.11363#S5.SS0.SSS0.Px1.p1.1),[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px1.p1.1),[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px5.p1.1)\.
- Masryet al\.\(2022\)A\. Masry, D\. X\. Long, J\. Q\. Tan, S\. Joty, and E\. HoqueChartQA: a benchmark for question answering about charts with visual and logical reasoning\.InFindings of the Association for Computational Linguistics: ACL 2022,pp\. 2263–2279\.Cited by:[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px1.p1.1)\.
- Müller and Unay \(2017\)H\. Müller and D\. UnayRetrieval from and understanding of large\-scale multi\-modal medical datasets: a review\.IEEE transactions on multimedia19\(9\),pp\. 2093–2104\.Cited by:[§1](https://arxiv.org/html/2610.11363#S1.p2.1)\.
- OpenAI \(2024\)OpenAIGPT\-4o system card\.External Links:2410\.21276,[Link](https://arxiv.org/abs/2410.21276)Cited by:[Table 9](https://arxiv.org/html/2610.11363#A1.T9.2.2.1),[Table 12](https://arxiv.org/html/2610.11363#A3.T12.6.1.3.2),[§1](https://arxiv.org/html/2610.11363#S1.p3.1),[§3\.1](https://arxiv.org/html/2610.11363#S3.SS1.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.11363#S3.SS1.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2610.11363#S3.SS1.SSS0.Px3.p1.1),[§3\.1](https://arxiv.org/html/2610.11363#S3.SS1.SSS0.Px3.p2.1),[§3\.1](https://arxiv.org/html/2610.11363#S3.SS1.SSS0.Px4.p1.1),[Table 1](https://arxiv.org/html/2610.11363#S4.T1.6.1.1.3),[Table 1](https://arxiv.org/html/2610.11363#S4.T1.6.1.3.2),[§5\.1](https://arxiv.org/html/2610.11363#S5.SS1.SSS0.Px1.p1.1)\.
- Radfordet al\.\(2023\)A\. Radford, J\. W\. Kim, T\. Xu, G\. Brockman, C\. McLeavey, and I\. SutskeverRobust speech recognition via large\-scale weak supervision\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 28492–28518\.Cited by:[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px1.p4.1)\.
- Ramoset al\.\(2003\)J\. Ramoset al\.Using tf\-idf to determine word relevance in document queries\.InProceedings of the first instructional conference on machine learning,Vol\.242,pp\. 29–48\.Cited by:[§5\.3](https://arxiv.org/html/2610.11363#S5.SS3.SSS0.Px1.p1.1)\.
- Singhet al\.\(2019\)A\. Singh, V\. Natarajan, M\. Shah, Y\. Jiang, X\. Chen, D\. Batra, D\. Parikh, and M\. RohrbachTowards VQA models that can read\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 8317–8326\.Cited by:[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px1.p1.1)\.
- Součeket al\.\(2024\)T\. Souček, D\. Damen, M\. Wray, I\. Laptev, and J\. SivicGenHowTo: learning to generate actions and state transformations from instructional videos\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[Table 10](https://arxiv.org/html/2610.11363#A1.T10.30.1.8.1),[§3](https://arxiv.org/html/2610.11363#S3.p1.1)\.
- Tanget al\.\(2024\)J\. Tang, H\. Lu, R\. Wu, X\. Xu, K\. Ma, C\. Fang, B\. Guo, J\. Lu, Q\. Chen, and Y\. ChenHAWK: learning to understand open\-world video anomalies\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.37,pp\. 139751–139785\.Cited by:[§1](https://arxiv.org/html/2610.11363#S1.p3.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2610.11363#S5.SS1.SSS0.Px2.p1.1)\.
- Touvronet al\.\(2023\)H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale, D\. Bikel, L\. Blecher, C\. C\. Ferrer, M\. Chen, G\. Cucurull, D\. Esiobu, J\. Fernandes, J\. Fu, W\. Fu, B\. Fuller, C\. Gao, V\. Goswami, N\. Goyal, A\. Hartshorn, S\. Hosseini, R\. Hou, H\. Inan, M\. Kardas, V\. Kerkez, M\. Khabsa, I\. Kloumann, A\. Korenev, P\. S\. Koura, M\. Lachaux, T\. Lavril, J\. Lee, D\. Liskovich, Y\. Lu, Y\. Mao, X\. Martinet, T\. Mihaylov, P\. Mishra, I\. Molybog, Y\. Nie, A\. Poulton, J\. Reizenstein, R\. Rungta, K\. Saladi, A\. Schelten, R\. Silva, E\. M\. Smith, R\. Subramanian, X\. E\. Tan, B\. Tang, R\. Taylor, A\. Williams, J\. X\. Kuan, P\. Xu, Z\. Yan, I\. Zarov, Y\. Zhang, A\. Fan, M\. Kambadur, S\. Narang, A\. Rodriguez, R\. Stojnic, S\. Edunov, and T\. ScialomLlama 2: open foundation and fine\-tuned chat models\.External Links:2307\.09288Cited by:[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px4.p1.1)\.
- Verma \(2024\)P\. VermaWhisper\-gpt: a hybrid representation audio large language model\.External Links:2412\.11449,[Link](https://arxiv.org/abs/2412.11449)Cited by:[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.SSS0.Px1.p4.1)\.
- Wanget al\.\(2024a\)B\. Wang, F\. Wu, X\. Han, J\. Peng, H\. Zhong, P\. Zhang, X\. Dong, W\. Li, W\. Li, J\. Wang, and C\. HeVIGC: visual instruction generation and correction\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 5309–5317\.Cited by:[§C\.3](https://arxiv.org/html/2610.11363#A3.SS3.p1.1),[Table 11](https://arxiv.org/html/2610.11363#A3.T11.4.1.3.1),[Table 12](https://arxiv.org/html/2610.11363#A3.T12.6.1.4.1),[Figure 2](https://arxiv.org/html/2610.11363#S1.F2),[§1](https://arxiv.org/html/2610.11363#S1.p2.1),[§1](https://arxiv.org/html/2610.11363#S1.p5.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2610.11363#S4.T1.6.1.4.1),[§4](https://arxiv.org/html/2610.11363#S4.p1.1),[§5\.1](https://arxiv.org/html/2610.11363#S5.SS1.SSS0.Px1.p1.1),[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.p1.1)\.
- Wanget al\.\(2024b\)K\. Wang, J\. Pan, W\. Shi, Z\. Lu, M\. Zhan, and H\. LiMeasuring multimodal mathematical reasoning with MATH\-Vision dataset\.InAdvances in Neural Information Processing Systems \(NeurIPS\) Datasets and Benchmarks Track,Cited by:[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px5.p1.1)\.
- Wanget al\.\(2023\)Y\. Wang, Y\. Kordi, S\. Mishra, A\. Liu, N\. A\. Smith, D\. Khashabi, and H\. HajishirziSelf\-instruct: aligning language models with self\-generated instructions\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 13484–13508\.Cited by:[§C\.3](https://arxiv.org/html/2610.11363#A3.SS3.p1.1),[Table 11](https://arxiv.org/html/2610.11363#A3.T11.4.1.2.1),[Table 12](https://arxiv.org/html/2610.11363#A3.T12.6.1.3.1),[Figure 2](https://arxiv.org/html/2610.11363#S1.F2),[§1](https://arxiv.org/html/2610.11363#S1.p2.1),[§1](https://arxiv.org/html/2610.11363#S1.p5.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2610.11363#S3.SS1.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2610.11363#S4.T1.6.1.3.1),[§5\.1](https://arxiv.org/html/2610.11363#S5.SS1.SSS0.Px1.p1.1),[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.p1.1)\.
- Wanget al\.\(2024c\)Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang,et al\.Mmlu\-pro: a more robust and challenging multi\-task language understanding benchmark\.InThe Thirty\-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track,Cited by:[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px6.p1.1)\.
- Wuet al\.\(2024\)S\. Wu, H\. Fei, L\. Qu, W\. Ji, and T\. ChuaNExT\-gpt: any\-to\-any multimodal llm\.InProceedings of the International Conference on Machine Learning,pp\. 53366–53397\.Cited by:[Table 10](https://arxiv.org/html/2610.11363#A1.T10.30.1.9.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p2.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p3.1),[§3](https://arxiv.org/html/2610.11363#S3.p1.1),[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.p1.1)\.
- Xieet al\.\(2025\)J\. Xie, W\. Mao, Z\. Bai, D\. J\. Zhang, W\. Wang, K\. Q\. Lin, Y\. Gu, Z\. Chen, Z\. Yang, and M\. Z\. ShouShow\-o: one single transformer to unify multimodal understanding and generation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p2.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px1.p3.1),[§4\.2](https://arxiv.org/html/2610.11363#S4.SS2.p1.1)\.
- Xuet al\.\(2022\)J\. Xu, X\. Liu, J\. Yan, D\. Cai, H\. Li, and J\. LiLearning to break the loop: analyzing and mitigating repetitions for neural text generation\.InAdvances in Neural Information Processing Systems,A\. H\. Oh, A\. Agarwal, D\. Belgrave, and K\. Cho \(Eds\.\),External Links:[Link](https://openreview.net/forum?id=sexfswCc7B)Cited by:[§4\.3](https://arxiv.org/html/2610.11363#S4.SS3.p1.1)\.
- Yeet al\.\(2023\)J\. Ye, A\. Hu, H\. Xu, Q\. Ye, M\. Yan, Y\. Dan, C\. Zhao, G\. Xu, C\. Li, J\. Tian, Q\. Qi, J\. Zhang, and F\. HuangMPLUG\-docowl: modularized multimodal large language model for document understanding\.External Links:2307\.02499Cited by:[Table 10](https://arxiv.org/html/2610.11363#A1.T10.30.1.7.1)\.
- Yinet al\.\(2023\)Z\. Yin, J\. Wang, J\. Cao, Z\. Shi, D\. Liu, M\. Li, X\. Huang, Z\. Wang, L\. Sheng, L\. Bai, J\. Shao, and W\. OuyangLAMM: language\-assisted multi\-modal instruction\-tuning dataset, framework, and benchmark\.InAdvances in Neural Information Processing Systems \(NeurIPS\) Datasets and Benchmarks Track,Cited by:[Table 10](https://arxiv.org/html/2610.11363#A1.T10.30.1.6.1),[§1](https://arxiv.org/html/2610.11363#S1.p2.1)\.
- Yueet al\.\(2024\)X\. Yue, Y\. Ni, K\. Zhang, T\. Zheng, R\. Liu, G\. Zhang, S\. Stevens, D\. Jiang, W\. Ren, Y\. Sun,et al\.Mmmu: a massive multi\-discipline multimodal understanding and reasoning benchmark for expert agi\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 9556–9567\.Cited by:[§5\.2](https://arxiv.org/html/2610.11363#S5.SS2.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2025\)L\. Zhang, Q\. Cui, B\. Zhao, and C\. YangOasis: one image is all you need for multimodal instruction data synthesis\.InProceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 3542–3551\.Cited by:[1st item](https://arxiv.org/html/2610.11363#A3.I2.i1.p1.1)\.
- Zhanget al\.\(2024\)W\. Zhang, Z\. Cheng, Y\. He, M\. Wang, Y\. Shen, Z\. Tan, G\. Hou, M\. He, Y\. Ma, W\. Lu, and Y\. ZhuangMultimodal self\-instruct: synthetic abstract image and visual reasoning instruction using language model\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 19228–19252\.Cited by:[Figure 2](https://arxiv.org/html/2610.11363#S1.F2),[§1](https://arxiv.org/html/2610.11363#S1.p5.1),[§2](https://arxiv.org/html/2610.11363#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2610.11363#S4.p1.1)\.

## Overview of the Appendix

This appendix is organized as follows:

- •Dataset construction\(Appendix[A](https://arxiv.org/html/2610.11363#A1)\): the topic pool that seeds generation, the external multimodal toolkit, dataset diversity, bias\-mitigation procedures, and modality extensibility\.
- •Methodology\(Appendix[B](https://arxiv.org/html/2610.11363#A2)\): a worked example of the instruction\-flow correction step and an analysis of why UniData improves over a vanilla GPT\-4o pipeline\.
- •Experiments\(Appendix[C](https://arxiv.org/html/2610.11363#A3)\): training and inference recipes, evaluation prompts, a user study, out\-of\-distribution results, and ablations on multimodal balance and user\-conditioned generation\.
- •Additional analyses: a scope comparison with recent instruction\-data pipelines \(Appendix[C\.7](https://arxiv.org/html/2610.11363#A3.SS7)\)\.
- •Future work\(Appendix[D](https://arxiv.org/html/2610.11363#A4)\) and additional qualitative examples \(Figures[10](https://arxiv.org/html/2610.11363#A4.F10)–[13](https://arxiv.org/html/2610.11363#A4.F13)\)\.

## Appendix ADataset Construction

Table 8:The 90\-domain topic pool used to seed UniDataset generation\. At sampling time two domains are intersected \(Eq\.[9](https://arxiv.org/html/2610.11363#A1.E9)\) to form a composite theme\.e\-commerceacademic discussionslifestylesportsmathematicsbusinesstechnologytravelshealthentertainmentarthistorycookingparentingfashionfinancepoliticsliteraturegardeningastronomymusiceducationcomputer scienceprogrammingdesignresearchinternshipfoodbeautyentrepreneurshipstartupsalgorithmbugsscholarITvisionsupervisionclassroomassignmentgamepsychologysocial mediafilmtheaterarchitecturephotographyenvironmentagriculturereal estatemarketingeconomicsphilosophylawmedicineengineeringphysicsbiologyroboticscybersecurityartificial intelligencefamilyspace explorationclimate changeurban planningtransportationhospitalitytourismfitnessrelationshipsholidayscharityDIY projectssurvival skillslifecartoonpetswildlifemarine biologygeneticsneuroscienceanthropologysociologylinguisticsculturemythologyarchaeologyforensic sciencecriminologymilitarydiplomacyTable 9:External tools used by UniData during multimodal realization\. Each tool is responsible for one modality, allowing the pipeline to be extended modularly\.ToolModalitySourceRole in the pipelineGPT\-4o[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28)<language\>OpenAI APIOrchestrator and language backbone; expands events, drafts dialogue, and validates outputs\.Google Search<link\>[https://www\.google\.com](https://www.google.com/)Provides up\-to\-date factual snippets and reference URLs\.Google Maps<map\>[https://www\.google\.com/maps](https://www.google.com/maps)Supplies geographical references and routing information\.DALL⋅\\cdotE 3<image\>[https://openai\.com/index/dall\-e\-3/](https://openai.com/index/dall-e-3/)Synthesizes photo\-realistic and stylized images from text prompts\.MusicGen[Copet et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib19)<music\>Open\-source modelGenerates instrumental clips from textual descriptions\.Apple Emoji Archive<emoji\>[https://www\.apple\.com/](https://www.apple.com/)Provides culturally familiar emoji symbols for affective cues\.### A\.1Topic Pool

We curate atopic poolof 90 domains spanning everyday life, science and engineering, business, the arts, and the humanities \(Table[8](https://arxiv.org/html/2610.11363#A1.T8)\)\. The breadth of this pool is the primary driver of UniDataset’s content diversity, as it determines the universe of scenarios from which subsequent events are drawn\.

For each instruction sample we draw two topicsQ1,Q2Q\_\{1\},Q\_\{2\}uniformly at random and merge them into a composite theme,

⟨topic⟩=Q1∩Q2\.\\langle\\textit\{topic\}\\rangle\\;=\\;Q\_\{1\}\\cap Q\_\{2\}\.\(9\)Intersecting unrelated domains \(for instance,*cooking*∩\\cap*chemistry*or*music*∩\\cap*marketing*\) introduces controlled stochasticity into the pipeline and yields interdisciplinary prompts that are difficult to obtain from any single\-domain source\.

### A\.2Multimodal Toolkit

To realize the any\-to\-any generation interface described in the main paper, UniData orchestrates a suite of off\-the\-shelf tools, each responsible for a single modality\. Table[9](https://arxiv.org/html/2610.11363#A1.T9)summarizes every component, its target modality, its source, and the specific role it plays in the pipeline\. We deliberately favor publicly available APIs and open\-source models so that the pipeline can be reproduced and extended by the community\.

### A\.3Data Diversity

Table[10](https://arxiv.org/html/2610.11363#A1.T10)compares UniDataset with widely\-used multimodal instruction datasets along three orthogonal axes: \(i\) the number of dialogue rounds per sample, \(ii\) the count of distinct input/output modalities, and \(iii\) the overall corpus size\. UniDataset supports nine modalities on both the input and the output side, the largest modality set in this comparison, and provides on average17\.517\.5rounds per dialogue \(vs\. at most4\.84\.8for prior work\)\.

Table 10:Comparison with current multimodal instruction datasets\.*Num*is the number of distinct modality types;*Class*lists the modalities themselves\. Legend: L:<language\>; I:<image\>; V:<video\>; 3D:<point cloud\>; LK:<link\>; A:<audio\>; M:<music\>; E:<emoji\>; C:<code\>; MP:<map\>; MT:<math\>; Q:<QR code\>\.DatasetInput ModalityOutput ModalityAvg\. \#Rounds\#InstructionNumClassNumClassLLaVA[Liu et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib39)2L/I1L≈\\approx1\.05KVideoChat[Li et al\. \(2023b\)](https://arxiv.org/html/2610.11363#bib.bib42)2L/V1L≈\\approx1\.811KFIRE[Li et al\. \(2024b\)](https://arxiv.org/html/2610.11363#bib.bib31)2L/I1L≈\\approx1\.0100KLAMM[Yin et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib32)3L/I/3D1L≈\\approx3\.3196KmPLUG\-DocOwl[Ye et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib30)4L/I/T/L1L\-\-GenHowTo[Souček et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib29)2L/I2L/I\-\-MosIT[Wu et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib36)4L/I/A/V4L/I/A/V≈\\approx4\.85KOurs \(UniDataset\)9L/I/M/E/C/MP/LK/MT/Q9L/I/M/E/C/MP/LK/MT/Q≈\\approx17\.520K

### A\.4Human Oversight for Bias Mitigation

To reduce the bias inherited from any single LLM, we insert lightweight human oversight at two carefully chosen points in the pipeline\.

#### Topic curation\.

The 90 seed topics are hand\-selected to balance category coverage and cultural breadth, preventing the corpus from skewing toward a narrow set of domains\.

#### Event filtering\.

After event expansion, annotators discard implausible events and remove near\-duplicates\. Cleaner events produce better downstream prompts and, importantly, mitigate the propagation of LLM\-specific stylistic biases into the final instructions\.

### A\.5Modality Extensibility

The pipeline is designed to absorb new modalities without architectural changes\.

#### Stage 3 \(raw\-instruction generation\)\.

The planner emits modality\-agnostic slots that are filled at realization time, so adding a modality beyond the current nine does not require changing the planner\.

#### Stage 4 \(multimodal realization\)\.

New APIs \(such as Text\-to\-Video or Text\-to\-3D models\) can be plugged in via a thin adapter that maps modality slots to API calls, with no change to upstream components\.

## Appendix BMethodology Details

### B\.1Instruction\-Flow Correction in Practice

Correction is performed*before*multimodal realization\. At this stage every multimodal slot is still represented by its textual prompt rather than the rendered media, so the corrector can reason about content, dependencies, and redundancy purely at the language level\. A representative snippet is shown below\.

Example: textualized multimodal slot during correctionQ: … A: This further stimulates economic growth by expanding markets and reducing food scarcity issues\. <\|image\|\>*Prompt: farmland used for marketing campaigns\.*<\|image\_end\|\>

Because the textual prompt is already available at this stage, the corrector can detect redundancy or topical drift without inspecting the final media, which keeps correction fast and cheap\. A tighter feedback loop that conditions the corrector on the rendered media itself is left to future work\.

### B\.2Why UniData Outperforms a Vanilla Pipeline

#### Richer multimodal toolkit\.

UniData couples GPT\-4o with an external toolkit \(DALL⋅\\cdotE for images, MusicGen for audio, search APIs for facts\), extending the effective modality range far beyond what a single language model can produce natively\.

#### Pipeline design\.

In Table[12](https://arxiv.org/html/2610.11363#A3.T12), Self\-Instruct runs on a GPT\-4 backbone while UniData runs on LLaMA\-3 7B, yet UniData scores higher on all four GPT\-guided metrics\. Under this comparison the gain is attributable to event expansion and instruction\-flow correction rather than to backbone capacity\.

## Appendix CExperimental Details

### C\.1Training and Inference

#### Training of𝐃𝐄𝐆\\mathbf\{DEG\}\.

The Diverse Event Generator \(DEG\) is fine\-tuned on top of LLaMA\-3 using⟨topic⟩→⟨event⟩\\langle\\textit\{topic\}\\rangle\\rightarrow\\langle\\textit\{event\}\\ranglepairs harvested from UniDataset\. Fine\-tuning takes roughly 40 hours on a single NVIDIA A100 \(80 GB\)\.

#### Training of𝐌𝐈𝐆\\mathbf\{MIG\}\.

The Multimodal Instruction Generator \(MIG\) is fine\-tuned on four NVIDIA A100 \(80 GB\) GPUs for roughly 40 hours\.

### C\.2Evaluation Prompt and Metrics

We define four GPT\-4o\-judged metrics \(*Reasonableness*,*Clarity*,*Detail*, and*Relevance*\) and query the judge with the unified prompt shown below\.

Unified GPT\-4o evaluation promptEvaluate the\{metric\}of the multimodal instruction\{generated\}, compared to the ground truth\{reference\}\. Return a single score in\[0,1\]\[0,1\]\. Example output:`\{"score": 0\.675\}`\. Definition of\{metric\}:\{explanation\}\.

The four metrics are operationalized as follows:

- •Reasonableness: logical consistency and factual accuracy with respect to the reference\.
- •Clarity: readability, well\-formedness, and unambiguity of the generated text\.
- •Detail: richness of information conveyed in each turn\.
- •Relevance: topical adherence to the input context across the full dialogue\.

The evaluator is asked to score each metric independently so that absolute and relative comparisons can be made across the four dimensions\.

### C\.3User Study

To complement the automatic GPT\-4o scores, we run a forced\-choice user study\. We recruit 15 evaluators from a mix of academic and industrial backgrounds, all with prior experience with large language models\. Each evaluator inspects 50 samples and, for every one of the four evaluation dimensions, selects which method among Self\-Instruct[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35), VIGC[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34), and UniData produced the best output\. The results, reported in Table[11](https://arxiv.org/html/2610.11363#A3.T11), show that UniData is preferred by a substantial margin on every dimension\.

Table 11:User study: percentage of samples \(%\) on which each method is selected as the best by 15 human evaluators across 50 samples \(per\-dimension forced choice\)\.Redindicates the best performance\.MethodReason\.\(↑\)\(\\mathbf\{\\uparrow\}\)Clarity\(↑\)\(\\mathbf\{\\uparrow\}\)Detail\(↑\)\(\\mathbf\{\\uparrow\}\)Relev\.\(↑\)\(\\mathbf\{\\uparrow\}\)Self\-Instruct[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35)18\.59\.79\.88\.5VIGC[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34)10\.35\.314\.410\.9Ours \(UniData\)71\.285\.075\.880\.6

Table 12:Quality of generated instructions on the out\-of\-distribution \(OOD\) set\.Boldmarks the best score in each column\.*\#Others*aggregates the emoji, code, math, map, link, and QR\-code modalities\.MethodBackbonesGPT\-Guided Metrics\(↑\)\(\\mathbf\{\\uparrow\}\)Avg\. \#Multimodalities\(↑\)\(\\mathbf\{\\uparrow\}\)Avg\. \#RoundsReasonablenessClarityDetailRelevance\#Image\#Music\#OthersSelf\-Instruct[Wang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib35)GPT\-4[OpenAI \(2024\)](https://arxiv.org/html/2610.11363#bib.bib28)0\.7350\.4750\.6080\.580\-\-\-≈\\approx3VIGC[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.11363#bib.bib34)Vicuna 7B[Chiang et al\. \(2023\)](https://arxiv.org/html/2610.11363#bib.bib11)0\.7140\.5070\.7320\.737\-\-\-≈\\approx1Ours \(UniData\)LLaMA\-3 7B[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2610.11363#bib.bib22)0\.7520\.7010\.8080\.8451\.040\.273\.82≈\\approx10

### C\.4Out\-of\-Distribution Performance

To probe generalization, we construct an out\-of\-distribution \(OOD\) test set of 100 events drawn from domains outside the seed topic pool \(collected via web scraping followed by manual screening\)\. Table[12](https://arxiv.org/html/2610.11363#A3.T12)shows that UniData remains the strongest method on every metric while simultaneously producing visibly richer multimodal content \(more images, music, and other modalities per dialogue\) and longer interactions\.

### C\.5Multimodal Distribution across Dialog Position

A common failure mode of existing pipelines is concentrating multimodal content in the opening turns of a dialogue\. Table[13](https://arxiv.org/html/2610.11363#A3.T13)shows that UniDataset instead distributes multimodal content nearly uniformly across the entire dialogue \(the five normalized buckets each carry roughly 20 % of the multimodal mass\)\. The qualitative examples in Figures[11](https://arxiv.org/html/2610.11363#A4.F11)–[13](https://arxiv.org/html/2610.11363#A4.F13)corroborate this: modalities appear consistently in early, middle, and late rounds\.

Table 13:Fraction of multimodal content \(%\) as a function of the normalized position within a dialogue\. Position is binned into five equal\-width buckets\.Position in dialogueMultimodal frequency \(%\)0 – 20 %20\.520 – 40 %19\.440 – 60 %19\.560 – 80 %21\.380 – 100 %19\.3

### C\.6Adaptation to User\-Specified Modalities

When the user steers the pipeline toward a particular modality, the generated content shifts accordingly \(Table[14](https://arxiv.org/html/2610.11363#A3.T14)\)\. For example, an image\-focused input yields roughly15×15\\timesmore image content than the general setting, while a music\-focused input boosts the music count by a similar factor\. The orchestration layer therefore responds to the requested modality rather than defaulting to a fixed mix\.

Table 14:Average count of each modality per dialogue when the user specifies a desired modality at input time\.Redindicates the targeted modality in each row\.Input type\#Image\#Music\#OthersGeneral1\.920\.747\.37Music\-related1\.1012\.302\.10Image\-related14\.900\.105\.20

### C\.7Comparison with Recent Instruction\-Data Pipelines

The two baselines in Table[1](https://arxiv.org/html/2610.11363#S4.T1)\(Self\-Instruct and VIGC\) are chosen because they represent the two dominant styles of universal instruction\-data generation: single\-modality seed\-based expansion and vision\-conditioned instruction generation\. For completeness, we position UniData against several recent instruction\-data pipelines that appeared alongside or after this work\.

- •Oasis[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.11363#bib.bib45)generates language\-side instruction–response pairs from a single conditioning image; assistant\-side output is text\-only\.
- •Instructify[Hansen et al\. \(2025\)](https://arxiv.org/html/2610.11363#bib.bib46)iteratively generates multi\-turn visual instruction conversations by aggregating over4040source datasets; assistant\-side output is again text\-only\.
- •MM\-IFEngine[Ding et al\. \(2025\)](https://arxiv.org/html/2610.11363#bib.bib47)produces constraint\-rich image–instruction–answer triples for SFT and DPO; assistant\-side output is text\-only\.
- •InterSyn[Feng et al\. \(2026\)](https://arxiv.org/html/2610.11363#bib.bib48)synthesizes single\- and multi\-turn interleaved image–text dialogues, with assistant\-side output covering language and image\.

These pipelines target different source data, tasks, and evaluation protocols, so their paper\-reported scores cannot be placed in a common “quality” column with those in Table[1](https://arxiv.org/html/2610.11363#S4.T1)without rerunning every method under an identical protocol\. We therefore treat them as scope comparisons rather than head\-to\-head numeric competitors\. The distinguishing feature of UniData relative to these pipelines is the combination of*symmetric*nine\-modality input/output coverage \(rather than a language\-only or language\+image assistant side\) and long multi\-round interactions \(17\.5 rounds on average\)\. Among the recent works listed, InterSyn[Feng et al\. \(2026\)](https://arxiv.org/html/2610.11363#bib.bib48)is closest in spirit but still limits assistant\-side output to language and image\.

### C\.8Additional Qualitative Results

Figure[10](https://arxiv.org/html/2610.11363#A4.F10)visualizes the output diversity ofDEG: three independently sampled events for the same keyword pair differ substantially in content, tone, and target modality\.

Figures[11](https://arxiv.org/html/2610.11363#A4.F11)–[13](https://arxiv.org/html/2610.11363#A4.F13)present three end\-to\-end examples produced byMIG\. Each example interleaves several modalities \(image, music, code, emoji\) across a multi\-round dialogue, illustrating both the modality breadth and the long\-horizon coherence achieved by UniData\.

## Appendix DFuture Work

We view UniData as a foundation for the automated production of multimodal training data, and identify three concrete directions for follow\-up work\.

#### Scaling backbones\.

Replace the LLaMA\-3 backbone ofMIGwith more recent and larger open\-source models to characterize how multimodal instruction quality scales with backbone capacity\.

#### Expanding modalities\.

Plug in tools for video, 3D, motion, and tabular generation so that UniDataset can extend beyond the current nine modalities, increasing applicability to embodied and scientific domains\.

#### User\-conditioned generation\.

Tighten the control loop so that UniData behaves as an interactive data factory: an end user provides their downstream specification, and UniData produces purpose\-built data aligned with that deployment scenario\.

Figure 10:Diverse event generation byDEG\. For the same keyword pair, three independently sampled events differ substantially in content, scenario, and intended modality, illustrating the diversity that drives downstream instruction variety\.![Refer to caption](https://arxiv.org/html/2610.11363v1/QualityE_Appendix.png)Figure 11:End\-to\-end qualitative example \(I\): a multi\-round, multimodal instruction produced by UniData\. Image, music, and text are interleaved into a coherent narrative\.![Refer to caption](https://arxiv.org/html/2610.11363v1/QualityE_Appendix_2.png)Figure 12:End\-to\-end qualitative example \(II\): a different topic pair, showing variation in dialogue length and modality mix while preserving topical coherence\.![Refer to caption](https://arxiv.org/html/2610.11363v1/QualityE_Appendix_3.png)Figure 13:End\-to\-end qualitative example \(III\): a long\-horizon dialogue that interleaves text, image, code, and emoji content, illustrating UniData’s behavior on extended interactions\.