面向全模态推理的以文本为中心的后训练

# Text-Centric Post-Training for Omni-Modal Reasoning
Source: [https://arxiv.org/html/2610.02819](https://arxiv.org/html/2610.02819)
## Text\-Centric Post\-Training for Omni\-Modal ReasoningThanks:Corresponding author\.

###### Abstract

Improving joint audio\-visual reasoning in Omni Large Language Models typically incurs substantial data construction and training costs\. Our diagnostics reveal multi\-hop reasoning difficulties despite correct answers to all corresponding single\-hop questions and suggest partial decoupling in the local optimization of perception and reasoning objectives\. This motivates post\-training with different emphases on these capabilities\. Text\-only reasoning training yields gains across data sources, model scales, and families\. With the best\-performing text\-only configuration, supervised fine\-tuning followed by reinforcement learning \(RL\) raises Qwen2\.5\-Omni\-7B’s geometric mean of nine reasoning scores by 25\.83% over the base model, outperforming the complete native audio\-visual route with 56\.6% fewer GPU\-hours\. Training on data synthesized entirely by a text\-only LLM raises this geometric mean by 21\.01% without audio\-visual data in construction or training\. However, text\-only training degrades perception\. We therefore propose a text\-centric post\-training paradigm: text\-only training provides the main reasoning optimization, and reduced\-data native audio\-visual RL then refines perception\. Refinement uses about 90% fewer input tokens than full\-data audio\-visual RL, restores perception above the base level, and retains 93\.5% of the best\-performing text\-only pipeline’s reasoning gain\.

## 1Introduction

Omni Large Language Models \(Omni\-LLMs\) have made remarkable progress in unified multimodal perception and understanding, enabling a single model to process text, audio, images, and video\([Xu et al\., 2025a](https://arxiv.org/html/2610.02819#bib.bib65);[Xu et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib66);[Team, 2026](https://arxiv.org/html/2610.02819#bib.bib55);[Deshmukh et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib15);[Team et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib53)\)\. However, strong perceptual capabilities do not necessarily translate into reliable multimodal reasoning\([Yao et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib72);[Wang et al\., 2026c](https://arxiv.org/html/2610.02819#bib.bib59)\)\. Existing Omni\-LLMs still struggle with complex audio\-visual tasks that require combining information across modalities and over time\([Chen et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib8);[Dai et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib14)\)\.

![Refer to caption](https://arxiv.org/html/2610.02819v1/figure_intro.png)Figure 1:\(a\) A diagnostic case of multi\-hop reasoning failure despite correct answers to corresponding single\-hop questions\. \(b\) Text\-only training provides the main reasoning optimization, followed by reduced\-data native audio\-visual RL to restore perception while retaining most reasoning gains\.To address this limitation, existing approaches typically use specially constructed multimodal reasoning data for supervised fine\-tuning \(SFT\) or reinforcement learning \(RL\)\([Xing et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib64);[Yang et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib70);[Kulkarni & Fazli, 2026](https://arxiv.org/html/2610.02819#bib.bib29);[Araujo et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib2)\)\. Constructing such data requires annotations consistent with audio\-visual content and adequate coverage of infrequent categories in long\-tailed distributions\([Cai et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib4);[Chen et al\., 2026c](https://arxiv.org/html/2610.02819#bib.bib11)\)\. These requirements and multimodal training costs limit scalability\([Guan et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib22)\)\. Meanwhile, text\-only reasoning training has improved performance on separate visual and audio tasks\([Liu et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib40);[Bousselham et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib3);[Rouditchenko et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib50)\)\. However, transfer to these tasks does not establish its effectiveness for joint audio\-visual reasoning, which requires integrating complementary evidence across modalities and over time\. This motivates us to ask: Can Omni\-LLM post\-training be organized to target perception and reasoning separately while reducing reliance on native audio\-visual training?

In Section[3](https://arxiv.org/html/2610.02819#S3), we investigate this possibility through a diagnostic study\. Across nine Omni\-LLMs, multi\-hop reasoning remains difficult despite correct answers to all corresponding single\-hop perception questions\. A diagnostic case is shown in Figure[1](https://arxiv.org/html/2610.02819#S1.F1)\(a\)\. Beyond this performance gap, we examine whether perception and reasoning admit different optimization directions\. Projected gradient updates retain most of the target loss reduction while substantially reducing changes to the other objective\. These responses support partial decoupling in the local optimization of the two mean\-loss objectives, motivating training strategies with different emphases on perception and reasoning\.

Building on these findings, we explore organizing Omni\-LLM training around the capabilities to be improved, rather than downstream input modalities\. We examine text\-only reasoning training across data sources\. On Qwen2\.5\-Omni\-7B, SFT followed by RL using the best\-performing text source improves all nine reasoning scores, raising their average by 9\.59 points and geometric mean by 25\.83% over the base model\. It outperforms the complete native audio\-visual route with 56\.6% fewer GPU\-hours\. Gains extend across the evaluated Qwen2\.5\-Omni, Qwen3\-Omni\([Xu et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib66)\), and Gemma 4\([Team et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib53)\)models of different scales\. Crucially, a text\-only LLM generates fictional scenario descriptions and reasoning questions and answers for SFT and RL, without native audio\-visual data in construction or training\. Training on these fully synthetic data raises the average reasoning score by 7\.70 points and geometric mean by 21\.01% over the same base model\.

However, text\-only reasoning training degrades perception\. We therefore propose the text\-centric post\-training paradigm in Figure[1](https://arxiv.org/html/2610.02819#S1.F1)\(b\): text\-only SFT followed by RL provides the main reasoning optimization, and reduced\-data native audio\-visual RL then refines perception\. On Qwen2\.5\-Omni\-7B, refinement uses about 90% fewer input tokens than full\-data audio\-visual RL\. It restores perception above the base level while retaining 93\.5% of the best\-performing text\-only pipeline’s gain in the geometric mean of reasoning scores, addressing the performance–compute trade\-off\.

Our contributions are as follows:

- •We conduct a diagnostic study revealing a gap between single\-hop perception and multi\-hop reasoning and suggesting partial decoupling in their local optimization\.
- •We systematically evaluate text\-only reasoning training across data sources, model scales, and families, demonstrating omni\-modal reasoning gains even with fully synthetic text requiring no audio\-visual data in construction or training\. Text\-only training outperforms the complete native audio\-visual route with lower training compute\.
- •We propose and validate a text\-centric post\-training paradigm: text\-only training provides the main reasoning optimization, and reduced\-data native audio\-visual RL then restores perception while retaining most of the reasoning gains\.

## 2Related Work

#### Text\-only reasoning transfer\.

We study joint audio\-visual reasoning across modalities and time, whereas text\-only transfer studies address visual\([Li et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib34);[Liu et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib40);[Bousselham et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib3)\)or audio tasks\([Rouditchenko et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib50)\)\. Our text\-only post\-training strengthens reasoning within Omni\-LLMs without external inference\-time reasoners\([Zhang et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib74)\)\. Unlike text\-derived multimodal supervision\([Yu et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib73);[Cheng et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib12)\), our fully synthetic setting uses textual SFT and RL data generated by a text\-only LLM without native audio\-visual inputs\.

#### Perception and reasoning in multimodal models\.

Cross\-modal evaluations reveal perception and integration limitations\([Wang et al\., 2026a](https://arxiv.org/html/2610.02819#bib.bib57);[Wang et al\., 2026b](https://arxiv.org/html/2610.02819#bib.bib58);[Mukherjee et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib46)\)\. Methods use multimodal training\([Liu et al\., 2026c](https://arxiv.org/html/2610.02819#bib.bib38);[Jia et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib27);[Liu et al\., 2026a](https://arxiv.org/html/2610.02819#bib.bib36);[Liu et al\., 2026b](https://arxiv.org/html/2610.02819#bib.bib37)\)or active inspection\([Xu et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib67);[Chen et al\., 2026b](https://arxiv.org/html/2610.02819#bib.bib10)\)\. VLM post\-training balances perception and reasoning\([Peng et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib47);[Chen et al\., 2026a](https://arxiv.org/html/2610.02819#bib.bib9);[Wu et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib60)\)\. Our diagnostics motivate text\-only post\-training that improves joint audio\-visual reasoning without native audio\-visual training data\.

## 3Preliminary Study

For this diagnostic study, we construct questions from annotations of real audio\-visual data and evaluate nine Omni\-LLMs on the corresponding inputs\. Single\-hop perception questions assess the direct extraction of information\. From the common subset answered correctly by all nine models, we construct 2\-hop and 3\-hop reasoning questions that combine the corresponding facts across audio and vision in two and three steps, respectively \(Figure[1](https://arxiv.org/html/2610.02819#S1.F1)\(a\)\)\. Details are provided in Appendix[A\.1](https://arxiv.org/html/2610.02819#A1.SS1)\.

Figure 2:Perception–reasoning gap and local update responses\.\(a\)Accuracy across hops\.\(b\)Gradient projection\.
### 3\.1The Gap Between Perception and Reasoning

Figure[2\(a\)](https://arxiv.org/html/2610.02819#S3.F2.sf1)reports macro\-averaged accuracy across models, with 94\.7% on the original 1\-hop set\. Despite correct answers to every corresponding single\-hop question, accuracy on the resulting multi\-hop questions is only 64\.1% for 2\-hop and 62\.3% for 3\-hop\. Across all five domains, these results reveal a gap between single\-hop perception and multi\-hop reasoning: recognizing individual facts does not ensure that models can combine them to reach the correct answer\.

### 3\.2Partial Decoupling in Perception and Reasoning Optimization

Beyond the performance gap, we measure improvement on one objective while constraining changes to the other\. In Qwen2\.5\-Omni\-3B\([Xu et al\., 2025a](https://arxiv.org/html/2610.02819#bib.bib65)\), we defineLEL\_\{E\}andLRL\_\{R\}as the mean cross\-entropy losses for predicting the correct answer token on single\-hop perception questions and 2\-hop and 3\-hop reasoning questions, respectively\. Under five\-fold cross\-validation by video, each split defines these objectives on its four training folds\. We compute gradients for the Thinker’s Transformer decoder layers and final normalization layer, with all other parameters fixed\.

Motivated by orthogonal gradient projection\([Huang et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib26);[Qiu et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib49)\), we compare original gradient updates with updates constrained to leave the other training\-fold mean loss unchanged to first order\. At the initial checkpointθ0\\theta\_\{0\}, letgt=∇Lt​\(θ0\)g\_\{t\}=\\nabla L\_\{t\}\(\\theta\_\{0\}\)for distinctt,s∈\{E,R\}t,s\\in\\\{E,R\\\}and define

rt⟂s=gt−gs⊤​gt‖gs‖22​gs,θt⟂s′=θ0−η​rt⟂s‖rt⟂s‖2\.r\_\{t\\perp s\}=g\_\{t\}\-\\frac\{g\_\{s\}^\{\\top\}g\_\{t\}\}\{\\\|g\_\{s\}\\\|\_\{2\}^\{2\}\}g\_\{s\},\\qquad\\theta^\{\\prime\}\_\{t\\perp s\}=\\theta\_\{0\}\-\\eta\\frac\{r\_\{t\\perp s\}\}\{\\\|r\_\{t\\perp s\}\\\|\_\{2\}\}\.\(1\)The original update isθt′=θ0−η​gt/‖gt‖2\\theta^\{\\prime\}\_\{t\}=\\theta\_\{0\}\-\\eta g\_\{t\}/\\\|g\_\{t\}\\\|\_\{2\}\. Both updates start fromθ0\\theta\_\{0\}, with parameter changes of the same Euclidean normη\\eta\. We pool held\-out loss reductions across folds and express them relative to each objective’s pooled initial loss; positive values indicate decreases and negative values indicate increases\. Implementation details and supplementary analyses appear in Appendix[A\.2](https://arxiv.org/html/2610.02819#A1.SS2)\.

In Figure[2\(b\)](https://arxiv.org/html/2610.02819#S3.F2.sf2), the projected reasoning update reduces reasoning loss by 6\.01% \(81\.7% of the original reduction\) and perception loss by 0\.06% \(original update: 2\.08%\)\. The projected perception update retains 74\.3% of the original target loss reduction and reduces reasoning loss by 0\.36% \(original update: 4\.33%\)\. These responses support partial decoupling of the two mean\-loss objectives in local optimization, motivating different post\-training emphases on perception and reasoning\.

## 4Study Design

Building on the findings in Section[3](https://arxiv.org/html/2610.02819#S3), we systematically address five research questions \(RQs\) to develop a post\-training paradigm that improves omni\-modal reasoning with lower training compute:

- •\(RQ1\) Reasoning Gains and Training Compute\.To what extent does text\-only reasoning training improve omni\-modal reasoning performance across diverse tasks and models, and how much training compute does it require?
- •\(RQ2\) Choice of Training Data\.How does the choice of training data affect the reasoning gains from text\-only reasoning training on omni\-modal tasks?
- •\(RQ3\) Synthetic Training Data\.Can text\-only reasoning training still yield omni\-modal reasoning gains when all training data are synthesized by a text\-only LLM?
- •\(RQ4\) Perceptual Effects\.How does text\-only reasoning training affect the perceptual capabilities of Omni\-LLMs relative to native audio\-visual training?
- •\(RQ5\) Training Paradigm\.How should text\-centric post\-training be staged to improve omni\-modal reasoning while maintaining perceptual capabilities?

### 4\.1Training Setup

#### Models\.

We conduct our main experiments on Qwen2\.5\-Omni\-7B\([Xu et al\., 2025a](https://arxiv.org/html/2610.02819#bib.bib65)\)\. To examine gains across model scales and families, we apply the same text\-only training pipeline to Qwen2\.5\-Omni\-3B\([Xu et al\., 2025a](https://arxiv.org/html/2610.02819#bib.bib65)\), Gemma\-4\-E4B\-IT, and Gemma\-4\-E2B\-IT\([Team et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib53)\)\. We also evaluate text\-only SFT on Qwen3\-Omni\-30B\-A3B\-Instruct\([Xu et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib66)\)\.

#### Training details\.

Text\-only and native audio\-visual post\-training start from the same base model and follow SFT with Group Relative Policy Optimization \(GRPO\)\([Shao et al\., 2024](https://arxiv.org/html/2610.02819#bib.bib51)\)\. We adopt the best\-performing text\-only configuration from the RQ2 comparison as the default, using Script\-based MCQs in both stages and CoT supervision for SFT\. Table[4\.2](https://arxiv.org/html/2610.02819#S4.SS2.SSS0.Px2)lists data types and sources for each RQ\. An overview of the training setup is provided in Appendix[B\.1](https://arxiv.org/html/2610.02819#A2.SS1)\.

### 4\.2Evaluation Setup

#### Benchmarks\.

For RQ1–RQ3, we evaluate omni\-modal reasoning on Daily\-Omni\([Zhou et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib76)\), Omni\-Cloze\([Ma et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib44)\), OmniVideoBench\([Li et al\., 2026a](https://arxiv.org/html/2610.02819#bib.bib32)\), SocialOmni\([Xie et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib63)\), Video\-Holmes\([Cheng et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib13)\), Video\-MME\-v2\([Fu et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib21)\), and WorldSense\([Hong et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib25)\)\. For RQ4, we evaluate perceptual capabilities on AVHBench\([Sung\-Bin et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib52)\), MUSIC\-AVQA v1/v2\([Li et al\., 2022](https://arxiv.org/html/2610.02819#bib.bib33);[Liu et al\., 2024](https://arxiv.org/html/2610.02819#bib.bib41)\), AVQA\([Yang et al\., 2022](https://arxiv.org/html/2610.02819#bib.bib69)\), and NExT\-QA\([Xiao et al\., 2021](https://arxiv.org/html/2610.02819#bib.bib62)\)\. For RQ5, we jointly evaluate reasoning and perception using these benchmarks\.

#### Baselines\.

We compare with the original Qwen2\.5\-Omni\-7B\([Xu et al\., 2025a](https://arxiv.org/html/2610.02819#bib.bib65)\)and six representative omni\-modal reasoning baselines: EchoInk\-R1\-7B\([Xing et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib64)\), AVATAR\-7B\([Kulkarni & Fazli, 2026](https://arxiv.org/html/2610.02819#bib.bib29)\), Omni\-R1\([Zhong et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib75)\), HumanSense\([Qin et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib48)\), HumanOmniV2\-7B\([Yang et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib70)\), and OmniVideo\-7B\([Cai et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib4)\)\.

Table 1:Data types and sources for text\-only and audio\-visual training\.StageData typeData sourcesRQ\[0pt\]\[0pt\]Text\-only trainingSFTGeneral reasoningSmallThoughts\([Wu et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib61)\); Mixture\-of\-Thoughts\([Face, 2025](https://arxiv.org/html/2610.02819#bib.bib17)\); Bespoke\-Stratos\-17k\([Labs, 2025](https://arxiv.org/html/2610.02819#bib.bib31)\); OpenThoughts\-114k\([Guha et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib23)\); OpenR1\-Math\-220k\([Lozhkov et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib42)\); OpenMathReasoning\([Moshkov et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib45)\)RQ2Script\-based CoTOmniVideo\-100K\([Cai et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib4)\)RQ1, 2, 4, 5Contextual QACoT\-Drop\([Liu et al\., 2025a](https://arxiv.org/html/2610.02819#bib.bib39)\); CoT\-Collection\([Kim et al\., 2023](https://arxiv.org/html/2610.02819#bib.bib28)\); CoT\-Scientific\-RAG\-Reasoning\([Kumar, 2025](https://arxiv.org/html/2610.02819#bib.bib30)\)RQ2GRPOScript\-based MCQOmniVideo\-100K\([Cai et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib4)\)RQ1, 2, 4, 5Temporal QAVideo\-T3 QA\([Li et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib34)\)RQ2General & math QANemotron\-CrossThink\([Akter et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib1)\)RQ2Contextual QAHotpotQA\([Yang et al\., 2018](https://arxiv.org/html/2610.02819#bib.bib71)\); MuSiQue\([Trivedi et al\., 2022](https://arxiv.org/html/2610.02819#bib.bib56)\); 2WikiMultiHopQA\([Ho et al\., 2020](https://arxiv.org/html/2610.02819#bib.bib24)\)RQ2SFT\+GRPOText\-only LLM synthesisQwen3\.7\-Max111[https://www\.alibabacloud\.com/help/en/model\-studio/qwen3\-7\-max](https://www.alibabacloud.com/help/en/model-studio/qwen3-7-max)RQ3\[0pt\]\[0pt\]Audio\-visual trainingSFTAudio\-visual reasoningAVRT\-20K\([Araujo et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib2)\); IntentTrain\([Yang et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib70)\)RQ1, 4, 5GRPOAudio\-visual MCQOmniVideo\-100K\([Cai et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib4)\)RQ1, 4, 5
#### Metrics\.

We report each benchmark’s official metric on a 0–100 scale\.Macro Avg\.is the unweighted mean of scores, computed separately for reasoning and perception\.Geometric Mean Improvementis the geometric mean of ratios to the corresponding base scores, minus one and expressed as a percentage\. For RQ1 and RQ5, we report training input tokens and training compute in GPU\-hours \(number of GPUs×\\timestraining time in hours\) separately for each training stage\.

## 5Findings

We summarize the findings from our five RQs and their implications for Omni\-LLM training:

- •\(RQ1\)Text\-only reasoning training yields broad omni\-modal reasoning gains with less training compute than the complete native audio\-visual route\.
- •\(RQ2\)Script\-based MCQs yield the strongest and broadest reasoning transfer\.
- •\(RQ3\)Training on data synthesized entirely by a text\-only LLM also improves omni\-modal reasoning, without native audio\-visual data in construction or training\.
- •\(RQ4\)Text\-only reasoning training incurs perceptual degradation\.
- •\(RQ5\)A text\-centric training paradigm addresses the performance–compute trade\-off by using text\-only training for reasoning optimization and reduced\-data native audio\-visual refinement to maintain perception while retaining the main reasoning gains\.

### 5\.1RQ1: Reasoning Gains and Training Compute

#### Text\-only reasoning training yields substantial and broad omni\-modal reasoning gains\.

In Table[2](https://arxiv.org/html/2610.02819#S5.T2), text\-only SFT followed by GRPO raises Qwen2\.5\-Omni\-7B’s Macro Avg\. from 39\.80 to 49\.39, a 9\.59\-point gain\. Its Macro Avg\. and geometric mean improvement of\+25\.83%\+25\.83\\%both rank first\. Trained only on text, it exceeds EchoInk\-R1\-7B and the complete native audio\-visual route by 4\.05 and 2\.02 Macro Avg\. points, respectively\. It ranks first on all three SocialOmni subtasks, Video\-MME\-v2, and WorldSense, and second on Daily\-Omni and OmniVideoBench\. All nine scores improve over the base model, whereas every external baseline declines on at least one\.

Table 2:Omni\-modal reasoning performance and training compute\. Parenthesized values show absolute changes from the base model\.Boldandunderlinedscores denote the best and second\-best results, respectively\.ModelOmni\-modal ReasoningAggregateTraining BudgetDaily\-OmniOmni\-ClozeOmniVideoSocialOmniVideo\-HolmesVideo\-MME\-v2WorldSenseMacroAvg\.Geo\. MeanImprov\.\(%\)Input Tok\.\(B\)↓\\downarrowGPU\-h↓\\downarrowWhoWhenHowQwen2\.5\-Omni\-7B62\.0721\.9029\.3036\.7557\.4256\.6724\.4024\.3145\.4039\.800\.00N/AExternal BaselinesEchoInk\-R1\-7B\([Xing et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib64)\)61\.0727\.3033\.5041\.8559\.8170\.4540\.7727\.7245\.6045\.34\+16\.510\.072191\.2AVATAR\-7B\([Kulkarni & Fazli, 2026](https://arxiv.org/html/2610.02819#bib.bib29)\)56\.1425\.9431\.6036\.2560\.2967\.2645\.1020\.4746\.0043\.23\+9\.370\.419385\.9Omni\-R1\([Zhong et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib75)\)55\.5626\.2832\.1033\.3062\.6872\.5532\.1225\.0441\.4042\.34\+7\.030\.055576\.0HumanSense\([Qin et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib48)\)56\.3126\.9831\.7038\.7066\.5175\.8633\.1524\.3342\.2043\.97\+10\.690\.208190\.3HumanOmniV2\-7B\([Yang et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib70)\)58\.4729\.3830\.5040\.2057\.8966\.0342\.9024\.9347\.1044\.16\+13\.654\.5062164\.1OmniVideo\-7B\([Cai et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib4)\)68\.3425\.4038\.3035\.4541\.1550\.4038\.0024\.5445\.1040\.74\+5\.381\.511201\.0Training VariantsAudio\-visual SFT54\.6428\.3232\.9037\.2563\.6456\.7636\.7426\.9643\.4042\.29\+9\.670\.09212\.2\+\+GRPO62\.9926\.4836\.7041\.4064\.5969\.7646\.6530\.2047\.6047\.37\+21\.961\.214642\.3Text\-only SFT52\.7225\.2531\.4039\.3053\.5966\.0727\.7122\.3440\.9039\.92\+1\.550\.04310\.2\+\+GRPO \(Ours\)65\.7527\.1737\.0041\.9568\.9079\.9344\.5830\.6648\.6049\.39\+25\.830\.463274\.0\(\+3\.68\)\(\+3\.68\)\(\+5\.27\)\(\+5\.27\)\(\+7\.70\)\(\+7\.70\)\(\+5\.20\)\(\+5\.20\)\(\+11\.48\)\(\+11\.48\)\(\+23\.26\)\(\+23\.26\)\(\+20\.18\)\(\+20\.18\)\(\+6\.35\)\(\+6\.35\)\(\+3\.20\)\(\+3\.20\)\(\+9\.59\)\(\+9\.59\)

Table 3:Generalization of text\-only reasoning training across model scales and families\. Parenthesized values show absolute changes from the corresponding base model\.ModelOmni\-modal ReasoningAggregateTraining BudgetDaily\-OmniOmni\-ClozeOmniVideoSocialOmniVideo\-HolmesVideo\-MME\-v2WorldSenseMacroAvg\.Geo\. MeanImprov\.\(%\)Input Tok\.\(B\)↓\\downarrowGPU\-h↓\\downarrowWhoWhenHowQwen2\.5\-Omni\-3B56\.2223\.5127\.3036\.5050\.2451\.2325\.2621\.3543\.5037\.230\.00N/AText\-only SFT51\.0423\.5232\.2032\.5555\.5058\.9332\.0120\.4640\.0038\.47\+3\.440\.12810\.1\+\+GRPO64\.1624\.4036\.0037\.1057\.4271\.8842\.9525\.1745\.3044\.93\+20\.390\.463259\.3\(\+7\.94\)\(\+7\.94\)\(\+0\.89\)\(\+0\.89\)\(\+8\.70\)\(\+8\.70\)\(\+0\.60\)\(\+0\.60\)\(\+7\.18\)\(\+7\.18\)\(\+20\.65\)\(\+20\.65\)\(\+17\.69\)\(\+17\.69\)\(\+3\.82\)\(\+3\.82\)\(\+1\.80\)\(\+1\.80\)\(\+7\.70\)\(\+7\.70\)Gemma\-4\-E4B\-IT50\.5425\.1628\.1035\.3052\.6369\.1732\.5521\.8135\.8039\.010\.00N/AText\-only SFT48\.3723\.8630\.6033\.0559\.3376\.4731\.9520\.3735\.2039\.91\+0\.430\.04212\.9\+\+GRPO57\.3125\.7733\.1036\.4061\.2479\.7342\.8421\.3039\.2044\.10\+11\.500\.572289\.0\(\+6\.77\)\(\+6\.77\)\(\+0\.61\)\(\+0\.61\)\(\+5\.00\)\(\+5\.00\)\(\+1\.10\)\(\+1\.10\)\(\+8\.61\)\(\+8\.61\)\(\+10\.56\)\(\+10\.56\)\(\+10\.29\)\(\+10\.29\)\(−0\.51\)\(\-0\.51\)\(\+3\.40\)\(\+3\.40\)\(\+5\.09\)\(\+5\.09\)Gemma\-4\-E2B\-IT48\.2026\.4528\.0034\.3551\.6766\.1830\.0019\.5633\.5037\.550\.00N/AText\-only SFT47\.4522\.7629\.0033\.0557\.8972\.7528\.1418\.6935\.8038\.39\-0\.050\.0429\.8\+\+GRPO51\.9624\.0330\.5034\.1562\.2073\.2833\.7020\.0936\.9040\.76\+6\.730\.565215\.2\(\+3\.76\)\(\+3\.76\)\(−2\.42\)\(\-2\.42\)\(\+2\.50\)\(\+2\.50\)\(−0\.20\)\(\-0\.20\)\(\+10\.53\)\(\+10\.53\)\(\+7\.10\)\(\+7\.10\)\(\+3\.70\)\(\+3\.70\)\(\+0\.53\)\(\+0\.53\)\(\+3\.40\)\(\+3\.40\)\(\+3\.21\)\(\+3\.21\)Qwen3\-Omni\-30B71\.8533\.5038\.4069\.2563\.6446\.3550\.4027\.0054\.0050\.490\.00N/AText\-only SFT64\.6636\.0841\.3066\.1559\.8173\.2854\.0029\.6252\.7453\.07\+6\.060\.13531\.1\(−7\.19\)\(\-7\.19\)\(\+2\.58\)\(\+2\.58\)\(\+2\.90\)\(\+2\.90\)\(−3\.10\)\(\-3\.10\)\(−3\.83\)\(\-3\.83\)\(\+26\.93\)\(\+26\.93\)\(\+3\.60\)\(\+3\.60\)\(\+2\.62\)\(\+2\.62\)\(−1\.26\)\(\-1\.26\)\(\+2\.58\)\(\+2\.58\)

#### The reasoning gains extend across model scales and families\.

In Table[3](https://arxiv.org/html/2610.02819#S5.T3), text\-only SFT followed by GRPO raises Qwen2\.5\-Omni\-3B’s Macro Avg\. from 37\.23 to 44\.93, with a geometric mean improvement of\+20\.39%\+20\.39\\%\. It improves all nine scores over the base model, including gains of 20\.65 and 17\.69 points on SocialOmni How and Video\-Holmes, respectively\. On Gemma\-4\-E4B\-IT and Gemma\-4\-E2B\-IT, Macro Avg\. rises by 5\.09 and 3\.21 points, and eight and seven scores improve, respectively\. For the latter, GRPO improves all nine scores over SFT despite uneven SFT gains\. Even on the stronger Qwen3\-Omni\-30B\-A3B\-Instruct, text\-only SFT alone raises the Macro Avg\. from 50\.49 to 53\.07\. These results support text\-only training across scales and families\.

#### Text\-only reasoning training achieves a superior performance–compute trade\-off\.

The complete text\-only route in Table[2](https://arxiv.org/html/2610.02819#S5.T2)improves Macro Avg\. \(49\.39 versus 47\.37\) with fewer GPU\-hours \(284\.17 versus 654\.52\) than the native audio\-visual route\. Across SFT and GRPO, it processes 0\.505B versus 1\.306B training input tokens\. This reduces input tokens by 61\.3% and training compute by 56\.6%\. It achieves the highest Macro Avg\. among compared methods with fewer GPU\-hours than measured for AVATAR\-7B, Omni\-R1, and HumanOmniV2\-7B\. Thus, text\-only training outperforms the complete native audio\-visual route with less training compute\.

### 5\.2RQ2: Choice of Training Data

Table 4:Effect of training data choice on omni\-modal reasoning\.Boldandunderlinedscores denote the best and second\-best results, respectively\.Training ConfigurationOmni\-modal ReasoningAggregateDaily\-OmniOmni\-ClozeOmniVideoSocialOmniVideo\-HolmesVideo\-MME\-v2WorldSenseMacroAvg\.Geo\. MeanImprov\.\(%\)WhoWhenHowQwen2\.5\-Omni\-7B62\.0721\.9029\.3036\.7557\.4256\.6724\.4024\.3145\.4039\.800\.00SFT:General reasoning55\.7226\.5831\.4037\.3556\.9469\.0237\.5125\.4945\.9042\.88\+9\.88\+\+GRPO:Script\-based MCQ66\.5828\.5636\.9041\.4563\.6475\.4247\.2530\.4150\.0048\.91\+25\.67\+\+GRPO:Temporal QA55\.2227\.0134\.0038\.6057\.8976\.0938\.5425\.7544\.9044\.22\+12\.96\+\+GRPO:General & math QA55\.5626\.8833\.7038\.6563\.6474\.5938\.1625\.9244\.7044\.64\+13\.73SFT:Script\-based CoT \(MCQ\)52\.7225\.2531\.4039\.3053\.5966\.0727\.7122\.3440\.9039\.92\+1\.55\+\+GRPO:Script\-based MCQ65\.7527\.1737\.0041\.9568\.9079\.9344\.5830\.6648\.6049\.39\+25\.83\+\+GRPO:Temporal QA57\.0625\.8833\.5737\.9057\.4271\.0836\.2522\.8142\.0942\.67\+8\.48\+\+GRPO:General & math QA55\.8125\.9731\.5637\.5555\.5067\.9335\.4925\.2242\.7241\.97\+7\.58SFT:Contextual QA52\.8826\.4632\.8038\.2554\.0770\.9227\.8221\.4439\.9040\.50\+2\.50\+\+GRPO:Contextual QA55\.1426\.4731\.2037\.1559\.8173\.9835\.4425\.2442\.3042\.97\+9\.20

#### Script\-based MCQ yields stronger omni\-modal reasoning transfer\.

In Table[4](https://arxiv.org/html/2610.02819#S5.T4), GRPO on Script\-based MCQs achieves the highest Macro Avg\. after SFT on either General reasoning data or Script\-based MCQs with CoT supervision, reaching 48\.91 and 49\.39, respectively\. Contextual QA in both phases yields 42\.97\. Both MCQ configurations improve all nine scores over the base model, whereas other complete configurations improve the Macro Avg\. but not all scores\. Tracking entities and event states and integrating evidence across segments may explain these broader gains in omni\-modal reasoning\. Further results on data mixtures appear in Appendix[C\.1](https://arxiv.org/html/2610.02819#A3.SS1)\.

![Refer to caption](https://arxiv.org/html/2610.02819v1/figure_caption.png)Figure 3:Text\-only LLM synthesis of scripts, MCQs, and reasoning explanations, without audio\-visual data in construction or training\. The illustration shows a fictional script, not generated video\.

### 5\.3RQ3: Synthetic Training Data

Figure 4:Omni\-modal reasoning with training data synthesized entirely by a text\-only LLM for both SFT and GRPO\. Results cover four models across different scales and families\.#### Text\-only LLM synthesis extends text\-only training to data construction\.

Building on RQ1, we test text\-only reasoning training on SFT and GRPO data synthesized entirely by a text\-only LLM\. Neither data construction nor training uses any native audio\-visual data\. Figure[3](https://arxiv.org/html/2610.02819#S5.F3)shows generation from specified scenarios, task types, and modality requirements\. The LLM creates fictional episode scripts with visual descriptions, speech, and non\-speech sounds, maintaining entity and temporal consistency across segments\. MCQs require entity tracking, event\-state updates, and cross\-segment evidence integration; joint audio\-visual questions require complementary audio and visual evidence\. Scripts, MCQs, and reasoning explanations undergo validation and human review\.

#### Text\-only LLM synthesis yields omni\-modal reasoning gains\.

In Figure[4](https://arxiv.org/html/2610.02819#S5.F4), training on synthesized text yields a 7\.70\-point Macro Avg\. gain for Qwen2\.5\-Omni\-7B, from 39\.80 to 47\.50\. Gains on Qwen2\.5\-Omni\-3B, Gemma\-4\-E4B\-IT, and Gemma\-4\-E2B\-IT are 6\.59, 3\.93, and 2\.79 points, respectively\. The Qwen models improve nine and eight scores over their base models and remain within 1\.89 and 1\.11 Macro Avg\. points of Script\-based MCQ\. Crucially, these gains require no native audio\-visual data in construction or training\. This opens a path to scaling up reasoning training data without collecting or annotating corresponding audio\-visual samples\.

Table 5:Perceptual effects of text\-only and audio\-visual training\. Parenthesized values show absolute changes from the base model\.ModelPerceptual CapabilitiesAggregateAVHBenchMUSIC\-AVQAAVQANExT\-QAMacroAvg\.Geo\. MeanImprov\.\(%\)OverallAudio\-drivenVideo HallucinationVideo\-drivenAudio Hallucinationv1v2Which / Come From /Happening / WhereDescriptiveQwen2\.5\-Omni\-7B75\.5580\.7073\.0071\.2269\.1088\.7982\.2177\.370\.00Audio\-visual SFT78\.1282\.6075\.9175\.9174\.6290\.7083\.1380\.50\+4\.22\+\+GRPO78\.3181\.8776\.5573\.1570\.3290\.5882\.9979\.07\+2\.21\(\+2\.76\)\(\+2\.76\)\(\+1\.17\)\(\+1\.17\)\(\+3\.55\)\(\+3\.55\)\(\+1\.93\)\(\+1\.93\)\(\+1\.22\)\(\+1\.22\)\(\+1\.79\)\(\+1\.79\)\(\+0\.78\)\(\+0\.78\)\(\+1\.70\)\(\+1\.70\)Text\-only SFT75\.7880\.3073\.5565\.0962\.9383\.7579\.2273\.35\-5\.37\+\+GRPO \(Ours\)73\.9479\.4271\.2369\.6068\.9987\.6881\.8576\.41\-1\.25\(−1\.61\)\(\-1\.61\)\(−1\.28\)\(\-1\.28\)\(−1\.77\)\(\-1\.77\)\(−1\.62\)\(\-1\.62\)\(−0\.11\)\(\-0\.11\)\(−1\.11\)\(\-1\.11\)\(−0\.36\)\(\-0\.36\)\(−0\.96\)\(\-0\.96\)

### 5\.4RQ4: Perceptual Effects

#### Text\-only reasoning training incurs perceptual degradation\.

In Table[5](https://arxiv.org/html/2610.02819#S5.T5), text\-only SFT followed by GRPO reduces Qwen2\.5\-Omni\-7B’s perceptual Macro Avg\. from 77\.37 to 76\.41, with a 1\.25% decrease in the geometric mean of scores\. All seven perceptual scores decline by 0\.11–1\.77 points\. In contrast, native audio\-visual SFT followed by GRPO improves all seven scores and raises the Macro Avg\. by 1\.70 points to 79\.07\. The text\-only route’s 9\.59\-point reasoning gain in RQ1 accompanies a 0\.96\-point perceptual Macro Avg\. decline\. This pattern is consistent with the partial decoupling of perception and reasoning suggested by our preliminary study\. Nevertheless, text\-only reasoning training does not fully replace native audio\-visual training for preserving perception\.

### 5\.5RQ5: Training Paradigm

Table 6:Reasoning, perception, and training budgets under staged training\. A→\\toV and V→\\toA denote audio\-driven video and video\-driven audio hallucination\. Parentheses show absolute changes from the base model\. Budgets are per audio\-visual stage; reduced\-data budgets are shared across initializations\.\(a\) Omni\-modal ReasoningTraining ConfigurationDaily\-OmniOmni\-ClozeOmniVideoSocialOmniVideo\-HolmesVideo\-MME\-v2WorldSenseMacroAvg\.Geo\. MeanImprov\.\(%\)Text\-onlyAudio\-visualWhoWhenHowQwen2\.5\-Omni\-7B62\.0721\.9029\.3036\.7557\.4256\.6724\.4024\.3145\.4039\.800\.00Reduced\-data Audio\-visual SFT–SFT51\.3825\.8631\.9037\.9056\.4668\.5234\.0820\.6940\.1040\.77\+3\.56SFTSFT55\.1425\.7437\.4037\.3059\.8171\.5935\.4923\.3841\.6043\.05\+9\.60SFT \+ GRPOSFT65\.3327\.0637\.4042\.1063\.6470\.5645\.1828\.0947\.4047\.42\+21\.71Reduced\-data Audio\-visual GRPOSFTGRPO62\.7426\.7537\.4740\.8066\.5172\.3543\.2227\.9847\.1047\.21\+20\.81SFT \+ GRPOGRPO64\.7527\.0837\.1741\.1566\.5175\.0045\.4530\.1649\.0248\.48\+24\.15\(\+2\.68\)\(\+2\.68\)\(\+5\.18\)\(\+5\.18\)\(\+7\.87\)\(\+7\.87\)\(\+4\.40\)\(\+4\.40\)\(\+9\.09\)\(\+9\.09\)\(\+18\.33\)\(\+18\.33\)\(\+21\.05\)\(\+21\.05\)\(\+5\.85\)\(\+5\.85\)\(\+3\.62\)\(\+3\.62\)\(\+8\.67\)\(\+8\.67\)\(b\) Perceptual CapabilitiesTraining ConfigurationAVHBenchMUSIC\-AVQAAVQANExT\-QAMacroAvg\.Geo\. MeanImprov\.\(%\)Text\-onlyAudio\-visualOverallA→\\toVV→\\toAv1v2Which / Come From /Happening / WhereDescriptiveQwen2\.5\-Omni\-7B75\.5580\.7073\.0071\.2269\.1088\.7982\.2177\.370\.00Reduced\-data Audio\-visual SFT–SFT76\.2580\.8373\.9874\.5071\.6490\.6783\.3579\.28\+2\.53SFTSFT76\.3181\.1073\.9474\.4171\.6790\.6483\.2079\.25\+2\.49SFT \+ GRPOSFT74\.8780\.0272\.3273\.1470\.4290\.3983\.8478\.53\+1\.49Reduced\-data Audio\-visual GRPOSFTGRPO75\.8480\.6573\.4674\.1971\.3089\.1482\.7078\.63\+1\.73SFT \+ GRPOGRPO76\.0681\.6073\.3273\.2270\.4989\.6983\.0678\.50\+1\.51\(\+0\.51\)\(\+0\.51\)\(\+0\.90\)\(\+0\.90\)\(\+0\.32\)\(\+0\.32\)\(\+2\.00\)\(\+2\.00\)\(\+1\.39\)\(\+1\.39\)\(\+0\.90\)\(\+0\.90\)\(\+0\.85\)\(\+0\.85\)\(\+1\.13\)\(\+1\.13\)\(c\) Training BudgetMetricFull\-data Audio\-visualReduced\-data Audio\-visualSFTGRPOSFTGRPOInput Tok\. \(B\)↓\\downarrow0\.0921\.2140\.0020\.122GPU\-h↓\\downarrow12\.2642\.30\.482\.7

#### RQ1–RQ4 motivate a text\-centric training paradigm\.

Building on the reasoning gains in RQ1–RQ3 and perceptual degradation in RQ4, we investigate text\-centric post\-training\. Once reasoning has been strengthened through text\-only training, we test whether reduced\-data native audio\-visual training is sufficient to preserve perception\. Specifically, we apply audio\-visual SFT or GRPO after either text\-only SFT or the complete text\-only SFT–GRPO pipeline, with reduced\-data audio\-visual SFT alone as a control\. Table[6](https://arxiv.org/html/2610.02819#S5.T6)reports reasoning, perception, and training budgets\.

#### Staged training improves reasoning while maintaining perception\.

Table[6](https://arxiv.org/html/2610.02819#S5.T6)shows higher reasoning Macro Avg\. scores from the complete text\-only SFT–GRPO pipeline than from text\-only SFT under either audio\-visual refinement method\. With this initialization, audio\-visual GRPO achieves the highest reasoning Macro Avg\. among the staged configurations at 48\.48 versus 47\.42 for audio\-visual SFT, with nearly identical perception averages of 78\.50 and 78\.53\. This refinement uses about 90% fewer training input tokens than full\-data native audio\-visual GRPO\.

Compared with the complete text\-only route in Tables[2](https://arxiv.org/html/2610.02819#S5.T2)and[5](https://arxiv.org/html/2610.02819#S5.T5), this configuration improves the perception Macro Avg\. by 2\.09 points while reducing reasoning by 0\.91 points\. It retains 93\.5% of the text\-only reasoning gain measured by geometric mean improvement \(24\.15% versus 25\.83% before refinement\)\. It exceeds the full\-data native audio\-visual SFT–GRPO route by 1\.11 reasoning Macro Avg\. points, with a 0\.57\-point lower perception average\. Together, these results establish our recommended text\-centric post\-training paradigm for Omni\-LLMs: text\-only SFT followed by GRPO for the main reasoning optimization, then reduced\-data native audio\-visual GRPO for perceptual refinement\. This paradigm retains the main reasoning gains while restoring perception above the base level, addressing the performance–compute trade\-off\.

## 6Conclusion

Building on our diagnostics, we studied text\-centric post\-training for Omni\-LLMs\. Text\-only training improves reasoning across data sources, model scales, and families, even with fully synthetic text requiring no audio\-visual data in construction or training\. On Qwen2\.5\-Omni\-7B, our best\-performing text\-only pipeline outperforms the complete native audio\-visual route with less compute\. Reduced\-data native audio\-visual RL then restores perception while retaining most reasoning gains\. This text\-centric paradigm organizes post\-training around the capabilities to be improved: text for reasoning optimization and native audio\-visual supervision for perceptual refinement\.

## References

- Akter et al\. \(2026\)Syeda Nahida Akter, Shrimai Prabhumoye, Matvei Novikov, Seungju Han, Ying Lin, Evelina Bakhturina, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, et al\.Nemotron\-crossthink: Scaling self\-learning beyond math reasoning\.In*Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pp\. 984–1002, 2026\.
- Araujo et al\. \(2026\)Edson Araujo, Saurabhchand Bhati, M Jehanzeb Mirza, Brian Kingsbury, Samuel Thomas, Rogerio Feris, James R Glass, and Hilde Kuehne\.Avrt: Audio\-visual reasoning transfer through single\-modality teachers\.*arXiv preprint arXiv:2604\.16617*, 2026\.
- Bousselham et al\. \(2026\)Walid Bousselham, Hilde Kuehne, and Cordelia Schmid\.Vold: Reasoning transfer from llms to vision\-language models via on\-policy distillation\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp\. 26209–26218, 2026\.
- Cai et al\. \(2026\)Xinyue Cai, Chaoyou Fu, Yi\-Fan Zhang, Ran He, and Caifeng Shan\.Omnivideo\-100k: A dataset for audio\-visual reasoning through structured scripts and evidence chains\.*arXiv preprint arXiv:2606\.14702*, 2026\.
- Cavalcanti et al\. \(2024\)Joao V Cavalcanti, Laurent Lessard, and Ashia C Wilson\.Adaptive backtracking line search\.*arXiv preprint arXiv:2408\.13150*, 2024\.
- Cawley & Talbot \(2010\)Gavin C Cawley and Nicola LC Talbot\.On over\-fitting in model selection and subsequent selection bias in performance evaluation\.*The Journal of Machine Learning Research*, 11:2079–2107, 2010\.
- Chaudhry et al\. \(2018\)Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny\.Efficient lifelong learning with a\-gem\.*arXiv preprint arXiv:1812\.00420*, 2018\.
- Chen et al\. \(2025\)Lichang Chen, Hexiang Hu, Mingda Zhang, Yiwen Chen, Zifeng Wang, Yandong Li, Pranav Shyam, Tianyi Zhou, Heng Huang, Ming\-Hsuan Yang, et al\.Omnixr: Evaluating omni\-modality language models on reasoning across modalities\.In*International Conference on Learning Representations*, volume 2025, pp\. 68100–68118, 2025\.
- Chen et al\. \(2026a\)Shuang Chen, Hangyu Guo, Zhaochen Su, Yafu Li, Jiacheng Chen, Yulun Wu, Weijie Wang, ZhiYuan Feng, Xiaoye Qu, and Yu Cheng\.Revisual\-r1: Advancing multimodal reasoning from optimized cold start to staged reinforcement learning\.In*International Conference on Learning Representations*, volume 2026, pp\. 32869–32888, 2026a\.
- Chen et al\. \(2026b\)Yu Chen, Caorui Li, Ziyu Xiong, Yidong Wang, Mingqi Gao, Shuman Liu, Biao Liu, Chunfeng Yang, Anxiang Zeng, Haibo Zhang, et al\.Omnireasoner: Thinking with long audio\-video via native tool use\.*arXiv preprint arXiv:2607\.19339*, 2026b\.
- Chen et al\. \(2026c\)Zhangquan Chen, Jiale Tao, Ruihuang Li, Yihao Hu, Ruitao Chen, Zhantao Yang, Xinlei Yu, Haodong Jing, Manyuan Zhang, Shuai Shao, et al\.Omnivideo\-r1: Reinforcing audio\-visual reasoning with query intention and modality attention\.*arXiv preprint arXiv:2602\.05847*, 2026c\.
- Cheng et al\. \(2026\)Dongjie Cheng, Yongqi Li, Zhixin Ma, Hongru Cai, Yupeng Hu, Wenjie Wang, Liqiang Nie, and Wenjie Li\.Omni\-r1: Towards the unified generative paradigm for multimodal reasoning\.In*Findings of the Association for Computational Linguistics: ACL 2026*, pp\. 38907–38924, 2026\.
- Cheng et al\. \(2025\)Junhao Cheng, Yuying Ge, Teng Wang, Yixiao Ge, Jing Liao, and Ying Shan\.Video\-holmes: Can mllm think like holmes for complex video reasoning?*arXiv preprint arXiv:2505\.21374*, 2025\.
- Dai et al\. \(2026\)Yifan Dai, Zhenhua Wu, Bohan Zeng, Daili Hua, Jialing Liu, Bozhou Li, Yuran Wang, Chengzhuo Tong, Hao Liang, Xiaochen Ma, et al\.Latentomni: Rethinking omni\-modal understanding via unified audio\-visual latent reasoning\.*arXiv preprint arXiv:2605\.22012*, 2026\.
- Deshmukh et al\. \(2026\)Amala Sanjay Deshmukh, Kateryna Chumachenko, Tuomas Rintamaki, Matthieu Le, Tyler Poon, Danial Mohseni Taheri, Ilia Karmanov, Guilin Liu, Jarno Seppanen, Arushi Goel, et al\.Nemotron 3 nano omni: efficient and open multimodal intelligence\.*arXiv preprint arXiv:2604\.24954*, 2026\.
- Du et al\. \(2026\)Yuwen Du, Rui Ye, Shuo Tang, Xinyu Zhu, Yijun Lu, Yuzhu Cai, and Siheng Chen\.Openseeker: Democratizing frontier search agents by fully open\-sourcing training data\.*arXiv preprint arXiv:2603\.15594*, 2026\.
- Face \(2025\)Hugging Face\.Open r1: A fully open reproduction of deepseek\-r1, January 2025\.URL[https://github\.com/huggingface/open\-r1](https://github.com/huggingface/open-r1)\.
- Farajtabar et al\. \(2020\)Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li\.Orthogonal gradient descent for continual learning\.In*International conference on artificial intelligence and statistics*, pp\. 3762–3773\. PMLR, 2020\.
- Farré et al\. \(2024\)Miquel Farré, Andi Marafioti, Lewis Tunstall, Leandro Von Werra, and Thomas Wolf\.Finevideo\.[https://huggingface\.co/datasets/HuggingFaceFV/finevideo](https://huggingface.co/datasets/HuggingFaceFV/finevideo), 2024\.
- Fox & Monette \(2026\)John Fox and Georges Monette\.cv: An r package for cross\-validating regression models\.*Journal of Statistical Software*, 116:1–41, 2026\.
- Fu et al\. \(2026\)Chaoyou Fu, Haozhi Yuan, Yuhao Dong, Yi\-Fan Zhang, Yunhang Shen, Xiaoxing Hu, Xueying Li, Jinsen Su, Chengwu Long, Xiaoyao Xie, et al\.Video\-mme\-v2: Towards the next stage in benchmarks for comprehensive video understanding\.*arXiv preprint arXiv:2604\.05015*, 2026\.
- Guan et al\. \(2026\)Yiran Guan, Sifan Tu, Dingkang Liang, Linghao Zhu, Jianzhong Ju, Zhenbo Luo, Jian Luan, Yuliang Liu, and Xiang Bai\.Thinkomni: Lifting textual reasoning to omni\-modal scenarios via guidance decoding\.*arXiv preprint arXiv:2602\.23306*, 2026\.
- Guha et al\. \(2026\)Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, et al\.Openthoughts: Data recipes for reasoning models\.In*International Conference on Learning Representations*, volume 2026, pp\. 108059–108130, 2026\.
- Ho et al\. \(2020\)Xanh Ho, Anh\-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa\.Constructing a multi\-hop qa dataset for comprehensive evaluation of reasoning steps\.In*Proceedings of the 28th International Conference on Computational Linguistics*, pp\. 6609–6625, 2020\.
- Hong et al\. \(2026\)Jack Hong, Shilin Yan, Jiayin Cai, Xiaolong Jiang, Yao Hu, and Weidi Xie\.Worldsense: Evaluating real\-world omnimodal understanding for multimodal llms\.In*International Conference on Learning Representations*, volume 2026, pp\. 52423–52443, 2026\.
- Huang et al\. \(2025\)Chengyue Huang, Junjiao Tian, Brisa Maneechotesuwan, Shivang Chopra, and Zsolt Kira\.Directional gradient projection for robust fine\-tuning of foundation models\.In*International Conference on Learning Representations*, volume 2025, pp\. 60524–60544, 2025\.
- Jia et al\. \(2026\)Yiduo Jia, Muzhi Zhu, Hao Zhong, Mingyu Liu, Yuling Xi, Hao Chen, Bin Qin, Yongjie Yang, Zhenbo Luo, and Chunhua Shen\.Omnijigsaw: Enhancing omni\-modal reasoning via modality\-orchestrated reordering\.*arXiv preprint arXiv:2604\.08209*, 2026\.
- Kim et al\. \(2023\)Seungone Kim, Se Joo, Doyoung Kim, Joel Jang, Seonghyeon Ye, Jamin Shin, and Minjoon Seo\.The cot collection: Improving zero\-shot and few\-shot learning of language models via chain\-of\-thought fine\-tuning\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pp\. 12685–12708, 2023\.
- Kulkarni & Fazli \(2026\)Yogesh Kulkarni and Pooyan Fazli\.Avatar: Reinforcement learning to see, hear, and reason over video\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp\. 7912–7922, 2026\.
- Kumar \(2025\)Abhinav Kumar\.CoT\-Scientific\-RAG\-Reasoning\.Hugging Face dataset, December 2025\.URL[https://huggingface\.co/datasets/abhinavdread/CoT\-Scientific\-RAG\-Reasoning/tree/9a883557fd4b5b958c1c8b8b431ba08f6ae39c42](https://huggingface.co/datasets/abhinavdread/CoT-Scientific-RAG-Reasoning/tree/9a883557fd4b5b958c1c8b8b431ba08f6ae39c42)\.Revision 9a88355; accessed 2026\-08\-25\.
- Labs \(2025\)Bespoke Labs\.Bespoke\-stratos: The unreasonable effectiveness of reasoning distillation\.https://www\.bespokelabs\.ai/blog/bespoke\-stratos\-the\-unreasonable\-effectiveness\-of\-reasoning\-distillation, 2025\.Accessed: 2025\-01\-22\.
- Li et al\. \(2026a\)Caorui Li, Yu Chen, Yiyan Ji, Jin Xu, Zhenyu Cui, Shihao Li, Yuanxing Zhang, Zhenghao Song, Dingling Zhang, Ying He, et al\.Omnivideobench: Towards audio\-visual understanding evaluation for omni mllms\.In*International Conference on Learning Representations*, volume 2026, pp\. 138214–138236, 2026a\.
- Li et al\. \(2022\)Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu, Ji\-Rong Wen, and Di Hu\.Learning to answer questions in dynamic audio\-visual scenarios\.In*2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pp\. 19086–19096\. IEEE, 2022\.
- Li et al\. \(2025\)Lei Li, Yuanxin Liu, Linli Yao, Peiyuan Zhang, Chenxin An, Lean Wang, Xu Sun, Lingpeng Kong, and Qi Liu\.Temporal reasoning transfer from text to video\.In*International Conference on Learning Representations*, volume 2025, pp\. 74070–74102, 2025\.
- Li et al\. \(2026b\)Xiaoxi Li, Wenxiang Jiao, Jiarui Jin, Haoxuan Li, Hao Wang, Shijian Wang, Guanting Dong, Jiajie Jin, Yinuo Wang, Yuan Lu, et al\.Omnigaia: Towards native omni\-modal ai agents\.*arXiv preprint arXiv:2602\.22897*, 2026b\.
- Liu et al\. \(2026a\)Che Liu, Lichao Ma, Xiangyu Tony Zhang, Yuxin Zhang, Haoyang Zhang, Xuerui Yang, and Fei Tian\.Boosting omni\-modal language models: Staged post\-training with visually debiased evaluation\.*arXiv preprint arXiv:2605\.12034*, 2026a\.
- Liu et al\. \(2026b\)Hongcheng Liu, Pingjie Wang, Heyang Liu, Zhiyuan Zhu, Yusheng Liao, Yanfeng Wang, and Yu Wang\.Anchornet: Adaptive anchor token enhancement in video\-grounded dialogue generation\.*IEEE Journal of Selected Topics in Signal Processing*, 2026b\.
- Liu et al\. \(2026c\)Hongcheng Liu, Yuhao Wang, Zhe Chen, Pingjie Wang, Zhiyuan Zhu, Yixuan Hou, Yanfeng Wang, and Yu Wang\.Cross\-modal coreference alignment: Enabling reliable information transfer in omni\-LLMs\.In Maria Liakata, Viviane P\. Moreira, Jiajun Zhang, and David Jurgens \(eds\.\),*Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pp\. 26430–26453, San Diego, California, United States, July 2026c\. Association for Computational Linguistics\.ISBN 979\-8\-89176\-390\-6\.doi:10\.18653/v1/2026\.acl\-long\.1217\.URL[https://aclanthology\.org/2026\.acl\-long\.1217/](https://aclanthology.org/2026.acl-long.1217/)\.
- Liu et al\. \(2025a\)Jiang Liu, Jialian Wu, Xiaodong Yu, Yusheng Su, Prakamya Mishra, Gowtham Ramesh, Sudhanshu Ranjan, Chaitanya Manem, Ximeng Sun, Ze Wang, et al\.Instella: Fully open language models with stellar performance\.*arXiv preprint arXiv:2511\.10628*, 2025a\.
- Liu et al\. \(2025b\)Qianchu Liu, Sheng Zhang, Guanghui Qin, Timothy Ossowski, Yu Gu, Ying Jin, Sid Kiblawi, Sam Preston, Mu Wei, Paul Vozila, et al\.X\-reasoner: Towards generalizable reasoning across modalities and domains\.*arXiv preprint arXiv:2505\.03981*, 2025b\.
- Liu et al\. \(2024\)Xiulong Liu, Zhikang Dong, and Peng Zhang\.Tackling data bias in music\-avqa: Crafting a balanced dataset for unbiased question\-answering\.In*Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*, pp\. 4478–4487, 2024\.
- Lozhkov et al\. \(2025\)Anton Lozhkov, Hynek Kydlíček, Loubna Ben Allal, Guilherme Penedo, Edward Beeching, Quentin Gallouédec, Nathan Habib, Lewis Tunstall, and Leandro von Werra\.Openr1\-math\-220k\.[https://huggingface\.co/datasets/open\-r1/OpenR1\-Math\-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k), 2025\.
- Lu et al\. \(2025\)Fanbin Lu, Zhisheng Zhong, Shu Liu, Chi\-Wing Fu, and Jiaya Jia\.Arpo: End\-to\-end policy optimization for gui agents with experience replay\.*arXiv preprint arXiv:2505\.16282*, 2025\.
- Ma et al\. \(2026\)Ziyang Ma, Ruiyang Xu, Zhenghao Xing, Yunfei Chu, Yuxuan Wang, Jinzheng He, Jin Xu, Pheng\-Ann Heng, Kai Yu, Junyang Lin, et al\.Omni\-captioner: Data pipeline, models, and benchmark for omni detailed perception\.In*International Conference on Learning Representations*, volume 2026, pp\. 132277–132304, 2026\.
- Moshkov et al\. \(2025\)Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman\.Aimo\-2 winning solution: Building state\-of\-the\-art mathematical reasoning models with openmathreasoning dataset\.*arXiv preprint arXiv:2504\.16891*, 2025\.
- Mukherjee et al\. \(2026\)Swapnanil Mukherjee, Agyeya Negi, Tanuja Ganu, and Ponnurangam Kumaraguru\.C3po: Evaluating cross\-modal composition and counterfactual performance in omnimodal models\.*arXiv preprint arXiv:2608\.05381*, 2026\.
- Peng et al\. \(2026\)Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang\.Lmm\-r1: Empowering 3b lmms with strong reasoning abilities with two\-stage rule\-based rl\.*Frontiers of Computer Science*, 2026\.
- Qin et al\. \(2026\)Zheng Qin, Ruobing Zheng, Yabing Wang, Tianqi Li, Yi Yuan, Jingdong Chen, and Le Wang\.Humansense: From multimodal perception to empathetic context\-aware responses through reasoning mllms\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, volume 40, pp\. 24973–24981, 2026\.
- Qiu et al\. \(2026\)Haomiao Qiu, Miao Zhang, Ziyue Qiao, Weili Guan, Min Zhang, and Liqiang Nie\.Splitlora: Balancing stability and plasticity in continual learning through gradient space splitting\.In*International Conference on Learning Representations*, volume 2026, pp\. 48458–48480, 2026\.
- Rouditchenko et al\. \(2025\)Andrew Rouditchenko, Saurabhchand Bhati, Edson Araujo, Samuel Thomas, Hilde Kuehne, Rogerio Feris, and James Glass\.Omni\-r1: Do you really need audio to fine\-tune your audio llm?In*2025 IEEE Automatic Speech Recognition and Understanding Workshop \(ASRU\)*, pp\. 1–7\. IEEE, 2025\.
- Shao et al\. \(2024\)Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al\.Deepseekmath: Pushing the limits of mathematical reasoning in open language models\.*arXiv preprint arXiv:2402\.03300*, 2024\.
- Sung\-Bin et al\. \(2025\)Kim Sung\-Bin, Oh Hyun\-Bin, Lee Jung\-Mok, Arda Senocak, Joon Son Chung, and Tae\-Hyun Oh\.Avhbench: A cross\-modal hallucination benchmark for audio\-visual large language models\.In*International Conference on Learning Representations*, volume 2025, pp\. 24244–24271, 2025\.
- Team et al\. \(2026\)Gemma Team, Sherif El Abd, Vaibhav Aggarwal, Robin Algayres, Alek Andreev, Olivier Bachem, Ian Ballantyne, Cormac Brick, Victor Cărbune, Michelle Casbon, et al\.Gemma 4 technical report\.*arXiv preprint arXiv:2607\.02770*, 2026\.
- Team et al\. \(2025\)MiroMind Team, Song Bai, Lidong Bing, Carson Chen, Guanzheng Chen, Yuntao Chen, Zhe Chen, Ziyi Chen, Jifeng Dai, Xuan Dong, et al\.Mirothinker: Pushing the performance boundaries of open\-source research agents via model, context, and interactive scaling\.*arXiv preprint arXiv:2511\.11793*, 2025\.
- Team \(2026\)Qwen Team\.Qwen3\.5\-omni technical report\.*arXiv preprint arXiv:2604\.15804*, 2026\.
- Trivedi et al\. \(2022\)Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal\.musique: Multihop questions via single\-hop question composition\.*Transactions of the Association for Computational Linguistics*, 10:539–554, 2022\.
- Wang et al\. \(2026a\)Xingrui Wang, Jiang Liu, Chao Huang, Xiaodong Yu, Ze Wang, Ximeng Sun, Jialian Wu, Alan Yuille, Emad Barsoum, and Zicheng Liu\.Xmodbench: Benchmarking cross\-modal capabilities and consistency in omni\-language models\.In*International Conference on Learning Representations*, volume 2026, pp\. 19508–19521, 2026a\.
- Wang et al\. \(2026b\)Yaoting Wang, Ziyi Zhang, Wenming Tu, Shaoxuan Xu, Wenjie Du, Cheng Liang, Weijun Wang, Yuanchao Li, Guangyao Li, Hao Fei, et al\.Avi\-bench: Toward human\-like audio\-visual intelligence of omni\-mllms\.*arXiv preprint arXiv:2606\.07643*, 2026b\.
- Wang et al\. \(2026c\)Yucheng Wang, Yifan Hou, Aydin Javadov, Mubashara Akhtar, and Mrinmaya Sachan\.Compose and fuse: Revisiting the foundational bottlenecks in multimodal reasoning\.In*International Conference on Learning Representations*, volume 2026, pp\. 11272–11301, 2026c\.
- Wu et al\. \(2026\)Juncheng Wu, Hardy Chen, Haoqin Tu, Xianfeng Tang, Freda Shi, Hui Liu, Hanqing Lu, Cihang Xie, and Yuyin Zhou\.From seeing to thinking: Decoupling perception and reasoning improves post\-training of vision\-language models\.*arXiv preprint arXiv:2605\.20177*, 2026\.
- Wu et al\. \(2025\)Yifan Wu, Jingze Shi, Bingheng Wu, Jiayi Zhang, Xiaotian Lin, Nan Tang, and Yuyu Luo\.Concise reasoning, big gains: Pruning long reasoning trace with difficulty\-aware prompting\.*arXiv preprint arXiv:2505\.19716*, 2025\.
- Xiao et al\. \(2021\)Junbin Xiao, Xindi Shang, Angela Yao, and Tat\-Seng Chua\.Next\-qa: Next phase of question\-answering to explaining temporal actions\.In*2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pp\. 9772–9781\. IEEE, 2021\.
- Xie et al\. \(2026\)Tianyu Xie, Jinfa Huang, Yuexiao Ma, Rongfang Luo, Yan Yang, Qingchuan Ma, Wang Chen, Yuhui Zeng, Yixuan Zou, Zhiqiang Lu, et al\.Socialomni: Benchmarking audio\-visual social interactivity in omni models\.*arXiv preprint arXiv:2603\.16859*, 2026\.
- Xing et al\. \(2025\)Zhenghao Xing, Xiaowei Hu, Chi\-Wing Fu, Wenhai Wang, Jifeng Dai, and Pheng\-Ann Heng\.Echoink\-r1: Exploring audio\-visual reasoning in multimodal llms via reinforcement learning\.*arXiv preprint arXiv:2505\.04623*, 2025\.
- Xu et al\. \(2025a\)Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin\.Qwen2\.5\-omni technical report\.*arXiv preprint arXiv:2503\.20215*, 2025a\.
- Xu et al\. \(2025b\)Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al\.Qwen3\-omni technical report\.*arXiv preprint arXiv:2509\.17765*, 2025b\.
- Xu et al\. \(2026\)Ke Xu, Yuhao Wang, Ziyang Cheng, Hongcheng Liu, Yanfeng Wang, and Yu Wang\.Agentic active omni\-modal perception for multi\-hop audio\-visual reasoning\.*arXiv preprint arXiv:2605\.28192*, 2026\.
- Yang et al\. \(2025a\)Chen Yang, Ran Le, Yun Xing, Zhenwei An, Zongchao Chen, Wayne Xin Zhao, Yang Song, and Tao Zhang\.Toolmind technical report: A large\-scale, reasoning\-enhanced tool\-use dataset\.*arXiv preprint arXiv:2511\.15718*, 2025a\.
- Yang et al\. \(2022\)Pinci Yang, Xin Wang, Xuguang Duan, Hong Chen, Runze Hou, Cong Jin, and Wenwu Zhu\.Avqa: A dataset for audio\-visual question answering on videos\.In*Proceedings of the 30th ACM international conference on multimedia*, pp\. 3480–3491, 2022\.
- Yang et al\. \(2025b\)Qize Yang, Shimin Yao, Weixuan Chen, Shenghao Fu, Detao Bai, Jiaxing Zhao, Boyuan Sun, Bowen Yin, Xihan Wei, and Jingren Zhou\.Humanomniv2: From understanding to omni\-modal reasoning with context\.*arXiv preprint arXiv:2506\.21277*, 2025b\.
- Yang et al\. \(2018\)Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning\.Hotpotqa: A dataset for diverse, explainable multi\-hop question answering\.In*Proceedings of the 2018 conference on empirical methods in natural language processing*, pp\. 2369–2380, 2018\.
- Yao et al\. \(2026\)Ruilin Yao, Bo Zhang, Jirui Huang, Xinwei Long, Yifang Zhang, Tianyu Zou, Shili Xiong, Yi Rong, Yufei Wu, Shichao Su, et al\.Lens: Multi\-level evaluation of multimodal reasoning with large language models\.In*International Conference on Learning Representations*, volume 2026, pp\. 2627–2654, 2026\.
- Yu et al\. \(2025\)Xiaomin Yu, Wenjie Zhang, Ziyue Qiao, Chengwei Qin, and Hui Xiong\.Text\-only data synthesis for vision language model training\.*arXiv preprint arXiv:2503\.22655*, 2025\.
- Zhang et al\. \(2025\)Weijia Zhang, Zijia Liu, Haoru Li, Haoqi Chen, and Jiaxuan You\.Seeingeye: Agentic information flow unlocks multimodal reasoning in text\-only llms\.*arXiv preprint arXiv:2510\.25092*, 2025\.
- Zhong et al\. \(2025\)Hao Zhong, Muzhi Zhu, Zongze Du, Zheng Huang, Canyu Zhao, Mingyu Liu, Wen Wang, Hao Chen, and Chunhua Shen\.Omni\-r1: Reinforcement learning for omnimodal reasoning via two\-system collaboration\.In D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(eds\.\),*Advances in Neural Information Processing Systems*, volume 38, Main Conference, pp\. 45266–45297\. Curran Associates, Inc\., 2025\.doi:10\.52202/085713\-1508\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2025/file/4086fe59dc3584708468fba0e459f6a7\-Paper\-Conference\.pdf](https://proceedings.neurips.cc/paper_files/paper/2025/file/4086fe59dc3584708468fba0e459f6a7-Paper-Conference.pdf)\.
- Zhou et al\. \(2025\)Ziwei Zhou, Rui Wang, Zuxuan Wu, and Yu\-Gang Jiang\.Daily\-omni: Towards audio\-visual reasoning with temporal alignment across modalities\.*arXiv preprint arXiv:2505\.17862*, 2025\.

## Appendix ADiagnostic Analysis

### A\.1Diagnostic Question Construction and Evaluation

This section details the construction and evaluation of the diagnostic questions used in Section[3](https://arxiv.org/html/2610.02819#S3)to examine perception and reasoning in Omni\-LLMs\.

#### Question taxonomy\.

We categorize questions by the number of reasoning steps specified during construction\. Single\-hop perception questions assess the direct extraction of information from audio\-visual inputs\. Multi\-hop questions combine facts from corresponding single\-hop QA pairs across audio and vision in two or three steps\. They must connect entities, events, attributes, or relations rather than merely group independent questions from the same video or topic\. Both question types cover five domains: Causal, Character, Modality, Temporal, and Theme\.

#### Data construction\.

We use FineVideo’s audio\-visual data and annotations\([Farré et al\., 2024](https://arxiv.org/html/2610.02819#bib.bib19)\)to generate QA pairs with Qwen3\.7\-Max and convert them into four\-option MCQs\. All nine models independently answer the original set of approximately 4,000 single\-hop questions, which remains the full 1\-hop evaluation set in Section[3](https://arxiv.org/html/2610.02819#S3)\. Only questions answered correctly by all nine models are used to construct multi\-hop questions\. Each multi\-hop question combines facts from these single\-hop QA pairs, supported by relevant audio or visual segments\. Together with the question and stated task rules, these facts must uniquely determine the answer without unstated specialized knowledge\. We remove ambiguous, underspecified, or unsupported questions, as well as cases solvable from a single cue or commonsense even with answer options\. Together, the original single\-hop and resulting multi\-hop sets draw on 2,703 videos; the latter contains approximately 5,000 questions, equally divided between 2\-hop and 3\-hop\.

#### Evaluation protocol\.

Table[7](https://arxiv.org/html/2610.02819#A1.T7)lists the evaluated models and their accuracy at each hop count\. Models receive the question and its four options with the corresponding audio\-visual inputs\. For each model, a multi\-hop question and its corresponding single\-hop questions use the same audio\-visual material and preprocessing settings, with each question answered in a separate call\. Multi\-hop evaluation provides no preceding single\-hop questions, answers, evidence hints, or conversation history\. All models use the original single\-hop set and the final multi\-hop sets defined above\. Accuracy is the fraction of correctly answered questions within each set\. For Figure[2\(a\)](https://arxiv.org/html/2610.02819#S3.F2.sf1), domain scores are averaged across models, and the macro\-average assigns equal weight to all five domains\.

Table 7:Accuracy \(%\) by model and question hop count in the diagnostic study\.Model1\-hop2\-hop3\-hopAVATAR\-7B\([Kulkarni & Fazli, 2026](https://arxiv.org/html/2610.02819#bib.bib29)\)93\.462\.060\.5HumanSense\([Qin et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib48)\)94\.664\.665\.2Omni\-R1\([Zhong et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib75)\)94\.065\.062\.5EchoInk\-R1\-7B\([Xing et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib64)\)95\.571\.068\.4HumanOmniV2\-7B\([Yang et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib70)\)96\.371\.872\.2OmniVideo\-7B\([Cai et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib4)\)95\.169\.567\.5Qwen2\.5\-Omni\-3B\([Xu et al\., 2025a](https://arxiv.org/html/2610.02819#bib.bib65)\)90\.937\.430\.0Qwen2\.5\-Omni\-7B\([Xu et al\., 2025a](https://arxiv.org/html/2610.02819#bib.bib65)\)93\.860\.058\.0Qwen3\-Omni\-30B\-A3B\-Instruct\([Xu et al\., 2025b](https://arxiv.org/html/2610.02819#bib.bib66)\)98\.575\.276\.0

### A\.2Local Optimization Analysis

#### Experimental setup\.

We use Qwen2\.5\-Omni\-3B with the full question sets defined above\. Each objective averages the cross\-entropy loss for predicting the correct answer token, using probabilities normalized over the full vocabulary\. We compute gradients for the Thinker’s Transformer decoder layers and final normalization layer, with all other parameters fixed\. Gradients are averaged across questions before normalization\.

We partition the data into five video\-level folds, keeping all questions from each video in the same fold\([Fox & Monette, 2026](https://arxiv.org/html/2610.02819#bib.bib20)\)\. For each split, gradient estimation for constructing update directions and update\-norm calibration use only the four training folds\. All updates start from the same initial checkpoint and are evaluated on the held\-out fold, which does not inform update construction or step\-size selection\([Cawley & Talbot, 2010](https://arxiv.org/html/2610.02819#bib.bib6)\)\.

#### First\-order analysis\.

Gradient projection is used to constrain parameter updates in multi\-objective fine\-tuning and continual learning\([Farajtabar et al\., 2020](https://arxiv.org/html/2610.02819#bib.bib18);[Chaudhry et al\., 2018](https://arxiv.org/html/2610.02819#bib.bib7);[Huang et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib26);[Qiu et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib49)\)\. Within each split, the following analysis uses the training\-fold objectives and gradients, with the split index omitted for clarity\. We write an update as−η​u\-\\eta u, whereu=gt/‖gt‖2u=g\_\{t\}/\\\|g\_\{t\}\\\|\_\{2\}for the original update andu=rt⟂s/‖rt⟂s‖2u=r\_\{t\\perp s\}/\\\|r\_\{t\\perp s\}\\\|\_\{2\}for the projected update in Equation[1](https://arxiv.org/html/2610.02819#S3.E1)\. Both directions have unit norm, soη\\etais the Euclidean norm of the parameter change\. The first\-order change in the other training objective is−η​gs⊤​u\-\\eta g\_\{s\}^\{\\top\}u; Equation[1](https://arxiv.org/html/2610.02819#S3.E1)sets this term to zero\. Forc=gE⊤​gR/\(‖gE‖2​‖gR‖2\)c=g\_\{E\}^\{\\top\}g\_\{R\}/\(\\\|g\_\{E\}\\\|\_\{2\}\\\|g\_\{R\}\\\|\_\{2\}\), the maximum first\-order decrease in the target loss per unit update norm satisfies

max‖u‖2≤1,gs⊤​u=0⁡gt⊤​u=‖rt⟂s‖2,‖rt⟂s‖2‖gt‖2=1−c2\.\\max\_\{\\\|u\\\|\_\{2\}\\leq 1,\\;g\_\{s\}^\{\\top\}u=0\}g\_\{t\}^\{\\top\}u=\\\|r\_\{t\\perp s\}\\\|\_\{2\},\\qquad\\frac\{\\\|r\_\{t\\perp s\}\\\|\_\{2\}\}\{\\\|g\_\{t\}\\\|\_\{2\}\}=\\sqrt\{1\-c^\{2\}\}\.\(2\)Hereuuranges over feasible directions\. For any suchuu,gt⊤​u=rt⟂s⊤​ug\_\{t\}^\{\\top\}u=r\_\{t\\perp s\}^\{\\top\}u; Cauchy–Schwarz gives the bound\. Whenrt⟂s≠0r\_\{t\\perp s\}\\neq 0, equality is attained atu=rt⟂s/‖rt⟂s‖2u=r\_\{t\\perp s\}/\\\|r\_\{t\\perp s\}\\\|\_\{2\}, whose unit norm matches the update in Equation[1](https://arxiv.org/html/2610.02819#S3.E1)\. Across the five splits, the mean gradient cosine similarity is 0\.5391, and the mean of the split\-specific first\-order retention ratios is 84\.23% at the same update norm\. Held\-out loss responses are measured after applying each split\-specific update; training\-gradient orthogonality does not impose a zero first\-order change on held\-out objectives\.

Table 8:First\-order retention \(%\) on held\-out objectives, averaged over three seeds\. Values are projected\-to\-original ratios of predicted loss decrease\.UpdateTargetretentionOtherretentionE1E\_\{1\}projected \(⟂E2\\perp E\_\{2\}\)56\.653\.6E2E\_\{2\}projected \(⟂E1\\perp E\_\{1\}\)60\.958\.5E1E\_\{1\}projected \(⟂R\\perp R\)71\.85\.7RRprojected \(⟂E1\\perp E\_\{1\}\)85\.013\.1We compare random perception halvesE1/E2E\_\{1\}/E\_\{2\}with a size\-matchedE1/RE\_\{1\}/Rpair, projecting in both directions\. The full reasoning set contains equal numbers of 2\-hop and 3\-hop questions\. Sampling is stratified by hop count and question type, with independent held\-out perception splits\. We pool held\-out directional derivatives across folds by question count before computing projected\-to\-original ratios\.Table[8](https://arxiv.org/html/2610.02819#A1.T8.fig1)shows similar retention on the two perception halves, whereas cross\-task projections show higher target retention and lower retention on the other objective\.

Table 9:Loss responses at two update norms, withRRalso reported separately for 2\-hop and 3\-hop questions\. The higher\-stepEE/RRresponses correspond to Figure[2\(b\)](https://arxiv.org/html/2610.02819#S3.F2.sf2)\. All values are percentages; positive reductions indicate loss decreases\.StepUpdate𝑬Ereduction𝑹RreductionTarget retentionOverall2\-hop3\-hopLowerE original\+1\.675\+1\.910\+1\.867\+1\.985100\.00LowerE projected\+1\.130\+0\.006\+0\.008\+0\.00367\.48LowerR original\+1\.176\+3\.331\+2\.718\+4\.397100\.00LowerR projected\-0\.040\+2\.860\+2\.077\+4\.22485\.89HigherE original\+3\.632\+4\.333\+4\.201\+4\.563100\.00HigherE projected\+2\.699\+0\.362\+0\.466\+0\.18174\.30HigherR original\+2\.080\+7\.358\+5\.694\+10\.254100\.00HigherR projected\+0\.060\+6\.013\+4\.150\+9\.25581\.72
Table 10:Loss reductions \(%\) by held\-out video\-level fold at both update norms\.𝑬Eoriginal𝑬Eprojected𝑹Roriginal𝑹RprojectedStep / held\-out fold𝑳𝑬L\_\{E\}𝑳𝑹L\_\{R\}𝑳𝑬L\_\{E\}𝑳𝑹L\_\{R\}𝑳𝑬L\_\{E\}𝑳𝑹L\_\{R\}𝑳𝑬L\_\{E\}𝑳𝑹L\_\{R\}Lower / 1\+1\.181\+1\.944\+0\.934\+0\.006\+0\.834\+3\.106\-0\.042\+2\.272Lower / 2\+2\.441\+1\.589\+1\.407\+0\.006\+1\.417\+3\.252\-0\.032\+2\.854Lower / 3\+1\.474\+1\.678\+0\.956\+0\.006\+1\.183\+3\.416\-0\.060\+2\.902Lower / 4\+1\.555\+1\.850\+1\.272\-0\.002\+0\.887\+2\.789\-0\.048\+2\.655Lower / 5\+1\.749\+2\.439\+1\.132\+0\.015\+1\.490\+4\.020\-0\.020\+3\.524Lower / Pooled\+1\.675\+1\.910\+1\.130\+0\.006\+1\.176\+3\.331\-0\.040\+2\.860Higher / 1\+2\.794\+3\.986\+2\.342\+0\.296\+1\.739\+7\.247\+0\.076\+6\.037Higher / 2\+4\.248\+4\.072\+3\.289\+0\.354\+2\.669\+7\.180\+0\.068\+5\.959Higher / 3\+3\.748\+4\.299\+2\.877\+0\.385\+1\.848\+7\.533\+0\.031\+5\.952Higher / 4\+3\.720\+4\.505\+2\.401\+0\.331\+1\.961\+6\.991\+0\.039\+5\.529Higher / 5\+3\.628\+4\.734\+2\.544\+0\.435\+2\.208\+7\.794\+0\.091\+6\.546Higher / Pooled\+3\.632\+4\.333\+2\.699\+0\.362\+2\.080\+7\.358\+0\.060\+6\.013
#### Step sizes\.

Within each split, we calibrate the higher update norm on 128 questions drawn from the four training folds: 64 perception questions and 64 reasoning questions, equally divided between 2\-hop and 3\-hop\. Starting from an initial candidate norm of approximately1\.44×10−21\.44\\times 10^\{\-2\}, we halve the candidate norm until both original update directions satisfy the Armijo condition on their respective calibration objectives, using the calibration losses and directional derivatives withσ=0\.1\\sigma=0\.1\([Cavalcanti et al\., 2024](https://arxiv.org/html/2610.02819#bib.bib5)\)\. This search selectsηhigh≈7\.20×10−3\\eta\_\{\\rm high\}\\approx 7\.20\\times 10^\{\-3\}in all five splits\. We then setηlow=ηhigh/2≈3\.60×10−3\\eta\_\{\\rm low\}=\\eta\_\{\\rm high\}/2\\approx 3\.60\\times 10^\{\-3\}to assess sensitivity to the update norm\. In each split, we verify that this lower norm also satisfies the Armijo condition for both original directions on their respective calibration objectives\. All four update directions use both norms, with each update starting fromθ0\\theta\_\{0\}\.

After fixing both norms, we check sufficient decrease on each held\-out fold for all four split\-specific update directions at both norms, without modifying either norm\. LetLtvalL\_\{t\}^\{\\rm val\}denote the held\-out objective andht=∇Ltval​\(θ0\)h\_\{t\}=\\nabla L\_\{t\}^\{\\rm val\}\(\\theta\_\{0\}\)\. ForDt=Ltval​\(θ0\)−Ltval​\(θ0−η​u\)D\_\{t\}=L\_\{t\}^\{\\rm val\}\(\\theta\_\{0\}\)\-L\_\{t\}^\{\\rm val\}\(\\theta\_\{0\}\-\\eta u\), the Armijo condition isDt≥σ​η​ht⊤​uD\_\{t\}\\geq\\sigma\\eta h\_\{t\}^\{\\top\}u, withσ=0\.1\\sigma=0\.1andht⊤​u\>0h\_\{t\}^\{\\top\}u\>0\. Across the 40 checks \(eight per split\),Dt/\(η​ht⊤​u\)D\_\{t\}/\(\\eta h\_\{t\}^\{\\top\}u\)ranges from 0\.328 to 0\.512, satisfying the Armijo condition in every case\. This establishes sufficient decrease on the held\-out objectives without requiring accurate first\-order predictions of the loss reduction\.

#### Measured responses\.

For cross\-validation, each question is evaluated only under the update constructed without its video\-level fold\. We pool question\-level baseline losses and loss reductions across the five held\-out folds before computing percentages, rather than averaging fold\-level percentages\. Writing the pooled mean loss reduction asD¯t\\bar\{D\}\_\{t\}and the pooled initial mean loss asL¯t​\(θ0\)\\bar\{L\}\_\{t\}\(\\theta\_\{0\}\), we report relative loss reduction as100​D¯t/L¯t​\(θ0\)100\\bar\{D\}\_\{t\}/\\bar\{L\}\_\{t\}\(\\theta\_\{0\}\)and target retention as100​D¯tprojected/D¯toriginal100\\bar\{D\}\_\{t\}^\{\\rm projected\}/\\bar\{D\}\_\{t\}^\{\\rm original\}\. ForRR, losses are pooled across 2\-hop and 3\-hop questions before computing percentages\. Table[A\.2](https://arxiv.org/html/2610.02819#A1.SS2.SSS0.Px2)reports both update norms; its higher\-step overall responses match Figure[2\(b\)](https://arxiv.org/html/2610.02819#S3.F2.sf2)\. At the lower step, the projected perception update retains 67\.48% of its target loss reduction and decreases reasoning loss by 0\.006%\. The projected reasoning update retains 85\.89% of its target loss reduction and increases perception loss by 0\.040%\. Thus, the lower step also supports local partial decoupling of the two mean\-loss objectives\. Table[10](https://arxiv.org/html/2610.02819#A1.T10)gives the held\-out\-fold breakdown\.

## Appendix BTraining and Evaluation Details

### B\.1Training Setup

#### Training data and preprocessing\.

Table[4\.2](https://arxiv.org/html/2610.02819#S4.SS2.SSS0.Px2)lists the training data types and sources\. Text\-only and audio\-visual GRPO use the same questions in the main comparison\. For RQ3, Qwen3\.7\-Max generates fictional scripts and MCQs with answers and reasoning explanations from abstract scenarios, task types, and aggregate distribution statistics, without native audio\-visual inputs \(Figure[3](https://arxiv.org/html/2610.02819#S5.F3)\)\. Scenarios cover daily life, work and craft, entertainment, sports, and social interaction\. Programmatic checks validate structure, timing, and answer formats; model\-based checks assess consistency, answer uniqueness, and distractor plausibility\. We manually review a subset of synthetic samples\. Generation and validation prompts appear in Appendix[D\.2](https://arxiv.org/html/2610.02819#A4.SS2)\.

#### Training settings\.

We apply LoRA \(r=16r=16,α=32\\alpha=32\) to the Thinker’s Transformer decoder in Qwen\-Omni and the language model in Gemma\-4\. Audio and vision encoders and multimodal projection modules remain frozen\. SFT and GRPO use AdamW with linear learning\-rate warmup over the first 5% of training steps, followed by cosine decay\. Table[11](https://arxiv.org/html/2610.02819#A2.T11)lists the hyperparameters\.

GRPO clips probability ratios to\[0\.8,1\.2\]\[0\.8,1\.2\]\(ϵ=0\.2\\epsilon=0\.2\)\. Qwen2\.5\-Omni\-7B uses the same SFT and GRPO hyperparameters for text\-only and audio\-visual training\. For audio\-visual training, we retain audio and sample frames at 1 FPS, with up to 180 frames per video and 200,704 pixels per frame\. Other settings, including sequence\-length limits, follow official base\-model defaults\.

Table 11:SFT and GRPO settings\. GRPO batch sizes count completions; “–” marks unused stages\.HyperparameterQwen2\.5\-Omni\-7BQwen2\.5\-Omni\-3BGemma\-4\-E4B\-ITGemma\-4\-E2B\-ITQwen3\-Omni\-30B\-A3B\-InstructSFTLearning rate2×10−62\\times 10^\{\-6\}5×10−65\\times 10^\{\-6\}5×10−55\\times 10^\{\-5\}5×10−55\\times 10^\{\-5\}10−410^\{\-4\}Global batch484848484848484888Epochs1133111111GRPOLearning rate10−510^\{\-5\}10−510^\{\-5\}10−510^\{\-5\}10−510^\{\-5\}–Global batch1616161616161616–Epochs11111111–Generations per prompt88888888–Temperature1\.01\.01\.01\.01\.01\.01\.01\.0–Top\-pp0\.990\.990\.990\.990\.950\.950\.950\.95–Top\-kk5050505064646464–KL coefficientβ\\beta0\.0010\.0010\.0010\.0010\.0010\.0010\.0010\.001–Clipping thresholdϵ\\epsilon0\.20\.20\.20\.20\.20\.20\.20\.2–Repetition penalty1\.01\.01\.01\.01\.01\.01\.01\.0–
#### Rewards\.

GRPO combines rule\-based accuracy and format rewards asR=Racc\+0\.1​RformatR=R\_\{\\mathrm\{acc\}\}\+0\.1R\_\{\\mathrm\{format\}\}\. The accuracy reward checks extracted final answers against references using task\-specific verification, including option matching for MCQs and answer verification for mathematical problems\. The format reward checks the required structure for reasoning and final answers\.

#### Implementation and compute accounting\.

We train on eight NVIDIA A100 PCIe 80 GB GPUs\. We reproduce external baselines with their official training datasets and original training stages, measuring GPU\-hours on the same hardware\. We count non\-padding training input tokens using each model’s tokenizer and training template, including special tokens and expanded audio and visual token positions\.ℐ\\mathcal\{I\}indexes sample occurrences across all epochs within each stage\.LiSFTL\_\{i\}^\{\\mathrm\{SFT\}\}denotes the complete sequence length including assistant targets, andLipromptL\_\{i\}^\{\\mathrm\{prompt\}\}denotes the prompt length before generation\. WithG=8G=8completions per prompt, counts in billions are

TSFT=10−9​∑i∈ℐLiSFT,TGRPO=10−9​G​∑i∈ℐLiprompt\.T\_\{\\mathrm\{SFT\}\}=10^\{\-9\}\\sum\_\{i\\in\\mathcal\{I\}\}L\_\{i\}^\{\\mathrm\{SFT\}\},\\qquad T\_\{\\mathrm\{GRPO\}\}=10^\{\-9\}G\\sum\_\{i\\in\\mathcal\{I\}\}L\_\{i\}^\{\\mathrm\{prompt\}\}\.\(3\)TGRPOT\_\{\\mathrm\{GRPO\}\}excludes generated completion tokens\.

Training compute isHGPU=∑jWj​tj/3600H\_\{\\mathrm\{GPU\}\}=\\sum\_\{j\}W\_\{j\}t\_\{j\}/3600GPU\-hours, withWjW\_\{j\}GPUs and wall\-clock durationtjt\_\{j\}\(seconds\) per intervaljj\. GRPO includes online rollout generation and policy optimization\.

### B\.2Evaluation Setup

We follow each benchmark’s official evaluation settings for decoding, maximum output length, and audio\-visual sampling\. For Gemma\-4, we retain the benchmark protocols but use official model inference defaults and set the maximum output length to 1,024 tokens\. Qwen3\.7\-Max assesses response quality on SocialOmni How and answers Omni\-Cloze’s cloze\-style MCQs using captions generated by the evaluated model\. For Video\-MME\-v2, we report average accuracy \(Avg Acc\)\. For perceptual evaluation, we restrict AVQA to the Which, Come From, Happening, and Where question types, and NExT\-QA to the Descriptive subset\. AVHBench Overall is the pooled accuracy across Audio\-driven Video Hallucination and Video\-driven Audio Hallucination\. Macro Avg\. is the unweighted mean of nine reasoning scores or five perception scores: AVHBench Overall, MUSIC\-AVQA v1 and v2, AVQA, and NExT\-QA\. Geometric Mean Improvement uses the same score sets\. Evaluation system prompts appear in Appendix[D\.1](https://arxiv.org/html/2610.02819#A4.SS1)\.

## Appendix CFurther Analyses

### C\.1Training Data Mixtures

To complement RQ2, we mix Script\-based MCQs with either open\-ended questions or General reasoning data during SFT, using CoT supervision\. The subsequent GRPO phase uses the same Script\-based MCQ data for all configurations\. Table[12](https://arxiv.org/html/2610.02819#A3.T12)reports results for these mixtures, while Table[4](https://arxiv.org/html/2610.02819#S5.T4)provides the reference results without data mixing\.

Table 12:Effect of SFT data mixtures on omni\-modal reasoning\. OE denotes open\-ended questions\. Parenthesized values show absolute changes from the base model\.Training ConfigurationOmni\-modal ReasoningAggregateDaily\-OmniOmni\-ClozeOmniVideoSocialOmniVideo\-HolmesVideo\-MME\-v2WorldSenseMacroAvg\.Geo\. MeanImprov\.\(%\)WhoWhenHowQwen2\.5\-Omni\-7B62\.0721\.9029\.3036\.7557\.4256\.6724\.4024\.3145\.4039\.800\.00SFT:Script\-based CoT \(MCQ\+OE\)56\.9827\.1232\.6037\.6557\.4275\.0039\.6324\.7645\.1044\.03\+12\.20\(−5\.09\)\(\-5\.09\)\(\+5\.22\)\(\+5\.22\)\(\+3\.30\)\(\+3\.30\)\(\+0\.90\)\(\+0\.90\)\(\+0\.00\)\(\+0\.00\)\(\+18\.33\)\(\+18\.33\)\(\+15\.23\)\(\+15\.23\)\(\+0\.45\)\(\+0\.45\)\(−0\.30\)\(\-0\.30\)\(\+4\.23\)\(\+4\.23\)\+\+GRPO:Script\-based MCQ64\.1627\.5536\.7039\.9558\.8569\.7143\.3328\.5847\.5046\.26\+19\.25\(\+2\.09\)\(\+2\.09\)\(\+5\.65\)\(\+5\.65\)\(\+7\.40\)\(\+7\.40\)\(\+3\.20\)\(\+3\.20\)\(\+1\.43\)\(\+1\.43\)\(\+13\.04\)\(\+13\.04\)\(\+18\.93\)\(\+18\.93\)\(\+4\.27\)\(\+4\.27\)\(\+2\.10\)\(\+2\.10\)\(\+6\.46\)\(\+6\.46\)SFT:Script\-based CoT \(MCQ\)\+\+General reasoning56\.3926\.0135\.0036\.4058\.8571\.8836\.9124\.4044\.7043\.39\+10\.62\(−5\.68\)\(\-5\.68\)\(\+4\.11\)\(\+4\.11\)\(\+5\.70\)\(\+5\.70\)\(−0\.35\)\(\-0\.35\)\(\+1\.43\)\(\+1\.43\)\(\+15\.21\)\(\+15\.21\)\(\+12\.51\)\(\+12\.51\)\(\+0\.09\)\(\+0\.09\)\(−0\.70\)\(\-0\.70\)\(\+3\.59\)\(\+3\.59\)\+\+GRPO:Script\-based MCQ65\.6628\.0337\.8040\.0061\.7270\.8347\.5230\.8048\.7047\.90\+23\.66\(\+3\.59\)\(\+3\.59\)\(\+6\.13\)\(\+6\.13\)\(\+8\.50\)\(\+8\.50\)\(\+3\.25\)\(\+3\.25\)\(\+4\.30\)\(\+4\.30\)\(\+14\.16\)\(\+14\.16\)\(\+23\.12\)\(\+23\.12\)\(\+6\.49\)\(\+6\.49\)\(\+3\.30\)\(\+3\.30\)\(\+8\.09\)\(\+8\.09\)

#### SFT data mixing does not improve final average performance\.

Mixing Script\-based MCQs with open\-ended questions raises the SFT Macro Avg\. from 39\.92 to 44\.03\. After GRPO, however, the mixture reaches 46\.26, below the 49\.39 obtained using MCQ data alone during SFT\. Mixing with General reasoning data similarly raises the SFT Macro Avg\. to 43\.39, but the final score of 47\.90 remains below the 49\.39 and 48\.91 obtained using the two SFT data types separately\. For the two mixtures, GRPO increases the Macro Avg\. by 2\.23 and 4\.51 points, compared with 9\.47 points after SFT on Script\-based MCQs alone \(Table[4](https://arxiv.org/html/2610.02819#S5.T4)\)\. For both mixtures, GRPO improves eight of the nine evaluation scores but decreases SocialOmni How, from 75\.00 to 69\.71 with open\-ended questions and from 71\.88 to 70\.83 with General reasoning data\. Both mixtures nevertheless improve all nine evaluation dimensions over the base model\. With the same subsequent GRPO training, SFT on Script\-based MCQs alone achieves a higher reasoning Macro Avg\. than either of the two tested SFT mixtures\.

### C\.2Agentic Reasoning Transfer

RQ1–RQ3 establish that text\-only reasoning training transfers to native audio\-visual reasoning tasks\. Here, we examine whether training on text\-only agent trajectories enables transfer to omni\-modal agentic tasks that require multi\-turn tool use and cross\-modal evidence integration\.#### Experimental setup\.

We train Qwen2\.5\-Omni\-7B with text\-only SFT followed by GRPO, using agent training data from ARPO\([Lu et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib43)\), MiroVerse\([Team et al\., 2025](https://arxiv.org/html/2610.02819#bib.bib54)\), OpenSeeker\([Du et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib16)\), and ToolMind\([Yang et al\., 2025a](https://arxiv.org/html/2610.02819#bib.bib68)\)\. SFT uses 16,000 training samples over 2 epochs, while GRPO uses 8,000 training samples over 1 epoch\. We follow the official OmniGAIA\([Li et al\., 2026b](https://arxiv.org/html/2610.02819#bib.bib35)\)evaluation setup and report Pass@1 \(0–100\) for Easy, Medium, Hard, and Overall\. We compare with the original Qwen2\.5\-Omni\-7B and OmniAtlas\-SFT with and without OmniDPO\([Li et al\., 2026b](https://arxiv.org/html/2610.02819#bib.bib35)\)\.Table 13:Agentic reasoning performance on OmniGAIA \(Pass@1\)\. Parenthesized values show absolute changes from the base model\.Boldandunderlinedscores denote the best and second\-best results, respectively\.ModelDifficulty LevelsOverallEasyMediumHardQwen2\.5\-Omni\-7B8\.21\.31\.33\.6Omni\-modal TrainingOmniAtlas\-SFT19\.79\.42\.611\.4\+\+OmniDPO22\.111\.33\.913\.3Text\-only TrainingText\-only SFT16\.48\.17\.710\.8\+\+GRPO \(Ours\)14\.815\.06\.413\.1\(\+6\.6\)\(\+6\.6\)\(\+13\.7\)\(\+13\.7\)\(\+5\.1\)\(\+5\.1\)\(\+9\.5\)\(\+9\.5\)#### Text\-only reasoning training transfers to omni\-modal agentic tasks\.

As shown in Table[13](https://arxiv.org/html/2610.02819#A3.T13), text\-only SFT followed by GRPO raises the OmniGAIA Overall score from 3\.6 to 13\.1, nearly matching the 13\.3 achieved by native omni\-modal training\. The gains are particularly pronounced on the Medium and Hard subsets, where the text\-only route reaches 15\.0 and 6\.4, respectively\. Together, these results suggest that text\-only reasoning training transfers to a more demanding form of omni\-modal reasoning: agentic tasks requiring multi\-turn tool use and cross\-modal evidence integration\. This result broadens the scope of text\-only reasoning training from benchmark reasoning to agentic reasoning in Omni\-LLMs\.

## Appendix DPrompts

### D\.1Training and Evaluation

Figures[5](https://arxiv.org/html/2610.02819#A4.F5)–[8](https://arxiv.org/html/2610.02819#A4.F8)show the Qwen\-Omni and Gemma\-4 prompts\. The\{input\}placeholder denotes textual context, such as scripts, caption sequences, and passages, or video with or without audio; it is omitted when absent\. Audio\-visual training is used only for Qwen2\.5\-Omni\-7B\. Gemma\-4 uses thinking mode with its native reasoning format\. Evaluation uses the video\-and\-audio template, while NExT\-QA uses video only\. For SocialOmni How, we augment the official prompt with our reasoning instructions\. Omni\-Cloze uses its official prompts\([Ma et al\., 2026](https://arxiv.org/html/2610.02819#bib.bib44)\)\.

\(a\)Training\(b\)Video\-and\-audio evaluation
Figure 5:Training and video\-and\-audio evaluation prompts for Qwen2\.5\-Omni\-3B, Qwen2\.5\-Omni\-7B, and Qwen3\-Omni\-30B\-A3B\-Instruct\.\(a\)Video\-only evaluation
\(b\)SocialOmni How

Figure 6:Video\-only and SocialOmni How evaluation prompts for Qwen2\.5\-Omni and Qwen3\-Omni\.\(a\)Training\(b\)Video\-and\-audio evaluation
Figure 7:Training and video\-and\-audio evaluation prompts for Gemma\-4\-E4B\-IT and Gemma\-4\-E2B\-IT\.\(a\)Video\-only evaluation
\(b\)SocialOmni How

Figure 8:Video\-only and SocialOmni How evaluation prompts for Gemma\-4\-E4B\-IT and Gemma\-4\-E2B\-IT\.### D\.2Synthetic Data Generation

Figures[D\.2](https://arxiv.org/html/2610.02819#A4.SS2)and[D\.1](https://arxiv.org/html/2610.02819#A4.SS1)show the prompts for episode planning, visual expansion, MCQ and reasoning generation, and quality assessment\. Named placeholders in braces denote variable content; JSON blocks illustrate output formats\.

\(a\)Episode planning\.
\(b\)Visual expansion\.

Table 14:Prompts for planning fictional episodes and expanding visual planning facts into descriptions\.

\(c\)Question and reasoning generation\.
\(d\)Quality assessment\.

Table 15:Prompts for generating MCQs and reasoning explanations from fictional scripts and assessing synthetic sample quality\.
## Appendix ECase Studies

Figures[E](https://arxiv.org/html/2610.02819#A5)–[D\.1](https://arxiv.org/html/2610.02819#A4.SS1)show examples from our finalQwen2\.5\-Omni\-7Bmodel in RQ5, obtained through text\-only reasoning training followed by reduced\-data audio\-visual refinement\.

![Refer to caption](https://arxiv.org/html/2610.02819v1/case_001_daily_omni.png)Table 16:Example from Daily\-Omni\.

![Refer to caption](https://arxiv.org/html/2610.02819v1/case_004_videoholmes.png)Table 17:Example from Video\-Holmes\.
![Refer to caption](https://arxiv.org/html/2610.02819v1/case_006_worldsense.png)Table 18:Example from WorldSense\.
![Refer to caption](https://arxiv.org/html/2610.02819v1/case_008_music_avqa.png)Table 19:Example from MUSIC\-AVQA v1\.