PRISM: Priority-aware Rubric Internalization via Structured Multimodal Data Synthesis
Summary
This paper introduces PRISM, a four-stage data synthesis framework for training multimodal LLMs to follow prioritized rubrics, and PRISM-Eval, a judge-free evaluation suite. With only 10K samples, PRISM lifts Qwen3-VL-4B from 9.5% to 30.1% Strict accuracy on PRISM-Eval while preserving general benchmark performance, and gains transfer to other open-source MLLMs.
View Cached Full Text
Cached at: 08/07/26, 07:49 AM
# PRISM: Priority-aware Rubric Internalization via Structured Multimodal Data Synthesis
Source: [https://arxiv.org/abs/2608.05249](https://arxiv.org/abs/2608.05249)
[View PDF](https://arxiv.org/pdf/2608.05249)
> Abstract:Real\-world multimodal instructions often bundle multiple requirements with unequal importance, yet most multimodal training data still reduce instruction following to answering one self\-contained question\. We study this gap through \\textbf\{rubric comprehension\}, which casts the model not as a generator measured against rubrics but as an \\textbf\{executor\} that follows them: given an image and a typed, prioritized rubric, the model must verify each rule before producing an overall judgment\. To support this setting, we propose \\textbf\{PRISM\}, a four\-stage data synthesis framework that produces persona\-\-task pairs, prefix\-guided rule sets, quality\-filtered rubrics, and structured verification traces\. We further introduce \\textbf\{PRISM\-Eval\}, whose Loose and Strict metrics use deterministic matching against fixed labels and therefore require no inference\-time judge model\. With only 10K synthesized samples, PRISM lifts Qwen3\-VL\-4B from 9\.5\\% to 30\.1\\% Strict accuracy on PRISM\-Eval while preserving average performance on general benchmarks, and the gains transfer to four additional open\-source MLLMs across dense and MoE architectures, suggesting that structured rubric supervision is a scalable path toward multi\-rule, priority\-aware multimodal instruction following\.
## Submission history
From: Xiaomin He \[[view email](https://arxiv.org/show-email/55986ff4/2608.05249)\] **\[v1\]**Wed, 5 Aug 2026 15:55:15 UTC \(14,284 KB\)Similar Articles
PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers
Introduces PRISM, a multi-dimensional benchmark for evaluating LLM-based peer reviewers across depth of analysis, novelty assessment, flaw identification, and constructiveness. Findings show LLMs match or beat humans on individual dimensions but lack balanced performance across all, suggesting they are best as supplements to human review.
PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning
PRISM is a large-scale benchmark of 10,372 human-calibrated instruction-code pairs for evaluating programmatic video generation, with a funnel-style framework of four metrics. Evaluation of seven LLMs reveals a significant gap between code executability and spatial coherence.
PRISM: Prosody-Integrated Multi-Agent Reasoning Framework for Empathetic Spoken Dialogue
PRISM is a multi-agent framework that decouples speech perception, response generation, and speech synthesis to improve empathetic spoken dialogue by integrating prosodic cues with LLM reasoning and external knowledge tools.
PReMISE: Policy Rubrics as Measurement Specifications for LLM Judges
Introduces PReMISE, a framework for discovering and auditing policy-level rubrics for LLM judges along four axes: structural adequacy, reliability, preference fit, and adversarial robustness.
PRISM: Perception Reasoning Interleaved for Sequential Decision Making
This paper introduces PRISM, a framework that integrates Vision-Language Models and Large Language Models through a dynamic question-answering pipeline to improve sequential decision-making in embodied AI tasks.