VoiceLongMemEval: Do Assistants Remember How You Sounded?

# VoiceLongMemEval: Do Assistants Remember How You Sounded?
Source: [https://arxiv.org/html/2609.00570](https://arxiv.org/html/2609.00570)
###### Abstract

With the growing scale of multi\-agent architectures and large language models, deployed AI assistants are increasingly tasked with reasoning over long, continuous, multi\-session conversation histories\. Current benchmarks evaluate this dialogue history as information retrieval over long horizon, temporal reasoning, or knowledge updates, while crucially ignoring the fundamental dynamics of human\-agent interaction,i\.e howthey said it\. To address this gap, we presentVoiceLongMemEval \(VLME\)benchmark, where every answer depends on paralinguistic metadata \(emotion labels, prosody descriptors, and voice events\) attached to conversational turns, which is otherwise unrecoverable from the words alone\. Every item passes a three\-stage adversarial gate, ensuring that a strong language model fails when given only the transcript\. Evaluating leading frontier and open\-weight models reveals a pervasive “affect gap"; providing text\-track paralinguistic metadata yields a \+0\.09 to \+0\.38 accuracy boost \(\+0\.61–0\.69 when prompted with evidence hints\), while standard ASR pipelines systematically discard this signal\. Additionally, audio\-native models successfully extract these cues directly from speech \(0\.354–0\.412 vs\. 0\.325 blind\)\. Code and dataset will be made available upon acceptance\.

## 1Introduction

A user tells their assistant, in a flat voice trailing into a sigh, “That was an okay restaurant"\. Weeks and a hundred thousand tokens later they ask:*How did I feel about that restaurant the other day?*and everything needed to answer was present at encoding time, but only in the delivery\. The words were logistics; the sadness was acoustic\. An assistant that transcribed it would answer, “You found it okay\!"\. But only the model that would have attended to the emotion would have known the user did not like the restaurant\.

While long\-term conversational memory\([Maharana et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib2);[Wu et al\., 2025](https://arxiv.org/html/2609.00570#bib.bib3);[Jiang et al\., 2025](https://arxiv.org/html/2609.00570#bib.bib5);[Wu et al\., 2026](https://arxiv.org/html/2609.00570#bib.bib4)\), and paralinguistic perception\([Ao et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib17);[Yang et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib12);[Wang et al\., 2025b](https://arxiv.org/html/2609.00570#bib.bib18)\)have both become measurable capabilities, the two axes have only been tested separately so far\. Their intersection,i\.eremembering how something was said, retaining it across sessions, updating any delivery shifts, and retrieving it against a specific question much later, is a vital cross\-modal dimension that current benchmarks lack\.

To this end, we introduceVoiceLongMemEval \(VLME\), a benchmark to fill this gap\. Every item embeds a paralinguistic needle in a haystack of conversational sessions: the answer is recoverable only from the delivery metadata, never from the words alone, enforced by a three\-part adversarial gate described in[Section3](https://arxiv.org/html/2609.00570#S3)\. Our contributions are:\.

1. 1\.VoiceLongMemEval \(VLME\) Benchmark:: 523 adversarially validated items, spanning six question types that factor paralinguistic memory into affect recall, affective preference, affect update, cross\-session affect, temporal\-affective reasoning, and prosody\-disambiguated interpretation, each with abstention variants\.
2. 2\.Systematic Analysis of the Affect Gap:: We evaluate eight models \(three proprietary, five open\-weight\), revealing a consistent affect gap of \+0\.09 to \+0\.38 across all systems \(all p < 0\.001\)\. Through fine\-grained component ablations and a five\-tier phrasing spectrum, we show that emotion tags provide the strongest signal, chain\-of\-thought reasoning cannot substitute for missing context, and controlled counterfactuals isolate a net\+0\.067\+0\.067accuracy gain driven strictly by metadata content\.

## 2Related Work

#### Long\-term conversational memory\.

A growing family of benchmarks probes whether assistants retain and use information across sessions\. MSC\([Xu et al\., 2022](https://arxiv.org/html/2609.00570#bib.bib1)\)established a multi\-session setting, LoCoMo\([Maharana et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib2)\)scaled it to very long persona\-grounded dialogues and showed that both long\-context reading and RAG lag humans substantially\. LongMemEval\([Wu et al\., 2025](https://arxiv.org/html/2609.00570#bib.bib3)\)and LongMemEval\-V2\([Wu et al\., 2026](https://arxiv.org/html/2609.00570#bib.bib4)\), on which we build directly, embeds curated questions in freely scalable haystacks, decomposing memory into extraction, multi\-session reasoning, temporal reasoning, knowledge update, and abstention\. Subsequent benchmarks broaden these evaluations to encompass personalized context and role conditioned dialogue\. PerLTQA\([Du et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib7)\)focuses on long\-term recall of social interactions and events; MemBench\([Tan et al\., 2025](https://arxiv.org/html/2609.00570#bib.bib9)\)categorizes memory into factual and reflective memory; and DialSim\([Kim et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib8)\)evaluates agents on answering spontaneous questions while role playing within scripted conversations\. Another stream of benchmarks targets*implicit*signals: PrefEval\([Zhao et al\., 2025](https://arxiv.org/html/2609.00570#bib.bib10)\)shows preference adherence drop below 10% after just a few thousand tokens, and PersonaMem\-v2\([Jiang et al\., 2025](https://arxiv.org/html/2609.00570#bib.bib5)\)finds frontier models achieve only 37–48% accuracy on implicit personalization, even when the evidence remain within the context cues\. Closest to our setting, A\-MBER\([Wen et al\., 2026](https://arxiv.org/html/2609.00570#bib.bib6)\)asks models to infer the user’s emotional state based on long\-term, multi\-session interaction history\. But, its evidence is purely lexical,i\.ethe emotion is*written in the words*\. Across this entire family, the memory being tested is a memory of*what*was said, never of*how*it was said\. Our benchmarks tries to bridge this gap, by adding the audio component to it\.

#### Paralinguistic understanding in speech\-LLMs\.

A complementary line evaluates whether audio\-language models perceive the non\-lexical cues at all\. Broader audio evaluation benchmarks like Dynamic\-SUPERB\([Huang et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib11)\), AIR\-Bench\([Yang et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib12)\), Audio2Tool\([Pahwa et al\., 2026](https://arxiv.org/html/2609.00570#bib.bib35)\), AudioBench\([Wang et al\., 2025a](https://arxiv.org/html/2609.00570#bib.bib13)\), MMAU\([Sakshi et al\., 2025](https://arxiv.org/html/2609.00570#bib.bib14)\), and MMSU\([Wang et al\., 2026](https://arxiv.org/html/2609.00570#bib.bib15)\)etc\. include emotion, prosody, and speaker\-attribute tasks among general audio understanding\. Other benchmarks go beyond recognition; SD\-Eval\([Ao et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib17)\)checks whether a reply changes appropriately with the speaker’s emotion, accent, age, and background noise; CP\-Bench\([Wang et al\., 2025b](https://arxiv.org/html/2609.00570#bib.bib18)\)targets contextual paralinguistic reasoning on in\-the\-wild data; and S2S\-Arena\([Jiang et al\., 2026](https://arxiv.org/html/2609.00570#bib.bib19)\)and ParaS2S\([Yang et al\., 2026](https://arxiv.org/html/2609.00570#bib.bib16)\)evaluate paralinguistic instruction following and response appropriateness in speech\-to\-speech models\. This work builds on prior affective\-computing work\([Busso et al\., 2008](https://arxiv.org/html/2609.00570#bib.bib20);[Poria et al\., 2019](https://arxiv.org/html/2609.00570#bib.bib21);[Castro et al\., 2019](https://arxiv.org/html/2609.00570#bib.bib22)\)and its recent LLM\-based successors\([Xu et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib23);[Lin et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib24)\), but all of them tests perception within a single utterance\. In contrast, VLME requires the model to retain paralinguistic information across sessions: the answer depends on how something was said up to∼\\sim100k tokens earlier in the conversation\.

#### Cascaded vs\. audio\-native pipelines\.

Production voice assistants remain largely cascaded \(ASR→\\rightarrowLLM\), a design that discards prosody at the transcription boundary\. Audio\-native models like GPT\-4o\([Hurst et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib29)\), Qwen2\-Audio\([Chu et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib28)\), Qwen2\.5\-Omni\([Xu et al\., 2025](https://arxiv.org/html/2609.00570#bib.bib30)\), Moshi\([Défossez et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib31)\), and GLM\-4\-Voice\([Zeng et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib32)\)take audio as input, and, in principle, both perceive and reproduce vocal nuance\. Prior cascade vs native comparisons were confined to single\-turn understanding\([Ao et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib17);[Wang et al\., 2025b](https://arxiv.org/html/2609.00570#bib.bib18);[Pahwa et al\., 2026](https://arxiv.org/html/2609.00570#bib.bib35)\), however, we measure the cascade’s paralinguistic loss at the*memory*level, where a cue transcribed away in one session silently corrupts answers weeks later\. Finally, unlike prior affective datasets, every item in our corpus passes an adversarial gate where a strong blind model given only the transcript must*fail*the question, ensuring the paralinguistic channel is effective and no question can be answered with words alone\.

## 3Benchmark Construction

Voice\-LongMemEval tests whether models remember*how*a user spoke long after the utterance\. Its 202\-question, adversarially gated core and two derived families total523 questionsover 326 paralinguistically annotated evidence sessions embedded in∼\\sim100k\-token histories \([Table1](https://arxiv.org/html/2609.00570#S3.T1)\)\. Every item obeys one invariant: the correct answer is recoverable from the paralinguistic channel but not from the words alone\. Construction has four stages: annotation \([Section3\.1](https://arxiv.org/html/2609.00570#S3.SS1)\), evidence authoring and hardening \([Sections3\.2](https://arxiv.org/html/2609.00570#S3.SS2)and[3\.3](https://arxiv.org/html/2609.00570#S3.SS3)\), question generation \([Section3\.4](https://arxiv.org/html/2609.00570#S3.SS4)\), and speech synthesis \([Section5\.6](https://arxiv.org/html/2609.00570#S5.SS6)\)\.

Table 1:Benchmark composition\. Taxonomy questions use full haystacks; nuanced and indirect questions target one evidence session but inherit its source history\.### 3\.1The paralinguistic layer

Each instance additively extends a LongMemEval\-compatible record\([Wu et al\., 2025](https://arxiv.org/html/2609.00570#bib.bib3)\), preserving compatibility with existing tooling\. Every*user*turn has five annotations: anemotionfrom 12 everyday labels spanning the valence–arousal plane\([Russell, 1980](https://arxiv.org/html/2609.00570#bib.bib25)\)\(*neutral, happy, excited, content, sad, disappointed, anxious, frustrated, angry, embarrassed, bored, affectionate*\); a categoricalprosodytuple covering rate, pitch, loudness, pauses, and emphasized words that must appear verbatim in the turn;voice eventsdrawn from five reliably synthesized nonverbals \(*laughs, sighs, coughs, clears\_throat, gasps*\);pragmatic flagsfor sarcasm and uncertainty; and a free\-textdelivery description\. Descriptions must be*acoustic\-only*: what a microphone captures, not an interpretation\. A lexical gate rejects emotion names, inflections, and∼\\sim60 interpretive glosses \(*relieved, wry, sarcastic, …*\); for example, “quick and light, laughs mid\-sentence” passes, whereas “relieved” does not\. Because descriptions enter the model’s text input, interpretive labels would reduce the task to string matching\.

The layer has three renders:*blind*\(transcript only, byte\-identical to the original\),*descriptive*\(transcript plus acoustic stage directions; a structured*tagged*variant also ships\), and*audio*\([Section5\.6](https://arxiv.org/html/2609.00570#S5.SS6)\)\. The blind render is both the control and the adversary’s view in[Section3\.3](https://arxiv.org/html/2609.00570#S3.SS3)\.

### 3\.2Evidence authoring and haystack assembly

An LLM authored 4–12\-turn evidence sessions \(≤\\leq26 evidence turns per instance\) in 14 themed batches \(two pilots, twelve 16\-item batches\)\. A protocol111Themes \(work, home, health, travel, money, community, …\) partition topics and persona names; collision scans ensure that no lexical topic recurs across instances\.enforced four validator\-checked invariants: \(i\) lexical flatness \(needles read as neutral logistics\), \(ii\) affect against pragmatics \(when possible, affect opposes the event’s prior\), \(iii\) question neutrality \(functional, valence\-free questions\), and \(iv\) annotation uniformity \(every user turn is fully annotated, so annotation presence cannot reveal the needle\)\.

For an instance withkkevidence sessions, a deterministic seeded assembler adds 40 topic\-screened LongMemEval filler sessions with synthetic neutral annotations, plus6\+2​\(k−1\)6\+2\(k\{\-\}1\)emotive, answer\-free distractors from a disjoint 48\-session pool; the distractor budget scales withkkto prevent emotive\-density shortcuts\. Needle positions are stratified \(early/middle/late\), and multi\-evidence arcs are distributed over time\. The corpus is released in*oracle*\(evidence only,≤1\{\\leq\}1k tokens\) and*full*\(∼\\sim100k tokens\) regimes, mirroring LongMemEvals\{\}\_\{\\textsc\{s\}\}\.

### 3\.3Adversarial validity gates

The main risk is*lexical leakage*: if an item is solvable from text alone, it is not measuring paralinguistic memory\. We audit each taxonomy item three ways\.G1\(blind\-unsolvable\): an adversary answers a blind render and an LLM judge applies a type\-specific rubric with the lexical\-only answer as an explicit trap\([Zheng et al\., 2023](https://arxiv.org/html/2609.00570#bib.bib33)\); any correct blind answer fails\.G2\(aware\-solvable\): the same model answers a descriptive render; we report results by solver strength \(not gating\), since failures may reflect model limits\.G3\(surface\-clean\): static checks for interpretive terms, stock phrases, and valence presuppositions\.

We iterated with a 7B judge–adversary, then gated with Qwen2\.5\-72B\-Instruct\-AWQ, requiring two consecutive clean runs on a frozen file to reduce nondeterminism\. The 72B blind adversary solved 8 items \(7\.5%\) that passed the 7B gate\. Post\-mortems identified five leak mechanisms—pragmatic\-prior leakage, outcome tells, gold\-matches\-prior, default\-recovery priors, and A/B gifts—now a checklist; later batches had zero authoring\-time leaks\. We rerun the terminal gate on the assembled corpus, since date/order shifts can affect marginal verdicts\. Final:0 of 175non\-abstention taxonomy items are blind\-solvable, G3 flags none, and the 72B aware\-solve rate is 57\.9%\. Derived families \([Section3\.4](https://arxiv.org/html/2609.00570#S3.SS4)\) are not separately blind\-attacked; they rely on gated evidence sessions and a mechanical invariant check\. Corpus probes add two controls: ranking sessions by emotive\-annotation density finds the needle in 13\.1% \(top\-1; random 3\.3%\), under the 15% budget, and a session\-length probe scores 0\.

### 3\.4Question generation

#### Taxonomy \(202\)\.

Six types isolate paralinguistic memory skills:*affect\-recall*\(the state expressed in one buried moment\),*affective\-preference*\(a rule keyed to a state expressed only in delivery\),*affect\-update*\(repeated wording with changed delivery; the latest reading wins\),*cross\-session\-affect*\(aggregation across sessions\),*temporal\-affective*\(affective ordering decoupled from lexical events\), and*prosody\-disambiguated*\(delivery resolves two lexically compatible readings\)\. Sarcasm is capped at one item per batch to prevent the last type from collapsing into sarcasm detection\. Across types, 27 abstention items \(\_abs\) presuppose an emotional episode that never occurred, penalizing affect hallucination\.

#### Nuanced \(181\)\.

To broaden single\-session delivery interpretations, an LLM generated three candidates per evidence session \(3×\\times326 = 978\), each with a question, gold answer, lexical\-only answer, category, and rationale\. To flatten a raw pool skewed 36% toward trajectory questions, we kept a verbatim, shuffled, category\-stratified sample: 30 each for*emotional\-trajectory*,*word\-tone\-contradiction*,*unspoken\-concern*,*confidence*,*implied\-preference*, and*sarcasm*, plus one residual item \(148 sessions, 114 source instances\)\. A keyword probe finds explicit delivery cues in over 96% of gold answers but rarely in lexical\-only ones\.

#### Indirect \(140\)\.

Nuanced questions ask what delivery*meant*; indirect questions ask what the assistant should*do*without mentioning voice \(e\.g\., whether to remind the user to decide about two stored items before an appointment\)\. Under the same schema, we sampled 30 each for*decision*,*attitude*,*factual\-intent*, and*preference*; 19 for*belief*; and one residual item \(126 sessions, 102 source instances\), mixing proactive assistance with stance and intent recall\. All gold answers rely on vocal delivery, no lexical\-only answer mentions acoustic cues, and the trap answer remains reachable from the words\.

Across all 523 items, mechanical checks confirm gold and lexical\-only answers differ \(maximum string similarity 0\.50\) and every item resolves to a valid evidence session in its source instance \(zero dangling references\)\.

### 3\.5Audio synthesis

We generate two\-speaker evidence\-session clips with Dia \(1\.6B\)\([Nari Labs, 2025](https://arxiv.org/html/2609.00570#bib.bib34)\)\. Because Dia has no emotion control, each clip is audio\-prompted with a trimmed, peak\-normalized RAVDESS reference\([Livingstone and Russo, 2018](https://arxiv.org/html/2609.00570#bib.bib26)\)for the target \(“needle”\) emotion \(fixed 12→\\to8 mapping\)\. The reference transcript becomes the first\[S1\]line, then the dialogue alternates\[S1\]/\[S2\]over a typically six\-turn, needle\-centered window; when possible, we start on an assistant turn to preserve alternation after the reference\. Sampling usesguidance\_scale3\.0, temperature 1\.8,222This is Dia’s native temperature\. Lowering it for “stability” breaks voice cloning under classifier\-free guidance and yields silence; pinned per\-clip seeds provide reproducibility\.top\-pp0\.9, and top\-kk45\. The manifest records all parameters, seeds, and references\.

We initially used a Whisper\-large\-v3 speech\-emotion\-recognition \(SER\) gate, but moved it to an advisory check after it reached only∼\\sim30% on acted RAVDESS and showed similar per\-emotion patterns across TTS backends, consistent with cross\-corpus SER bias\([Schuller et al\., 2010](https://arxiv.org/html/2609.00570#bib.bib27)\)\. Quality control is now a human listen\-through; each annotator records pass/fail in append\-only sidecar logs\. The human annotator passed 91/104 clips \(87\.5%\); failed clips will be regenerated\. All audio is machine\-generated \(no real recordings\), with timbre cloned from acted RAVDESS references\.

## 4Experimental Setup

We evaluate eight LLMs on our benchmark dataset consisting of three proprietary frontier models: Claude Opus 4\.8, Claude Sonnet 4\.6, GPT\-5\.5, and five open\-weight models: Llama 4 Maverick, Qwen3\.5\-122B\-A10B, Qwen3\-Next\-80B, Llama 3\.3\-70B, Gemma 3\-12B\.

### 4\.1Evaluation Protocol:

Each test case places target evidence withinnd=5n\_\{d\}=5randomly sampled distractor sessions, creating a context window of roughly 10k\-15k tokens\. Models process the context followed by the query to generate free\-text responses, which an LLM judge evaluates against ground\-truth answers using task\-specific rubrics\. We run the experiments for three random seeds, controlling distractor selection and arrangement, and report performance as mean accuracy±\\pmstandard deviation\. We primarily compare two input formats: blind \(plain transcripts without non\-verbal metadata\) and descriptive \(transcripts enriched with natural\-language stage directions detailing vocal delivery\)\. Additional ablations in \([Section5\.2](https://arxiv.org/html/2609.00570#S5.SS2)\) isolate individual metadata components\.

We evaluated for three different question\-set conditions: Nuanced 181Q, Original 202Q and Indirect 140Q\. All pairwise comparisons use paired bootstrap resampling\([Efron et al\., 2000](https://arxiv.org/html/2609.00570#bib.bib37)\)\(10,000 iterations\) and McNemar’s test\([McNemar, 1947](https://arxiv.org/html/2609.00570#bib.bib36)\)for matched\-pair binary outcomes\. We reportpp\-values; all reported gaps are significant atp<0\.001p<0\.001\.

## 5Results

In this section, we present our experimental findings on the benchmark, and show that models consistently benefit from access to paralinguistic information\. Through[Sections5\.1](https://arxiv.org/html/2609.00570#S5.SS1)and[5\.6](https://arxiv.org/html/2609.00570#S5.SS6), we show the affect gap on the original 202Q, examine how question phrasing can influence the performance, examine whether prompting can recover the missing signals and compare audio\-native models with transcript\-based cascades\.

### 5\.1The Affect Gap on Original 202Q

Table 2:Accuracy on Original 202Q \(nd=5n\_\{d\}=5, 3 seeds\)\. The affect gapΔ\\Delta= descriptive−\-blind, computed as the mean of paired per\-seed differences \(not the difference of marginal means\)\. All gaps are positive across all seeds; all 3\-seed mean gaps are significant atp<0\.001p<0\.001\(paired bootstrap \+ McNemar\)\.We presents our core findings in[Table2](https://arxiv.org/html/2609.00570#S5.T2)and show a positive and a statistically significant affect gap between blind and descriptive conditions\. Across all the models, we observe a consistently positive affect gap, indicating that the effect generalizes to both proprietary and open\-weight systems\. Moreover, the magnitude of this gap increases with model capability, rising from\+0\.089\+0\.089for Gemma 3\-12B to\+0\.383\+0\.383for Opus 4\.8\. Importantly, comparing performances between Llama 4 Maverick \(∼\{\\sim\}400B MoE\) and Llama 3\.3\-70B, show that the size of the model alone doesn’t inform about the model’s performance\. Given that blind accuracy is uniformly low \(0\.09–0\.18\) across models, one might hypothesize that the affect gap is merely an artifact of overall capability\. However, normalizing by headroom recovered,Δ/\(1−blind\)\\Delta/\(1\-\\text\{blind\}\), preserves the same ranking between models\. The uniformly low blind accuracy further supports the interpretation that the adversarial gates effectively remove items that can be solved via text alone\.

A per\-type analysis \([Table3](https://arxiv.org/html/2609.00570#S5.T3)\) shows that the affect gap is maximal for question types in which delivery most directly encodes the correct response, and minimal for types requiring integration across multiple sessions\. The resulting type\-level ordering is consistent across both models, suggesting that the associated difficulty hierarchy is inherent to the question types rather than contingent on model\-specific behavior\.

Table 3:Per\-type affect gap on Original 202Q \(nd=5n\_\{d\}=5, 3\-seed mean±\\pmstd\)\. Types sorted by Opus gap\.
### 5\.2What Metadata Component Matters?

Table 4:Ablation study on Original 202Q \(nd=5n\_\{d\}=5, seed=42\)\. Each row renders a different subset of the paralinguistic metadata\. Results for two frontier models\.To understand which paralinguistic cues drive the affect gap, we evaluate Claude Opus 4\.8 and GPT\-5\.5 on nine render conditions \([Table4](https://arxiv.org/html/2609.00570#S5.T4)\)\. Several findings emerge, consistent across both models:Models genuinely use metadata\.The wrong\-metadata condition \(Opus: 0\.228, GPT: 0\.183\) is barely above blind \(0\.193, 0\.129\), confirming that models do not simply benefit from the*presence*of metadata annotations; they read and use the content\.

Emotion labels are the single most informative cue\.Emotion\-only \(Opus: 0\.614, GPT: 0\.535\) surpasses the full descriptive condition \(0\.589, 0\.475\) in both models, despite containing far less information\. Explicit categorical labels are easier for models to integrate into reasoning than free\-text acoustic descriptions\.

Structured tags outperform natural language\.The tagged condition \(Opus: 0\.757, GPT: 0\.624\) exceeds descriptive by \+0\.168 \(Opus\) and \+0\.149 \(GPT\), indicating that frontier models extract paralinguistic information more reliably from structured formats\.

CoT helps but cannot compensate\.Chain\-of\-thought prompting\([Wei et al\., 2022](https://arxiv.org/html/2609.00570#bib.bib38)\)without metadata \(cot\-blind: 0\.302, 0\.203\) improves over blind but falls far short of any metadata\-equipped condition\. Adding CoT to descriptive input \(cot\-descriptive: 0\.767, 0\.668\) yields the best overall accuracy\.

Prosody and events provide partial signal\.Events\-only and prosody\-only each exceed blind substantially, but neither alone approaches the performance of emotion labels\. The ranking of conditions is identical across both models, suggesting the hierarchy of cue informativeness is model\-independent\.

### 5\.3The Question Explicitness Spectrum

Table 5:Affect gap \(Δ\\Delta= descriptive−\-blind\) across five question\-set conditions\. The gap spans an order of magnitude depending on how explicitly the question cues paralinguistic evidence\.We evaluate three question\-set conditions, from explicit paralinguistic cues to fully natural phrasing in[Table5](https://arxiv.org/html/2609.00570#S5.T5)\. Thenuancedset, whose questions explicitly reference voice, tone, or delivery, produces the largest affect gap \(\+0\.61 to \+0\.69\)\. Theindirectset \(140 items, fully natural, open\-ended\) shows the smallest gap \(\+0\.11 to \+0\.18\) indicating that models struggle to connect natural questions to paralinguistic evidence\. The final row previews the prompting result detailed in[Section5\.4](https://arxiv.org/html/2609.00570#S5.SS4)showing that adding a retrieval\-time hint nearly triples the indirect gap\.

### 5\.4Can Prompting Fix the Indirect Gap?

Table 6:Effect of a retrieval\-time prompt hint on indirect questions \(140 items, no paralinguistic cues in question phrasing\)\. All results are 3\-seed mean±\\pmstd\. The hint consistently improves accuracy across all 8 models\.The indirect result poses a practical question: if models*have*paralinguistic metadata in context but fail to attend to it, can a simple prompt intervention close the gap? We test this by prepending a single instruction to the descriptive condition: “When answering, consider not just*what*was said but*how*it was said\.” Table[6](https://arxiv.org/html/2609.00570#S5.T6)shows that a retrieval\-time hint substantially improves accuracy on natural questions across all eight models\. Critically, the hint also lifts the*blind*condition: on indirect v1, hint\-on\-blind raises Opus from0\.1690\.169to0\.557±0\.0310\.557\{\\pm 0\.031\}\(\+0\.388\+0\.388\) and GPT\-5\.5 from0\.1430\.143to0\.536±0\.0260\.536\{\\pm 0\.026\}\(\+0\.393\+0\.393\), exceeding even unprompted descriptive \(0\.3050\.305,0\.2850\.285\)\. This reveals thatprompting and annotation are partially interchangeable: prompting for affective reasoning recovers much of the signal that metadata provides\.

To determine whether this reflects genuine reasoning or judge reward hacking, we run three controls: \(1\)*Scrambled context*: hint with wrong evidence sessions collapses to0\.0430\.043, ruling out plausible made\-up guessing\. \(2\)*Cross\-judge*: re\-judging hint\-on\-blind outputs with GPT\-5\.5 yields0\.6000\.600\(vs\.0\.5360\.536with Opus 4\.5\), ruling out self\-preference bias\. \(3\)*Wrong\-metadata \+ hint*: randomized annotations with the hint score0\.5640\.564, comparable to blind\+hint \(0\.5360\.536\), confirming that the hint operates on conversational content rather than annotation content\. The tightest estimate of metadata’s*content*contribution comes from comparing descriptive\+hint \(0\.6310\.631\) against wrong\-metadata\+hint \(0\.5640\.564\): a clean\+0\.067\+0\.067, clean by annotation presence or prompt effects\.

Crucially, the interchangeability is question\-set\-dependent\.On the adversarially\-gated Original 202Q \(Opus, seed=42\), the hint lifts blind only modestly \(0\.188→0\.2670\.188\\rightarrow 0\.267,\+0\.079\+0\.079\), and the affect gap*grows*under the hint \(descriptive\+hint0\.7330\.733minus blind\+hint0\.2670\.267=\+0\.465\+0\.465, vs\.\+0\.376\+0\.376without hint\)\. On the LLM\-generated indirect v1 \(3\-seed means\), the hint lifts blind dramatically \(0\.169→0\.5570\.169\\rightarrow 0\.557,\+0\.388\+0\.388\), and the gap narrows to\+0\.074\+0\.074for Opus and\+0\.026\+0\.026for GPT\-5\.5\. This divergence reflects the adversarial gates: 202Q items were authored to resist text\-only reasoning, making metadata genuinely irreplaceable; indirect v1 items, generated without such gates, are more amenable to general affective reasoning\. The headline affect gap on the gated benchmark is robust to prompting\. This result also serves as an empirical validation of the adversarial gates themselves: gated items \(202Q\) resist the strongest known prompting attack \(\+0\.079\+0\.079blind hint lift\), while ungated items \(indirect v1\) do not \(\+0\.367\+0\.367\)\. The observed leak is confined to the LLM\-generated question sets that were not subjected to the adversarial gates The core 202Q benchmark, which passed these gates, remains robust to the same prompting intervention\.

### 5\.5Distractor Scaling

Table 7:Effect of distractor count on accuracy \(Original 202Q, seed=42\)\. The affect gap persists across haystack sizes for all model types\.Table[7](https://arxiv.org/html/2609.00570#S5.T7)shows that increasing distractors from 3 to 10 mildly reduces descriptive accuracy across all model types, but blind accuracy remains flat\. The affect gap persists at all scales for both frontier and open\-weight models, confirming that the benchmark’s difficulty is not an artifact of haystack size\.

### 5\.6Audio\-Native Evaluation

To measure the paralinguistic\-memory deficit of cascaded pipelines, we synthesize the 114 indirect v2 evidence sessions with Dia TTS\([Nari Labs, 2025](https://arxiv.org/html/2609.00570#bib.bib34)\), conditioned on emotion\-matched RAVDESS reference clips\([Livingstone and Russo, 2018](https://arxiv.org/html/2609.00570#bib.bib26)\)across four speaker voices\. We evaluate two audio\-native models \(Qwen2\-Audio\-7B\([Chu et al\., 2024](https://arxiv.org/html/2609.00570#bib.bib28)\), Qwen2\.5\-Omni\-7B\) under four conditions each, against a cascade \(Whisper large\-v3→\\rightarrowOpus 4\.8 or GPT\-5\.5\) on the same clips \(Table[8](https://arxiv.org/html/2609.00570#S5.T8)\)\. All conditions are regime\-matched \(evidence\-only\); text baselines use all 114 items, while audio and cascade rows use the 104 with valid TTS output\.

Table 8:Audio\-native and cascade evaluation on indirect v2, evidence\-only regime\. 3\-seed mean±\\pmstd; cascade runs via Databricks\. Text baselines: 114 items; audio and cascade rows: 104 items with valid TTS\.ModalityConditionContextQwen2\-AudioOmniCascadeTextBlindevidence—0\.325TextDescriptiveevidence—0\.675CascadeWhisper→\\rightarrowOpusevidence—0\.254±0\.0150\.254\{\\pm 0\.015\}CascadeWhisper→\\rightarrowOpus \+ hintevidence—0\.515±0\.0220\.515\{\\pm 0\.022\}CascadeWhisper→\\rightarrowGPTevidence—0\.468±0\.0270\.468\{\\pm 0\.027\}CascadeWhisper→\\rightarrowGPT \+ hintevidence—0\.552±0\.0150\.552\{\\pm 0\.015\}AudioAudio onlyevidence0\.354±0\.0100\.354\{\\pm 0\.010\}0\.412±0\.0090\.412\{\\pm 0\.009\}—AudioAudio \+ hintevidence0\.401±0\.0100\.401\{\\pm 0\.010\}0\.444±0\.0130\.444\{\\pm 0\.013\}—Audio\+TextAudio \+ metadataevidence0\.509±0\.0180\.509\{\\pm 0\.018\}0\.541±0\.0100\.541\{\\pm 0\.010\}—Audio\+TextAudio \+ meta \+ hintevidence0\.541±0\.0100\.541\{\\pm 0\.010\}0\.582±0\.0100\.582\{\\pm 0\.010\}—Three findings emerge\. First,audio\-native models hear paralinguistic cues: Qwen2\-Audio \(0\.3540\.354\) and Qwen2\.5\-Omni \(0\.4120\.412\) outperform the blind text baseline \(0\.3250\.325\), and supplementary metadata lifts both further \(0\.5090\.509,0\.5410\.541\), approaching the descriptive upper bound \(0\.6750\.675\)\. Second,the cascade loses this signal: Whisper→\\rightarrowOpus 4\.8 scores0\.2540\.254, below both 7B audio\-native models and even the blind baseline, despite a frontier\-scale capability advantage\. The loss has two sources: Whisper model strips all delivery cues \(transcript analysis finds no bracketed voice events, fillers, or other paralinguistic markers in any of the 104 clips\) and introduces content errors relative to the ground\-truth transcript\. Third,the hint compensates for cascade loss, raising Opus from0\.2540\.254to0\.5150\.515\(\+0\.261\+0\.261\) and GPT\-5\.5 from0\.4680\.468to0\.5520\.552\(\+0\.084\+0\.084\); per[Section5\.4](https://arxiv.org/html/2609.00570#S5.SS4), this reflects general affective\-reasoning gains rather than recovery of delivery cues, since comparable lifts appear on blind text\.

## 6Discussion and Limitations

#### Implications for memory system design\.

Memory systems should retain*structured*paralinguistic metadata \(tagged: 0\.757 vs\. descriptive NL: 0\.589 for Opus\) and explicitly elicit affective reasoning during retrieval\. The cascade shortfall is an architectural issue rather than a capability ceiling: Whisper→\\rightarrowOpus \(0\.2540\.254\) underperforms compared with 7B audio\-native models \(0\.3540\.354–0\.4120\.412\) on the same clips\. For adversarially\-gated items, metadata remains indispensable even with strong prompting; for generated items, prompting and annotation each add distinct, complementary benefits \(\+0\.067\+0\.067controlled metadata contribution\)\.

#### Limitations\.

\(1\) Synthetically authored items; emotional distribution may differ from naturalistic conversation\. \(2\) Oracle\-regime evaluation only \(∼\\sim10–15k tokens\); the full∼\\sim100k\-token regime is untested\. \(3\) Audio evaluation limited to two 7B models\. \(4\) LLM\-generated question sets may introduce distributional biases\. \(5\) LLM\-as\-judge may exhibit biases on affect\-laden content; human evaluation would strengthen results\. \(6\) G1 gates certify items against an*unprompted*72B adversary; prompted frontier models reach0\.2670\.267on 202Q blind \(seed=42\), so certification is prompting\-dependent\.

## 7Conclusion

VoiceLongMemEval demonstrates that paralinguistic metadata improves conversational memory across eight models, with the affect gap persisting across question types, distractor scales, and audio modalities\. Prompting and annotation are partially interchangeable: a retrieval\-time hint recovers much of the signal, but metadata contributes an additional\+0\.067\+0\.067\(controlled\)\. For practitioners: prompt first, annotate second, do both\. Audio\-native 7B models outperform cascaded frontier models on identical clips, quantifying the ASR pipeline’s paralinguistic deficit\. We release the benchmark to support research on the paralinguistic dimension of long\-term memory\.

## References

- \[1\]A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang\(2024\)Evaluating very long\-term conversational memory of LLM agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 13851–13870\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.747)Cited by:[§1](https://arxiv.org/html/2609.00570#S1.p2.1),[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px1.p1.1)\.
- \[2\]D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu\(2025\)LongMemEval: benchmarking chat assistants on long\-term interactive memory\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2410\.10813Cited by:[§1](https://arxiv.org/html/2609.00570#S1.p2.1),[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2609.00570#S3.SS1.p1.1)\.
- \[3\]B\. Jiang, Y\. Yuan, M\. Shen, Z\. Hao, Z\. Xu, Z\. Chen, Z\. Liu, A\. R\. Vijjini, J\. He, H\. Yu, R\. Poovendran, G\. Wornell, L\. Ungar, D\. Roth, S\. Chen, and C\. J\. Taylor\(2025\)PersonaMem\-v2: towards personalized intelligence via learning implicit user personas and agentic memory\.arXiv preprint arXiv:2512\.06688\.Cited by:[§1](https://arxiv.org/html/2609.00570#S1.p2.1),[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px1.p1.1)\.
- \[4\]D\. Wu, Z\. Ji, A\. Kawatkar, B\. Kwan, J\. Gu, N\. Peng, and K\. Chang\(2026\)LongMemEval\-V2: evaluating long\-term agent memory toward experienced colleagues\.arXiv preprint arXiv:2605\.12493\.Cited by:[§1](https://arxiv.org/html/2609.00570#S1.p2.1),[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px1.p1.1)\.
- \[5\]J\. Ao, Y\. Wang, X\. Tian, D\. Chen, J\. Zhang, L\. Lu, Y\. Wang, H\. Li, and Z\. Wu\(2024\)SD\-Eval: a benchmark dataset for spoken dialogue understanding beyond words\.InAdvances in Neural Information Processing Systems \(NeurIPS\) Datasets and Benchmarks Track,External Links:2406\.13340Cited by:[§1](https://arxiv.org/html/2609.00570#S1.p2.1),[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px3.p1.1)\.
- \[6\]Q\. Yang, J\. Xu, W\. Liu, Y\. Chu, Z\. Jiang, X\. Zhou, Y\. Leng, Y\. Lv, Z\. Zhao, C\. Zhou, and J\. Zhou\(2024\)AIR\-Bench: benchmarking large audio\-language models via generative comprehension\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1979–1998\.Cited by:[§1](https://arxiv.org/html/2609.00570#S1.p2.1),[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[7\]Q\. Wang, H\. B\. Sailor, T\. Liu, W\. Zhang, M\. Huzaifah, N\. Lertcheva, S\. Sun, N\. F\. Chen, J\. Wu, and A\. Aw\(2025\)Benchmarking contextual and paralinguistic reasoning in speech\-LLMs: a case study with in\-the\-wild data\.InFindings of the Association for Computational Linguistics: EMNLP 2025,pp\. 14133–14148\.Cited by:[§1](https://arxiv.org/html/2609.00570#S1.p2.1),[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px3.p1.1)\.
- \[8\]J\. Xu, A\. Szlam, and J\. Weston\(2022\)Beyond goldfish memory: long\-term open\-domain conversation\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Dublin, Ireland,pp\. 5180–5197\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.356)Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px1.p1.1)\.
- \[9\]Y\. Du, H\. Wang, Z\. Zhao, B\. Liang, B\. Wang, W\. Zhong, Z\. Wang, and K\. Wong\(2024\)PerLTQA: a personal long\-term memory dataset for memory classification, retrieval, and fusion in question answering\.InProceedings of the 10th SIGHAN Workshop on Chinese Language Processing \(SIGHAN\-10\),Bangkok, Thailand,pp\. 152–164\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px1.p1.1)\.
- \[10\]H\. Tan, Z\. Zhang, C\. Ma, X\. Chen, Q\. Dai, and Z\. Dong\(2025\)MemBench: towards more comprehensive evaluation on the memory of LLM\-based agents\.InFindings of the Association for Computational Linguistics: ACL 2025,Vienna, Austria,pp\. 19336–19352\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.989)Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px1.p1.1)\.
- \[11\]J\. Kim, W\. Chay, H\. Hwang, D\. Kyung, H\. Chung, E\. Cho, Y\. Kwon, Y\. Jo, and E\. Choi\(2024\)DialSim: a dialogue simulator for evaluating long\-term multi\-party dialogue understanding of conversational agents\.arXiv preprint arXiv:2406\.13144\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px1.p1.1)\.
- \[12\]S\. Zhao, M\. Hong, Y\. Liu, D\. Hazarika, and K\. Lin\(2025\)Do LLMs recognize your preferences? evaluating personalized preference following in LLMs\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2502\.09597Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px1.p1.1)\.
- \[13\]D\. Wen, K\. Sun, and Y\. Wang\(2026\)A\-MBER: affective memory benchmark for emotion recognition\.arXiv preprint arXiv:2604\.07017\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px1.p1.1)\.
- \[14\]C\. Huang, K\. Lu, S\. Wang, C\. Hsiao, C\. Kuan, H\. Wu, S\. Arora, K\. Chang, J\. Shi, Y\. Peng, R\. Sharma, S\. Watanabe, B\. Ramakrishnan, S\. Shehata, and H\. Lee\(2024\)Dynamic\-SUPERB: towards a dynamic, collaborative, and comprehensive instruction\-tuning benchmark for speech\.InIEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[15\]R\. Pahwa, A\. Beedu, P\. Priye, R\. Gandhi, S\. Takawale, A\. Baijal, and Z\. Yang\(2026\)Audio2Tool: speak, call, act–a dataset for benchmarking speech tool use\.arXiv preprint arXiv:2604\.22821\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px3.p1.1)\.
- \[16\]B\. Wang, X\. Zou, G\. Lin, S\. Sun, Z\. Liu, W\. Zhang, Z\. Liu, A\. Aw, and N\. F\. Chen\(2025\)AudioBench: a universal benchmark for audio large language models\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 4297–4316\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[17\]S\. Sakshi, U\. Tyagi, S\. Kumar, A\. Seth, R\. Selvakumar, O\. Nieto, R\. Duraiswami, S\. Ghosh, and D\. Manocha\(2025\)MMAU: a massive multi\-task audio understanding and reasoning benchmark\.InThe Thirteenth International Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[18\]D\. Wang, J\. Li, J\. Wu, D\. Yang, X\. Chen, T\. Zhang, and H\. Meng\(2026\)MMSU: a massive multi\-task spoken language understanding and reasoning benchmark\.InThe Fourteenth International Conference on Learning Representations \(ICLR\),Note:arXiv:2506\.04779Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[19\]F\. Jiang, Z\. Lin, Y\. Liu, L\. Xue, F\. Bu, Y\. Du, X\. Chen, B\. Wang, and H\. Li\(2026\)S2S\-Arena: evaluating paralinguistic instruction following in speech\-to\-speech models\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Note:arXiv:2503\.05085Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[20\]S\. Yang, M\. Tu, A\. T\. Liu, X\. Qu, H\. Lee, L\. Lu, Y\. Wang, and Y\. Wu\(2026\)ParaS2S: benchmarking and aligning spoken language models for paralinguistic\-aware speech\-to\-speech interaction\.InThe Fourteenth International Conference on Learning Representations \(ICLR\),Note:arXiv:2511\.08723Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[21\]C\. Busso, M\. Bulut, C\. Lee, A\. Kazemzadeh, E\. Mower, S\. Kim, J\. N\. Chang, S\. Lee, and S\. S\. Narayanan\(2008\)IEMOCAP: interactive emotional dyadic motion capture database\.Language Resources and Evaluation42\(4\),pp\. 335–359\.External Links:[Document](https://dx.doi.org/10.1007/s10579-008-9076-6)Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[22\]S\. Poria, D\. Hazarika, N\. Majumder, G\. Naik, E\. Cambria, and R\. Mihalcea\(2019\)MELD: a multimodal multi\-party dataset for emotion recognition in conversations\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,Florence, Italy,pp\. 527–536\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1050)Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[23\]S\. Castro, D\. Hazarika, V\. Pérez\-Rosas, R\. Zimmermann, R\. Mihalcea, and S\. Poria\(2019\)Towards multimodal sarcasm detection \(an \_obviously\_ perfect paper\)\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,Florence, Italy,pp\. 4619–4629\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1455)Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[24\]Y\. Xu, H\. Chen, J\. Yu, Q\. Huang, Z\. Wu, S\. Zhang, G\. Li, Y\. Luo, and R\. Gu\(2024\)SECap: speech emotion captioning with large language model\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19323–19331\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v38i17.29902)Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[25\]G\. Lin, P\. G\. Shivakumar, A\. Gandhe, C\. H\. Yang, Y\. Gu, S\. Ghosh, A\. Stolcke, H\. Lee, and I\. Bulyko\(2024\)Paralinguistics\-enhanced large language modeling of spoken dialogue\.InIEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 10316–10320\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px2.p1.1)\.
- \[26\]A\. Hurst, A\. Lerer, A\. P\. Goucher,et al\.\(2024\)GPT\-4o system card\.arXiv preprint arXiv:2410\.21276\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px3.p1.1)\.
- \[27\]Y\. Chu, J\. Xu, Q\. Yang, H\. Wei, X\. Wei, Z\. Guo, Y\. Leng, Y\. Lv, J\. He, J\. Lin, C\. Zhou, and J\. Zhou\(2024\)Qwen2\-Audio technical report\.arXiv preprint arXiv:2407\.10759\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px3.p1.1),[§5\.6](https://arxiv.org/html/2609.00570#S5.SS6.p1.1)\.
- \[28\]J\. Xu, Z\. Guo, J\. He, H\. Hu, T\. He, S\. Bai, K\. Chen, J\. Wang, Y\. Fan, K\. Dang, B\. Zhang, X\. Wang, Y\. Chu, and J\. Lin\(2025\)Qwen2\.5\-Omni technical report\.arXiv preprint arXiv:2503\.20215\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px3.p1.1)\.
- \[29\]A\. Défossez, L\. Mazaré, M\. Orsini, A\. Royer, P\. Pérez, H\. Jégou, E\. Grave, and N\. Zeghidour\(2024\)Moshi: a speech\-text foundation model for real\-time dialogue\.arXiv preprint arXiv:2410\.00037\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px3.p1.1)\.
- \[30\]A\. Zeng, Z\. Du, M\. Liu, K\. Wang, S\. Jiang, L\. Zhao, Y\. Dong, and J\. Tang\(2024\)GLM\-4\-Voice: towards intelligent and human\-like end\-to\-end spoken chatbot\.arXiv preprint arXiv:2412\.02612\.Cited by:[§2](https://arxiv.org/html/2609.00570#S2.SS0.SSS0.Px3.p1.1)\.
- \[31\]J\. A\. Russell\(1980\)A circumplex model of affect\.Journal of Personality and Social Psychology39\(6\),pp\. 1161–1178\.Cited by:[§3\.1](https://arxiv.org/html/2609.00570#S3.SS1.p1.1)\.
- \[32\]L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica\(2023\)Judging LLM\-as\-a\-judge with MT\-Bench and chatbot arena\.InAdvances in Neural Information Processing Systems \(NeurIPS\) Datasets and Benchmarks Track,External Links:2306\.05685Cited by:[§3\.3](https://arxiv.org/html/2609.00570#S3.SS3.p1.1)\.
- \[33\]Nari Labs\(2025\)Dia: a 1\.6b\-parameter dialogue text\-to\-speech model\.Hugging Face / GitHub\.Note:[https://huggingface\.co/nari\-labs/Dia\-1\.6B](https://huggingface.co/nari-labs/Dia-1.6B)Cited by:[§3\.5](https://arxiv.org/html/2609.00570#S3.SS5.p1.1),[§5\.6](https://arxiv.org/html/2609.00570#S5.SS6.p1.1)\.
- \[34\]S\. R\. Livingstone and F\. A\. Russo\(2018\)The ryerson audio\-visual database of emotional speech and song \(RAVDESS\): a dynamic, multimodal set of facial and vocal expressions in north american english\.PLoS ONE13\(5\),pp\. e0196391\.Cited by:[§3\.5](https://arxiv.org/html/2609.00570#S3.SS5.p1.1),[§5\.6](https://arxiv.org/html/2609.00570#S5.SS6.p1.1)\.
- \[35\]B\. Schuller, B\. Vlasenko, F\. Eyben, M\. Wöllmer, A\. Stuhlsatz, A\. Wendemuth, and G\. Rigoll\(2010\)Cross\-corpus acoustic emotion recognition: variances and strategies\.IEEE Transactions on Affective Computing1\(2\),pp\. 119–131\.Cited by:[§3\.5](https://arxiv.org/html/2609.00570#S3.SS5.p2.1)\.
- \[36\]B\. Efron R\. J\. Tibshiraniet al\.\(2000\)An introduction to the bootstrap\.Boca Raton, Florida\.Cited by:[§4\.1](https://arxiv.org/html/2609.00570#S4.SS1.p2.1)\.
- \[37\]Q\. McNemar\(1947\)Note on the sampling error of the difference between correlated proportions or percentages\.Psychometrika12\(2\),pp\. 153–157\.Cited by:[§4\.1](https://arxiv.org/html/2609.00570#S4.SS1.p2.1)\.
- \[38\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§5\.2](https://arxiv.org/html/2609.00570#S5.SS2.p4.1)\.

## Appendix AError Analysis

We categorize all 202 items by the joint outcome of blind and descriptive conditions for Claude Opus 4\.8 \(Table[9](https://arxiv.org/html/2609.00570#A1.T9)\)\.

Table 9:Error analysis: joint outcome categories for Claude Opus 4\.8 on Original 202Q\.The 85gap contributors\(42% of items\) are the benchmark’s core: items where paralinguistic metadata makes the difference between success and failure\. The 78hard\-for\-bothitems represent a ceiling challenge: even with full metadata, the model fails, often on temporal\-affective or cross\-session\-affect types requiring integration across multiple sessions\. The 29easy/lexicalitems suggest residual text signal that survived the adversarial gates; these are candidates for future tightening\. The 10metadata\-hurtsitems are predominantly abstention variants where the model, given rich emotional metadata, hallucinates an affective episode that the question presupposes but that never occurred; metadata increases the temptation to fabricate answers\.

## Appendix BTaxonomy\.

Six question types factor the competence \([Table10](https://arxiv.org/html/2609.00570#A2.T10)\) and \([Figure1](https://arxiv.org/html/2609.00570#A2.F1)\)\. Each type has an abstention variant \(\_abs, 15% of items\) whose question presupposes an emotional episode that never occurred; the gold answer is that it was never expressed, punishing affect hallucination\.

Table 10:The six question types\. Each row shows a question, what the*words alone*suggest \(wrong\), and what the*delivery*reveals \(correct\)\. The gap between the two is what the benchmark measures\.Figure 1:Original Benchmark Questions
## Appendix CPrompts

#### Standard prompt \(text conditions\)\.

> You are a personal assistant with access to the user’s conversation history\. === CONVERSATION HISTORY === \{history\} === END HISTORY === Today’s date is \{question\_date\}\. Answer the user’s question based on the history above\. Be concise \(1–3 sentences\)\. If the history does not contain enough information to answer, say so instead of guessing\. Question: \{question\}

#### Hint prompt \(retrieval\-time intervention\)\.

Identical to the standard prompt, with one sentence prepended to the answer instruction:

> When answering, consider not just*what*was said but*how*it was said\. Delivery cues like tone, pace, volume, pauses, laughter, and sighs often reveal the speaker’s true feelings, intentions, and preferences\.

#### Audio prompt\.

> Listen to this conversation\. Answer concisely \(1–3 sentences\)\. Question: \{question\}

## Appendix DQualitative Examples: Hint vs\. No\-Hint

Below are examples from the indirect v1 set where the hint prompt yields a correct answer \(green\) while the standard prompt yields an incorrect answer \(red\)\. Model: Claude Opus 4\.8, descriptive condition with 3 distractors\. Each example shows the evidence session with its paralinguistic metadata \(the model also sees 3 distractor sessions, omitted for space\)\.

#### Example 1 \(belief\): “Should I expect this coverage period to be manageable?”

Evidence session \(descriptive render\):

> User \(plain, conversational\):talked with my manager about her leave today Assistant:The eight weeks starting next month? User \(slow, quiet; long hesitations; sighs\):I told her yes\. I’ll cover the vendor approvals and the monday scheduling while she’s out, and she’ll do a handover doc before she goes Assistant:Coverage confirmed: vendor approvals and Monday scheduling for eight weeks, with a handover doc to come\.*\[…6 more turns\]*

Gold:Challenging\. Hesitations and sighs when accepting suggest you’re already feeling the weight of it\. No hint:The coverage is fairly contained: you’re taking on vendor approvals and Monday scheduling for eight weeks, with a clear handover document coming… With hint:Your delivery when agreeing suggested otherwise—the slow, quiet tone, long hesitations, and sighs pointed to real reservations, even though the words sounded agreeable…

#### Example 2 \(belief\): “Am I actually going to use all ten yoga sessions?”

Evidence session \(descriptive render\):

> User \(slow, flat, quiet; long pauses\):upgraded to level two and bought a ten\-pack\. expires in ninety days Assistant:Ten sessions in ninety days — want me to block two mornings a week? User \(even pace, no stress\):sure\. tuesdays and thursdays*\[…\]*

Gold:Unlikely\. Delivery was slow, flat, and resigned, suggesting obligation rather than enthusiasm\. No hint:Based on your history, I can’t predict that\. You bought the ten\-pack with a 90\-day expiration and seem to have planned sessions… With hint:It’s uncertain\. The way you spoke about the yoga plan was flat and low\-energy with long pauses, which doesn’t scream enthusiasm…

#### Example 3 \(preference\): “Should I plan anything special to celebrate the certification?”

Evidence session \(descriptive render\):

> User \(slow, flat; long pause; sighs\):name was on the pass list this morning\. state inspector cert is done Assistant:Congratulations — that’s the full certification now\. Want me to update your credentials file?*\[…\]*

Gold:Probably not\. The passing news came with flat, dragging speech and a sigh, suggesting relief rather than celebration\. No hint:You passed the state inspector exam, which is certainly worth celebrating\! However, there’s no mention of plans… With hint:Based on how you shared that news—flat, dragging, with a long sigh—you didn’t sound celebratory; it read more like relief or crossing off a to\-do…

## NeurIPS Paper Checklist

1. 1\.Claims
2. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
3. Answer:\[Yes\]
4. Justification: The abstract and Section 1 state the contributions — the VLME benchmark, the three\-stage adversarial gating protocol \(G1–G3\), and the empirical findings on paralinguistic memory across blind, descriptive, cascade, and audio\-native conditions — and these match the experiments reported in Section 5\. Claims are scoped to the evaluated context regimes\.
5. Guidelines: - •The answer\[N/A\]means that the abstract and introduction do not include the claims made in the paper\. - •The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations\. A\[No\]or\[N/A\]answer to this question will not be perceived well by the reviewers\. - •The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings\. - •It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper\.
6. 2\.Limitations
7. Question: Does the paper discuss the limitations of the work performed by the authors?
8. Answer:\[Yes\]
9. Justification: The paper includes a dedicated, numbered Limitations section covering, among others, the evaluated context\-length regime, the reliance on LLM\-generated sessions and TTS\-rendered audio, the fact that derived question families are not separately blind\-gated, and sensitivity to prompting\.
10. Guidelines: - •The answer\[N/A\]means that the paper has no limitation while the answer\[No\]means that the paper has limitations, but those are not discussed in the paper\. - •The authors are encouraged to create a separate “Limitations” section in their paper\. - •The paper should point out any strong assumptions and how robust the results are to violations of these assumptions \(e\.g\., independence assumptions, noiseless settings, model well\-specification, asymptotic approximations only holding locally\)\. The authors should reflect on how these assumptions might be violated in practice and what the implications would be\. - •The authors should reflect on the scope of the claims made, e\.g\., if the approach was only tested on a few datasets or with a few runs\. In general, empirical results often depend on implicit assumptions, which should be articulated\. - •The authors should reflect on the factors that influence the performance of the approach\. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting\. Or a speech\-to\-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon\. - •The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size\. - •If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness\. - •While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper\. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community\. Reviewers will be specifically instructed to not penalize honesty concerning limitations\.
11. 3\.Theory assumptions and proofs
12. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete \(and correct\) proof?
13. Answer:\[N/A\]
14. Justification: The paper is an empirical benchmark and evaluation study and contains no theoretical results\.
15. Guidelines: - •The answer\[N/A\]means that the paper does not include theoretical results\. - •All the theorems, formulas, and proofs in the paper should be numbered and cross\-referenced\. - •All assumptions should be clearly stated or referenced in the statement of any theorems\. - •The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition\. - •Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material\. - •Theorems and Lemmas that the proof relies upon should be properly referenced\.
16. 4\.Experimental result reproducibility
17. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper \(regardless of whether the code and data are provided or not\)?
18. Answer:\[Yes\]
19. Justification: Sections 3–4 and the appendix specify the item taxonomy and schema, the full gating protocol \(adversary and judge models, type\-specific rubrics, and the two\-consecutive\-clean\-runs criterion\), the audio synthesis recipe \(Dia TTS conditioned on RAVDESS reference clips\), model versions and endpoints, evaluation prompts, and the multi\-seed evaluation protocol\.
20. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •If the paper includes experiments, a\[No\]answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not\. - •If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable\. - •Depending on the contribution, reproducibility can be accomplished in various ways\. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model\. In general\. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model \(e\.g\., in the case of a large language model\), releasing of a model checkpoint, or other means that are appropriate to the research performed\. - •While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution\. For example 1. \(a\)If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm\. 2. \(b\)If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully\. 3. \(c\)If the contribution is a new model \(e\.g\., a large language model\), then there should either be a way to access this model for reproducing the results or a way to reproduce the model \(e\.g\., with an open\-source dataset or instructions for how to construct the dataset\)\. 4. \(d\)We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility\. In the case of closed\-source models, it may be that access to the model is limited in some way \(e\.g\., to registered users\), but it should be possible for other researchers to have some path to reproducing or verifying the results\.
21. 5\.Open access to data and code
22. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
23. Answer:\[Yes\]
24. Justification: An anonymized repository containing the benchmark \(items, prompts, gating rubrics, and audio synthesis scripts and manifests\) and the full evaluation code, with instructions to reproduce the main results, is provided at[ANONYMIZED\-URL\-HERE](https://anonymized-url-here/)\. The de\-anonymized version will be released publicly upon publication\.
25. Guidelines: - •The answer\[N/A\]means that paper does not include experiments requiring code\. - • - •While we encourage the release of code and data, we understand that this might not be possible, so\[No\]is an acceptable answer\. Papers cannot be rejected simply for not including code, unless this is central to the contribution \(e\.g\., for a new open\-source benchmark\)\. - •The instructions should contain the exact command and environment needed to run to reproduce the results\. See the NeurIPS code and data submission guidelines \([https://neurips\.cc/public/guides/CodeSubmissionPolicy](https://neurips.cc/public/guides/CodeSubmissionPolicy)\) for more details\. - •The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc\. - •The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines\. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why\. - •At submission time, to preserve anonymity, the authors should release anonymized versions \(if applicable\)\. - •Providing as much information as possible in supplemental material \(appended to the paper\) is recommended, but including URLs to data and code is permitted\.
26. 6\.Experimental setting/details
27. Question: Does the paper specify all the training and test details \(e\.g\., data splits, hyperparameters, how they were chosen, type of optimizer\) necessary to understand the results?
28. Answer:\[Yes\]
29. Justification: Section 4 specifies the evaluation conditions, context regimes, models, decoding settings, and number of seeds; full prompts and per\-condition details appear in the appendix\. No models are trained, so no optimizer or training hyperparameters apply\.
30. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them\. - •The full details can be provided either with the code, in appendix, or as supplemental material\.
31. 7\.Experiment statistical significance
32. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
33. Answer:\[Yes\]
34. Justification: Main results are reported as mean±\\pmstandard deviation \(1\-sigma\) over 3 evaluation seeds with fixed items, capturing decoding and judging nondeterminism; this is stated in the corresponding table captions\.
35. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The authors should answer\[Yes\]if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper\. - •The factors of variability that the error bars are capturing should be clearly stated \(for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions\)\. - •The method for calculating the error bars should be explained \(closed form formula, call to a library function, bootstrap, etc\.\) - •The assumptions made should be given \(e\.g\., Normally distributed errors\)\. - •It should be clear whether the error bar is the standard deviation or the standard error of the mean\. - •It is OK to report 1\-sigma error bars, but one should state it\. The authors should preferably report a 2\-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified\. - •For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range \(e\.g\., negative error rates\)\. - •If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text\.
36. 8\.Experiments compute resources
37. Question: For each experiment, does the paper provide sufficient information on the computer resources \(type of compute workers, memory, time of execution\) needed to reproduce the experiments?
38. Answer:\[Yes\]
39. Justification: All experiments are inference\-only evaluations: frontier models are accessed via hosted API endpoints, and open\-weight models \(7B–72B, AWQ\-quantized\) run on a single GPU node; the appendix reports the inference setup and approximate total evaluation cost, including gating reruns\.
40. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage\. - •The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute\. - •The paper should disclose whether the full research project required more compute than the experiments reported in the paper \(e\.g\., preliminary or failed experiments that didn’t make it into the paper\)\.
41. 9\.Code of ethics
43. Answer:\[Yes\]
44. Justification: We have reviewed the NeurIPS Code of Ethics and the research conforms to it: the benchmark consists of synthetic sessions and TTS\-rendered audio, involves no human subjects or personal data, and the submission preserves anonymity\.
45. Guidelines: - •The answer\[N/A\]means that the authors have not reviewed the NeurIPS Code of Ethics\. - •If the authors answer\[No\], they should explain the special circumstances that require a deviation from the Code of Ethics\. - •The authors should make sure to preserve anonymity \(e\.g\., if there is a special consideration due to laws or regulations in their jurisdiction\)\.
46. 10\.Broader impacts
47. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
48. Answer:\[Yes\]
49. Justification: The paper discusses positive impacts \(memory systems that respond to how something was said, benefiting conversational and assistive applications\) and negative ones: persistent storage of affective and paralinguistic metadata about users raises privacy and profiling risks, which the paper notes alongside mitigations such as user control over stored metadata\.
50. Guidelines: - •The answer\[N/A\]means that there is no societal impact of the work performed\. - •If the authors answer\[N/A\]or\[No\], they should explain why their work has no societal impact or why the paper does not address societal impact\. - •Examples of negative societal impacts include potential malicious or unintended uses \(e\.g\., disinformation, generating fake profiles, surveillance\), fairness considerations \(e\.g\., deployment of technologies that could make decisions that unfairly impact specific groups\), privacy considerations, and security considerations\. - •The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments\. However, if there is a direct path to any negative applications, the authors should point it out\. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation\. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster\. - •The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from \(intentional or unintentional\) misuse of the technology\. - •If there are negative societal impacts, the authors could also discuss possible mitigation strategies \(e\.g\., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML\)\.
51. 11\.Safeguards
52. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse \(e\.g\., pre\-trained language models, image generators, or scraped datasets\)?
53. Answer:\[N/A\]
54. Justification: The paper releases no models; the benchmark is a small evaluation set of synthetic text sessions and TTS\-rendered audio \(no scraped data\) and poses no high risk for misuse\.
55. Guidelines: - •The answer\[N/A\]means that the paper poses no such risks\. - •Released models that have a high risk for misuse or dual\-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters\. - •Datasets that have been scraped from the Internet could pose safety risks\. The authors should describe how they avoided releasing unsafe images\. - •We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort\.
56. 12\.Licenses for existing assets
57. Question: Are the creators or original owners of assets \(e\.g\., code, data, models\), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
58. Answer:\[Yes\]
59. Justification: All existing assets are cited — RAVDESS, Dia TTS, Whisper large\-v3, Qwen2\-Audio\-7B, Qwen2\.5\-Omni\-7B, Qwen2\.5\-72B\-Instruct, and the hosted frontier\-model APIs — and are used in accordance with their respective licenses and terms of service, listed in the appendix\.
60. Guidelines: - •The answer\[N/A\]means that the paper does not use existing assets\. - •The authors should cite the original paper that produced the code package or dataset\. - •The authors should state which version of the asset is used and, if possible, include a URL\. - •The name of the license \(e\.g\., CC\-BY 4\.0\) should be included for each asset\. - •For scraped data from a particular source \(e\.g\., website\), the copyright and terms of service of that source should be provided\. - •If assets are released, the license, copyright information, and terms of use in the package should be provided\. For popular datasets,[paperswithcode\.com/datasets](https://paperswithcode.com/datasets)has curated licenses for some datasets\. Their licensing guide can help determine the license of a dataset\. - •For existing datasets that are re\-packaged, both the original license and the license of the derived asset \(if it has changed\) should be provided\. - •If this information is not available online, the authors are encouraged to reach out to the asset’s creators\.
61. 13\.New assets
62. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
63. Answer:\[Yes\]
64. Justification: The VLME benchmark introduced here is documented in Section 3 and the appendix \(item schema and taxonomy, gating rubrics and prompts, audio synthesis recipe\), and this documentation is included in the anonymized repository \(see Question 5\)\. All released content is synthetic \(LLM\-authored text and TTS\-rendered audio\); voice conditioning uses reference clips from the licensed RAVDESS corpus\.
65. Guidelines: - •The answer\[N/A\]means that the paper does not release new assets\. - •Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates\. This includes details about training, license, limitations, etc\. - •The paper should discuss whether and how consent was obtained from people whose asset is used\. - •At submission time, remember to anonymize your assets \(if applicable\)\. You can either create an anonymized URL or include an anonymized zip file\.
66. 14\.Crowdsourcing and research with human subjects
67. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation \(if any\)?
68. Answer:\[N/A\]
69. Justification: The paper involves neither crowdsourcing nor research with human subjects; sessions are LLM\-generated, and all audits are performed by the authors via automated adversary–judge gates and static checks\.
70. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper\. - •According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector\.
71. 15\.Institutional review board \(IRB\) approvals or equivalent for research with human subjects
72. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board \(IRB\) approvals \(or an equivalent approval/review based on the requirements of your country or institution\) were obtained?
73. Answer:\[N/A\]
74. Justification: No human\-subject research was conducted, so no IRB review or equivalent was required\.
75. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Depending on the country in which research is conducted, IRB approval \(or equivalent\) may be required for any human subjects research\. If you obtained IRB approval, you should clearly state this in the paper\. - •We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution\. - •For initial submissions, do not include any information that would break anonymity \(if applicable\), such as the institution conducting the review\.
76. 16\.Declaration of LLM usage
77. Question: Does the paper describe the usage of LLMs if it is an important, original, or non\-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does*not*impact the core methodology, scientific rigor, or originality of the research, declaration is not required\.
78. Answer:\[Yes\]
79. Justification: LLMs are a core methodological component and are fully described in Sections 3–4: benchmark items are LLM\-authored and validated through an LLM adversary–judge gating protocol, evaluation scoring uses an LLM judge with a cross\-judge control, and \(audio\-\)LLMs are the systems under evaluation\.
80. Guidelines: - •The answer\[N/A\]means that the core method development in this research does not involve LLMs as any important, original, or non\-standard components\. - •Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described\.