EmpaAva: An Open-source Agentic 3D-Avatar Empathetic Live Chatbot

# EmpaAva: An Open-source Agentic 3D-Avatar Empathetic Live Chatbot
Source: [https://arxiv.org/html/2608.04709](https://arxiv.org/html/2608.04709)
Jie Yang1, Wenhao Xu1, Shuhui Lin2, Hao Fei3 1National University of Singapore2Tsinghua University3University of Oxford e1554543@u\.nus\.edu, haofei7419@gmail\.com

###### Abstract

This paper presentsEmpaAva, to our knowledge the first open\-source, agentic 3D\-avatar empathetic chatbot, which carries empathetic response generation \(ERG\) from text\-only exchanges into live, face\-to\-face interaction\. Through a video\-call\-like interface, a user speaks to a 3D digital human that reads their affect from speech and optional vision, and replies with emotional speech, lip\-synced facial motion, and photorealistic 3D Gaussian rendering\. At its core, an LLM coordinates aTri\-Agent Architecture, in which perception, empathetic response planning, and embodied rendering form a closed loop, paired with aResponse Planninglayer that compiles each reply into an executable multimodal plan, keeping voice, expression, and rendering on one empathetic intent\. Building on strong open\-source modules,EmpaAvasupplies the intelligence that binds them into one controllable, inspectable experience\. In automatic and human evaluations, EmpaAva surpasses text\-only, 2D talking\-face, and multimodal avatar baselines in emotion understanding, response quality, and audio\-visual consistency\. We open\-source EmpaAva111[https://empaava\.top](https://empaava.top/)with an online live demo222[https://empaava\.top/demo](https://empaava.top/demo)\.

EmpaAva: An Open\-source Agentic 3D\-Avatar Empathetic Live Chatbot

Jie Yang1, Wenhao Xu1, Shuhui Lin2, Hao Fei3††thanks:Corresponding author\.1National University of Singapore2Tsinghua University3University of Oxforde1554543@u\.nus\.edu, haofei7419@gmail\.com

## 1Introduction

The past few years have turned large language models\(LLMs; OpenAI,[2022](https://arxiv.org/html/2608.04709#bib.bib16); Chunget al\.,[2022](https://arxiv.org/html/2608.04709#bib.bib17)\)into remarkably fluent conversational partners, yet fluency alone does not make a machine feel human\. Human\-level interaction also demands*empathy*: the capacity to read another’s feelings and reply with comfort and support\. This goal drives empathetic response generation\(ERG; Rashkinet al\.,[2019](https://arxiv.org/html/2608.04709#bib.bib1)\)and a line of research that infers a user’s affect before answering\(Linet al\.,[2019](https://arxiv.org/html/2608.04709#bib.bib2); Majumderet al\.,[2020](https://arxiv.org/html/2608.04709#bib.bib3); Liet al\.,[2020](https://arxiv.org/html/2608.04709#bib.bib4)\), already backing scenarios from mental\-health companionship to emotional\-support dialogue\(Liuet al\.,[2021](https://arxiv.org/html/2608.04709#bib.bib18)\)\.

Human empathy, however, travels through far more than words\. A trembling voice, a softened gaze, a hesitant pause carry affective weight that plain text can never hold\. Recent research therefore pushes ERG into a multimodal setting, jointly modeling what a user says, how they say it, and how their face changes\(Feiet al\.,[2024](https://arxiv.org/html/2608.04709#bib.bib6); Zhanget al\.,[2025](https://arxiv.org/html/2608.04709#bib.bib7)\)\. In parallel, avatar\-driven video generation has matured dramatically, with audio\-driven portrait models\(Zhanget al\.,[2022](https://arxiv.org/html/2608.04709#bib.bib9); Tianet al\.,[2024](https://arxiv.org/html/2608.04709#bib.bib22); Xuet al\.,[2024c](https://arxiv.org/html/2608.04709#bib.bib23)\)synthesizing lifelike talking faces\. A face\-to\-face digital human is arguably the most natural vessel for empathy: it lets a user*see*the listener nod, smile, and react, closing a loop that a chat bubble leaves open\.

![Refer to caption](https://arxiv.org/html/2608.04709v1/illus/booth_empaava.png)Figure 1:EmpaAva\. In a video\-call\-like booth, the user speaks to a 3D digital human that senses their emotion and replies face to face, with emotional speech, lip\-synced facial motion, and photorealistic 3D\-avatar rendering\.Turning this vision into a deployable system, though, is far from settled, and two obstacles stand out\. The first isexpressive fidelity\. Many empathetic\-avatar efforts still rest on 2D talking\-face synthesis\(Prajwalet al\.,[2020](https://arxiv.org/html/2608.04709#bib.bib8); Zhanget al\.,[2022](https://arxiv.org/html/2608.04709#bib.bib9)\), which copes poorly with three\-dimensional consistency, natural head movement, and spatial realism\. For a system meant to sit across from a user in a video call, a 3D avatar is the more faithful embodiment: FLAME\(Liet al\.,[2017](https://arxiv.org/html/2608.04709#bib.bib11)\)offers controllable geometry and expression, while 3D Gaussian Splatting\(Kerblet al\.,[2023](https://arxiv.org/html/2608.04709#bib.bib12); Qianet al\.,[2023](https://arxiv.org/html/2608.04709#bib.bib13); Choet al\.,[2024](https://arxiv.org/html/2608.04709#bib.bib24)\)renders photorealistic, view\-consistent heads beyond the reach of 2D pipelines\. The second isinteraction intelligence\. Existing 3D\-avatar methods mostly animate a talking head from given speech, and stop short of the full arc from perceiving a user’s emotion, to planning an empathetic reply, to voicing it through an embodied face\. No*open\-source, live*system, to our knowledge, wraps this arc into something the community can readily run and extend\.

We close this gap withEmpaAva, to our knowledge the first open\-source, agentic 3D\-avatar empathetic live chatbot\. As previewed in Figure[1](https://arxiv.org/html/2608.04709#S1.F1), a user steps into a video\-call\-like booth and talks with a 3D digital human that listens, gauges the user’s affect, and answers with synchronized speech, facial motion, and emotionally consistent rendering\. For perception and generation, EmpaAva stands on strong open\-source shoulders, e\.g\., 3DGS\+FLAME rendering and the emotional TTS engine EmotiVoice\(NetEase Youdao,[2023](https://arxiv.org/html/2608.04709#bib.bib26)\), so that its contribution centers on the intelligence orchestrating these parts\.

That intelligence is arranged as an agentic system with an LLM at its reasoning core, echoing the shift toward LLMs as controllers that plan and dispatch specialized tools\(Yaoet al\.,[2023](https://arxiv.org/html/2608.04709#bib.bib19); Shenet al\.,[2023](https://arxiv.org/html/2608.04709#bib.bib21); Xiet al\.,[2023](https://arxiv.org/html/2608.04709#bib.bib20)\)\. Two designs carry the novelty\. TheTri\-Agent Architecturesplits the longperceive–decide–expresschain into three cooperating agents,PerceptionAgent,ResponseAgent, andRenderAgent, mirroring how a person first reads a situation, then decides how to respond, and finally conveys it through voice and expression\. Beyond keeping each stage modular and independently upgradable, the split turns a flat pipeline into a closed empathetic loop that tracks the user’s emotional state across turns\.

The second is aResponse Planninglayer between the LLM and the rendering back\-end\. Rather than forward a bare line of text, ResponseAgent emits a structured*reply plan*fixing who speaks, in which voice and emotional tone, against which background, and which modules to invoke\. This plan acts as one shared expressive intent that every generator obeys, keeping the comforting wording, the gentle voice, and the softened expression aligned, and granting fine control over how strongly the avatar smiles, nods, or slows its pace\.

Overall, we make the following contributions:

1. 1\)We presentEmpaAva, to our knowledge the firstopen\-source, liveagentic 3D\-avatar empathetic chatbot, delivering face\-to\-face empathetic interaction over text, speech, and vision\.
2. 2\)We propose aTri\-Agent Architecturethat decomposes empathetic interaction into perception, response planning, and embodied rendering, forming a closed, controllable, and state\-aware empathetic loop\.
3. 3\)We introduce aResponse Planninglayer that translates an LLM reply into an executable multimodal expression plan, enforcing cross\-modal consistency and controllable avatar behavior\.
4. 4\)We release the full implementation, an online live demo, and the evaluation setup to support research on embodied empathetic agents\.

Table 1:Positioning ofEmpaAvaagainst representative prior work\.OS: open\-source;LLM: LLM\-based reasoning core;MM: multimodal \(text\+speech\+vision\) input;3D: embodied 3D\-avatar output;Emp\.: empathy\-oriented;Live: interactive live system\.SystemOSLLMMM3DEmp\.LiveText ERG\(Linet al\.,[2019](https://arxiv.org/html/2608.04709#bib.bib2)\)✓×\\times×\\times×\\times✓×\\times2D Talking\-Face\(Zhanget al\.,[2022](https://arxiv.org/html/2608.04709#bib.bib9)\)✓×\\times×\\times×\\times×\\times×\\times3D Avatar\(Qianet al\.,[2023](https://arxiv.org/html/2608.04709#bib.bib13)\)✓×\\times×\\times✓×\\times×\\timesAvaMERG\(Zhanget al\.,[2025](https://arxiv.org/html/2608.04709#bib.bib7)\)×\\times✓✓×\\times✓×\\timesEmpathyEar\(Feiet al\.,[2024](https://arxiv.org/html/2608.04709#bib.bib6)\)✓✓✓×\\times✓✓EmpaAva\(Ours\)✓✓✓✓✓✓

## 2Related Work

##### Empathetic Dialogue, from Text to Multimodal\.

Empathetic response generation asks a dialogue system to recognize how a user feels and reply with comfort and support\. Early work lives entirely in text: EmpatheticDialogues\(Rashkinet al\.,[2019](https://arxiv.org/html/2608.04709#bib.bib1)\)set up a widely used benchmark, models such as MoEL, MIME, and EmpDG encode user emotion to sharpen affective quality and relevance\(Linet al\.,[2019](https://arxiv.org/html/2608.04709#bib.bib2); Majumderet al\.,[2020](https://arxiv.org/html/2608.04709#bib.bib3); Liet al\.,[2020](https://arxiv.org/html/2608.04709#bib.bib4)\), and emotional\-support dialogue adds strategy\-guided comforting\(Liuet al\.,[2021](https://arxiv.org/html/2608.04709#bib.bib18)\)\. Reading and writing only words, these methods let slip the prosody, expression, speaking rate, and micro\-expressions through which people reveal how they truly feel\. A more recent thread therefore reaches into vision and speech\(Poriaet al\.,[2019](https://arxiv.org/html/2608.04709#bib.bib5)\), with avatar\-oriented systems such as EmpathyEar and AvaMERG jointly modeling text, speech, and vision\(Feiet al\.,[2024](https://arxiv.org/html/2608.04709#bib.bib6); Zhanget al\.,[2025](https://arxiv.org/html/2608.04709#bib.bib7)\)\. Their replies, however, still surface as text, speech, or 2D talking faces, leaving embodied feedback beyond reach\.

##### Talking\-Face and 3D\-Avatar Generation\.

Expressing empathy also calls for the right voice, facial motion, and head gesture, motivating a parallel line on avatar generation\. 2D approaches such as Wav2Lip and SadTalker synthesize lip\-synchronized faces from speech or a single image\(Prajwalet al\.,[2020](https://arxiv.org/html/2608.04709#bib.bib8); Zhanget al\.,[2022](https://arxiv.org/html/2608.04709#bib.bib9)\), DEEPTalk injects emotionally expressive dynamics\(Kimet al\.,[2024](https://arxiv.org/html/2608.04709#bib.bib10)\), and recent diffusion\-based portrait models push photorealism and head motion further\(Tianet al\.,[2024](https://arxiv.org/html/2608.04709#bib.bib22); Xuet al\.,[2024c](https://arxiv.org/html/2608.04709#bib.bib23),[b](https://arxiv.org/html/2608.04709#bib.bib25)\)\. Tied to a 2D image plane, all remain fragile on three\-dimensional consistency, natural head pose, and spatial realism\. 3D avatar techniques attack this directly, coupling FLAME\(Liet al\.,[2017](https://arxiv.org/html/2608.04709#bib.bib11)\)for controllable geometry and expression with 3D Gaussian Splatting\(Kerblet al\.,[2023](https://arxiv.org/html/2608.04709#bib.bib12)\)for high\-fidelity rendering; GaussianAvatars, GaussianTalker, When Words Smile, and A2\-LLM showcase controllable, audio\-driven 3D heads\(Qianet al\.,[2023](https://arxiv.org/html/2608.04709#bib.bib13); Choet al\.,[2024](https://arxiv.org/html/2608.04709#bib.bib24); Xuet al\.,[2024a](https://arxiv.org/html/2608.04709#bib.bib14); Huet al\.,[2026](https://arxiv.org/html/2608.04709#bib.bib15)\)\. The connective tissue stays missing: these systems animate a talking head from given speech, and none assembles an open, LLM\-driven loop from perceiving user emotion, through empathetic planning, to embodied 3D expression\.EmpaAvaoccupies exactly this space, uniting speech recognition, speech emotion recognition, LLM\-based empathetic planning, structured reply plans, emotional speech synthesis, audio\-driven facial motion, FLAME control, and 3DGS rendering in a single agentic system \(Table[1](https://arxiv.org/html/2608.04709#S1.T1)\)\.

![Refer to caption](https://arxiv.org/html/2608.04709v1/x1.png)Figure 2:Workflow of theEmpaAvasystem\.![Refer to caption](https://arxiv.org/html/2608.04709v1/x2.png)Figure 3:The Tri\-Agent architecture of EmpaAva\.PerceptionAgentunderstands the user,ResponseAgentplans an empathetic reply, andRenderAgentturns the plan into an embodied 3D\-avatar video\.

## 3System Workflow

EmpaAvaruns entirely in the browser as a video\-call\-like emotional booth, and Figure[2](https://arxiv.org/html/2608.04709#S2.F2)traces one full round of interaction, which we walk through from the user’s side\.

1. ▶\\blacktrianglerightEntry\(Steps 1–3\)\. A visitor joins as a guest with no sign\-in, picks a digital avatar together with its voice and background from a pop\-up, and lands straight in the call, where the avatar fills the main window and the microphone and optional camera come online\.
2. ▶\\blacktrianglerightTalking\(Step 4\)\. The interaction mirrors a real call: microphone and camera stay on, and voice\-activity detection segments each utterance and submits it automatically, so no record, send, or stop button is needed\. The spoken language is detected on the fly and drives recognition, reasoning, and synthesis, letting the user converse in English or Chinese\.
3. ▶\\blacktrianglerightResponding\(Steps 5–6\)\. For every turn, the backend reads the user’s affect, plans an empathetic reply, and renders it as an embodied avatar video with synchronized emotional speech and facial motion\. The reply is placed beside the user’s view at matching size, and the user may replay it or rotate the 3D head to inspect the avatar from new viewpoints\.
4. ▶\\blacktrianglerightHistory\(Steps 7–8\)\. The dialogue continues or stops at any moment, and each turn is archived as a paired user\-and\-avatar record for later replay\. A lightweight registration is requested only when the user exports the full history\.

## 4Implementation Specification

EmpaAva turns a stream of user audio\-visual input into an embodied avatar reply that can be seen, heard, and answered back\. We avoid handing this whole job to a single end\-to\-end model, and instead factor it into three cooperating agents coordinated by an LLM \(Figure[3](https://arxiv.org/html/2608.04709#S2.F3)\):PerceptionAgentsenses the user,ResponseAgentdecides what and how to reply, andRenderAgentrealizes the reply as a 3D\-avatar video\. The agents talk to each other through a shared, human\-readable state instead of opaque tensors, so every stage stays independently testable and swappable; a stronger ASR model, a larger LLM, or a higher\-fidelity renderer can be dropped in without disturbing the rest, and the same transparency is what makes the pipeline easy to ablate in Section[5](https://arxiv.org/html/2608.04709#S5)\.

### 4\.1PerceptionAgent

PerceptionAgentconverts raw user input into a structured percept\. After light preprocessing \(format conversion, resampling, denoising\), automatic speech recognition transcribes the utterance, while speech emotion recognition reads the acoustic affect carried by prosody, pitch, and pace\. That second signal matters because words alone can mislead: a flat sentence may still sound tired or anxious, and the avatar ought to answer the feeling and not only the text\. When the camera is enabled, a few lightweight frames are sampled as visual context\. The agent then packs the transcript, the speech\-emotion label, the sampled frames, dialogue history, and input metadata into a*dialogue schema*and hands it toResponseAgent\.

### 4\.2ResponseAgent: Empathetic Response Planning

ResponseAgentis the decision center, and the place where our response\-planning idea lives\. Conditioned on the percept, the LLM first reasons over the user’s emotion, its likely cause, and the conversation so far, and then produces more than an utterance\. Its output is a structured*reply plan*that couples the words to be spoken with how, and by whom, they should be delivered:

Reply Planreply text: what the avatar says •emotion & tone: target affect and delivery •avatar: which digital human speaks •voice: speaker timbre for TTS •background: interaction scene •evidence: the user emotion and inferred cause being addressed\.

This plan is a single expressive contract that every downstream generator obeys, which is what keeps the comforting wording, the voice, and the facial expression aimed at the same emotional target\. It also carries an empathetic strategy rather than a lone answer, typically acknowledging the feeling, naming its cause, validating it, and only then offering a low\-burden suggestion\. Because each field points back to the perception evidence it responds to, the plan stays grounded and interpretable, andResponseAgentcan revise it turn by turn as the user’s state drifts\.

### 4\.3RenderAgent

RenderAgentexecutes the plan and produces the avatar video\. Emotional speech is first synthesized with EmotiVoice\(NetEase Youdao,[2023](https://arxiv.org/html/2608.04709#bib.bib26)\)under the plan’s tone, so a comforting turn sounds soft and steady while an encouraging one sounds brighter\. This speech then drives an audio\-to\-motion module that predicts frame\-level FLAME\(Liet al\.,[2017](https://arxiv.org/html/2608.04709#bib.bib11)\)parameters for jaw, lip shape, head pose, and expression\. We deliberately keep appearance and motion apart: a 3D Gaussian Splatting\(Kerblet al\.,[2023](https://arxiv.org/html/2608.04709#bib.bib12)\)head carries the identity\-specific geometry and texture, FLAME acts as a low\-dimensional control space, and a FLAME\-to\-Gaussian transfer step applies the predicted motion to the rigged Gaussian head, letting the avatar speak while holding its personalized look\. Three levers govern output quality: \(i\) expression, jaw, and lip ranges are clamped to stay expressive without tipping into exaggeration; \(ii\) temporal smoothing and offset correction tighten lip\-sync and suppress jitter; and \(iii\) 3DGS rendering with background compositing sets the avatar in a realistic call scene\. The final clip carries the intended empathy across speech, motion, and appearance\.

Table 2:Automatic results on EmpatheticDialogues\. Acc\.: emotion\-cue accuracy; Dist\-1/2: diversity\.BlenderBotEmpathyEarAvaMERGEmpaAva

Dist\-2Emo\.AccCause\.M02020404060608080Score45\.8145\.8153\.8553\.8574\.8174\.81

Figure 4:Automatic evaluation on the AvaMERG test set\. Higher scores indicate better response diversity, emotion understanding, and emotion\-cause modeling\.

## 5Evaluation and Analysis

We evaluateEmpaAvaon text\-level response generation, end\-to\-end multimodal interaction, and embodied expression quality, plus two multi\-turn case studies\. Subjective scores are averaged over human raters on a five\-point scale\.

Table 3:Human evaluation scores from a questionnaire study with 10 participants, each evaluating 5 user turns with four anonymized system responses\. Emp\.: Response Empathy; Rel\.: Relevance; Spec\.: Specificity; Pref\.: Overall Preference votes out of 50\.Table 4:Automatic evaluation of avatar embodied expression quality on the AvaMERG evaluation set\. Higher LSE\-C and A\-V Cos are better, while lower LSE\-D is better\. Best results are in bold\. The underlined value denotes the second\-best LSE\-D score\.### 5\.1Empathetic System Comparison

We compareEmpaAvaat the text and end\-to\-end embodied\-avatar levels\.

#### 5\.1\.1Text\-Level Response Comparison

##### Protocol\.

We first evaluate EmpaAva’s text response module on the full EmpatheticDialogues test split\(Rashkinet al\.,[2019](https://arxiv.org/html/2608.04709#bib.bib1)\)\(5,255 samples with emotion labels and references\), one response per system per sample\. Baselines cover non\-LLM ERG methods \(MoEL, MIME, EmpDG, CASE, ESCM\), LLMs \(Alpaca, Flan\-T5, ChatGLM, Qwen, and a GPT\-style LLM\), and text modules of multimodal systems \(EmpathyEar, AvaMERG\)\. For EmpaAva, we use the same deterministic decoding setting for all test samples\.

##### Results\.

Table[2](https://arxiv.org/html/2608.04709#S4.T2)shows EmpaAva leading on all three metrics \(Acc\. 10\.26, Dist\-1 4\.01, Dist\-2 30\.54\), indicating stronger emotion\-cue alignment and lexical diversity than non\-LLM, LLM, and avatar\-based baselines\.

![Refer to caption](https://arxiv.org/html/2608.04709v1/x3.png)Figure 5:Qualitative multi\-turn case studies ofEmpaAva\.

#### 5\.1\.2End\-to\-End Avatar Comparison

##### Protocol\.

We evaluate EmpaAva’s complete multimodal output \(text, emotional speech, 3D avatar video\) on the AvaMERG test set\(Zhanget al\.,[2025](https://arxiv.org/html/2608.04709#bib.bib7)\), against BlenderBot\(Rolleret al\.,[2020](https://arxiv.org/html/2608.04709#bib.bib27)\)with neutral TTS and avatar, EmpathyEar\(Feiet al\.,[2024](https://arxiv.org/html/2608.04709#bib.bib6)\)with 2D talking\-face, and AvaMERG\(Zhanget al\.,[2025](https://arxiv.org/html/2608.04709#bib.bib7)\)\. We report automatic metrics for diversity, emotion understanding, and cause modeling, plus a questionnaire where 10 participants each rate 5 turns over four anonymized responses \(50 preference votes\); MECS and AV\-Agree serve as cross\-modal diagnostics \(Appendix[E](https://arxiv.org/html/2608.04709#A5.SS0.SSS0.Px4)\)\.

##### Results\.

In Figure[4](https://arxiv.org/html/2608.04709#S4.F4), EmpaAva attains the best Dist\-2 \(45\.81\), Emo\.Acc \(53\.85\), and Cause\.M \(74\.81\); on 100 complete avatar outputs it further reaches 22\.0% MECS and 40\.0% AV\-Agree\. Human evaluation in Table[3](https://arxiv.org/html/2608.04709#S5.T3)echoes the trend: highest Empathy \(4\.42\), Relevance \(4\.65\), Specificity \(4\.35\), and preference \(18/50\)\.

### 5\.2Avatar Embodied Expression Quality

##### Protocol\.

This experiment isolates avatar generation, fixing response text, target emotion, speech input, avatar identity, and video format\. On 2,308 AvaMERG samples, we compare TTS\-only playback, Wav2Lip\(Prajwalet al\.,[2020](https://arxiv.org/html/2608.04709#bib.bib8)\), SadTalker\(Zhanget al\.,[2022](https://arxiv.org/html/2608.04709#bib.bib9)\), DEEPTalk\(Kimet al\.,[2024](https://arxiv.org/html/2608.04709#bib.bib10)\), a 3DGS Avatar\(Qianet al\.,[2023](https://arxiv.org/html/2608.04709#bib.bib13)\), and EmpaAva, reporting SyncNet LSE\-C/LSE\-D for lip synchronization and A\-V Cos for speech\-expression emotional consistency\.

##### Results\.

In Table[4](https://arxiv.org/html/2608.04709#S5.T4), EmpaAva obtains the highest LSE\-C \(7\.649\) and A\-V Cos \(0\.346\)\. Wav2Lip wins LSE\-D \(7\.940\) via explicit lip\-sync optimization, yet EmpaAva stays close \(8\.242\) with better emotional consistency, giving the best overall sync–affect balance\.

### 5\.3Qualitative Multi\-turn Case Study

##### Case 1: Academic Stress and Self\-doubt\.

The user \(Figure[5](https://arxiv.org/html/2608.04709#S5.F5), top\) gradually reveals assignment pressure, poor sleep, and self\-doubt; EmpaAva tracks this drift, links it to the earlier stressors, and separates exhaustion from ability\.

##### Case 2: Interpersonal Conflict and Emotional Invalidation\.

The user \(Figure[5](https://arxiv.org/html/2608.04709#S5.F5), bottom\) is hurt and called “too sensitive”; EmpaAva names the invalidation, validates the feelings, reduces self\-blame, and suggests boundary communication without judging the friend\.

## 6Conclusion

We presented EmpaAva, an open\-source agentic 3D\-avatar chatbot for live, face\-to\-face empathetic dialogue: its LLM\-drivenTri\-Agent Architecturecloses the loop from affect perception to embodied rendering, andResponse Planningkeeps speech, facial motion, and rendering on one empathetic intent\. Released with an online demo, it offers an inspectable testbed for embodied empathetic agents; richer bodily expression, longer emotional memory, and lower latency remain future work\.

## Ethical Considerations and Broader Impact

##### Privacy, consent, and data handling\.

EmpaAva takes microphone and optional camera input through a video\-call\-like interface, and both channels are strictly opt\-in: the browser requests each permission explicitly, the purpose of every permission is stated before interaction, and the system remains fully functional with the camera disabled\. Audio is used only for speech recognition and speech emotion recognition, and sampled video frames only for turn\-level multimodal perception; neither stream is used to train models or shared with any third party\. The research demo may produce intermediate artifacts, including perception results, reply plans, synthesized speech, facial\-motion parameters, and rendered videos, solely for debugging, reproducibility, history review, and user\-initiated export\. All examples shown in this paper are selected and anonymized with the consent of the recorded participants, and expose no real user identities, raw recordings, or sensitive information\.

##### Embodied presence and identity safety\.

A photorealistic talking avatar creates stronger social presence than a chat window, which raises risks of deception and emotional over\-reliance\. EmpaAva therefore always presents itself as an AI companion and never impersonates a real person\. Avatar identities and TTS voices derive from publicly released research assets and synthetic sources; the system provides no capability for cloning the face or voice of a user or any third party\. We further avoid manipulative emotional cues and engagement\-maximizing behaviors, and the interface allows the user to stop or leave a session at any moment\.

##### Not a substitute for professional care\.

EmpaAva is a companion\-oriented research demonstration, not a medical or psychological service; its prompts forbid clinical or diagnostic advice and steer replies toward warm, validating, low\-burden support\. When a conversation signals severe distress or self\-harm, the system avoids prescriptive language and encourages the user to seek trusted people or professional help\. Any deployment in real wellbeing contexts would require clinical oversight and safety review beyond this demo\.

##### Human evaluation\.

The questionnaire study involved voluntary on\-campus participants who were informed of the research purpose, rated fully anonymized system outputs presented in randomized order, and contributed no personal data beyond their ratings\.

##### Responsible release\.

We release the code and demo for research purposes under terms that prohibit deceptive impersonation, harassment, and unauthorized use of anyone’s likeness or voice\. We also document known limitations, e\.g\., speech emotion recognition can misread affect and generated comfort can be generic, so that downstream users can assess risks before adoption\.

## References

- K\. Cho, J\. Lee, H\. Yoon, Y\. Hong, J\. Ko, S\. Ahn, and S\. Kim \(2024\)GaussianTalker: real\-time high\-fidelity talking head synthesis with audio\-driven 3d gaussian splatting\.InProceedings of the 32nd ACM International Conference on Multimedia,pp\. 10985–10994\.Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p3.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1)\.
- H\. W\. Chung, L\. Hou, S\. Longpre, B\. Zoph, Y\. Tay, W\. Fedus, E\. Li, X\. Wang, M\. Dehghani, S\. Brahma, A\. Webson, S\. S\. Gu, Z\. Dai, M\. Suzgun, X\. Chen, A\. Chowdhery, S\. Narang, G\. Mishra, A\. Yu, V\. Y\. Zhao, Y\. Huang, A\. M\. Dai, H\. Yu, S\. Petrov, E\. H\. Chi, J\. Dean, J\. Devlin, A\. Roberts, D\. Zhou, Q\. V\. Le, and J\. Wei \(2022\)Scaling instruction\-finetuned language models\.arXiv preprint arXiv:2210\.11416\.Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p1.1)\.
- H\. Fei, H\. Zhang, B\. Wang, L\. Liao, Q\. Liu, and E\. Cambria \(2024\)EmpathyEar: an open\-source avatar multimodal empathetic chatbot\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations,pp\. 61–71\.Cited by:[Table 1](https://arxiv.org/html/2608.04709#S1.T1.17.17.17.2),[§1](https://arxiv.org/html/2608.04709#S1.p2.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px1.p1.1),[§5\.1\.2](https://arxiv.org/html/2608.04709#S5.SS1.SSS2.Px1.p1.1)\.
- X\. Hu, H\. Yuan, X\. Sang, B\. Yan, Z\. Yu, C\. Huang, and K\. Chen \(2026\)A2\-LLM: an end\-to\-end conversational audio avatar large language model\.arXiv preprint arXiv:2602\.04913\.Cited by:[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1)\.
- B\. Kerbl, G\. Kopanas, T\. Leimkühler, and G\. Drettakis \(2023\)3D gaussian splatting for real\-time radiance field rendering\.ACM Transactions on Graphics42\(4\),pp\. 1–14\.Cited by:[Table 5](https://arxiv.org/html/2608.04709#A1.T5.1.8.7.3.1.1),[§1](https://arxiv.org/html/2608.04709#S1.p3.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1),[§4\.3](https://arxiv.org/html/2608.04709#S4.SS3.p1.1)\.
- J\. Kim, J\. Cho, J\. Park, S\. Hwang, D\. E\. Kim, G\. Kim, and Y\. Yu \(2024\)DEEPTalk: dynamic emotion embedding for probabilistic speech\-driven 3d face animation\.arXiv preprint arXiv:2408\.06010\.Cited by:[Table 5](https://arxiv.org/html/2608.04709#A1.T5.1.6.5.3.1.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1),[§5\.2](https://arxiv.org/html/2608.04709#S5.SS2.SSS0.Px1.p1.1)\.
- Q\. Li, H\. Chen, Z\. Ren, P\. Ren, Z\. Tu, and Z\. Chen \(2020\)EmpDG: multi\-resolution interactive empathetic dialogue generation\.InProceedings of the 28th International Conference on Computational Linguistics,pp\. 4454–4466\.Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p1.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px1.p1.1)\.
- T\. Li, T\. Bolkart, M\. J\. Black, H\. Li, and J\. Romero \(2017\)Learning a model of facial shape and expression from 4d scans\.ACM Transactions on Graphics36\(6\),pp\. 1–17\.Cited by:[Table 5](https://arxiv.org/html/2608.04709#A1.T5.1.7.6.3.1.1),[§1](https://arxiv.org/html/2608.04709#S1.p3.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1),[§4\.3](https://arxiv.org/html/2608.04709#S4.SS3.p1.1)\.
- Z\. Lin, A\. Madotto, J\. Shin, P\. Xu, and P\. Fung \(2019\)MoEL: mixture of empathetic listeners\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,pp\. 121–132\.Cited by:[Table 1](https://arxiv.org/html/2608.04709#S1.T1.4.4.4.5),[§1](https://arxiv.org/html/2608.04709#S1.p1.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Liu, C\. Zheng, O\. Demasi, S\. Sabour, Y\. Li, Z\. Yu, Y\. Jiang, and M\. Huang \(2021\)Towards emotional support dialog systems\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing,pp\. 3469–3483\.Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p1.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px1.p1.1)\.
- N\. Majumder, P\. Hong, S\. Peng, J\. Lu, D\. Ghosal, A\. Gelbukh, R\. Mihalcea, and S\. Poria \(2020\)MIME: MIMicking emotions for empathetic response generation\.arXiv preprint arXiv:2010\.01454\.Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p1.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px1.p1.1)\.
- NetEase Youdao \(2023\)EmotiVoice: a multi\-voice and prompt\-controlled TTS engine\.Note:[https://github\.com/netease\-youdao/EmotiVoice](https://github.com/netease-youdao/EmotiVoice)Cited by:[Table 5](https://arxiv.org/html/2608.04709#A1.T5.1.5.4.3.1.1),[§1](https://arxiv.org/html/2608.04709#S1.p4.1),[§4\.3](https://arxiv.org/html/2608.04709#S4.SS3.p1.1)\.
- OpenAI \(2022\)Introducing ChatGPT\.Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p1.1)\.
- S\. Poria, D\. Hazarika, N\. Majumder, G\. Naik, E\. Cambria, and R\. Mihalcea \(2019\)MELD: a multimodal multi\-party dataset for emotion recognition in conversations\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 527–536\.Cited by:[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px1.p1.1)\.
- K\. R\. Prajwal, R\. Mukhopadhyay, V\. P\. Namboodiri, and C\. V\. Jawahar \(2020\)A lip sync expert is all you need for speech to lip generation in the wild\.InProceedings of the 28th ACM International Conference on Multimedia,pp\. 484–492\.Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p3.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1),[§5\.2](https://arxiv.org/html/2608.04709#S5.SS2.SSS0.Px1.p1.1)\.
- S\. Qian, T\. Kirschstein, L\. Schoneveld, D\. Davoli, S\. Giebenhain, and M\. Nießner \(2023\)GaussianAvatars: photorealistic head avatars with rigged 3d gaussians\.arXiv preprint arXiv:2312\.02069\.Cited by:[Table 5](https://arxiv.org/html/2608.04709#A1.T5.1.8.7.3.1.1),[Table 1](https://arxiv.org/html/2608.04709#S1.T1.13.13.13.5),[§1](https://arxiv.org/html/2608.04709#S1.p3.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1),[§5\.2](https://arxiv.org/html/2608.04709#S5.SS2.SSS0.Px1.p1.1)\.
- H\. Rashkin, E\. M\. Smith, M\. Li, and Y\. Boureau \(2019\)Towards empathetic open\-domain conversation models: a new benchmark and dataset\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 5370–5381\.Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p1.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px1.p1.1),[§5\.1\.1](https://arxiv.org/html/2608.04709#S5.SS1.SSS1.Px1.p1.1)\.
- S\. Roller, E\. Dinan, N\. Goyal, D\. Ju, M\. Williamson, Y\. Liu, J\. Xu, M\. Ott, K\. Shuster, E\. M\. Smith, Y\. Boureau, and J\. Weston \(2020\)Recipes for building an open\-domain chatbot\.External Links:2004\.13637,[Link](https://arxiv.org/abs/2004.13637)Cited by:[§5\.1\.2](https://arxiv.org/html/2608.04709#S5.SS1.SSS2.Px1.p1.1)\.
- Y\. Shen, K\. Song, X\. Tan, D\. Li, W\. Lu, and Y\. Zhuang \(2023\)HuggingGPT: solving AI tasks with ChatGPT and its friends in hugging face\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p5.1)\.
- L\. Tian, Q\. Wang, B\. Zhang, and L\. Bo \(2024\)EMO: emote portrait alive – generating expressive portrait videos with audio2video diffusion model under weak conditions\.InProceedings of the European Conference on Computer Vision,Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p2.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Xi, W\. Chen, X\. Guo, W\. He, Y\. Ding, B\. Hong, M\. Zhang, J\. Wang, S\. Jin, E\. Zhou,et al\.\(2023\)The rise and potential of large language model based agents: a survey\.arXiv preprint arXiv:2309\.07864\.Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p5.1)\.
- H\. Xu, M\. Zhang, H\. Ju, Z\. Zheng, E\. Cambria, M\. Zhang, and H\. Fei \(2024a\)When words smile: generating diverse emotional facial expressions from text\.arXiv preprint arXiv:2412\.02508\.Cited by:[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1)\.
- M\. Xu, H\. Li, Q\. Su, H\. Shang, L\. Zhang, C\. Liu, J\. Wang, Y\. Yao, and S\. Zhu \(2024b\)Hallo: hierarchical audio\-driven visual synthesis for portrait image animation\.arXiv preprint arXiv:2406\.08801\.Cited by:[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Xu, G\. Chen, Y\. Guo, J\. Yang, C\. Li, Z\. Zang, Y\. Zhang, X\. Tong, and B\. Guo \(2024c\)VASA\-1: lifelike audio\-driven talking faces generated in real time\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p2.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2608.04709#S1.p5.1)\.
- H\. Zhang, Z\. Meng, M\. Luo, H\. Han, L\. Liao, E\. Cambria, and H\. Fei \(2025\)Towards multimodal empathetic response generation: a rich text\-speech\-vision avatar\-based benchmark\.arXiv preprint arXiv:2502\.04976\.Cited by:[Table 1](https://arxiv.org/html/2608.04709#S1.T1.16.16.16.4),[§1](https://arxiv.org/html/2608.04709#S1.p2.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px1.p1.1),[§5\.1\.2](https://arxiv.org/html/2608.04709#S5.SS1.SSS2.Px1.p1.1)\.
- W\. Zhang, X\. Cun, X\. Wang, Y\. Zhang, X\. Shen, Y\. Guo, Y\. Shan, and F\. Wang \(2022\)SadTalker: learning realistic 3d motion coefficients for stylized audio\-driven single image talking face animation\.arXiv preprint arXiv:2211\.12194\.Cited by:[Table 1](https://arxiv.org/html/2608.04709#S1.T1.9.9.9.6),[§1](https://arxiv.org/html/2608.04709#S1.p2.1),[§1](https://arxiv.org/html/2608.04709#S1.p3.1),[§2](https://arxiv.org/html/2608.04709#S2.SS0.SSS0.Px2.p1.1),[§5\.2](https://arxiv.org/html/2608.04709#S5.SS2.SSS0.Px1.p1.1)\.

## Appendix ASystem Implementation Details

Table[5](https://arxiv.org/html/2608.04709#A1.T5)lists the backbone of each Tri\-Agent module\. EmpaAva runs in the browser: a lightweight frontend renders the call booth and streams audio/video input, while the backend orchestrates the three agents and returns the rendered clip\.

Table 5:Backbone modules of EmpaAva\.##### Deployment and latency\.

The backend uses FastAPI with persistent local workers\. On an NVIDIA H200, warm\-start per\-turn latency averages 45\.8s, dominated by Gaussian rendering/export \(41\.5s\), with 2\.1s perception, 0\.002s planning, 0\.27s TTS, and 0\.05s audio\-to\-motion\.

## Appendix BAgent Prompt Designs

We expose the full agent prompt templates; angle\-bracket placeholders \(e\.g\.,<user\-text\>\) are filled at run time from the dialogue schema\.

##### Global system role\.

System Prompt \(shared\)You are the reasoning core of EmpaAva, an empathetic 3D\-avatarcompanion\. Your goal is to understand how the user feels, whythey feel that way, and to respond with warmth, validation, andgentle, low\-burden support\. Never give clinical or diagnosticadvice\. Always keep the reply short enough to be spoken aloudby an avatar in a video call\.

##### PerceptionAgent: state fusion\.

This prompt fuses transcript, acoustic emotion, and optional visual cues into a compact user\-state summary\.

PerceptionAgent PromptInputs:\- transcript: <user\-text\>\- speech emotion \(SER\): <speech\-emotion\>\- visual cues \(optional\): <face\-cues\>\- dialogue history: <history\>Summarize the user’s current state in 1\-2 sentences:\(a\) the dominant emotion, \(b\) its most likely cause, and\(c\) whether it intensifies or eases relative to prior turns\.Return only the summary\.

##### ResponseAgent: empathetic planning\.

It reasons over the fused state and emits a structured*reply plan*\(Appendix[C](https://arxiv.org/html/2608.04709#A3)\) rather than a bare utterance\.

ResponseAgent PromptUser state: <perception\-summary\>Speech emotion: <speech\-emotion\>Dialogue history: <history\>Available avatars: <avatar\-list\>Available voices: <voice\-list\>Available backgrounds: <bg\-list\>Think step by step, then produce a reply plan:1\. Name the user’s emotion and its likely cause\.2\. Choose an empathetic strategy: acknowledge \-\> addresscause \-\> validate \-\> offer one low\-burden suggestion\.3\. Write the reply text to be spoken \(up to 3 sentences\)\.4\. Set the target emotion and tone, avatar, voice, andbackground for the delivery\.5\. Cite the perception evidence each choice responds to\.Output the reply plan as a single JSON object followingthe schema\. Do not add any text outside the JSON\.

## Appendix CReply Plan Schema

The reply plan is the machine\-readable contract passed fromResponseAgenttoRenderAgent, specifying the spoken response, delivery style, selected avatar, TTS speaker, background, and links to intermediate perception artifacts\.

##### Example reply plan\.

Reply Plan Example\{"schema": 1, "agent": "ResponseAgent", "strategy": "acknowledge\_and\_support", "emotion": "anxiety", "cause": "multiple approaching deadlines", "evidence": "I have several deadlines this week\.", "reply\_text": "That sounds overwhelming\. Let’s take one small step first\.", "tone": "warm", "expression": "concerned", "selected\_avatar\_id": "306", "selected\_tts\_speaker\_id": "6224", "background": null\}

## Appendix DAvatar Identity Gallery

Figure[6](https://arxiv.org/html/2608.04709#A4.F6)shows representative renderings by EmpaAva, demonstrating visually consistent 3D digital humans with diverse identities and expressions\.

![[Uncaptioned image]](https://arxiv.org/html/2608.04709v1/x4.png)

Figure 6:Qualitative avatar renderings across different identities\.

## Appendix EExperimental Settings

##### Datasets\.

We use AvaMERG and EmpatheticDialogues as complementary evaluation sources\. AvaMERG provides 6,288 valid multimodal instances after filtering, covering user input, emotions, responses, and avatar resources, and serves both response\-level and avatar\-level evaluation\. EmpatheticDialogues contains about 25K text conversations with 32 emotion labels and is used for text\-response and response\-planning evaluation\. We sample from both datasets with balanced emotion distribution, remove invalid cases, and build the avatar evaluation subset from AvaMERG by selecting samples with valid text, emotion labels, avatar identities, and renderable resources\.

##### Human evaluation protocol\.

We conduct a small pilot human evaluation with on\-campus students from NLP, multimodal learning, or psychology backgrounds\. Participants view anonymized system responses in randomized order with matched player size and normalized audio, rate*Response Empathy*,*Relevance*, and*Specificity*on a 1–5 Likert scale, and select one overall preferred response\. A representative rater instruction follows\.

Rater Instruction \(excerpt\)You will watch a short clip of a user turn and the avatar’sresponse\. Rate 1\-\-5 on each dimension:\- Response Empathy: is the reply warm, validating, andsupportive?\- Relevance: does the reply directly address the user’ssituation and emotional state?\- Specificity: does the reply provide concrete andcontext\-specific support rather than a generic response?After rating the four anonymized system responses for thesame user turn, select the one you prefer overall\.Judge only the presented response; ignore minor differencesin video resolution or audio volume\.

##### Evaluation metrics\.

Acc\\displaystyle\\mathrm\{Acc\}=100×1N​∑i=1N𝕀​\[y^ie=yie\],\\displaystyle=100\\times\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\\bigl\[\\hat\{y\}\_\{i\}^\{e\}=y\_\{i\}^\{e\}\\bigr\],\(1\)Dist​\-​n\\displaystyle\\mathrm\{Dist\}\\text\{\-\}n=100×\|⋃ingramn⁡\(ri\)\|∑i\|ngramn⁡\(ri\)\|,n∈\{1,2\},\\displaystyle=100\\times\\frac\{\\bigl\|\\bigcup\_\{i\}\\operatorname\{ngram\}\_\{n\}\(r\_\{i\}\)\\bigr\|\}\{\\sum\_\{i\}\\bigl\|\\operatorname\{ngram\}\_\{n\}\(r\_\{i\}\)\\bigr\|\},\\quad n\\in\\\{1,2\\\},\(2\)Cause\.M\\displaystyle\\mathrm\{Cause\.M\}=100×1N​∑i𝕀​\[y^ic=yic\],\\displaystyle=100\\times\\frac\{1\}\{N\}\\sum\_\{i\}\\mathbb\{I\}\\bigl\[\\hat\{y\}^\{c\}\_\{i\}=y^\{c\}\_\{i\}\\bigr\],\(3\)MECS\\displaystyle\\mathrm\{MECS\}=100×1N​∑i𝕀​\[y^it=y^ia=y^iv=yi\],\\displaystyle=100\\times\\frac\{1\}\{N\}\\sum\_\{i\}\\mathbb\{I\}\\bigl\[\\hat\{y\}^\{t\}\_\{i\}=\\hat\{y\}^\{a\}\_\{i\}=\\hat\{y\}^\{v\}\_\{i\}=y\_\{i\}\\bigr\],\(4\)Score​\(m\)\\displaystyle\\mathrm\{Score\}\(m\)=1J​N​∑j=1J∑i=1Nsj,im,\\displaystyle=\\frac\{1\}\{JN\}\\sum\_\{j=1\}^\{J\}\\sum\_\{i=1\}^\{N\}s\_\{j,i\}^\{m\},\(5\)LSE​\-​D\\displaystyle\\mathrm\{LSE\}\\text\{\-\}\\mathrm\{D\}=1M​∑i,t‖ai,t−vi,t‖2,\\displaystyle=\\tfrac\{1\}\{M\}\\sum\_\{i,t\}\\\|a\_\{i,t\}\-v\_\{i,t\}\\\|\_\{2\},\(6\)LSE​\-​C\\displaystyle\\mathrm\{LSE\}\\text\{\-\}\\mathrm\{C\}=1M∑i,t\(medτ≠0∥ai,t−vi,t\+τ∥2\\displaystyle=\\tfrac\{1\}\{M\}\\sum\_\{i,t\}\\bigl\(\\operatorname\{med\}\_\{\\tau\\neq 0\}\\\|a\_\{i,t\}\-v\_\{i,t\+\\tau\}\\\|\_\{2\}−∥ai,t−vi,t∥2\)\.\\displaystyle\\qquad\-\\\|a\_\{i,t\}\-v\_\{i,t\}\\\|\_\{2\}\\bigr\)\.\(7\)

##### Full automatic results\.

Table[6](https://arxiv.org/html/2608.04709#A5.T6)reports the complete numerical automatic results corresponding to Figure[4](https://arxiv.org/html/2608.04709#S4.F4)\.

Table 6:Complete numerical results for end\-to\-end automatic evaluation on theAvaMERGtest set\. Dist\-2: response diversity; Emo\.Acc: emotion understanding accuracy; Cause\.M: emotion\-cause modeling\.