Causal Analysis and Mitigation of Spurious Onsets in Full-Duplex Speech LLMs

# Causal Analysis and Mitigation ofSpurious Onsets in Full-Duplex Speech LLMs
Source: [https://arxiv.org/html/2609.13445](https://arxiv.org/html/2609.13445)
###### Abstract

Speech\-to\-speech LLMs like Moshi, and its derivative PersonaPlex, can listen and speak concurrently through full\-duplex generation\. However, they can begin speaking inappropriately during prolonged user silence: under digital\-zero input, Moshi and PersonaPlex initiate speech in 12/40 and 11/40 five\-minute continuations, respectively\. What causes this spurious speech? We investigate two hypotheses: either repeated sampling selects speech despite persistently low onset probabilities, or conditioning on the model’s nonspeech outputs causes an abrupt spike in onset probability\. We find that, at every observed onset, speech probability spikes by over nine orders of magnitude in one 80\-ms frame, supporting the latter hypothesis\. Then, to suppress these onsets without blocking genuine responses, we ask a causal counterfactual question: is the model responding to user speech, or would its next\-token distribution remain similar if the preceding user input were muted? Accordingly, we suppress onsets whose distributions change little under this intervention\. Across 40 held\-out trials per model with realistic microphone noise, our method suppresses 13/13 Moshi and 9/9 PersonaPlex spurious onsets, while preserving 40/40 genuine responses per model\. Our inference\-time method requires no retraining and runs in real\-time, with 95th\-percentile decision time below 61 ms, within the 80\-ms frame budget\. Our code is available at[https://github\.com/KentoNishi/icassp27\-spurious\-onsets](https://github.com/KentoNishi/icassp27-spurious-onsets)\.

###### Index Terms:

full\-duplex, speech AI, multimodal AI, large language models, causal analysis

††address:⋆Massachusetts Institute of Technology†Comcast Applied AI Research, Speech AI Team## 1Introduction

Most spoken dialogue systems have explicit turn boundaries\. That is to say, a voice activity detector \(VAD\) closes the user’s “turn,” an automatic speech recognition service \(ASR\) produces a transcript, a large language model \(LLM\) generates a response, and a text\-to\-speech synthesizer \(TTS\) renders the response as audio\. Streaming components can overlap some of these computations, but the cascaded architecture still converts spoken interaction into alternating segments of listening and speaking\. As a result, contemporary systems cannot respond while the user is speaking, handle interruptions, or understand and produce overlapping speech\. Passing through text also discards non\-linguistic information that can affect meaning, including prosody, emotion, laughter, and other non\-speech vocalizations\. Moreover, the sequence of independent components introduces additional latency between the user and the model\[[5](https://arxiv.org/html/2609.13445#bib.bib5),[22](https://arxiv.org/html/2609.13445#bib.bib4),[10](https://arxiv.org/html/2609.13445#bib.bib7)\]\.

usermodelobservedcounterfactual𝐩\\mathbf\{p\}𝐪\\mathbf\{q\}D⁡\(𝐩,𝐪\)D\(\\mathbf\{p\},\\mathbf\{q\}\)1 frame = 80 msemitsuppress\>τ\>\\\!\\tau≤τ\\leq\\\!\\tau

Figure 1:To suppress spurious speech in full\-duplex LLMs, we compare each proposed onset against a matched counterfactual\.The observed continuation receives the newly arrived user audio, while the counterfactual continuation replaces the corresponding frames with silence; both preserve the preceding conversation and generated model history\. We emit the proposed lexical onset when the resulting divergence exceeds a selected threshold and suppress it otherwise\. The comparison occurs within one 80\-ms frame\.Full\-duplex language modeling aims to address these challenges by absorbing speech recognition, language generation, and speech synthesis into a single learned model that processes incoming audio while generating its own output\. For example, SyncLLM represents the user and model as synchronized language chunks\[[22](https://arxiv.org/html/2609.13445#bib.bib4)\], while Moshi and PersonaPlex jointly advance user audio, model text, and model audio streams under a shared autoregressive clock\[[5](https://arxiv.org/html/2609.13445#bib.bib5),[17](https://arxiv.org/html/2609.13445#bib.bib6)\]\. As a result, these models can respond before the user finishes speaking, react to interruptions, coordinate overlapping speech, and preserve vocal information that would be discarded by transcription\.

While elegant, delegating responsibility for speech onset to the generative model introduces a new failure mode\. Specifically, we find that after either Moshi or PersonaPlex finishes answering a user and the user remains completely silent, the model keeps advancing its synchronized streams; after several seconds or minutes of nonlexical output, it often breaks the silence by hallucinating a new utterance despite receiving no new user speech\. Controlling these onsets is difficult because they arise from the very continuous generative process that enables timely responses, interruptions, and overlapping conversation\. Of course, turn\-based gating could suppress model speech after a sufficiently long silence; however, the duration of preceding silence cannot determine whether the next onset arises autonomously or resumes a response to the user\. For example, a full\-duplex model may acknowledge a difficult question before pausing momentarily and continuing with its answer, thereby affecting latency measures such as “retrieval delay” and “end\-to\-end keyword delay”\[[4](https://arxiv.org/html/2609.13445#bib.bib10)\]\. Imagine, though, that this pause outlasts the fixed activity window\. In such a scenario, the gate would suppress the continued answer, even though the answer remains a response to the user’s request\. Gates such as these return responsibility for speech timing to an external turn detector, defeating the purpose of full\-duplex modeling\.

What causes these aforementioned full\-duplex models to escape silence when they should stay silent? Even when the user supplies no new speech, Moshi and PersonaPlex continue sampling model outputs, appending them to the generated history and conditioning each subsequent prediction on that history\. Continuing to build history during silence may produce spurious onsets in two ways: small probabilities of lexical onset may accumulate across repeated samples until the model begins speaking, or alternatively, self\-conditioning on the generated nonlexical outputs may change the model’s activations, causing an abrupt spike in onset probability\. These competing hypotheses imply different remedies: if onset comes from repeatedly sampling a small lexical tail, a decoding rule such as top\-kkcould remove that tail before it is ever sampled; meanwhile, if the model instead assigns nearly all probability to lexical onset at certain frames, restricting low\-probability tokens cannot prevent it, and a more elaborate method is needed to distinguish desirable and undesirable onsets\. We therefore investigate whether onset probability remains small throughout silence, or spikes immediately before speech\.

This work\.In this paper, we first survey recent developments in speech\-to\-speech language modeling \([Sec\.2](https://arxiv.org/html/2609.13445#S2)\)\. Then, we empirically document spurious speech onset across two publicly available frontier full\-duplex models, Moshi and PersonaPlex, and distinguish whether continued history construction produces these onsets through accumulated sampling from small probabilities or spikes caused by self\-conditioning \([Sec\.3](https://arxiv.org/html/2609.13445#S3)\)\. Next, having explained how these models escape silence, we develop a method for deciding which lexical onsets should be emitted or suppressed \([Sec\.4](https://arxiv.org/html/2609.13445#S4)\)\. Finally, we evaluate this intervention on held\-out rollouts containing a genuine response followed by user silence and find that it successfully suppresses spurious onsets while allowing desirable ones \([Sec\.5](https://arxiv.org/html/2609.13445#S5)\)\. Not only do we arrive at a scientific understanding of the failure, but we also derive an actionable intervention from it\.

## 2Related Work

Streaming cascades\.Before end\-to\-end full\-duplex models, incremental spoken dialogue systems sought to reduce latency and relax rigid turn boundaries by closely chaining recognition, dialogue, and synthesis components\. For example, NUMBERS propagated partial updates across every component, allowing the system to provide feedback while the user spoke and react to feedback while producing its own speech\[[20](https://arxiv.org/html/2609.13445#bib.bib2)\]\. More recently, NeuralFSM and DuplexCascade use language models to emit control tokens that determine whether a modular system should wait, respond, or interrupt\[[23](https://arxiv.org/html/2609.13445#bib.bib3),[24](https://arxiv.org/html/2609.13445#bib.bib1)\]\. Systems in this category are commonly used in industry settings, including to automate customer service calls at DoorDash and GE Appliances and to process voice commands at Comcast\[[2](https://arxiv.org/html/2609.13445#bib.bib20),[1](https://arxiv.org/html/2609.13445#bib.bib21),[21](https://arxiv.org/html/2609.13445#bib.bib22)\]\. Though useful, these pipelines are inherently limited by the fact that recognition, language generation, and synthesis are separate modules\.

Full\-duplex spoken language modeling\.To place listening and speaking within one generative process, recent spoken language models jointly represent user and model streams\. For instance, SyncLLM represents dialogue as synchronized user and model language chunks and predicts model chunks while new user chunks arrive\[[22](https://arxiv.org/html/2609.13445#bib.bib4)\]\. Whereas SyncLLM operates over language chunks, Moshi jointly advances user\-audio, model\-text, and model\-audio streams under a shared autoregressive clock\[[5](https://arxiv.org/html/2609.13445#bib.bib5)\]\. Furthermore, directly extending and fine\-tuning Moshi, PersonaPlex simultaneously improves pause\-handling behaviors and adds the ability to specify conversational personas\[[17](https://arxiv.org/html/2609.13445#bib.bib6)\]\. There also exist closed systems such as GPT\-Live\[[14](https://arxiv.org/html/2609.13445#bib.bib23)\]and Gemini Live\[[6](https://arxiv.org/html/2609.13445#bib.bib24)\], but they do not expose weights for research purposes\. Alongside these developments, researchers have proposed benchmarks for interruption handling\[[10](https://arxiv.org/html/2609.13445#bib.bib7),[13](https://arxiv.org/html/2609.13445#bib.bib9)\], overlapping speech\[[22](https://arxiv.org/html/2609.13445#bib.bib4),[9](https://arxiv.org/html/2609.13445#bib.bib8)\], backchanneling\[[10](https://arxiv.org/html/2609.13445#bib.bib7),[13](https://arxiv.org/html/2609.13445#bib.bib9)\], and changes in speaker activity\[[10](https://arxiv.org/html/2609.13445#bib.bib7)\]\.

Free\-running autoregressive generation\.At inference time, an autoregressive model conditions each new prediction on its own previously sampled outputs\. Accordingly, researchers studying exposure bias characterize the mismatch between teacher\-forced training prefixes and free\-running sampled prefixes\[[18](https://arxiv.org/html/2609.13445#bib.bib11),[3](https://arxiv.org/html/2609.13445#bib.bib12),[7](https://arxiv.org/html/2609.13445#bib.bib13),[16](https://arxiv.org/html/2609.13445#bib.bib14)\]\. The failure studied here is related in that self\-conditioning can drive generated speech away from its ordinary response trajectory over long rollouts\.

Causal analysis through controlled interventions\.Those seeking causal explanations of complex model behaviors distinguish between competing accounts by holding most of a computation fixed while intervening on one candidate source of evidence\[[11](https://arxiv.org/html/2609.13445#bib.bib15),[8](https://arxiv.org/html/2609.13445#bib.bib16),[12](https://arxiv.org/html/2609.13445#bib.bib17)\]\. For instance, matched counterfactuals can test whether a model output depends on a particular feature or input\. In full\-duplex speech, the relevant source of evidence is user audio\. We therefore apply this approach by comparing the model’s next\-token distribution under observed user audio and under counterfactual silent audio\.

## 3Inspecting Trained Moshi and PersonaPlex

![Refer to caption](https://arxiv.org/html/2609.13445v1/mechanism-readable.png)Figure 2:Spurious onsets follow abrupt spikes in onset probability during self\-conditioned generation\.In each panel, the thick step curve shows cumulative onset incidence \(left axis\), the dotted curve shows the corresponding constant\-probability prediction, and the thin curves show frame\-level onset probabilities from all 40 continuations \(logarithmic right axis\)\. Onset probability remains near zero before spiking to nearly one at every observed onset; these spikes account for the steps in cumulative incidence\.Experimental setup\.Throughout, we use the official PyTorch BF16 7B checkpoints for Moshi and PersonaPlex, the models’ native 12\.5\-Hz frame rate, text sampling temperature0\.70\.7with top\-k=25k=25, and audio sampling temperature0\.80\.8with top\-k=250k=250\. Text token IDs below 4 are padding or control outputs\. As for how we classify a frame as a new onset, we first record 40 calibration responses per checkpoint to the fixed spoken request “ Tell me one funny joke right now in one sentence” and identify the final lexical frame of each response\. The longest nonlexical gaps within the Moshi and PersonaPlex responses span 231 and 13 frames, respectively\. Then, we define an onset to be a lexical token following one additional nonlexical frame\. In these experiments, we reset all model and random\-number\-generator states; generate a deterministic greeting under seed 0; play the fixed spoken request; and allow the model to complete its response\. To test whether a new user event is necessary to trigger spurious speech or whether it can arise completely autonomously as we suspect, we reseed the decoder and supply 3,750 digital\-zero PCM frames, corresponding to five minutes\. During this interval, the input audio is completely devoid of any audio events\. We measure time from when the model completes its first response, as determined by the calibrated nonlexical gap\.

Onsets under silence\.In[Fig\.2](https://arxiv.org/html/2609.13445#S3.F2), we plot the empirical cumulative incidence of lexical onsets across 40 five\-minute continuations under uninterrupted digital\-zero input\. Moshi produces a lexical onset in 12 of 40 continuations, yielding a five\-minute cumulative incidence of 30\.0% and a two\-sided exact95%95\\%confidence interval of 16\.6–46\.5%\. PersonaPlex produces an onset in 11 continuations, with a lower cumulative incidence of 27\.5%\. Its median onset occurs after 169\.76 seconds rather than Moshi’s 161\.96 seconds, showing that PersonaPlex exhibits less frequent and later spurious speech but still fails to avoid spurious speech completely\. Considering that every frame of user audio contains digital zero in this controlled setting, no new acoustic or conversational event can be held responsible for these onsets\. We therefore reject the account under which either model begins a new utterance only in response to a clear triggering event\.

Distribution of onset times\.For these models, onset times span 26\.56 to 268\.00 seconds\. If either model consistently assigned a small probability of lexical onset at each frame, repeated sampling would begin accumulating incidence immediately, and the waiting time to first onset would follow a geometric distribution or an exponential distribution under a continuous\-time approximation\. However, the observed curves underrepresent onsets at earlier times before rising later, departing from the corresponding constant\-hazard distributions in[Fig\.2](https://arxiv.org/html/2609.13445#S3.F2)\. We therefore hypothesize that onset probability varies both as individual histories develop and across histories\.

Extracting history\-dependent onset probabilities\.Can we directly determine whether onset probability remains small throughout silence or spikes immediately before speech? Indeed, we can: at every framettof continuationii, each model produces the temperature\-scaled, top\-kknext\-token distribution used for sampling\. Lettingℒ\\mathcal\{L\}denote the set of lexical token IDs, the probability of lexical onset is

hi,t=∑v∈ℒpi,t​\(v\),h\_\{i,t\}=\\sum\_\{v\\in\\mathcal\{L\}\}p\_\{i,t\}\(v\),\(1\)wherepi,t​\(v\)p\_\{i,t\}\(v\)is the probability assigned to tokenvvat framett\. For continuationii, the probability of sampling at least one lexical token before frameTT, conditional on its evolving nonlexical history, is

Ci​\(T\)=1−∏t=1T−1\(1−hi,t\),C\_\{i\}\(T\)=1\-\\prod\_\{t=1\}^\{T\-1\}\(1\-h\_\{i,t\}\),\(2\)before frameTT\. If small probabilities accumulate, thenCi​\(Tonset\)C\_\{i\}\(T\_\{\\mathrm\{onset\}\}\)should become appreciable before onset\. If self\-conditioning instead produces a spike, then bothhi,th\_\{i,t\}andCi​\(t\)C\_\{i\}\(t\)should remain near zero until the onset frame\. Consistent with the latter account, every observed onset follows an abrupt spike\. Across all 23 onsets,hi,th\_\{i,t\}is at most3\.46×10−103\.46\\times 10^\{\-10\}on the immediately preceding frame and at least0\.9600\.960on the onset frame\. Correspondingly,Ci​\(Tonset\)C\_\{i\}\(T\_\{\\mathrm\{onset\}\}\)never exceeds3\.40×10−73\.40\\times 10^\{\-7\}\. Thus, neither model escapes silence by eventually sampling a lexical token assigned a persistently small probability\. Instead, self\-conditioning causes onset probability to spike by more than nine orders of magnitude within one frame, after which lexical speech is nearly certain\. Because the proposed onset already dominates the sampled distribution, removing low\-probability tokens through top\-kkcannot suppress the failure\. A more elaborate intervention is needed\.

## 4Mitigating Spurious Speech

So far, we have shown that self\-conditioning during silence can produce abrupt spikes in onset probability\. Can we simply suppress all spikes? The answer is a definite no, since genuine responses should also produce spikes in onset probability when the model decides to speak at an appropriate time\. Spikes indicatewhenthe model intends to speak, but they cannot tell uswhy\. We must instead distinguish whether each proposed onset depends on the user’s incoming request\.

Distinguishing between user\-dependent and spurious onsets\.For each lexical onset, there is a simple question that one can ask, the answer to which we use to distinguish responses that depend on the user’s inputted speech from spurious onsets\. That is this:Does the model’s spoken response depend on the user’s most recent speech input, or would the model predict a similar next\-token distribution even if that audio were muted?If the response causally depends on this user input, we classify it as genuine; otherwise, as spurious\. We can implement this idea by forking histories from the frame immediately preceding that user utterance, saving the model’s streaming state, and constructing two continuations, as illustrated in[Fig\.1](https://arxiv.org/html/2609.13445#S1.F1)\. The observed continuation receives the user audio, while the counterfactual continuation receives the corresponding frames without user speech—we can then compare between the model’s predictions given each of the forked continuations\. As an aside, the position of the forking frame is known from the request waveform in our controlled experiments; online, a streaming voice activity detector \(VAD\) may identify the frames of user speech, and the decoder may branch from the preceding state\. We emphasize that the VAD here may select the counterfactual forking point, but critically, unlike in cascaded systems with explicit turns, the full\-duplex model still itself decides when it wants to speak\.

Formalization\.Letptp\_\{t\}denote the model’s next\-token distribution for a lexical proposal at framettunder the observed user audio, and letqtq\_\{t\}denote the corresponding distribution when that audio is muted\. We condition both predictions on the generated model\-text history through framet−1t\-1, such that they differ only in the supplied user audio\. Concretely, the observed continuation samples model\-text tokens, the counterfactual continuation is teacher\-forced with those tokens, and each continuation generates its own model\-audio tokens under its assigned user audio\. To compare these distributions symmetrically, we define their midpointmt=\(pt\+qt\)/2m\_\{t\}=\(p\_\{t\}\+q\_\{t\}\)/2and measure their Jensen–Shannon divergence,

Dt=12KL\(pt∥mt\)\+12KL\(qt∥mt\)\.D\_\{t\}=\\frac\{1\}\{2\}\\operatorname\{KL\}\(p\_\{t\}\\\|m\_\{t\}\)\+\\frac\{1\}\{2\}\\operatorname\{KL\}\(q\_\{t\}\\\|m\_\{t\}\)\.\(3\)Once a thresholdτ\\tauis selected, we suppress lexical proposals withDt≤τD\_\{t\}\\leq\\tauand preserve those withDt\>τD\_\{t\}\>\\tau\. We study how this choice trades off suppression of spurious onsets against preservation of genuine responses in[Sec\.5](https://arxiv.org/html/2609.13445#S5)\.

## 5Experiments and Results

Threshold selection\.Using seeds 0–39, we sweep decision thresholds by comparing the first genuine response onset near the start of each rollout with the first spurious onset that occurs within a five\-minute window of user silence following the end of the genuine response\. We select three thresholds: the smallest that preserves every genuine response without failing to suppress an additional spurious onset, the largest that suppresses every spurious onset without suppressing an additional genuine response, and the midpoint of the equivalence interval that maximizes the mean response\-preservation and onset\-suppression rate\. We evaluate these “preserve,” “suppress,” and “balanced” threshold points on held\-out seeds 10000–10039\.

Evaluation conditions\.To simulate realistic microphone conditions, we use the BRD2601 Background Noise recording, an audio clip of silence captured by the development board’s microphone, as idle input and mix it into spoken requests\[[19](https://arxiv.org/html/2609.13445#bib.bib18)\]\. The noise and speech waveforms have RMS levels of−96\.0\-96\.0and−19\.1\-19\.1dBFS, respectively\.

Onset suppression and preservation results\.In[Fig\.3](https://arxiv.org/html/2609.13445#S5.F3), the two continuations make similar predictions at spurious onsets; for genuine responses, however, muting the user’s speech changes those predictions\. The calibration scores separate remarkably cleanly: every spurious onset receives a smaller score than every genuine response\. The balanced threshold suppresses 13/13 Moshi and 9/9 PersonaPlex spurious onsets while preserving 40/40 genuine responses from each model \([Tab\.1](https://arxiv.org/html/2609.13445#S5.T1)\)\. Across all three threshold points, every observed spurious onset is suppressed; only the suppression\-focused threshold rejects genuine responses: two for Moshi and one for PersonaPlex\.

Figure 3:A threshold on distributional divergence cleanly separates spurious onsets from genuine responses\.Spurious onsets generally produce smaller distributional changes than genuine responses across both Moshi and PersonaPlex, with the calibration distributions being perfectly disjoint\. The vertical lines mark the response\-preserving, balanced, and suppression\-focused threshold points\. The gray basin yields equal performance, but in deployment, one can choose a higher decision threshold to prioritize suppressing more spurious onsets at the potential cost of suppressing more genuine responses, or alternatively, a lower threshold to prioritize emitting genuine responses at the potential cost of emitting more spurious ones as well\.Generalizing beyond Moshi to PersonaPlex\.We are particularly interested in PersonaPlex, as the authors claim it improves upon Moshi’s pause handling via fine\-tuning\[[17](https://arxiv.org/html/2609.13445#bib.bib6)\]\. Our results verify that PersonaPlex is indeed better behaved, but only marginally: it initiates speech in 11 rather than 12 of 40 continuations, and its median onset occurs after 169\.76 rather than 161\.96 seconds\. Despite the extra fine\-tuning PersonaPlex underwent, it still escapes silence frequently and requires inference\-time mitigation in practice\. Earlier, we showed that our intervention substantially suppresses spurious onsets while preserving desirable responses with respect to both Moshi and PersonaPlex; that confirms that our mitigation method is not specific to certain Moshi checkpoints and transfers well\.

Table 1:Our mitigation can suppress spurious onsets while preserving genuine ones\.It is effective for both Moshi and PersonaPlex\.Running in real\-time\.To run in real\-time, our method must complete all computations within one frame\. Indeed, every counterfactual comparison completes within the required 80\-ms frame at the 95th percentile\. We carried out this benchmark by waiting for 100 warm\-up frames, then measuring 1,000 paired forward steps on one NVIDIA RTX PRO 6000 96GB GPU\. The resulting 95th\-percentile decision times are 45\.60 and 60\.61 ms for Moshi and PersonaPlex, respectively, below 80 ms\.

## 6Limitations

Conversational and acoustic conditions\.We tested one spoken request and microphone noise from one device\. Other questions, voices, microphones, and environments may produce different model predictions, so broader evaluations across conversational and recording conditions may be needed to tune the method for deployment\.

Selecting user input to mute\.Suppose a user says “Speak in ten seconds; ignore me until then\.” The model replies “Got it,” and the user keeps talking\. Muting the most recent speech leaves the earlier request intact, so our method could suppress the delayed response\. Accordingly, selecting the forking point may require reasoning across turns, beyond VAD\. We focus on practical, training\-free interventions on existing models; future work could explore a small trained selector\.

Generalizability across full\-duplex architectures\.Although PersonaPlex changes Moshi’s voice and conversational role, it retains Moshi’s organization of user audio, Inner Monologue, and model audio\[[5](https://arxiv.org/html/2609.13445#bib.bib5),[17](https://arxiv.org/html/2609.13445#bib.bib6)\]\. Other full\-duplex systems use substantially different representations and generation procedures\[[22](https://arxiv.org/html/2609.13445#bib.bib4),[24](https://arxiv.org/html/2609.13445#bib.bib1)\], and such differences may change both how spurious speech arises and how it can be suppressed\. As new full\-duplex architectures emerge, we recommend studying each architecture on its own terms and asking whether our explanation of spurious speech and our proposed intervention apply\. At least for the time being, our results establish the spurious speech phenomenon and our mitigation across both Moshi and PersonaPlex\.

Generalizability to asynchronous tool interactions\.Our counterfactual analysis and intervention both assume that new external input arrives only through the user’s microphone\. However, modern agentic systems also receive tool outputs and retrieved context\[[4](https://arxiv.org/html/2609.13445#bib.bib10),[15](https://arxiv.org/html/2609.13445#bib.bib19)\]\. For example, Qwen Audio Agent allows conversation to continue while a background agent executes tasks and returns results to the ongoing context\[[15](https://arxiv.org/html/2609.13445#bib.bib19)\]\. These rapidly evolving architectures are not always publicly available; Qwen Audio Agent relies on the hosted Qwen Audio 3\.0 Realtime model, whose weights have not been released at the time of writing\. We therefore study open speech models without committing to any particular tool interface\. Future work should extend our counterfactual comparisons across tool and model states to distinguish onsets caused by user audio from those caused by newly returned information\. For the underlying speech models, however, our results nevertheless distinguish spurious from desirable speech through causal dependence on external microphone input\.

## 7Conclusion

In summary, we show that under prolonged user silence, Moshi and PersonaPlex initiate spurious speech \([Sec\.3](https://arxiv.org/html/2609.13445#S3)\)\. By supplying digital\-zero input, we remove new user\-supplied events and show that these onsets arise autonomously rather than in response to an acoustic trigger \([Sec\.3](https://arxiv.org/html/2609.13445#S3)\)\. Then, we show that Moshi and PersonaPlex escape silence because self\-conditioning on generated history causes the probability of lexical onset to spike, as evidenced by our finding that this probability increases by more than nine orders of magnitude at every observed onset and that these spikes, occurring at different times across continuations, produce the gradual cumulative incidence in[Fig\.2](https://arxiv.org/html/2609.13445#S3.F2)\. This result rejects accumulated sampling from small probabilities and shows why pruning a low\-probability sampling tail cannot prevent the failure\. However, genuine responses also produce spikes, so a spike identifies when the model intends to speak but cannot determine whether that speech depends on the user\. We therefore compare each proposed onset against the distribution obtained when the most recent user audio is muted \([Sec\.4](https://arxiv.org/html/2609.13445#S4)\)\. Across held\-out trials with Moshi and PersonaPlex, this intervention suppresses spurious onsets while preserving responses to user speech under realistic microphone noise while running in real\-time \([Sec\.5](https://arxiv.org/html/2609.13445#S5)\)\.

Taken together, our work advances both understanding and methodology pertaining to full\-duplex models\. We hope our contributions help make these models more reliable without sacrificing the free\-flowing interactions they enable\. As this modeling paradigm evolves, we invite researchers to trace failures in speech timing to the mechanisms that produce them and design interventions accordingly\. We look forward to systems whose conversational flexibility is matched by principled control over when and why they speak\.

## 8Compliance with Ethical Standards

We evaluate full\-duplex models under controlled audio inputs\. We do not involve human participants, record private conversations, or use personally identifiable data; as such, no ethical approval was required\.

Our goal is to make full\-duplex voice agents speak more reliably, but that does not ensure the generated speech is factual, safe, or fair\. Furthermore, calibration under one set of voices, microphones, and acoustic conditions may result in biased behaviors\. We therefore recommend participatory engagement with real users and environments, maintaining independent safeguards for response content, and minimizing retention of user data\. Finally, since improved speech timing may make automated agents appear more human, developers should clearly disclose that users are interacting with a synthetic agent\.

## 9Funding Acknowledgement

Comcast provided funding and computational resources during an internship under Akshat Pandey, Karun Kumar, and Tony Braskich\.

## References

- \[1\]Amazon Web ServicesAmazon Lex customers\.Note:Customer case studiesAccessed August 13, 2026External Links:[Link](https://aws.amazon.com/lex/customers/)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p1.1.2)\.
- \[2\]Amazon Web ServicesBuilding a generative AI contact center solution for DoorDash using Amazon Bedrock, Amazon Connect Customer, and Anthropic’s Claude\.Note:Customer case studyAccessed August 13, 2026External Links:[Link](https://aws.amazon.com/solutions/case-studies/doordash-bedrock-case-study/)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p1.1.2)\.
- \[3\]S\. Bengio, O\. Vinyals, N\. Jaitly, and N\. Shazeer\(2015\)Scheduled sampling for sequence prediction with recurrent neural networks\.InAdvances in Neural Information Processing Systems,Vol\.28\.External Links:[Link](https://proceedings.neurips.cc/paper/2015/hash/e995f98d56967d946471af29d7bf99f1-Abstract.html)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p3.1)\.
- \[4\]C\. Chien, M\. Orsini, E\. Kharitonov, N\. Zeghidour, K\. Livescu, and A\. Défossez\(2026\)MoshiRAG: asynchronous knowledge retrieval for full\-duplex speech language models\.arXiv preprint arXiv:2604\.12928\.External Links:[Link](https://arxiv.org/abs/2604.12928)Cited by:[§1](https://arxiv.org/html/2609.13445#S1.p3.1),[§6](https://arxiv.org/html/2609.13445#S6.p4.1)\.
- \[5\]A\. Défossez, L\. Mazaré, M\. Orsini, A\. Royer, P\. Pérez, H\. Jégou, E\. Grave, and N\. Zeghidour\(2024\)Moshi: a speech\-text foundation model for real\-time dialogue\.arXiv preprint arXiv:2410\.00037\.External Links:[Link](https://arxiv.org/abs/2410.00037)Cited by:[§1](https://arxiv.org/html/2609.13445#S1.p1.1),[§1](https://arxiv.org/html/2609.13445#S1.p2.1),[§2](https://arxiv.org/html/2609.13445#S2.p2.1),[§6](https://arxiv.org/html/2609.13445#S6.p3.1)\.
- \[6\]Google DeepMindGemini Audio: live dialogue\.Note:Accessed August 13, 2026External Links:[Link](https://deepmind.google/models/gemini-audio/live-dialogue/)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p2.1)\.
- \[7\]A\. M\. Lamb, A\. Goyal, Y\. Zhang, S\. Zhang, A\. C\. Courville, and Y\. Bengio\(2016\)Professor forcing: a new algorithm for training recurrent networks\.InAdvances in Neural Information Processing Systems,Vol\.29\.External Links:[Link](https://proceedings.neurips.cc/paper/2016/hash/16026d60ff9b54410b3435b403afd226-Abstract.html)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p3.1)\.
- \[8\]K\. Li, A\. K\. Hopkins, D\. Bau, F\. Viégas, H\. Pfister, and M\. Wattenberg\(2023\)Emergent world representations: exploring a sequence model trained on a synthetic task\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=DeG07_TcZvT)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p4.1)\.
- \[9\]G\. Lin, S\. S\. Kuan, Q\. Wang, J\. Lian, T\. Li, S\. Watanabe, and H\. Lee\(2026\)Full\-Duplex\-Bench v1\.5: evaluating overlap handling for full\-duplex speech models\.In2026 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 19447–19451\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP55912.2026.11463576)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p2.1)\.
- \[10\]G\. Lin, J\. Lian, T\. Li, Q\. Wang, G\. Anumanchipalli, A\. H\. Liu, and H\. Lee\(2025\)Full\-duplex\-bench: a benchmark to evaluate full\-duplex spoken dialogue models on turn\-taking capabilities\.In2025 IEEE Automatic Speech Recognition and Understanding Workshop \(ASRU\),pp\. 1–8\.External Links:[Document](https://dx.doi.org/10.1109/ASRU65441.2025.11433838)Cited by:[§1](https://arxiv.org/html/2609.13445#S1.p1.1),[§2](https://arxiv.org/html/2609.13445#S2.p2.1)\.
- \[11\]N\. Nanda, L\. Chan, T\. Lieberum, J\. Smith, and J\. Steinhardt\(2023\)Progress measures for grokking via mechanistic interpretability\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=9XFSbDPmdW)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p4.1)\.
- \[12\]K\. Nishi, R\. Ramesh, M\. Okawa, M\. Khona, H\. Tanaka, and E\. S\. Lubana\(2025\)Representation shattering in transformers: a synthetic study with knowledge editing\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 46525–46553\.External Links:[Link](https://proceedings.mlr.press/v267/nishi25a.html)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p4.1)\.
- \[13\]A\. Ohashi, N\. Zeghidour, A\. Défossez, and E\. Kharitonov\(2026\)Multi\-faceted interactivity alignment in full\-duplex speech models\.arXiv preprint arXiv:2606\.11167\.External Links:[Link](https://arxiv.org/abs/2606.11167)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p2.1)\.
- \[14\]OpenAI\(2026\)GPT\-Live system card\.External Links:[Link](https://deploymentsafety.openai.com/gpt-live/gpt-live.pdf)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p2.1)\.
- \[15\]Qwen TeamQwen Audio Agent\.Note:GitHub repositoryAccessed August 13, 2026External Links:[Link](https://github.com/QwenAudio/qwen-audio-agent)Cited by:[§6](https://arxiv.org/html/2609.13445#S6.p4.1)\.
- \[16\]S\. Ross, G\. J\. Gordon, and J\. A\. Bagnell\(2011\)A reduction of imitation learning and structured prediction to no\-regret online learning\.InProceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics,Proceedings of Machine Learning Research, Vol\.15,pp\. 627–635\.External Links:[Link](https://proceedings.mlr.press/v15/ross11a.html)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p3.1)\.
- \[17\]R\. Roy, J\. Raiman, S\. Lee, T\. Ene, R\. Kirby, S\. Kim, J\. Kim, and B\. Catanzaro\(2026\)PersonaPlex: voice and role control for full duplex conversational speech models\.In2026 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 16137–16141\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP55912.2026.11463413)Cited by:[§1](https://arxiv.org/html/2609.13445#S1.p2.1),[§2](https://arxiv.org/html/2609.13445#S2.p2.1),[§5](https://arxiv.org/html/2609.13445#S5.p4.1),[§6](https://arxiv.org/html/2609.13445#S6.p3.1)\.
- \[18\]F\. Schmidt\(2019\)Generalization in generation: a closer look at exposure bias\.InProceedings of the 3rd Workshop on Neural Generation and Translation,pp\. 157–167\.External Links:[Document](https://dx.doi.org/10.18653/v1/D19-5616)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p3.1)\.
- \[19\]Silicon LaboratoriesBRD2601 Background Noise\.Note:MLTK 0\.20\.0 documentationAccessed August 13, 2026External Links:[Link](https://siliconlabs.github.io/mltk/docs/python_api/datasets/audio/background_noise/brd2601.html)Cited by:[§5](https://arxiv.org/html/2609.13445#S5.p2.1)\.
- \[20\]G\. Skantze and D\. Schlangen\(2009\)Incremental dialogue processing in a micro\-domain\.InProceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics,pp\. 745–753\.External Links:[Document](https://dx.doi.org/10.3115/1609067.1609150)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p1.1.2)\.
- \[21\]R\. Tang, Y\. Mao, K\. Kumar, and F\. Ture\(2026\)Systems and methods for managing cascading models\.Note:U\.S\. Patent Application Publication US 2026/0120708 A1External Links:[Link](https://patents.google.com/patent/US20260120708A1/en)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p1.1.2)\.
- \[22\]B\. Veluri, B\. N\. Peloquin, B\. Yu, H\. Gong, and S\. Gollakota\(2024\)Beyond turn\-based interfaces: synchronous LLMs as full\-duplex dialogue agents\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 21390–21402\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.1192)Cited by:[§1](https://arxiv.org/html/2609.13445#S1.p1.1),[§1](https://arxiv.org/html/2609.13445#S1.p2.1),[§2](https://arxiv.org/html/2609.13445#S2.p2.1),[§6](https://arxiv.org/html/2609.13445#S6.p3.1)\.
- \[23\]P\. Wang, S\. Lu, Y\. Tang, S\. Yan, W\. Xia, and Y\. Xiong\(2024\)A full\-duplex speech dialogue scheme based on large language model\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 13372–13403\.External Links:[Document](https://dx.doi.org/10.52202/079017-0427)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p1.1.2)\.
- \[24\]J\. Yang, Y\. Fujita, and Y\. Sudo\(2026\)DuplexCascade: full\-duplex speech\-to\-speech dialogue with VAD\-free cascaded ASR\-LLM\-TTS pipeline and micro\-turn optimization\.arXiv preprint arXiv:2603\.09180\.External Links:[Link](https://arxiv.org/abs/2603.09180)Cited by:[§2](https://arxiv.org/html/2609.13445#S2.p1.1.2),[§6](https://arxiv.org/html/2609.13445#S6.p3.1)\.