少读多写：一种用于流式多模态解码器的闭式带宽调节器

# A Closed-Form Bandwidth Dial for Streaming Multimodal Decoders
Source: [https://arxiv.org/html/2609.20845](https://arxiv.org/html/2609.20845)
## Reading Less While Writing: A Closed\-Form Bandwidth Dial for Streaming Multimodal Decoders

Kashif Javed University of Engineering and Technology \(UET\) Lahore, Pakistan kashif\.javed@uet\.edu\.pk

###### Abstract

When a model translates a source—be it video or audio—into text, it conventionally looks at the whole input before writing a single word\. In offline settings this is merely more than the task requires; in live mode, it is impossible: a caption cannot wait for a football match to end\. The streaming methods used in practice bolt on a scheduler that reveals the input by a fixed rule \(for instance, the wait\-kkfamily\) and always waits for the same number of input tokens before each word, regardless of the length or pace of the input\. We take a simpler route\. Unlike a fixed offset, our model, ZENDAYA \(Zero\-leakage Non\-Decreasing Aligned & Yoked Architecture\), dynamically shapes its reading to each input by leveraging a single knob,γ\\gamma, that slides the visible window smoothly with the sentence and in proportion to the input’s own predicted length\. So, while the schedule remains deterministic, with almost nothing learned from the input except for an auxiliary length estimate, it is not entirely blind to the input the way a constantkkis\. Thus, turnγ\\gammaone way and the model reads everything upfront, as usual\. Then, turn it the other way and it keeps pace with a live feed\. This knob also fixes, in closed form, the average fraction of the source consumed per word written, and it carries a proof: in streaming operation, no output can ever depend on input that has not yet arrived\. The result is counterintuitive: seeing less can produce better text, since a flood of input dilutes attention at the very moment the model decides what to say first\. Trained from scratch on three public benchmarks spanning two modalities \(Charades\-STA and ActivityNet Captions for video and LibriHeavy for audiobook speech\) our compact 29M\-parameter decoder matches or outperforms the fixed schedule while reading less of the source\. Its sharpest gains fall exactly where live systems operate: in our lowest\-latency settings, text generation begins after a single source token, in contrast to a larger fixed wait\. Even there, ZENDAYA keeps pace with the stream while holding quality, delivering streaming METEOR improvements that remain statistically significant on all three datasets\. On ActivityNet Captions, it reaches this quality with a fraction of the parameters that comparable video captioners use, signalling that a principled schedule, not sheer size, is doing the work\.

## 1Introduction

A person interpreting a speech does not wait for the speaker to finish\. They begin almost at once, a few words behind, and let their output flow alongside the incoming words\. Machines that turn one sequence into another, for instance speech into a transcript or video into a caption, usually do the opposite\. A standard decoder consults theentireinput before committing to its first word\. For a short clip stored on disk, this costs nothing\. For a live broadcast it is impossible as the caption cannot wait for the match to end before announcing the goal\.

This gap between how machines translate and how the world arrives has two faces, and both trace back to the same habit of looking at everything at once\.

1\. Too much input can hurt\.It is tempting to assume that more context is always better\. It is not\. In long documents, language models quietly neglect the middle of what they are givenLiu et al\. \([2024](https://arxiv.org/html/2609.20845#bib.bib10)\)\. In video, piling on frames can lower caption quality rather than raise it, since consecutive frames make visual tokens redundant, as recently addressed byLee et al\. \([2025](https://arxiv.org/html/2609.20845#bib.bib9)\)\. A decoder that stares at the whole source is most exposed to this dilution at the very first word, when it has the least of its own output to anchor on\. And once that word is out, a left\-to\-right model cannot take it back\.

2\. Looking at everything forecloses streaming\.If the first word depends on the whole input, the model cannot start until the whole input exists\. The usual fix is to attach a separate controller that doles out the input gradually\. A wait\-kkrule, for example, readskktokens, writes one, reads another, and so on\. These schedulers work, but they are rigid in a telling way\. The amount they wait is a fixed constant, the same for a two\-second clip and a two\-minute one, blind to how long the input is or how quickly its meaning unfolds\. A single setting cannot suit every input, and, as we show, no single setting suits even a single dataset once the balance between input and output shifts\.

Our fix: One knob, tuned to the data\.That balance is measurable by how many input tokens there are per output token\. In some settings words race ahead of the signal, as in live transcription, where speech is written down almost as fast as it is spoken, so the model must keep pace\. In others a long input funds a short summary, as in captioning a lengthy video in one sentence\. Here revealing the input too eagerly is just the old look\-at\-everything habit in disguise\. No fixed amount of waiting can be right for both settings\. ZENDAYA replaces the fixed rule with a single continuous control,γ\\gamma, that decides how much of the source the decoder may look atas a function of how far it has gotten, letting the visible window grow smoothly with the sentence instead of by a rigid step\. All of this comes from one mechanism, with no separate controller to train and nothing specific to audio or video baked in\.

What makes this more than a convenient dial is that the same number carries three meanings at once\. As alatency settingit says how far the model trails the live input\. As aninterpretable budgetit fixes in closed form the average fraction of source consumed per word, so turning it up provably reads less\. And as a guarantee, it ensures that, when streaming, no output can depend on input that has not yet arrived\. This last one is structural, holding for trained and untrained models alike and extending to unbounded, arbitrarily\-arriving streams\. There is also a bonus here\. Because the model cannot peek ahead, the latency measured offline is provably the latency shown live\.

We put the idea to work with a deliberately small model: a 29M\-parameter decoder trained from scratch, with no pretraining and no fine\-tuned encoders\. Across two video datasets it beats its own full\-context counterpart, and on audio it matches it, all while looking at less of the source\. It also meets or exceeds the fixed\-schedule alternative, with the sharpest gains at the lowest latencies where the fixed rule falls apart\. The scale is worth pausing on\. On ActivityNet Captions, the tri\-modal captioner ofIashin and Rahtu \([2020](https://arxiv.org/html/2609.20845#bib.bib6)\)reports BLEU\-4 of 1\.81 and METEOR of 10\.09 on ground\-truth proposals with 179 million parameters\. Our decoder reaches 1\.97 BLEU\-4 and 12\.25 METEOR offline with 29 million, excluding the frozen encoders it reads from — the figures our three\-seed replication supports \(Appendix[E](https://arxiv.org/html/2609.20845#A5)\); the single\-run sweep of Table[2](https://arxiv.org/html/2609.20845#S5.T2)reports 2\.10 and 12\.64 at a smallerγ\\gammathat the seeds do not confirm\. Their evaluation averages over tIoU thresholds and adds audio and ASR text, so we read this not as a state\-of\-the\-art claim but as evidence that a principled schedule, not sheer size, does the work\.

Contributions\.This paper contributes:\(i\)a single\-parameter, progress\-indexed cross\-attention schedule whose endpoints are physically meaningful \(i\.e\., offline, real\-time, and deliberately starved\) and whose mean source exposure obeys the closed formE¯​\(γ\)≈1/\(1\+γ\)\\bar\{E\}\(\\gamma\)\\approx 1/\(1\{\+\}\\gamma\), makingγ\\gammaat once a latency dial and an exposure budget, with a practical rule for setting it from a dataset’s input\-to\-output ratio;\(ii\)a structural dependency theorem showing that, for any schedule fixed in advance and non\-decreasing, the emitted stream depends only on already\-seen input \(trained and untrained weights alike\), with a corollary extending this to unbounded streams under arbitrary asynchronous arrival;\(iii\)ZENDAYA\-inf, a windowed extension for continuous streams whose only change to the decoder’s training objective is a loss mask; and\(iv\)evidence across two modalities and three datasets that a compact from\-scratch decoder, seeing less of the source, matches or beats a fixed\-offset policy across the latency–quality frontier\. We develop the finite\-segment method \(Sec\.[2](https://arxiv.org/html/2609.20845#S2)\) before extending it to unbounded streams \(Sec\.[3](https://arxiv.org/html/2609.20845#S3)\)\.

## 2ZENDAYA\-fix: A Tunable Horizon for a Finite Segment

ZENDAYA\-fix operates on a fully available segment, where source features are extracted from the complete input before decoding begins and the effective output lengthN^\\hat\{N\}is predicted from that same context\. It addresses context dilution but does not by itself claim end\-to\-end streaming\. ZENDAYA\-inf \(Sec\.[3](https://arxiv.org/html/2609.20845#S3)\) removes the assumption by predictingN^k\\hat\{N\}\_\{k\}per window from already\-arrived data\. Every guarantee below carries over verbatim to that per\-window setting\.

### The Core Mechanism

LetX=\(X0,…,XF−1\)X=\(X\_\{0\},\\dots,X\_\{F\-1\}\)beFFsource tokens produced by a frozen modality encoder for the input segment,N^\\hat\{N\}an effective text length predicted per instance, andi∈\{1,…,N^\}i\\in\\\{1,\\dots,\\hat\{N\}\\\}the current generation step\. Forγ≥0\\gamma\\geq 0we define the normalized progress ratio and the source horizon as

ri=iN^,Ωi=⌈F⋅riγ⌉\.r\_\{i\}=\\frac\{i\}\{\\hat\{N\}\},\\qquad\\Omega\_\{i\}=\\left\\lceil F\\cdot r\_\{i\}^\{\\,\\gamma\}\\right\\rceil\.\(1\)At stepii, the cross\-attention mask permits attention strictly to source tokens\{0,…,Ωi−1\}\\\{0,\\dots,\\Omega\_\{i\}\-1\\\}and blocks all indices beyond\. This single structural change, i\.e\., a dynamically growing cross\-attention window governed by one scalar, is the only modification ZENDAYA\-fix makes to a standard causal decoder\. The ceiling guaranteesΩ1≥1\\Omega\_\{1\}\\geq 1, so the first token always sees at least one source token and the schedule is non\-decreasing iniifor allγ≥0\\gamma\\geq 0\.

### The Bandwidth Dial

The exponentγ\\gammahas a direct physical reading:the rate at which source information becomes available relative to the rate at which text is produced\. Rather than a switch between an offline mode and a streaming mode, it traces a continuum of deployment conditions\.

1. 1\.γ→0\\gamma\\to 0\(offline, full bandwidth\)\.For anyri∈\(0,1\]r\_\{i\}\\in\(0,1\],riγ→1r\_\{i\}^\{\\gamma\}\\to 1, soΩi→F\\Omega\_\{i\}\\to Fat every step: the model sees the whole segment from step one\.The standard offline decoder is this family’sγ=0\\gamma=0endpoint, not a separate comparison point\.
2. 2\.γ∈\(0,1\)\\gamma\\in\(0,1\)\(buffered streaming\)\.Sincexγ\>xx^\{\\gamma\}\>xon\(0,1\)\(0,1\), the feed runs ahead of real time, simulating a prefetch buffer\. This is a safe default when text outpaces the source, giving early context for entity identification\.
3. 3\.γ=1\\gamma=1\(real\-time\)\.Source tokens accumulate linearly with the generation step, matching a constant arrival rate:Ωi=⌈F⋅i/N^⌉\\Omega\_\{i\}=\\lceil F\\cdot i/\\hat\{N\}\\rceil\.
4. 4\.γ\>1\\gamma\>1\(starved reveal\)\.Sincexγ<xx^\{\\gamma\}<xon\(0,1\)\(0,1\), the feed starts behind real time and accelerates late\. Classically this models a degraded link, and as Sec\.[5](https://arxiv.org/html/2609.20845#S5)shows it is also the natural range when the source floods a short target\.

Exposure budget\.We define the mean source exposure as the average fraction of the source visible per emitted token,

E¯​\(γ\)=1N^​∑i=1N^ΩiF=1N^​∑i=1N^\(iN^\)γ\+O​\(1F\)\.\\bar\{E\}\(\\gamma\)\\;=\\;\\frac\{1\}\{\\hat\{N\}\}\\sum\_\{i=1\}^\{\\hat\{N\}\}\\frac\{\\Omega\_\{i\}\}\{F\}\\;=\\;\\frac\{1\}\{\\hat\{N\}\}\\sum\_\{i=1\}^\{\\hat\{N\}\}\\left\(\\frac\{i\}\{\\hat\{N\}\}\\right\)^\{\\\!\\gamma\}\+\\,O\\\!\\left\(\\tfrac\{1\}\{F\}\\right\)\.\(2\)The rounding contributes anO​\(1/F\)O\(1/F\)term per step; since the source\-to\-target ratioρ=F/N^\\rho=F/\\hat\{N\}is bounded below in our regimes, this error is absorbed\. The sum is a Riemann sum ofxγx^\{\\gamma\}on\[0,1\]\[0,1\], so

E¯​\(γ\)⟶∫01xγ​𝑑x=11\+γ\.\\bar\{E\}\(\\gamma\)\\;\\longrightarrow\\;\\int\_\{0\}^\{1\}x^\{\\gamma\}\\,dx\\;=\\;\\frac\{1\}\{1\+\\gamma\}\.\(3\)The dial therefore has a second, budget reading:γ=1\\gamma=1halves the average exposure,γ=0\.3\\gamma=0\.3trims it to 77%,γ=2\\gamma=2to 33%, andγ→0\\gamma\\to 0recovers the full\-exposure decoder\.

Choosingγ\\gammafrom the data’s geometry\.The useful range of the dial is set by the source\-to\-target ratioρ=F/N^\\rho=F/\\hat\{N\}, and it does not end at11\. When text outpaces source \(ρ<1\\rho<1, e\.g\., speech transcription\),γ∈\(0,1\)\\gamma\\in\(0,1\)supplies a prefetch buffer at modest latency, and the schedule’s first\-step horizonΩ1=⌈F​\(1/N^\)γ⌉\\Omega\_\{1\}=\\lceil F\(1/\\hat\{N\}\)^\{\\gamma\}\\rceilis naturally small\. When the source floods a short target \(ρ≫1\\rho\\gg 1, e\.g\., captioning long video segments\), even moderateγ\\gammafront\-loads a largeabsolutenumber of source tokens, and the just\-in\-time operating points migrate above11\. A practical rule of thumb, however, makes this quantitative: to target a first\-token budget ofBBsource tokens, set

γ∗≈ln⁡\(F/B\)ln⁡N^\.\\gamma^\{\*\}\\;\\approx\\;\\frac\{\\ln\(F/B\)\}\{\\ln\\hat\{N\}\}\.\(4\)For our audio geometry \(F≈100F\\approx 100,N^≈165\\hat\{N\}\\approx 165\), a lean buffer ofB≈10B\\approx 10source tokens calls forγ∗≈0\.45\\gamma^\{\*\}\\approx 0\.45, while a more generousB≈30B\\approx 30calls forγ∗≈0\.24\\gamma^\{\*\}\\approx 0\.24\. The longer one is willing to buffer, the gentler the exponent\. The experiments of Sec\.[5](https://arxiv.org/html/2609.20845#S5)confirm this prediction, along with a sharp negative one\. Where the source is theslowerstream \(ρ<1\\rho<1\), a schedule that deliberately runs behind it \(γ\>1\\gamma\>1\) starves permanently\. This is the deeper contrast with a fixed offset\. Wait\-kkowns one constant that buys a single point on the latency axis per trained model, with lag identical for every instance, whileγ\\gammarepositions an entire per\-instance, length\-adaptive schedule\.

### Streaming\-Compatible Dependency

We now make the dependency guarantee precise for arbitrary fixed parameters, letting the decoder consist of:

- ∙\\bullettoken and positional embeddings;
- ∙\\bulletLLlayers, each applying causal self\-attention, then cross\-attention in which positioniimay attend only to source indices\{0,…,Ωi−1\}\\\{0,\\dots,\\Omega\_\{i\}\-1\\\}\(masked logits are set to−∞\-\\inftybefore the softmax\), then position\-wise residual, normalization, and feed\-forward maps;
- ∙\\bulleta linear output projection\.

We say a quantitydepends only onX<kX\_\{<k\}if it is unchanged under arbitrary modification ofXXon indices≥k\\geq k\. IfΩi=0\\Omega\_\{i\}=0the attended set is empty, and we adopt the convention that the cross\-attention block outputs zero at positionii\. Every statement below is preserved under this\.

###### Theorem 1\.

For any horizon schedulei↦Ωii\\mapsto\\Omega\_\{i\}that is fixed before decoding begins and non\-decreasing inii, the output logits at stepiiare a function only of the text prefixy0:iy\_\{0:i\}and the source prefixX<ΩiX\_\{<\\Omega\_\{i\}\}\.

###### Proof sketch\.

By induction on layers\. The base case is immediate:hi\(0\)h\_\{i\}^\{\(0\)\}is a function ofy0:iy\_\{0:i\}alone\. At layerℓ\+1\\ell\+1, causal self\-attention reads only\{hi′\(ℓ\):i′≤i\}\\\{h\_\{i^\{\\prime\}\}^\{\(\\ell\)\}:i^\{\\prime\}\\leq i\\\}, and monotonicity givesΩi′≤Ωi\\Omega\_\{i^\{\\prime\}\}\\leq\\Omega\_\{i\}, so every input depends only onX<ΩiX\_\{<\\Omega\_\{i\}\}\. In cross\-attention the masked logits equal−∞\-\\infty, so each indexj≥Ωij\\geq\\Omega\_\{i\}receives softmax weight exactly zero and is absent from the normalizer, leaving∑j<Ωipi​j​Vj\\sum\_\{j<\\Omega\_\{i\}\}p\_\{ij\}V\_\{j\}\. The remaining maps are position\-wise\. Hencelogitsi\\mathrm\{logits\}\_\{i\}depends only ony0:iy\_\{0:i\}andX<ΩiX\_\{<\\Omega\_\{i\}\}\. A further induction over decoding steps extends this to the emitted stream\. The full argument is in Appendix[A](https://arxiv.org/html/2609.20845#A1)\. ∎

In ZENDAYA\-fix the schedule is itself a function ofN^​\(X\)\\hat\{N\}\(X\), so the guarantee is conditional on that schedule\. ZENDAYA\-inf removes the conditioning by computingN^k\\hat\{N\}\_\{k\}from arrived data alone \(Cor\.[1](https://arxiv.org/html/2609.20845#Thmcorollary1)\)\.

### Auxiliary Length Head

The schedule’s scale is set per instance by a predicted effective lengthN^\\hat\{N\}\. A lightweight head \(global average pooling over the source features followed by two dense layers\) is trained jointly with the main decoder, using a cross\-entropy loss over discretized length classes weighted byλlen=0\.1\\lambda\_\{\\mathrm\{len\}\}=0\.1\. During training the schedule itself is built from the ground\-truth target lengthN=\|Y\|N=\|Y\|\(Alg\.[1](https://arxiv.org/html/2609.20845#alg1)\)\. The head is optimized only as an auxiliary task, and its predictionN^\\hat\{N\}is consulted solely at inference, whenNNis unavailable\. PredictingN^\\hat\{N\}from a pooled summary is the one place ZENDAYA\-fix looks at the whole input ahead of decoding, and ZENDAYA\-inf removes even this \(Sec\.[3](https://arxiv.org/html/2609.20845#S3)\)\.

Algorithm 1ZENDAYA\-fix: training and inference1:Train\(X,Y\)\(X,Y\): \{parallel, teacher\-forced\}

2:

N←\|Y\|N\\leftarrow\|Y\|\{target length sets the schedule\}

3:

Ωi←⌈F⋅\(i/N\)γ⌉\\Omega\_\{i\}\\leftarrow\\lceil F\\cdot\(i/N\)^\{\\gamma\}\\rceilfor

i=1​…​Ni=1\\dots N
4:

Mcross​\[i,j\]←𝟏​\[j<Ωi\]M\_\{\\mathrm\{cross\}\}\[i,j\]\\leftarrow\\mathbf\{1\}\[\\,j<\\Omega\_\{i\}\\,\]for all

i,ji,j
5:

logits←Decoder​\(Yin,X;Mcross\)\\mathrm\{logits\}\\leftarrow\\mathrm\{Decoder\}\(Y\_\{\\mathrm\{in\}\},X;\\,M\_\{\\mathrm\{cross\}\}\)
6:

ℒ←ℒtext\+λlen⋅ℒlength\\mathcal\{L\}\\leftarrow\\mathcal\{L\}\_\{\\mathrm\{text\}\}\+\\lambda\_\{\\mathrm\{len\}\}\\cdot\\mathcal\{L\}\_\{\\mathrm\{length\}\}\{

ℒlength\\mathcal\{L\}\_\{\\mathrm\{length\}\}: aux\. length loss\}

7:Infer\(X\)\(X\): \{sequential,

NNpredicted\}

8:

N^←LengthHead​\(GAP​\(X\)\)\\hat\{N\}\\leftarrow\\mathrm\{LengthHead\}\(\\mathrm\{GAP\}\(X\)\)
9:

Kv,Vv←WK​X,WV​XK\_\{v\},V\_\{v\}\\leftarrow W\_\{K\}X,\\,W\_\{V\}X
10:for

i=1​…​N^i=1\\dots\\hat\{N\}do

11:

Ωi←⌈F⋅\(i/N^\)γ⌉\\Omega\_\{i\}\\leftarrow\\lceil F\\cdot\(i/\\hat\{N\}\)^\{\\gamma\}\\rceil
12:

yi←Decode​\(y0:i,Kv,Vv;horizon​\[0,Ωi\)\)y\_\{i\}\\leftarrow\\mathrm\{Decode\}\(y\_\{0:i\},\\,K\_\{v\},V\_\{v\};\\,\\text\{horizon \}\[0,\\Omega\_\{i\}\)\)
13:if

yi=\[EOS\]y\_\{i\}=\\texttt\{\[EOS\]\}then break

14:endfor

15:return

y1​…​yN^y\_\{1\}\\dots y\_\{\\hat\{N\}\}

### Training: 2D Masking

During teacher\-forced training allN^\\hat\{N\}positions are computed in parallel\. We precompute the boolean maskMcross∈𝔹N^×FM\_\{\\mathrm\{cross\}\}\\in\\mathbb\{B\}^\{\\hat\{N\}\\times F\}, whose rowiiis true exactly on\[0,Ωi\)\[0,\\Omega\_\{i\}\), and pass it to every cross\-attention layer \(see Alg\.[1](https://arxiv.org/html/2609.20845#alg1)\)\. A single forward pass thus reproduces the dependency structure of Theorem[1](https://arxiv.org/html/2609.20845#Thmtheorem1)across the whole batch, with no sequential loop\.

## 3ZENDAYA\-inf: Unbounded Streams via Context\-Conditioned Chunking

ZENDAYA\-fix operates on a finite, fully available segment\. Genuinely long or unbounded streams, such as live commentary, continuous interpretation, or rolling transcription, cannot be held in memory\. ZENDAYA\-inf extends the bandwidth dial to a sliding source window of widthWW, carrying into each window the textYhistoryY\_\{\\mathrm\{history\}\}generated for the one before it\.

### Prefix Forcing with Loss Masking

Training presents the model with consecutive target sentences drawn from the same continuous stream\. The earlier sentence acts as historical context: its loss weight is0and its source horizon is fixed atr=1r=1, i\.e\., fully open, since it lies entirely in the past relative to the active sentence\. The progressive horizonΩk,j\\Omega\_\{k,j\}applies only to the active target sentence in windowkk\. This prefix forcing is the only change to the decoder’s training objective relative to ZENDAYA\-fix\. For each pair of consecutive ground\-truth sentences\(Sk−1,Sk\)\(S\_\{k\-1\},S\_\{k\}\), the loss is masked to zero onSk−1S\_\{k\-1\}and applied fully toSkS\_\{k\}\. Following standard practice in streaming captioning,YhistoryY\_\{\\mathrm\{history\}\}carries the most recently completed sentence rather than the entire transcript, so the deployed condition matches the training condition exactly\.

### Per\-Window Length Prediction \(Strictly Causal\)

Theorem[1](https://arxiv.org/html/2609.20845#Thmtheorem1)guarantees thatgenerationat stepjjin windowkkdepends only on the firstΩk,j\\Omega\_\{k,j\}source tokens ofwkw\_\{k\}, butΩk,j\\Omega\_\{k,j\}is derived fromN^k\\hat\{N\}\_\{k\}, which must itself be predicted without future information\. We predictN^k\\hat\{N\}\_\{k\}from three pooled summaries, each available before windowkkis decoded: \(i\) the firstBBsource tokens of windowkk, whereB=⌈W⋅\(1/Nmax\)γ⌉B=\\lceil W\\cdot\(1/N\_\{\\max\}\)^\{\\gamma\}\\rceilandNmaxN\_\{\\max\}is the longest decode permitted within a window \(Sec\.[5](https://arxiv.org/html/2609.20845#S5)\), so the buffer never exceeds what the first token would license; and \(ii\) the features of the preceding windowwk−1w\_\{k\-1\}; \(iii\) the textYhistoryY\_\{\\mathrm\{history\}\}generated for that preceding window\. Every input to this prediction has already arrived by the time it is used, so no component depends on future source data\.

### Dependency Under Asynchronous Arrival

Within windowkk, decoding proceeds over stepsj=1,…,N^kj=1,\\dots,\\hat\{N\}\_\{k\}with prescribed horizonΩk,j=⌈W⋅\(j/N^k\)γ⌉\\Omega\_\{k,j\}=\\lceil W\\cdot\(j/\\hat\{N\}\_\{k\}\)^\{\\gamma\}\\rceil, so the schedule never asks for more thanΩk,j\\Omega\_\{k,j\}tokens at stepjjand decoding may begin once the firstΩk,1\\Omega\_\{k,1\}have arrived\. What it cannot control is whether arrival keeps pace\. LetAk​\(t\)∈\{0,…,W\}A\_\{k\}\(t\)\\in\\\{0,\\dots,W\\\}denote the number of tokens of windowkkencoded and cached by wall\-clock timett, and lettjt\_\{j\}be the wall\-clock time at which decoding reaches stepjj\. We define theeffective horizon

Ωk,jeff:=min⁡\(Ωk,j,Ak​\(tj\)\),\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}:=\\min\\bigl\(\\Omega\_\{k,j\},\\,A\_\{k\}\(t\_\{j\}\)\\bigr\),\(5\)the source prefix actually licensed at stepjjonce both the schedule and physical arrival are accounted for\.

Algorithm 2ZENDAYA\-inf: unbounded streaming inference1:

Yhistory←∅Y\_\{\\mathrm\{history\}\}\\leftarrow\\varnothing;

k←0k\\leftarrow 0
2:whilestream continuesdo

3:buffer first

BBtokens of window

kk
4:

wk←X\[kW:\(k\+1\)W\]w\_\{k\}\\leftarrow X\[kW:\(k\{\+\}1\)W\]
5:

N^k←LengthHead​\(buffer​\(wk\);wk−1;Yhistory\)\\hat\{N\}\_\{k\}\\leftarrow\\mathrm\{LengthHead\}\(\\mathrm\{buffer\}\(w\_\{k\}\);\\,w\_\{k\-1\};\\,Y\_\{\\mathrm\{history\}\}\)
6:

Kv,k,Vv,k←K\_\{v,k\},V\_\{v,k\}\\leftarrowprojections of

wkw\_\{k\}
7:

Sk←∅S\_\{k\}\\leftarrow\\varnothing
8:for

j=1​…​N^kj=1\\dots\\hat\{N\}\_\{k\}do

9:

Ωk,j←⌈W⋅\(j/N^k\)γ⌉\\Omega\_\{k,j\}\\leftarrow\\lceil W\\cdot\(j/\\hat\{N\}\_\{k\}\)^\{\\gamma\}\\rceil
10:decode next token attending to

YhistoryY\_\{\\mathrm\{history\}\}and

wkw\_\{k\}masked to

\[0,Ωk,j\)\[0,\\Omega\_\{k,j\}\)
11:append token to

SkS\_\{k\}
12:iftoken

=\[EOS\]=\\texttt\{\[EOS\]\}then break

13:endfor

14:

Yhistory←SkY\_\{\\mathrm\{history\}\}\\leftarrow S\_\{k\};

k←k\+1k\\leftarrow k\+1
15:endwhile

###### Corollary 1\.

For any non\-decreasing arrival processAkA\_\{k\}, the mapj↦Ωk,jeffj\\mapsto\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}is non\-decreasing, and Theorem[1](https://arxiv.org/html/2609.20845#Thmtheorem1)applies verbatim withΩk,jeff\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}in place ofΩk,j\\Omega\_\{k,j\}: generation at stepjjis a function only of the text so far, the pre\-decoding inputs that set the schedule, and the source tokens ofwkw\_\{k\}with index<Ωk,jeff<\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}\. No emitted token ever depends on a source token that has not arrived by the time it is produced\.

###### Proof\.

The mapj↦Ak​\(tj\)j\\mapsto A\_\{k\}\(t\_\{j\}\)is non\-decreasing as a composition of non\-decreasing maps, and the pointwise minimum of two non\-decreasing maps is non\-decreasing, soj↦Ωk,jeffj\\mapsto\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}is a valid horizon schedule\. Being non\-decreasing is the only property Theorem[1](https://arxiv.org/html/2609.20845#Thmtheorem1)requires\. The schedule is fixed byN^k\\hat\{N\}\_\{k\}, which itself is computed from theBB\-token buffer, the preceding windowwk−1w\_\{k\-1\}, andYhistoryY\_\{\\mathrm\{history\}\}\. All three are available before window\-kkdecoding begins and consist exclusively of arrived data\. Applying Theorem[1](https://arxiv.org/html/2609.20845#Thmtheorem1)to the window\-kkdecode with scheduleΩk,jeff\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}yields the dependency claim, andΩk,jeff≤Ak​\(tj\)\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}\\leq A\_\{k\}\(t\_\{j\}\)ensures that every licensed source token has arrived by timetjt\_\{j\}\. ∎

The guarantee holds however the prescribed schedule relates to true arrival\. When tokens arrive at least as fast as the schedule asks, the effective horizon is the prescribed one and the model restricts itself as designed\. Our experiments occupy this regime, and Algorithm[2](https://arxiv.org/html/2609.20845#alg2)instantiates it\. When arrival falls behind, as on a degraded link, the model is provably limited to what has arrived, with no separate\-case logic\.

## 4Related Work

Our contribution meets three active lines of work\.

Variable attention masking in streaming ASR\.Training an encoder under masks that expose differing amounts of future audio, so a single model covers both streaming and full\-context use, is established practice in speech recognitionSwietojanski et al\. \([2023](https://arxiv.org/html/2609.20845#bib.bib19)\); Moritz et al\. \([2021](https://arxiv.org/html/2609.20845#bib.bib14)\)\. Theprincipleof mask\-induced streaming compatibility is therefore not new, but its target is different as these methods mask a single encoder’s self\-attention over its own modality, indexed byinput\-frame position\. ZENDAYA maskscross\-modalattention from decoder to whichever source encoder is in use, indexed bygeneration progress,i/N^i/\\hat\{N\}\.

Simultaneous generation via alignment policies\.Simultaneous translation schedules a learned or scheduled alignment between source consumption and target emission, such as wait\-kkprefix policiesMa et al\. \([2019](https://arxiv.org/html/2609.20845#bib.bib12)\), multi\-path wait\-kktrainingElbayad et al\. \([2020](https://arxiv.org/html/2609.20845#bib.bib3)\), and monotonic attention that learns when to read versus writeChiu and Raffel \([2018](https://arxiv.org/html/2609.20845#bib.bib2)\); Ma et al\. \([2020](https://arxiv.org/html/2609.20845#bib.bib13)\)\. Further, in cross\-modal settings SimulSLT couples wait\-kkwith a learned boundary predictorYin et al\. \([2021](https://arxiv.org/html/2609.20845#bib.bib21)\), andSun et al\. \([2024](https://arxiv.org/html/2609.20845#bib.bib18)\)drive an adaptive policy from a learned translatable\-length estimate\. ZENDAYA is related in spirit \(theγ\\gamma\-schedule reads as a continuous, length\-normalized generalization of a linear prefix policy, moved into the cross\-attention mask\) but differs in kind\. Its visible prefix is a closed\-form function of\(γ,i,N^\)\(\\gamma,i,\\hat\{N\}\), needing no policy module, no boundary predictor, and no learned alignment, and Theorem[1](https://arxiv.org/html/2609.20845#Thmtheorem1)follows from the mask structure alone\.

Low\-latency captioning and frame selection\.Hori et al\. \([2021](https://arxiv.org/html/2609.20845#bib.bib5)\)train on partial inputs to mimic a full\-context teacher with a learned timing detector, retaining 94% of quality from 28% of frames—the trade\-off they optimize empirically is the one our dial exposes analytically, and their guarantee is behavioral rather than structural\.Zhou et al\. \([2024](https://arxiv.org/html/2609.20845#bib.bib22)\)handle unbounded video via clustering\-based fixed\-size memory, addressingunbounded lengthwhere ZENDAYA addressesper\-step dependency structurewithin a window\. That excess source tokens can hurt quality is documented in captioningLee et al\. \([2025](https://arxiv.org/html/2609.20845#bib.bib9)\), countered by input\-adaptive selection applied once the whole input is in hand\. ZENDAYA instead reduces exposuredynamically across generation, never requires the full input up front, and yields Theorem[1](https://arxiv.org/html/2609.20845#Thmtheorem1)as a by\-product rather than a separate guarantee to establish\.

## 5Experiments

![Refer to caption](https://arxiv.org/html/2609.20845v1/x1.png)Figure 1:\(a\) Offline BLEU\-4 versusγ\\gamma; blue dashed is the full\-exposure baseline\. \(b, c\) Streaming BLEU\-4 and METEOR versus AL; ZENDAYA in red \(labeled byγ\\gamma, best enlarged\), wait\-kkin gray\.Our experiments answer three questions\.\(Q1\)Offline: does restricting exposure cost or gain quality against the full\-exposure member of the same family, at matched capacity?\(Q2\)Streaming: how does ZENDAYA’s latency–quality frontier compare with wait\-kk, the canonical fixed\-offset policy, in both source–target regimes?\(Q3\)Do the analytic dials—the exposure budget of Eq\. \([3](https://arxiv.org/html/2609.20845#S2.E3)\) and theγ∗\\gamma^\{\*\}rule of Eq\. \([4](https://arxiv.org/html/2609.20845#S2.E4)\)—predict where each method is strong?

### Setup

Table 1:Datasets and configuration\.FFandNNare per\-corpus truncation caps andρ\\rhois their ratio; the schedule uses each instance’s own lengths\. One annotated segment per window, soW=FW=FandNmax=NN\_\{\\max\}=N\.Models\.To attribute differences to the cross\-attention schedule rather than to representation quality or pretraining, every decoder is trainedfrom scratch:∼\\sim29M parameters, 4 layers,d=768d=768, 8 heads, feed\-forward 1536, dropout 0\.1, a per\-dataset 8k BPE vocabulary, and AdamWLoshchilov and Hutter \([2019](https://arxiv.org/html/2609.20845#bib.bib11)\)with cosine decay and warmup to2×10−42\\times 10^\{\-4\}for at most 40 epochs111Training uniformly converges in fewer than 20 epochs, batch sizes kept strictly uniform across both ZENDAYA and the baseline\.\. All models are implemented in TensorFlow and trained on a single T4 GPU on either Kaggle or Google Colab\. Source features come from the frozen public encoders of Table[1](https://arxiv.org/html/2609.20845#S5.T1)and are never updated\. All models decode greedily to at mostNmaxN\_\{\\max\}steps, stopping at \[EOS\]\. Training is unseeded by default, so exact replication of any individual run is not expected; every headline margin instead carries a paired\-bootstrap interval\. The fixed\-window sweep is additionally repeated under three fixed seeds in Appendix[E](https://arxiv.org/html/2609.20845#A5), while the streaming results reported below are single runs\.

Datasets\.The three corpora span the source\-to\-target ratioρ\\rhoon both sides of unity\. Charades\-STA’s action segments overlap heavily in time, so we exclude overlapping consecutive segments when forming streaming\(Sk−1,Sk\)\(S\_\{k\-1\},S\_\{k\}\)pairs, keeping the causal assumption of Sec\.[3](https://arxiv.org/html/2609.20845#S3)intact by construction\. ActivityNet Captions supplies the most source\-dense geometry we test as its long and heterogeneous segments each carry a single human\-written caption\. We use the publicly released C3D features so that the visual front end is fixed and independently reproducible\. LibriHeavy is large enough that both families would eventually excel, so we train on a 6k\-utterance subset with a book\-disjoint test split, placing every model in a data\-austere regime where the schedule, not scale, must do the work\. Table[1](https://arxiv.org/html/2609.20845#S5.T1)gives each corpus’s encoder, geometry, and regime\.

Baselines\.Every comparison stays inside one family\. The offline baseline is theγ→0\\gamma\\to 0endpoint of the identical architecture\. The wait\-kkbaselines are the identical decoder trained under a wait\-kkcross\-attention mask revealingg​\(i\)=k\+⌈s​\(i−1\)⌉g\(i\)=k\+\\lceil s\\,\(i\-1\)\\rceilsource tokens at stepii, with strides=ρs=\\rhoso that a full decode consumes the whole source\. This length\-ratio catch\-up is STACL’s strong variantMa et al\. \([2019](https://arxiv.org/html/2609.20845#bib.bib12)\), making the baseline the competitive form of wait\-kkrather than a strawman\. It carries no length head, since a fixed offset needs no length estimate, whereas ZENDAYA must predict the target length, from only the first tokens of a window when streaming\. The auxiliary loss might also regularize the shared trunk\. We disclose this asymmetry rather than adjudicate its net sign\.

Protocol and metrics\.Offline results \(Table[2](https://arxiv.org/html/2609.20845#S5.T2), Fig\.[1](https://arxiv.org/html/2609.20845#S5.F1)a\) are produced by ZENDAYA\-fix\. Streaming results \(Table[3](https://arxiv.org/html/2609.20845#S5.T3), Fig\.[1](https://arxiv.org/html/2609.20845#S5.F1)b, c\) are generated by ZENDAYA\-inf with per\-windowN^k\\hat\{N\}\_\{k\}predicted causally from aBB\-token buffer \(Sec\.[3](https://arxiv.org/html/2609.20845#S3)\)\. We report corpus BLEU\-4Papineni et al\. \([2002](https://arxiv.org/html/2609.20845#bib.bib15)\)\(smoothed\) and METEOR222We report both metrics because they fail differently: short single\-reference captions make 4\-gram overlap brittle on video, while BLEU\-4 suits the long fluent audio transcripts\.Banerjee and Lavie \([2005](https://arxiv.org/html/2609.20845#bib.bib1)\)throughout, and in the streaming case Average LaggingMa et al\. \([2019](https://arxiv.org/html/2609.20845#bib.bib12)\)on the decode\-step clock withg​\(i\)=Ωig\(i\)=\\Omega\_\{i\}\. Because the streaming schedule is causal givenN^k\\hat\{N\}\_\{k\}, this AL is the one the system exhibits live \(Cor\.[1](https://arxiv.org/html/2609.20845#Thmcorollary1)\)\. For wait\-kk, whose lag is content\-blind, AL grows monotonically withkk, though per\-instanceρ\\rhovariation under a dataset\-constant stride shifts and spreads it belowkk\. Since no analytical map couples a linearkkto a power\-lawγ\\gamma, we sweep both and compare frontiers directly:γ∈\{0\.1,0\.2,0\.3,0\.4,0\.7,1\.0\}\\gamma\\in\\\{0\.1,0\.2,0\.3,0\.4,0\.7,1\.0\\\}offline, andγ∈\{0\.2,0\.3,0\.4,1\.0,1\.5\}\\gamma\\in\\\{0\.2,0\.3,0\.4,1\.0,1\.5\\\}againstk∈\{5,10,15,20,25,30,40\}k\\in\\\{5,10,15,20,25,30,40\\\}when streaming\. Anchor pairs are matched on lag where available, and best\-versus\-best otherwise\. The three headline pairs carry a paired\-bootstrap 95% interval over 1000 resamples\. The offline sweep is reported as a trend, its shape visible in Fig\.[1](https://arxiv.org/html/2609.20845#S5.F1)a\.

### Offline Setting: Dilution \(Q1\)

Table[2](https://arxiv.org/html/2609.20845#S5.T2)and Figure[1](https://arxiv.org/html/2609.20845#S5.F1)a show the offline response\. The best configuration is not the full\-exposure endpoint onanyof the three datasets, and the margin grows withρ\\rho: \+19\.6% and \+46\.9% on the two source\-dense video datasets, where a flood of source tokens gives dilution the most room to operate\. Shielding the earliest, least\-anchored generation steps from the full source does not merely preserve quality but improves it\. Both BLEU\-4 margins clear a paired bootstrap \(p=0\.026p\{=\}0\.026andp<0\.001p\{<\}0\.001\), and METEOR moves with them, significantly on ActivityNet\. On LibriHeavy, where the audio is already text\-dense, neither metric shifts significantly, exactly as that geometry predicts, and quality stays within roughly 4% across the bandγ∈\[0\.1,0\.7\]\\gamma\\in\[0\.1,0\.7\]before collapsing atγ=1\\gamma=1\. On the source\-dense video datasets, every setting across the swept band exceeds the baseline\. That breadth makes the dial practical, sinceγ\\gammacan be raised for latency well before quality degrades\. These offline numbers come from single unseeded runs\. A three\-seed replication of the entire fixed\-window sweep \(Appendix[E](https://arxiv.org/html/2609.20845#A5), 63 training runs\) supports the geometric claim on all three corpora and sharpens its form, but it does not support theγ\\gammaselected here for ActivityNet, and under seeded means the two margins quoted above become\+34%\+34\\%and\+20%\+20\\%, reversing their order\. Appendix[E](https://arxiv.org/html/2609.20845#A5)states precisely which entries are affected and what replaces them\.

Table 2:Fixed\-window results\. Best ZENDAYA is selected by BLEU\-4; Rel\. is that margin\. Bootstrap intervals for every cell are in Appendix[C](https://arxiv.org/html/2609.20845#A3)\.Table 3:Streaming anchors\. z\-γ\\gammais ZENDAYA at thatγ\\gamma, w\-kkis wait\-kk\. Best in row is bold\.Table 4:Paired\-bootstrap 95% intervals \(1000 resamples\), ZENDAYA \(z\-γ\\gamma\) vs\. wait\-kk\(w\-kk\)\. Deltas are computed on unrounded scores\. Bold excludes zero\.
### Streaming: One Dial, Two Regimes \(Q2\)

We now turn to the streaming setting\. Figure[1](https://arxiv.org/html/2609.20845#S5.F1)b, c plots the latency–quality frontiers, Table[3](https://arxiv.org/html/2609.20845#S5.T3)the anchor points that carry the argument, and Table[4](https://arxiv.org/html/2609.20845#S5.T4)their bootstrap intervals\.

Charades\-STA\.Atγ=0\.2\\gamma=0\.2, ZENDAYA exceeds the strongest offset found anywhere on the wait\-kkcurve on both quality metrics while arriving5\.7 source tokens earlier\. The METEOR margin is significant, the BLEU margin positive but within noise\. Against wait\-20, its nearest neighbour in lag, the same setting yields\+36\.8% BLEUand\+14\.6% METEOR\. Neither knob traces a tidy single\-peaked curve, the two differing in kind rather than in tuning \(Sec\.[2](https://arxiv.org/html/2609.20845#S2)\)\. Paired decodes appear in Appendix[D](https://arxiv.org/html/2609.20845#A4)\.

ActivityNet Captions\.This geometry makes a sharp prediction via Eq\. \([4](https://arxiv.org/html/2609.20845#S2.E4)\): with a lean first\-token budget the just\-in\-time operating points should migrate above11\. At comparable negative lag, where a fixed offset must starve itself,γ=1\.5\\gamma=1\.5beats wait\-5 by\+28% BLEUand\+12\.6% METEOR\. In best\-versus\-best,γ=0\.4\\gamma=0\.4exceeds wait\-10 on both metrics with METEOR advantage significant\. Extending the offset to wait\-40 does not recover the gap: it returns 0\.0269 BLEU\-4 and 0\.1446 METEOR at an average lagging of 24\.2, inside the existing frontier and below wait\-10 on both metrics\.

LibriHeavy\.In text\-dense streams the transcript outruns the signal, exactly where content\-blindness costs most\. At the low\-lag end the fixed offset collapses while the schedule holds:γ=1\.0\\gamma=1\.0retains 0\.1363 BLEU at a lag of1\.7 source tokens, 64% of the offline optimum and 45% above the nearest fixed offset, which needs 2\.6 more tokens of lag to reach 0\.0939\. With METEOR up 18\.5%, both margins are decisive under the paired bootstrap \(Table[4](https://arxiv.org/html/2609.20845#S5.T4)\)\. The regime symmetry also carries a sharp negative prediction\. Since the source is here theslowerstream, a schedule that deliberately runs behind it \(γ=1\.5\\gamma=1\.5\) starves permanently and collapses to BLEU 0\.0039, visible at the foot of Fig\.[1](https://arxiv.org/html/2609.20845#S5.F1)b, confirming that the dial’s useful range is set byρ\\rho\.

### Synthesis \(Q3\)

Where a fixed offset must be small relative to the instance, content\-blindness is expensive and ZENDAYA is clearly ahead\. Where a single offset happens to suit a dataset well, ZENDAYA still beats it on METEOR at lower lag \(Charades\), and at none of the three anchor pairs is it significantly behind on either metric\. Theγ∗\\gamma^\{\*\}rule predicted, before the runs, both regimes’ operating points and the collapse ofγ\>1\\gamma\>1where the source is slower\. One mechanism, one theorem, one training recipe servedρ=0\.61\\rho=0\.61andρ=3\.38\\rho=3\.38unmodified\.

## 6Conclusion

A single number decides how much of a stream the decoder may see while it writes\. Turn it down and it reads everything upfront like a usual decoder\. Then turn it up and it keeps pace with a live feed and never leans on words that have not arrived\. It also fixes, in closed form, how much source each written word consumes, and its guarantee is structural, not learned\. So, the latency measured offline is the latency delivered live\. The finding we did not go looking for is the one worth keeping\. A compact decoder trained from scratch, shown less of its input, writes better than the same decoder shown all of it, and holds that advantage across corpora whose source and target run at opposite speeds\. Seeing less tunably does better\.

Future work\.We set the dial by hand, once per corpus, though the right horizon surely varies window to window: a crowded minute of speech and a static shot do not deserve the same one\. Moreover, learningγ\\gammaper window is theorem\-preserving, since fixing it before a window decodes keeps the schedule monotone, so a per\-window learnedγ\\gammainherits every guarantee\.

Limitations\.The fixed\-window sweep is replicated over three seeds \(Appendix[E](https://arxiv.org/html/2609.20845#A5)\), and that replication revises the ActivityNet operating point of Table[2](https://arxiv.org/html/2609.20845#S5.T2); the streaming results remain single runs, and the optimisation noise measured offline plausibly reaches theirγ\\gammaandkkselections too\. Learned adaptive policies remain unevaluated head\-to\-head\. And while Corollary[1](https://arxiv.org/html/2609.20845#Thmcorollary1)covers arbitrary arrival, we ran only its window\-synchronized case\. Thus, the asynchronous regime is proved but not measured\.

## References

- Banerjee and Lavie \(2005\)Satanjeev Banerjee and Alon Lavie\.METEOR: An automatic metric for MT evaluation with improved correlation with human judgments\.In*Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization*, pages 65–72, 2005\.
- Chiu and Raffel \(2018\)Chung\-Cheng Chiu and Colin Raffel\.Monotonic chunkwise attention\.In*International Conference on Learning Representations \(ICLR\)*, 2018\.
- Elbayad et al\. \(2020\)Maha Elbayad, Laurent Besacier, and Jakob Verbeek\.Efficient wait\-k models for simultaneous machine translation\.In*Interspeech*, pages 1461–1465, 2020\.
- Gao et al\. \(2017\)Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia\.TALL: Temporal activity localization via language query\.In*Proceedings of the IEEE International Conference on Computer Vision \(ICCV\)*, pages 5267–5275, 2017\.
- Hori et al\. \(2021\)Chiori Hori, Takaaki Hori, and Jonathan Le Roux\.Optimizing latency for online video captioning using audio\-visual transformers\.In*Interspeech*, 2021\.
- Iashin and Rahtu \(2020\)Vladimir Iashin and Esa Rahtu\.Multi\-modal dense video captioning\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops \(CVPRW\)*, pages 958–959, 2020\.
- Kang et al\. \(2024\)Wei Kang, Xiaoyu Yang, Zengwei Yao, Fangjun Kuang, Yifan Yang, Liyong Guo, Long Lin, and Daniel Povey\.LibriHeavy: A 50,000 hours ASR corpus with punctuation casing and context\.In*IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, 2024\.
- Krishna et al\. \(2017\)Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei\-Fei, and Juan Carlos Niebles\.Dense\-captioning events in videos\.In*IEEE International Conference on Computer Vision \(ICCV\)*, pages 706–715, 2017\.
- Lee et al\. \(2025\)Sangho Lee, Il Yong Chun, and Hogun Park\.MAMS: Model\-agnostic module selection framework for video captioning\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, 2025\.
- Liu et al\. \(2024\)Nelson F\. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang\.Lost in the middle: How language models use long contexts\.*Transactions of the Association for Computational Linguistics*, 12:157–173, 2024\.
- Loshchilov and Hutter \(2019\)Ilya Loshchilov and Frank Hutter\.Decoupled weight decay regularization\.In*International Conference on Learning Representations \(ICLR\)*, 2019\.
- Ma et al\. \(2019\)Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, and Haifeng Wang\.STACL: Simultaneous translation with implicit anticipation and controllable latency using prefix\-to\-prefix framework\.In*Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics \(ACL\)*, pages 3025–3036, 2019\.
- Ma et al\. \(2020\)Xutai Ma, Juan Pino, James Cross, Liezl Puzon, and Jiatao Gu\.Monotonic multihead attention\.In*International Conference on Learning Representations \(ICLR\)*, 2020\.
- Moritz et al\. \(2021\)Niko Moritz, Takaaki Hori, and Jonathan Le Roux\.Dual causal/non\-causal self\-attention for streaming end\-to\-end speech recognition\.In*Interspeech*, 2021\.
- Papineni et al\. \(2002\)Kishore Papineni, Salim Roukos, Todd Ward, and Wei\-Jing Zhu\.BLEU: A method for automatic evaluation of machine translation\.In*Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics \(ACL\)*, pages 311–318, 2002\.
- Radford et al\. \(2021\)Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever\.Learning transferable visual models from natural language supervision\.In*International Conference on Machine Learning \(ICML\)*, 2021\.
- Radford et al\. \(2023\)Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever\.Robust speech recognition via large\-scale weak supervision\.In*International Conference on Machine Learning \(ICML\)*, 2023\.
- Sun et al\. \(2024\)Tong Sun, Biao Fu, Cong Hu, Liang Zhang, Ruiquan Zhang, Xiaodong Shi, Jinsong Su, and Yidong Chen\.Adaptive simultaneous sign language translation with confident translation length estimation\.In*Proceedings of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING\)*, pages 372–384, 2024\.
- Swietojanski et al\. \(2023\)Pawel Swietojanski, Stefan Braun, Dogan Can, Thiago Fraga da Silva, Arnab Ghoshal, Takaaki Hori, Roger Hsiao, Henry Mason, Erik McDermott, Honza Silovsky, Ruchir Travadi, and Xiaodan Zhuang\.Variable attention masking for configurable transformer transducer speech recognition\.In*IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, 2023\.
- Tran et al\. \(2015\)Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri\.Learning spatiotemporal features with 3D convolutional networks\.In*IEEE International Conference on Computer Vision \(ICCV\)*, pages 4489–4497, 2015\.
- Yin et al\. \(2021\)Aoxiong Yin, Zhou Zhao, Jinglin Liu, Weike Jin, Meng Zhang, Xingshan Zeng, and Xiaofei He\.SimulSLT: End\-to\-end simultaneous sign language translation\.In*Proceedings of the 29th ACM International Conference on Multimedia*, pages 4118–4127, 2021\.
- Zhou et al\. \(2024\)Xingyi Zhou, Anurag Arnab, Shyamal Buch, Shen Yan, Austin Myers, Xuehan Xiong, Arsha Nagrani, and Cordelia Schmid\.Streaming dense video captioning\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pages 18243–18252, 2024\.

## Appendices

These appendices give the full proofs of Theorem[1](https://arxiv.org/html/2609.20845#Thmtheorem1)and Corollary[1](https://arxiv.org/html/2609.20845#Thmcorollary1), paired\-bootstrap intervals behind every entry of Table[2](https://arxiv.org/html/2609.20845#S5.T2), eighteen streaming decode clips on Charades\-STA, and a three\-seed replication of the entire fixed\-window sweep\. That replication supports the central claim on all three corpora and sharpens its geometric form, but it does not support theγ\\gammaselected for ActivityNet in Section[5](https://arxiv.org/html/2609.20845#S5); Appendix[E](https://arxiv.org/html/2609.20845#A5)lists precisely which entries are affected and what replaces them\. Theorem, corollary and remark numbering follows the main text, as does notation\.

## Appendix AFull Proof of Theorem 1

#### Setting\.

LetMMbe a decoder built from token and positional embeddings, thenLLidentical layers, then a linear output projection\. Each layer applies causal self\-attention, then cross\-attention in which positioniimay attend only to source indices\{0,…,Ωi−1\}\\\{0,\\dots,\\Omega\_\{i\}\-1\\\}, with masked logits set to−∞\-\\inftybefore the softmax, then position\-wise residual, normalization, and feed\-forward maps\. Parameters are arbitrary, trained or untrained, and no property of the training procedure is used anywhere below\. The schedulei↦Ωii\\mapsto\\Omega\_\{i\}is fixed before decoding begins\. In ZENDAYA\-fix it is determined byN^​\(X\)\\hat\{N\}\(X\), so the guarantee is conditional on that schedule; in ZENDAYA\-inf the schedule is determined by arrived data alone, which removes the conditioning\.

#### Definition\.

A quantity*depends only on*X<kX\_\{<k\}if, for any two source sequencesXXandX′X^\{\\prime\}agreeing on indices\{0,…,k−1\}\\\{0,\\dots,k\-1\\\}, the quantity takes the same value, howeverXXandX′X^\{\\prime\}differ on indices≥k\\geq k\.

###### Theorem 1\.

For any horizon schedulei↦Ωii\\mapsto\\Omega\_\{i\}that is fixed before decoding begins and non\-decreasing inii, the output logits at stepiiare a function only of the text prefixy0:iy\_\{0:i\}and the source prefixX<ΩiX\_\{<\\Omega\_\{i\}\}\.

###### Proof\.

Writehi\(ℓ\)h^\{\(\\ell\)\}\_\{i\}for the hidden state at positioniiafter layerℓ\\ell, withhi\(0\)h^\{\(0\)\}\_\{i\}the input embedding\. We prove by induction onℓ\\ellthathi\(ℓ\)h^\{\(\\ell\)\}\_\{i\}depends only onX<ΩiX\_\{<\\Omega\_\{i\}\}, for every positioniisimultaneously\.

Base case\.hi\(0\)h^\{\(0\)\}\_\{i\}is a function ofy0:iy\_\{0:i\}alone and is independent ofXX, hence depends only onX<ΩiX\_\{<\\Omega\_\{i\}\}vacuously\.

Inductive step\.Supposehi′\(ℓ\)h^\{\(\\ell\)\}\_\{i^\{\\prime\}\}depends only onX<Ωi′X\_\{<\\Omega\_\{i^\{\\prime\}\}\}for every positioni′i^\{\\prime\}\. Fix a positionii\.

\(i\) Causal self\-attention\.The output at positioniiis a function of\{hi′\(ℓ\):i′≤i\}\\\{h^\{\(\\ell\)\}\_\{i^\{\\prime\}\}:i^\{\\prime\}\\leq i\\\}\. For each suchi′i^\{\\prime\}the schedule is non\-decreasing, soΩi′≤Ωi\\Omega\_\{i^\{\\prime\}\}\\leq\\Omega\_\{i\}and therefore\{0,…,Ωi′−1\}⊆\{0,…,Ωi−1\}\\\{0,\\dots,\\Omega\_\{i^\{\\prime\}\}\-1\\\}\\subseteq\\\{0,\\dots,\\Omega\_\{i\}\-1\\\}\. A quantity determined by a subset ofX<ΩiX\_\{<\\Omega\_\{i\}\}is determined byX<ΩiX\_\{<\\Omega\_\{i\}\}, so every input to the block, and hence its outputh~i\\tilde\{h\}\_\{i\}, depends only onX<ΩiX\_\{<\\Omega\_\{i\}\}\. This step is where monotonicity is indispensable: without it a positioniicould inherit, through an earlier positioni′i^\{\\prime\}withΩi′\>Ωi\\Omega\_\{i^\{\\prime\}\}\>\\Omega\_\{i\}, information about source tokens beyond its own horizon\.

\(ii\) Masked cross\-attention\.Letei​je\_\{ij\}denote the pre\-softmax logit for source indexjj\. By constructionei​j=−∞e\_\{ij\}=\-\\inftyfor everyj≥Ωij\\geq\\Omega\_\{i\}, soexp⁡\(ei​j\)=0\\exp\(e\_\{ij\}\)=0and those indices contribute nothing to either the numerator or the normalizer of the softmax\. The attention weights therefore satisfypi​j=0p\_\{ij\}=0for allj≥Ωij\\geq\\Omega\_\{i\}, and the block output reduces exactly to

∑j=0F−1pi​j​Vj=∑j<Ωipi​j​Vj\.\\sum\_\{j=0\}^\{F\-1\}p\_\{ij\}\\,V\_\{j\}\\;=\\;\\sum\_\{j<\\Omega\_\{i\}\}p\_\{ij\}\\,V\_\{j\}\.Each surviving keyKj=Xj​WKK\_\{j\}=X\_\{j\}W\_\{K\}and valueVj=Xj​WVV\_\{j\}=X\_\{j\}W\_\{V\}withj<Ωij<\\Omega\_\{i\}depends only onX<ΩiX\_\{<\\Omega\_\{i\}\}, and the query is derived fromh~i\\tilde\{h\}\_\{i\}, which depends only onX<ΩiX\_\{<\\Omega\_\{i\}\}by \(i\)\. Every factor entering the block output is therefore accounted for\.

\(iii\) Position\-wise maps\.The statehi\(ℓ\+1\)h^\{\(\\ell\+1\)\}\_\{i\}is obtained fromh~i\\tilde\{h\}\_\{i\}and the cross\-attention output by residual addition, normalization, and a feed\-forward map, all deterministic and applied at positioniialone\. It is thus a function of quantities already shown to depend only onX<ΩiX\_\{<\\Omega\_\{i\}\}\.

Closing the induction at layerLL, the statehi\(L\)h^\{\(L\)\}\_\{i\}depends only onX<ΩiX\_\{<\\Omega\_\{i\}\}, and therefore so do the logitslogitsi=Wout​hi\(L\)\\mathrm\{logits\}\_\{i\}=W\_\{\\mathrm\{out\}\}h^\{\(L\)\}\_\{i\}\.

Extension to closed\-loop decoding\.The argument above concerns teacher\-forced logits, wherey0:iy\_\{0:i\}is given\. Under a deterministic decoding rule, greedy selection in our experiments, it extends to generated text by a second induction, this time over steps\. The first token is produced from a fixed start symbol, so the base case is immediate\. Suppose every emitted tokenyi′y\_\{i^\{\\prime\}\}withi′<ii^\{\\prime\}<idepends only onX<Ωi′X\_\{<\\Omega\_\{i^\{\\prime\}\}\}\. Monotonicity givesX<Ωi′⊆X<ΩiX\_\{<\\Omega\_\{i^\{\\prime\}\}\}\\subseteq X\_\{<\\Omega\_\{i\}\}, so the whole prefixy0:iy\_\{0:i\}is a function ofX<ΩiX\_\{<\\Omega\_\{i\}\}\. Combining this with the layer induction,logitsi\\mathrm\{logits\}\_\{i\}and henceyiy\_\{i\}are functions ofX<ΩiX\_\{<\\Omega\_\{i\}\}alone, and the emitted stream up to stepiiinherits the same dependency\. ∎

#### Remark on the degenerate case\.

IfΩi=0\\Omega\_\{i\}=0the attended set is empty and we adopt the convention that the cross\-attention block outputs zero at positionii, a pure language\-modeling step, under which every statement above is preserved\. This case cannot arise under the ceiling constructionΩi=⌈F⋅\(i/N^\)γ⌉\\Omega\_\{i\}=\\lceil F\\cdot\(i/\\hat\{N\}\)^\{\\gamma\}\\rceil, sinceF≥1F\\geq 1and\(i/N^\)γ\>0\(i/\\hat\{N\}\)^\{\\gamma\}\>0giveΩi≥Ω1≥1\\Omega\_\{i\}\\geq\\Omega\_\{1\}\\geq 1for every admissibleFF,N^\\hat\{N\}andγ\\gamma\.333Our fixed\-window implementation rounds down rather than up, and so admitsΩ1=0\\Omega\_\{1\}=0whereF<N^γF<\\hat\{N\}^\{\\gamma\}\. The threshold grows withγ\\gammaand so is easiest to meet at the largest exponents we sweep, as for instance on LibriHeavy atγ=1\\gamma=1, whose source is generally shorter than its target\. The convention above governs those steps\.

## Appendix BFull Proof of Corollary 1

Within windowkk, decoding proceeds over stepsj=1,…,N^kj=1,\\dots,\\hat\{N\}\_\{k\}with prescribed horizonΩk,j=⌈W⋅\(j/N^k\)γ⌉\\Omega\_\{k,j\}=\\lceil W\\cdot\(j/\\hat\{N\}\_\{k\}\)^\{\\gamma\}\\rceil, non\-decreasing injjby Remark[1](https://arxiv.org/html/2609.20845#Thmremark1)\. The schedule never asks for more thanΩk,j\\Omega\_\{k,j\}source tokens at stepjj, so decoding may begin once the firstΩk,1\\Omega\_\{k,1\}have arrived\. What the schedule cannot control is whether arrival keeps pace with it, and this section supplies the guarantee for the case where it does not\.

LetAk​\(t\)∈\{0,…,W\}A\_\{k\}\(t\)\\in\\\{0,\\dots,W\\\}denote the number of source tokens of windowkkencoded and cached by wall\-clock timett\. It is non\-decreasing intt, since tokens cannot un\-arrive\. Lettjt\_\{j\}denote the wall\-clock time at which decoding reaches stepjj; since generation proceeds strictly forward in time,tjt\_\{j\}is non\-decreasing injj\. Define theeffective horizon

Ωk,jeff:=min⁡\(Ωk,j,Ak​\(tj\)\),\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}:=\\min\\big\(\\Omega\_\{k,j\},\\,A\_\{k\}\(t\_\{j\}\)\\big\),\(6\)the source prefix actually licensed at stepjjonce both the schedule and physical arrival are accounted for\.

###### Corollary 1\.

For any non\-decreasing arrival processAkA\_\{k\}, the mapj↦Ωk,jeffj\\mapsto\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}is non\-decreasing, and Theorem 1 applies verbatim withΩk,jeff\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}in place ofΩk,j\\Omega\_\{k,j\}\. Generation at stepjjis then a function only of the text generated so far, the pre\-decoding inputs that fix the schedule, and the source tokens ofwkw\_\{k\}with index<Ωk,jeff<\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}\. In particular no emitted token depends on a source token that has not arrived by the time that token is produced\.

###### Proof\.

The mapj↦Ak​\(tj\)j\\mapsto A\_\{k\}\(t\_\{j\}\)is non\-decreasing, being a composition of the non\-decreasing mapsj↦tjj\\mapsto t\_\{j\}andt↦Ak​\(t\)t\\mapsto A\_\{k\}\(t\)\. Forj≤j′j\\leq j^\{\\prime\}we then haveΩk,j≤Ωk,j′\\Omega\_\{k,j\}\\leq\\Omega\_\{k,j^\{\\prime\}\}by monotonicity of the schedule andAk​\(tj\)≤Ak​\(tj′\)A\_\{k\}\(t\_\{j\}\)\\leq A\_\{k\}\(t\_\{j^\{\\prime\}\}\)by the above, somin⁡\(Ωk,j,Ak​\(tj\)\)≤min⁡\(Ωk,j′,Ak​\(tj′\)\)\\min\(\\Omega\_\{k,j\},A\_\{k\}\(t\_\{j\}\)\)\\leq\\min\(\\Omega\_\{k,j^\{\\prime\}\},A\_\{k\}\(t\_\{j^\{\\prime\}\}\)\), that isΩk,jeff≤Ωk,j′eff\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}\\leq\\Omega^\{\\mathrm\{eff\}\}\_\{k,j^\{\\prime\}\}\. Theorem 1 further asks that the schedule be fixed before decoding begins, andΩeff\\Omega^\{\\mathrm\{eff\}\}is not, since it moves with the arrival process as that process unfolds\. We therefore apply the theorem pathwise: fix any realization ofAkA\_\{k\}together with the realized step timestjt\_\{j\}, and along that pathj↦Ωk,jeffj\\mapsto\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}is a determinate non\-decreasing integer sequence that no generated token can alter, which is all that either induction of Theorem 1 uses\. The dependency claim therefore holds on every path, hence unconditionally\.

It remains to check that the quantities fixing the schedule are themselves free of future information\. These areN^k\\hat\{N\}\_\{k\}, and through it the initial buffer ofBBsource tokens of windowkk, the pooled features of the preceding windowwk−1w\_\{k\-1\}, and the textYhistoryY\_\{\\mathrm\{history\}\}generated for that preceding window\. All three are available before window\-kkdecoding begins, andB≤Ωk,1B\\leq\\Omega\_\{k,1\}becauseNmax≥N^kN\_\{\\max\}\\geq\\hat\{N\}\_\{k\}, so the buffer never exceeds what the first token already licenses\. The text side of the induction concerns onlyYhistoryY\_\{\\mathrm\{history\}\}and the target prefix, neither of which involvesAkA\_\{k\}\.

Applying Theorem 1 to the window\-kkdecode with scheduleΩk,jeff\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}yields the dependency claim, andΩk,jeff≤Ak​\(tj\)\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}\\leq A\_\{k\}\(t\_\{j\}\)ensures that every licensed source token has arrived by timetjt\_\{j\}\. ∎

#### Extension across windows\.

The corollary is stated for a single window\. Extending it to the unbounded stream is an induction overk=0,1,2,…k=0,1,2,\\dots\. The base casek=0k=0is the corollary with empty history\. For the inductive step, the quantities fixing windowkk’s schedule consist of the preceding window and the text generated for it, both of which the inductive hypothesis places in the past, so the corollary applies to windowkkunchanged\. Since each window’s guarantee is stated relative to data already arrived, the conjunction over allkkgives the guarantee for the stream as a whole\.

#### Relation to the evaluated system\.

The guarantee holds regardless of the relationship between the prescribed schedule and true arrival\. Where tokens arrive at least as fast as the schedule requests them,Ωk,jeff=Ωk,j\\Omega^\{\\mathrm\{eff\}\}\_\{k,j\}=\\Omega\_\{k,j\}and the model self\-restricts exactly as prescribed, which is the regime our experiments evaluate, precomputed features making arrival effectively instantaneous\. Where arrival falls behind, as on a degraded link, the model is automatically and provably limited to what has actually arrived, with no separate\-case logic required\. The fully asynchronous regime is therefore proved here but not measured, as the limitations note in Section 6\.

## Appendix CPaired\-Bootstrap Intervals, Fixed\-Window Setting

Table[A1](https://arxiv.org/html/2609.20845#A3.T1)gives the interval behind every entry of Table[2](https://arxiv.org/html/2609.20845#S5.T2)\. For each dataset we compare ZENDAYA at its selectedγ\\gammaagainst theγ→0\\gamma\\to 0endpoint of the identical architecture, on the same test set in the same order, resampling clips with replacement 1000 times\. Reported values are means over resamples and so differ from Table[2](https://arxiv.org/html/2609.20845#S5.T2)in the fourth decimal\.

Both margins shrink monotonically as the source\-to\-target ratioρ\\rhofalls, and neither metric moves on the text\-dense corpus: withρ<1\\rho<1there is no surplus source to withhold, so restricting exposure can neither help nor hurt\.

Table A1:Fixed\-window paired bootstrap, 1000 resamples\. Bold marks intervals excluding zero\. Two\-sidedpponΔ\\DeltaBLEU\-4: 0\.026 Charades\-STA,<<0\.001 ActivityNet, 0\.560 LibriHeavy\.
## Appendix DQualitative Sample

Table[A2](https://arxiv.org/html/2609.20845#A4.T2)reports eighteen Charades\-STA clips with both systems’ decodes on each, referred to from the Q2 discussion of Section 5\. The protocol is symmetric: we score every decode of both systems with sentence\-level BLEU\-4, take the nine clips on which each system scores highest keeping one per distinct reference, and report both systems’ decodes of each\. Each half is chosen on one system’s own best terms\.

Table A2:Streaming decodes on identical Charades\-STA clips\.z: ZENDAYAγ=0\.2\\gamma\{=\}0\.2;w: wait\-40\.
## Appendix ESeeded Replication of the Fixed\-Window Sweep

The fixed\-window sweep was repeated under three seeds\. Seven configurations per corpus, theγ→0\\gamma\\to 0baseline and six values ofγ\\gamma, across three corpora resulted in 63 training runs in all\. A seed fixes weight initialisation, dropout and batch order\. Splits and vocabulary were already fixed upstream by the preprocessing stage\. Figure[A1](https://arxiv.org/html/2609.20845#A5.F1)and Table[A3](https://arxiv.org/html/2609.20845#A5.T3)report the outcome\.

The seeds pull the offline picture towards our geometry rather than away from it\. We argue that a larger source\-to\-target ratio should move the useful operating point to largerγ\\gamma, and our single runs did not show that\. They put ActivityNet, the corpus of highestρ\\rho, atγ=0\.10\\gamma=0\.10, the smallest exponent we sweep\. Under three seeds its supported settings sit atγ=0\.7\\gamma=0\.7, againstγ=0\.3\\gamma=0\.3on Charades\-STA, which is the order thatρ=3\.38\\rho=3\.38andρ=2\.75\\rho=2\.75ask for\. The negative prediction survives as well\. On LibriHeavy, whereρ<1\\rho<1leaves no surplus source to withhold, noγ\\gammaseparates from the baseline at all, and the collapse atγ=1\\gamma=1is plain on every seed\.

Where the seeds disagree with us is theγ\\gammawe chose for ActivityNet\. That selection came from a single unseeded run, which is the scope those experiments declare, and it landed atγ=0\.10\\gamma=0\.10\. Across three seeds the setting sits a little under the baseline on both metrics,0\.01450\.0145against0\.01650\.0165and0\.11230\.1123against0\.11500\.1150, with both gaps inside the noise\. Three of our statements lean on that run\. The best\-against\-best entry for ActivityNet in Table[2](https://arxiv.org/html/2609.20845#S5.T2)does not hold at theγ\\gammait names\. The claim that every swept setting exceeds the baseline fails there atγ=0\.1\\gamma=0\.1andγ=0\.3\\gamma=0\.3\. And the margins we quote,\+19\.6%\+19\.6\\%for Charades\-STA against\+46\.9%\+46\.9\\%for ActivityNet, become\+34%\+34\\%and\+20%\+20\\%, which reverses their order\. We would replace that magnitude claim with the location claim above, which is the sharper statement and the one the seeds support\. At the corrected setting ActivityNet returns1\.971\.97BLEU\-4 and12\.2512\.25METEOR\. On Charades\-STA and LibriHeavy theγ\\gammawe selected stands\.

Two things temper the picture\. The useful settings are not contiguous: Charades\-STA gains on both metrics atγ∈\{0\.1,0\.2,0\.3\}\\gamma\\in\\\{0\.1,0\.2,0\.3\\\}and again at0\.70\.7, but slips just below the baseline on METEOR at0\.40\.4\. That is the multi\-modal response overγ\\gammaour future work anticipates, and a reason to readγ\\gammaas a region to search rather than a value to tune\. And only the fixed\-window setting was repeated\. The streaming results remain single runs, and the optimisation noise we measure here plausibly reaches theirγ\\gammaandkkselections too\. We have not measured that\.

Table[A1](https://arxiv.org/html/2609.20845#A3.T1)and this section are not in conflict\. The bootstrap resamples test clips within one training run and so measures sampling noise, while the seeds vary the run and so measure optimisation noise\. On ActivityNet the second turns out to be the larger of the two, which no single\-run interval could have revealed\. Where they disagree, we would trust the seeded means\.

![Refer to caption](https://arxiv.org/html/2609.20845v1/x2.png)Figure A1:Offline BLEU\-4 againstγ\\gamma, mean and standard deviation over three seeds\. The dashed line and shaded band are the baseline mean and its own seed spread\. The enlarged marker is the bestγ\\gammaby mean on each corpus; on LibriHeavy that margin lies inside the seed spread, as Table[A3](https://arxiv.org/html/2609.20845#A5.T3)shows\.Table A3:Fixed\-window sweep over three seeds, mean±\\pmstandard deviation\. Bold marks a configuration that beats its baseline on every seed by a mean paired difference of at least three times its standard error\. With three seeds these are effect sizes rather than significance tests, and we report them as such\.