ZAYA1-8B Technical Report

# ZAYA1-8B Technical Report
Source: [https://arxiv.org/html/2605.05365](https://arxiv.org/html/2605.05365)
Robert Washbourne\*, Rishi Iyer, Tomas Figliolia, Henry Zheng, Ryan Lorig\-Roach, Sungyeon Yang, Pritish Yuvraj, Quentin Anthony, Yury Tokpanov, Xiao Yang, Ganesh Nanduru, Stephen Ebert, Praneeth Medepalli, Skyler Szot, Srivatsan Rajagopal, Alex Ong, Bhavana Mehta, Beren Millidge\* Zyphra San Francisco, CA

###### Abstract

We present ZAYA1\-8B, a reasoning\-focused mixture\-of\-experts \(MoE\) model with 700M active and 8B total parameters, built on Zyphra’s MoE\+\+ architecture\. ZAYA1\-8B’s core pretraining, midtraining, and supervised fine\-tuning \(SFT\) were performed on a full\-stack AMD compute, networking, and software platform\. With under 1B active parameters, ZAYA1\-8B matches or exceeds DeepSeek\-R1\-0528 on several challenging mathematics and coding benchmarks, and remains competitive with substantially larger open\-weight reasoning models\. ZAYA1\-8B was trained from scratch for reasoning, with reasoning data included from pretraining onward using an answer\-preserving trimming scheme\. Post\-training uses a four\-stage RL cascade: reasoning warmup on math and puzzles; a 400\-task RLVE\-Gym curriculum; math and code RL with test\-time compute traces and synthetic code environments built from competitive\-programming references; and behavioral RL for chat and instruction following\. We also introduce Markovian RSA, a test\-time compute method that recursively aggregates parallel reasoning traces while carrying forward only bounded\-length reasoning tails between rounds\. In TTC evaluation, Markovian RSA raises ZAYA1\-8B to 91\.9% on AIME’25 and 89\.6% on HMMT’25 while carrying forward only a 4K\-token tail, narrowing the gap to much larger reasoning models including Gemini\-2\.5 Pro, DeepSeek\-V3\.2, and GPT\-5\-High\.

## IIntroduction

![Refer to caption](https://arxiv.org/html/2605.05365v1/x1.png)Figure 1:ZAYA1\-8B with Markovian RSA test\-time compute vs\. substantially larger reasoning models on AIME’25, HMMT’25, and LCB\-v6\. Hatched bars show the boost from Markovian RSA over single\-rollout ZAYA1\-8B\. With 0\.7B active parameters and the 40K/4K Markovian RSA configuration \(Section[VI\-C](https://arxiv.org/html/2605.05365#S6.SS3)\), ZAYA1\-8B reaches 91\.9% on AIME’25 and 89\.6% on HMMT’25, narrowing the gap to larger proprietary and open\-weight reasoning models\. ZAYA1\-8B numbers \(single\-rollout and TTC\) are evaluated in the Zyphra harness on the pre\-behavioral checkpoint after math\+code\+TTC RL and before the final lightweight behavioral\-RL polishing stage; comparator numbers are taken from official release materials \(see Table[XI](https://arxiv.org/html/2605.05365#S6.T11)for sources\)\. The final behavioral stage targets chat style, instruction following, and preference behavior rather than math/code/TTC capability\.![Refer to caption](https://arxiv.org/html/2605.05365v1/x2.png)Figure 2:Active\-parameter scaling across HMMT’26, AIME’26, and LiveCodeBench\-v6\. ZAYA1\-8B is shown at 0\.7B active parameters and compared against larger open\-weight and frontier models where available\. Bubble area denotes total parameter count where available\.In this paper, we introduce ZAYA1\-8B, a 700M\-active, 8B\-total parameter mixture\-of\-experts \(MoE\) model\. With under 1B active parameters, ZAYA1\-8B matches or exceeds DeepSeek\-R1\-0528 on several challenging mathematics and coding benchmarks, while remaining competitive with substantially larger open\-weight reasoning models including OLMo\-3\.1\-32B\-Think, Nemotron\-3\-Nano\-30B\-A3B, Mistral\-Small\-4\-119B\-2603, and Intellect\-3\-12A\-106B\(NVIDIA,[2025](https://arxiv.org/html/2605.05365#bib.bib6); Teamet al\.,[2025c](https://arxiv.org/html/2605.05365#bib.bib3); Team,[2025a](https://arxiv.org/html/2605.05365#bib.bib5); Mistral AI,[2026](https://arxiv.org/html/2605.05365#bib.bib4)\)\.

Moreover, using our test\-time compute scheme, Markovian RSA, ZAYA1\-8B narrows the gap on AIME’25 and HMMT’25 to substantially larger reasoning models including Gemini\-2\.5 Pro, DeepSeek\-V3\.2, Qwen3\-235B\-A22B\-Thinking\-2507, and GPT\-5\-High\(Comaniciet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib7); DeepSeek\-AI,[2025c](https://arxiv.org/html/2605.05365#bib.bib9); Team,[2025b](https://arxiv.org/html/2605.05365#bib.bib10); OpenAI,[2025](https://arxiv.org/html/2605.05365#bib.bib11)\)\. These results suggest that competitive mathematical reasoning can be reached with under 1B active parameters when model architecture, reasoning\-heavy training, verifiable RL, and test\-time aggregation are co\-designed\.

The system combines five design choices that we found important in practice:

Architecture:ZAYA1\-8B builds on Zyphra’s MoE\+\+ architecture\(Anthonyet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib156)\), with three main changes relative to standard transformer MoE designs\. First, ZAYA1\-8B uses Compressed Convolutional Attention \(CCA\)\(Figlioliaet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib193)\), a FLOP\- and memory\-efficient attention variant that performs sequence mixing in a compressed latent space\. Prior work showed that CCA performs well on perplexity and standard language modeling at small scale; ZAYA1\-8B evaluates its behavior at larger scale and on more challenging reasoning and long\-context tasks\. Second, ZAYA1\-8B uses the ZAYA1 router, which replaces the standard linear MoE router with a multi\-layer MLP\-based design, substantially increasing its expressiveness\. In our experiments we find that increasing router capacity and expressiveness is a strong use of marginal parameters\. A small number of router parameters controls a much larger number of expert parameters, and better routing decisions significantly reduce balancing instability and improve model quality\. Third, ZAYA1\-8B applies learned residual scaling to both the residual stream and the layer input at each block, which controls residual\-norm growth through depth at negligible parameter and FLOP cost\.

Reasoning\-aware training across stages:We designed ZAYA1\-8B from scratch for reasoning\. Motivated by evidence that including reasoning data during pretraining can produce gains that post\-training alone does not recover\(Akteret al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib165)\), we include long chain\-of\-thought \(CoT\) data in all pretraining phases and during midtraining\. To train on reasoning traces that exceed the pretraining context length, we introduce a novel answer\-preserving trimming methodology, which truncates the tail of the reasoning trace while preserving the final answer, or drops the example if the answer alone does not fit\. Unlike prior length\-control methods that operate during inference or RL rollout generation\(Khatriet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib163); Yanget al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib202)\), AP\-trimming is applied during training\-data construction\.

Cascaded reinforcement learning pipeline:Post\-training for ZAYA1\-8B uses a four\-stage RL cascade: reasoning warmup, a 400\-task adaptive difficulty curriculum over the RLVE\-Gym environment suite\(Zenget al\.,[2025a](https://arxiv.org/html/2605.05365#bib.bib164)\), math and code RL with test\-time compute traces, and a final behavioral RL stage\. The cascade uses asynchronous PipelineRL\(Pichéet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib155); Khatriet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib163)\)with DPPO Binary\-TV trust\-region masking\(Qiet al\.,[2026](https://arxiv.org/html/2605.05365#bib.bib162)\), Dr\-GRPO sequence\-level loss aggregation\(Liuet al\.,[2024](https://arxiv.org/html/2605.05365#bib.bib148)\), MaxRL advantage estimation\(Tajwaret al\.,[2026](https://arxiv.org/html/2605.05365#bib.bib13)\), and no KL regularization in the reward\. Stable training required substantial precision, verifier, and data curation work, which we document throughout the report\.

Test\-time compute methods:We introduce Markovian RSA, a novel test\-time compute method that combines the recursive candidate\-aggregation structure of RSA\(Venkatramanet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib157)\)with the bounded\-workspace principle of Markovian Thinking\(Aghajohariet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib158)\)\. Markovian RSA turns long reasoning into staged batched inference: each stage generatesNNcandidates in parallel, each candidate has bounded decode lengthβ\\beta, and aggregation prefill depends only onCCcarried\-forward tails of lengthτ\\tau, not on the full reasoning history\. Crucially, we also integrate Markovian RSA into training: SFT data is constructed by reshuffling expert\-model rollouts into aggregation examples, and RL stages train both expert\-model and policy\-self\-aggregation variants\. The resulting model is trained for the Markovian RSA workflow at inference and we achieve substantial performance uplift by doing so\.

AMD training stack:Building on our prior work with AMD MI300X GPUs and AMD Pensando Pollara 400 networking for large\-scale pretraining\(Anthonyet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib156)\), ZAYA1\-8B was pretrained, midtrained, and supervised fine\-tuned on this GPU/networking stack\. This provides evidence that the stack can support sustained pretraining, long\-context midtraining, and supervised fine\-tuning for an 8B\-total\-parameter MoE reasoning model\. We validate this stack at the ZAYA1\-8B scale; validation for substantially larger models and broader parallelism regimes remains future work\.

The remainder of this report is organized as follows: Section[II](https://arxiv.org/html/2605.05365#S2)describes the ZAYA1\-8B architecture\. Section[III](https://arxiv.org/html/2605.05365#S3)describes pretraining, midtraining, and answer\-preserving trimming\. Section[IV](https://arxiv.org/html/2605.05365#S4)describes the SFT stage and RL cascade, including infrastructure, precision, optimizer, and stability\-monitoring choices\. Section[V](https://arxiv.org/html/2605.05365#S5)reports benchmark results and comparisons\. Section[VI](https://arxiv.org/html/2605.05365#S6)describes our test\-time compute approach\. Section[VII](https://arxiv.org/html/2605.05365#S7)concludes with observations from training and open questions\.

## IIModel

TABLE I:ZAYA1\-8B model configuration\. Exact parameter counts are shown; the rounded release convention refers to the model as 0\.7B active and 8B total\. Architectural constants follow the ZAYA1 base configuration used for pretraining and continued post\-training\.### II\-AArchitecture

ZAYA1\-8B uses an MoE architecture with three changes relative to contemporary MoE models: \(1\) CCA for the attention block, \(2\) the ZAYA1 router, and \(3\) residual scaling\. In our ablations, these changes improve per\-parameter perplexity relative to classical MoE architectures\(Shazeeret al\.,[2016](https://arxiv.org/html/2605.05365#bib.bib93); Feduset al\.,[2022](https://arxiv.org/html/2605.05365#bib.bib53)\)using MLA or GQA attention and a linear router\(Daiet al\.,[2024](https://arxiv.org/html/2605.05365#bib.bib184)\)\. CCA also improves training speed relative to GQA and MLA and reduces prefill FLOPs while maintaining comparable KV\-cache compression rates\.

![Refer to caption](https://arxiv.org/html/2605.05365v1/figures/Screenshot_2026-05-04.png)Figure 3:ZAYA1\-8B model architecture\. Two of the three main architectural changes are shown here: CCA for the attention block and the ZAYA1 router\. The ZAYA1 router replaces the linear router with an MLP\-based router consisting of a down\-projection, EDA, and a three\-layer MLP\.#### II\-A1Compressed Convolutional Attention \(CCA\)

CCA performs sequence mixing in a compressed latent space using a lightweight convolutional downprojector\. This reduces compute requirements for training and prefill and reduces KV\-cache size for long\-context decoding\. CCA is competitive with attention variants such as MLA and GQA\(Ainslieet al\.,[2023](https://arxiv.org/html/2605.05365#bib.bib152); DeepSeek\-AI,[2025a](https://arxiv.org/html/2605.05365#bib.bib182)\)\. ZAYA1\-8B’s reasoning and long\-context performance provides evidence that CCA remains effective at this scale and can support reasoning, in\-context learning \(ICL\), and long\-range recall\. CCA also supports our long\-context midtraining workloads at lower compute and communication cost, which was important for training ZAYA1\-8B during midtraining and RL phases\. Appendix[C](https://arxiv.org/html/2605.05365#A3)provides additional details\.

#### II\-A2ZAYA1 Router

We replace the standard linear router used in many large\-scale MoE models with a more expressive router\. First, we use an MLP in place of the linear router\. Second, we mix the router representation with the previous layer’s routing representation usingExponential Depth Averaging \(EDA\), a variant of Depth\-Weighted Averaging\(Pagliardiniet al\.,[2024](https://arxiv.org/html/2605.05365#bib.bib199)\)\.

Given the residual stream inputxl∈ℝB×S×Dx\_\{l\}\\in\\mathbb\{R\}^\{B\\times S\\times D\}, whereDDis the residual stream dimension, the ZAYA1 router first down\-projects the residual stream to a smaller router dimensionRRusing a learned weight matrixWdown∈ℝR×DW\_\{\\text\{down\}\}\\in\\mathbb\{R\}^\{R\\times D\}:

rl=Wdown​xl,\\displaystyle r\_\{l\}=W\_\{\\text\{down\}\}x\_\{l\}\\,,\(1\)such thatrl∈ℝB×S×Rr\_\{l\}\\in\\mathbb\{R\}^\{B\\times S\\times R\}\. For ZAYA1\-8B we setR=256R=256\. We then apply EDA, which combines the representation with that of the previous layer using a learned coefficientγ\\gamma:

rl=rl\+γ​rl−1\.\\displaystyle r\_\{l\}=r\_\{l\}\+\\gamma r\_\{l\-1\}\\,\.\(2\)The EDA operation is followed by a three\-layer MLP with GeLU activations to produce the final router scoress∈ℝB×S×Es\\in\\mathbb\{R\}^\{B\\times S\\times E\}, whereEEis the number of experts:

sl=softmax​\(MLP​\(RMSnorm​\(rl\)\)\)\.\\displaystyle s\_\{l\}=\\text\{softmax\}\(\\text\{MLP\}\(\\text\{RMSnorm\}\(r\_\{l\}\)\)\)\\,\.\(3\)The scores are then used to select experts through a top\-k operation:

eidx=topk​\(sl\+bl\),\\displaystyle e\_\{\\text\{idx\}\}=\\text\{topk\}\(s\_\{l\}\+b\_\{l\}\)\\,,\(4\)whereblb\_\{l\}are learned bias\-balancing vectors and topk selects thekkexperts with the largest biased router scores for each token\. In ZAYA1\-8B,k=1k=1, so \([4](https://arxiv.org/html/2605.05365#S2.E4)\) reduces to selectingarg​maxe⁡\(sl,e\+bl,e\)\\operatorname\*\{arg\\,max\}\_\{e\}\(s\_\{l,e\}\+b\_\{l,e\}\)for each token\. The ZAYA1 router uses a bias\-balancing scheme building on\(DeepSeek\-AI,[2025a](https://arxiv.org/html/2605.05365#bib.bib182)\)\. Routing biases are updated using a scheme inspired by proportional–integral–derivative \(PID\) controllers from classical control theory\(Åström and Hägglund,[2006](https://arxiv.org/html/2605.05365#bib.bib210)\)\. The router enforces balancing across a global batch of expert choices\. Our PID optimizer uses AdamW internally, where the error signal passed to the optimizer is the difference between the empirical routing probability distribution and the uniform distribution\. Specifically, the gradient∇bl,e\\nabla b\_\{l,e\}, for experteeat layerll, is computed as:

∇bl,e=pl,e−1E,\\displaystyle\\nabla b\_\{l,e\}=p\_\{l,e\}\-\\frac\{1\}\{E\}\\,,\(5\)wherepl,ep\_\{l,e\}is the actual fraction of tokens routed to experteein the current batch, andEEis the total number of experts\. This gradient signal is then used by AdamW to update the bias terms, penalizing over\-utilized experts and boosting under\-utilized ones\. This improved the convergence speed and stability of the PID loop relative to the classical DeepSeek implementation\.

In our experiments, the MLP router and EDA improve MoE performance and make balancing \(Figure[4](https://arxiv.org/html/2605.05365#S2.F4)\) and expert specialization easier\. The additional MLP adds some FLOPs and parameters, but parameter\-matched ablations show that the router is a strong target for marginal parameters compared with the experts or attention\. The added router parameters and FLOPs remain small because the MLP operates in the down\-projected latent space rather than in the full embedding dimension\. Figure[4](https://arxiv.org/html/2605.05365#S2.F4)illustrates the average balancing across layers from initialization of an experiment\-sized model\. Empirically, reduced time to convergence translated to increased recovery speed in the face of perturbations such as data distribution shifts throughout phases of training\. This yields an improved router\-load entropy convergence in the reported 1\.8B ablation and reduced balancing failures in our training runs compared to linear routers\.

![Refer to caption](https://arxiv.org/html/2605.05365v1/figures/average_entropy_recovery.png)Figure 4:Normalized router\-load entropy, averaged over MoE layers, as a function of training step from initialization of a 1\.8B\-parameter experimental model\. For each global batch and layer, letpip\_\{i\}denote the fraction of routed tokens assigned to expertii, withEEtotal experts\. We reportH​\(p\)/ln⁡EH\(p\)/\\ln E, whereH​\(p\)=−∑i=1Epi​ln⁡piH\(p\)=\-\\sum\_\{i=1\}^\{E\}p\_\{i\}\\ln p\_\{i\}is the Shannon entropy of the empirical expert\-load distribution\.
#### II\-A3ZAYA1 Residual Scaling

The final architectural change in ZAYA1\-8B is residual scaling\. We apply a learned biasblb\_\{l\}and gating coefficientα∈ℝD\\alpha\\in\\mathbb\{R\}^\{D\}both to the residual stream and to the output of each layer before the residual connection:

Res\-scale​\(x\)\\displaystyle\\text\{Res\-scale\}\(x\)=α​x\+β,\\displaystyle=\\alpha x\+\\beta,\(6\)xl\+1=Res\-scaler​e​s​\(xl\)\\displaystyle x\_\{l\+1\}=\\text\{Res\-scale\}\_\{res\}\(x\_\{l\}\)\+Res\-scaleo​u​t​\(Layer​\(RMSnorm​\(xl\)\)\)\\displaystyle\+\\text\{Res\-scale\}\_\{out\}\\left\(\\text\{Layer\}\(\\text\{RMSnorm\}\(x\_\{l\}\)\)\\right\)
Different gating coefficients and biases are applied to the residual stream and to the layer outputs\. Residual scaling lets the model downweight parts of the residual stream and control how much prior residual information is retained\. In our experiments, residual scaling provides similar benefits to Qwen’s attention gating scheme\(Qiuet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib211)\), without the parameter or FLOP overhead of an explicit gating matrix\. Residual scaling also helps control residual\-norm growth through network depth, without observing any gradient vanishing\. We initializeα\\alphato ones andβ\\betato zeros, as this initializes the model with default residual connections\. Because residual scaling adds only4×L×D4\\times L\\times Dparameters, its parameter and FLOP overhead are comparable to LayerNorm and are negligible\.

Beyond these architectural changes, we trained with 16 experts and a hidden\-dimension expansion factor of 2\. This relatively fine\-grained expert configuration improved performance at fixed parameter count, consistent with prior work\(Teamet al\.,[2025b](https://arxiv.org/html/2605.05365#bib.bib141); DeepSeek\-AI,[2025b](https://arxiv.org/html/2605.05365#bib.bib188); Daiet al\.,[2024](https://arxiv.org/html/2605.05365#bib.bib184); Tianet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib198)\)\.

Unlike many contemporary MoEs, we trained with top\-k equal to 1 and without residual experts\(Rajbhandariet al\.,[2022](https://arxiv.org/html/2605.05365#bib.bib54); DeepSeek\-AI,[2025a](https://arxiv.org/html/2605.05365#bib.bib182)\)\. In our experiments, the improved routing expressiveness of the ZAYA1 router and the resulting expert specialization make a residual expert unnecessary\. FLOP\-matched experiments also favored top\-1 over higher top\-k when using the ZAYA1 router\. We hypothesize that the ZAYA1 router assigns more certain expert choices, with better expert specialization, so additional experts in parallel via top\-k are less useful\. When larger values ofkkare used, their contribution is further reduced by multiplication with the routing probability\. ZAYA1\-8B produces lower\-entropy routing probabilities per token than linear routers, consistent with more confident routing\. As a sanity check on expert redundancy, Appendix[D](https://arxiv.org/html/2605.05365#A4)measures within\-layer expert subspace overlap for ZAYA1\-8B and public MoE baselines\. ZAYA1\-8B is not an outlier toward higher expert overlap: its first\-projection input overlap is1\.45×1\.45\\timesthe random\-subspace baseline, close to Qwen3\-30B\-A3B’s1\.48×1\.48\\times, while its output\-projection overlap is intermediate among the compared MoEs\. For attention, we used CCGQA with a query compression rate of2×2\\timesand a KV compression rate of8×8\\times\. We applied RoPE\(Suet al\.,[2023](https://arxiv.org/html/2605.05365#bib.bib120)\)to half the channels in each head, leaving the other half without position embeddings\. ZAYA1\-8B was trained with the Gemma3 tokenizer\.

Table[I](https://arxiv.org/html/2605.05365#S2.T1)summarizes core architectural hyperparameters of the final release configuration\.

## IIIPretraining and Midtraining

ZAYA1\-8B was initialized from Zyphra’s ZAYA1 base architecture and trained through pretraining, context\-extension midtraining, and SFT on an AMD MI300X cluster equipped with the AMD Pensando Pollara networking stack\. Full details of the base\-model pretraining system, hardware, checkpointing, context parallelism, and AMD\-specific optimizer and kernel work are provided in\(Anthonyet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib156)\)\.

Table[II](https://arxiv.org/html/2605.05365#S3.T2)summarizes the main phases\. Base pretraining used a broad web\-crawl distribution with code, math, multilingual, and reasoning data mixed in progressively\. The second base pretraining phase upweighted code, math, reasoning, and instruction\-formatted data while still training at 4K context length\. We then ran a reasoning\-focused midtrain phase at 32K context for 1\.2T tokens at a RoPE base frequency of 1M\. This was followed by an SFT phase at 131K context for 660B tokens at a RoPE base frequency of 5M\. We believe that training for a large number of tokens at longer contexts significantly improves the model’s native long\-context capabilities and thus provides a stronger base for post\-training and RL\. The substantial reduction in prefill FLOPs we obtained through using CCA was instrumental in making this feasible at our compute scale\.

TABLE II:Training recipe summary\. Base\-pretraining details are summarized here for context and described in detail in\(Anthonyet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib156)\)\.Table[III](https://arxiv.org/html/2605.05365#S3.T3)reports coarse data categories for the reasoning\-focused midtrain and SFT\. Percentages are normalized over the nonzero mixture weights in the data cards; we report only category\-level proportions rather than individual dataset names\. To specialize the model for reasoning and provide as strong a base for RL as possible we utilized a very high fraction of long\-CoT reasoning traces in the midtrain and SFT\.

TABLE III:Coarse midtraining data mixtures\. The 32K context\-extension mixture was trained for approximately 1\.2T tokens, while SFT was trained for approximately 660B tokens; percentages denote normalized mixture weights\. Individual source datasets are omitted\.For context extension, we used all\-gather KV context parallelism with two ranks at 32K and eight ranks at 131K\. CCA’s compressed KV representation kept activation and KV\-cache memory overhead low, while short asynchronous point\-to\-point exchanges handled the convolution and value\-shift boundary conditions introduced by CCA\. Across these phases, we trained with the Muon optimizer using AdamW RMS matching\(Jordanet al\.,[2024](https://arxiv.org/html/2605.05365#bib.bib209); Liuet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib167)\)\.

### III\-AReasoning\-aware pretraining and answer\-preserving trimming

Recent work suggests that introducing long chain\-of\-thought reasoning data during pretraining and midtraining, rather than only during post\-training, can produce gains that subsequent fine\-tuning does not recover\(Akteret al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib165)\)\. We follow this approach throughout ZAYA1\-8B’s training pipeline: every pretraining and midtraining phase included long\-CoT data and it was a majority of the mix for the midtraining phases\.

Including reasoning data at short pretraining contexts creates a practical challenge: reasoning traces from strong teacher models often exceed 10K tokens, with a long tail beyond 30K\. At the initial 4K context length, each example must be handled in one of three ways: \(i\) drop it entirely, losing the reasoning signal; \(ii\) truncate naively, often preserving the reasoning prefix while losing the answer and thereby training the model on reasoning that never reaches a conclusion; or \(iii\) preserve the answer while truncating part of the reasoning\. We use the third option and call the resulting scheme*answer\-preserving \(AP\) trimming*\.

Given a sample containing one or more assistant messages with<think\>\.\.\.</think\>reasoning blocks followed by a final\-answer section, AP\-trimming applies the following procedure to fit the sample within a target context budgetCC:

1. 1\.Keep unchanged\.If the full conversation fits withinCC, retain it as\-is\.
2. 2\.Trim the tail of the last reasoning block\.If the conversation does not fit, truncate the final assistant turn’s reasoning trace from the tail, immediately before the answer\. This preserves the start of the reasoning trace and the full answer section\. The retained reasoning length is chosen so that the full sample fits withinCC\.
3. 3\.Drop prior reasoning blocks\.For multi\-turn conversations, if step 2 is insufficient, remove the<think\>blocks of earlier assistant turns while preserving their answer sections, then re\-apply step 2\.
4. 4\.Drop the sample\.If the answer sections alone exceedCC, discard the sample\.

The core idea is to truncate from the tail of the reasoning trace rather than from the middle\. The beginning of a reasoning trace often contains problem decomposition, planning, and exploration of multiple approaches\. The tail is usually more local, consolidating the selected approach into the final answer\. Removing tail tokens therefore preserves more of the planning and decomposition signal while producing partial but coherent reasoning sequences whose beginning, truncated end, and final answer remain causally aligned\. The transition between truncated reasoning and the answer is distributionally artificial, but in practice we did not observe obvious artifacts: pass\-rate evaluations on reasoning benchmarks after pretraining and midtraining remained strong, and we did not identify a truncation\-specific failure mode in downstream evaluations\.

##### Stage\-aware re\-trimming

AP\-trimming is applied offline to each dataset at each context length where the data is used\. As the training pipeline advances through 4K pretraining, 32K midtraining, and 131K context\-extension SFT, we re\-trim each dataset to the corresponding context length and progressively retain longer reasoning traces\. Most reasoning datasets fit fully at 131K context, so late midtraining operates on near\-complete traces; early pretraining uses the most aggressive trimming\.

##### Relation to prior work

The closest related techniques operate during inference or RL rollout generation rather than during pretraining data construction\.\(Khatriet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib163)\)use forced length interruptions during RL rollouts: when a thinking trace approaches the budget, the environment appends an end\-of\-thinking phrase that forces the model to produce a final answer\.\(Yanget al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib202)\)use a similar mechanism for inference\-time thinking\-budget control\. Both methods operate on rollouts during training or generation, not on training data before consumption\. The closest training\-data analogue is the answer\-length\-filtered subset of\(Akteret al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib165)\), which retains examples whose answer length exceeds 4K tokens as a proxy for reasoning depth\. That is a selection strategy rather than a truncation strategy\. AP\-trimming addresses the complementary problem of using long\-CoT reasoning data at training contexts shorter than the natural trace length by truncating reasoning while preserving the answer section\.

## IVPost\-training

![Refer to caption](https://arxiv.org/html/2605.05365v1/x3.png)Figure 5:Schematic of our post\-training process for ZAYA1\-8B\. Post\-training progressed through SFT followed by four sequential RL stages\. The first stage built general reasoning capabilities on math and puzzles and was then followed by two stages of code \+ TTC training\. The model was then polished through a short behavioral RLHF phase which focused more on chat and user interaction\.Post\-training begins with SFT, followed by a four\-stage RL cascade\. The first three RL stages are almost entirely verifiable reasoning: a math/puzzle/TTC warmup, an RLVE\-Gym adaptive difficulty curriculum, and a two\-phase math\+code\+TTC stage\. We defer general chat, style, and instruction\-following optimization to the final behavioral RL stage\. This ordering prioritizes capability extraction from verifiable signals before applying preference and instruction\-following rewards\.

Two aspects of this ordering differ from common post\-training recipes\. First, reasoning RL is front\-loaded: most RL compute before behavioral RL is spent on verifiable math, puzzles, synthetic environments, and code\. Second, the code stage uses several synthetic auxiliary environments constructed from competitive\-programming references, including code input/output prediction, code reconstruction from test cases, and falsification\.

### IV\-ASupervised Fine\-Tuning

The SFT phase establishes the chat template used in subsequent post\-training, improves instruction following, and continues reasoning supervision at 131K context\. The stage consumed 660B tokens\. We use a supervised mixture spanning chat, instruction following, code, math, reasoning, tool\-calling traces, and TTC aggregation examples, but do not report individual dataset details\.

Because the SFT stage trains at 131K context, packing strategy mattered\. We use optimized best\-fit decreasing bin packing\(Dinget al\.,[2024](https://arxiv.org/html/2605.05365#bib.bib247)\)rather than naively streaming examples into fixed\-length windows and truncating at arbitrary boundaries\. The packer fills each 131K window with complete examples whenever possible; over\-length examples are handled by dataset\-specific preprocessing before packing rather than by training on arbitrary suffixes created by a fixed\-boundary truncation pass\. This avoided hallucination artifacts we observed when models were trained on endings of mechanically truncated packed sequences\.

SFT also introduces aggregation\-based examples used by Markovian RSA\. These examples present the model with a problem and several candidate reasoning tails, then train it to produce a single improved solution\. Section[VI\-B](https://arxiv.org/html/2605.05365#S6.SS2)describes this construction in detail\.

### IV\-BReinforcement Learning Cascade

Post\-training reinforcement learning is organized as a four\-stage cascade\. The cascade uses a shared algorithmic spine described in Section[IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1); individual stages differ in data, reward signal, and stage length\.

TABLE IV:RL cascade summary\. The first three stages emphasize verifiable reasoning\. Behavioral RL is run last to tune chat, style, and instruction\-following behavior\.#### IV\-B1Algorithmic Spine

All RL stages and subphases share a common algorithmic spine\. Per\-stage differences are confined to data, reward signal, and a small number of hyperparameters\.

##### PipelineRL

Rollout generation and gradient updates run fully asynchronously on disjoint GPU pools\(Pichéet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib155); Khatriet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib163)\)\. We allocate 2–5×\\timesmore rollout workers than trainer workers, balanced per workload to match average response length to actor update time so that neither pool stalls\. Trainer\-to\-rollout weight sync happens in place every 2 trainer iterations; in steady state, the rollout policy is bounded at 2 trainer updates behind the trainer policy\.

##### Trust region

DPPO Binary\-TV\. We replace PPO’s per\-token ratio clipping with Binary Total\-Variation trust\-region masking\(Qiet al\.,[2026](https://arxiv.org/html/2605.05365#bib.bib162)\)\. Tokens for which the policy\-divergence estimate exceeds a thresholdδ\\deltaare masked from the gradient while remaining tokens contribute as in standard policy\-gradient updates\. We useδ=0\.1\\delta=0\.1in production\. We tune this threshold against preserving the reward\-growth trajectory of an unconstrained baseline, selecting the largest value that did not produce unconstrained reward growth in early training\. The Binary\-TV variant uses a deterministic indicator over a single divergence threshold rather than the continuous TV penalty or the Top\-KKapproximation, and adds negligible overhead relative to standard PPO\.

##### Loss aggregation

Dr\-GRPO SMTSN\. Loss aggregation follows Dr\-GRPO\(Liuet al\.,[2024](https://arxiv.org/html/2605.05365#bib.bib148)\): sequence\-mean over token\-sum\-norm \(SMTSN\)\. Token\-level losses are summed within each rollout and then averaged across rollouts in the batch, rather than averaged per\-token\. This avoids the implicit length normalization in standard GRPO, which biases the gradient toward longer responses\.

##### Advantage estimation

MaxRL\. Advantages are computed as in\(Tajwaret al\.,[2026](https://arxiv.org/html/2605.05365#bib.bib13)\)\. For each prompt, we sample a group ofGGrollouts with task rewardsri∈\{0,1\}r\_\{i\}\\in\\\{0,1\\\}using dynamic sampling\(Yuet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib8)\)\. The advantage normalizes by the per\-prompt mean reward rather than the per\-prompt reward standard deviation:

A^i=ri−r¯r¯,\\hat\{A\}\_\{i\}=\\frac\{r\_\{i\}\-\\bar\{r\}\}\{\\bar\{r\}\},\(7\)wherer¯=1G​∑j=1Grj\\bar\{r\}=\\tfrac\{1\}\{G\}\\sum\_\{j=1\}^\{G\}r\_\{j\}is the group mean reward\. This corresponds to the variance\-reduced MaxRL estimator \(Tajwaret al\.\([2026](https://arxiv.org/html/2605.05365#bib.bib13)\), Algorithm 1\), which is unbiased for a truncated maximum\-likelihood objective rather than for expected reward and produces stronger gradient signal on harder prompts\. We use this normalization in all RL stages except the final behavioral RL stage \(Section[IV\-B7](https://arxiv.org/html/2605.05365#S4.SS2.SSS7)\), which uses standard GRPO with reward standard\-deviation normalization\.

##### Reward shape and length reward

Task rewards are binary across the cascade, with the exception of \(i\) RLVE\-Gym environments that yield continuous solve rates near difficulty thresholds \(Section[IV\-B4](https://arxiv.org/html/2605.05365#S4.SS2.SSS4)\) and \(ii\) the behavioral RL stage, which uses a normalized reward\-model score \(Section[IV\-B7](https://arxiv.org/html/2605.05365#S4.SS2.SSS7)\)\. All RL stages except behavioral RL also include the difficulty\-scaled length reward of Section[IV\-B2](https://arxiv.org/html/2605.05365#S4.SS2.SSS2), applied additively asΔ​ri\\Delta r\_\{i\}to the task reward at the numerator of the advantage only — the denominatorr¯\\bar\{r\}uses the unmodified task reward to avoid scale blow\-up\. The length\-reward coefficientccramps from a small initial value during reasoning warmup toc=1\.0c=1\.0during the math\+code\+TTC stage, where the production reasoning length is established\.

##### No KL in reward

The cascade applies no KL regularization to the reward; the trust region is enforced entirely by DPPO Binary\-TV\. In stress\-testing with high KL\-penalty coefficients, we observed a length\-dependent bias attributable to applying a signed sequence\-level log\-ratio reward term to stale or mixed\-policy rollouts under in\-flight weight sync; Section[IV\-F](https://arxiv.org/html/2605.05365#S4.SS6)describes the mechanism and possible mitigations\. The production cascade avoids this configuration entirely by relying on the DPPO trust region alone\.

##### Optimizer

All RL stages use Muon with momentum set to zero\(GLM\-5\-Teamet al\.,[2026](https://arxiv.org/html/2605.05365#bib.bib29)\), extending the GLM\-5 prescription of resetting the optimizer at each weight\-sync boundary into a fully momentum\-free regime\. Section[IV\-E](https://arxiv.org/html/2605.05365#S4.SS5)describes this choice in detail and discusses its motivation and memory implications\.

##### Hyperparameters

Across all five stages, the cascade uses minibatches of 128 prompts with rollout group sizeG=16G=16responses per prompt\. Per\-rollout maximum response length is 81,920 tokens, except for the first half of the reasoning\-warmup stage, which uses 65K\. The maximum aggregation\-prompt length is 20,480 tokens, sized to fit Markovian RSA round\-1 prompts containingC=4C=4candidate tails\. Trainer\-to\-rollout weight sync occurs every 2 trainer iterations\. The trainer requires 2 batches of completed rollouts to be available in the buffer before pulling, and the buffer is capped with oldest\-sample eviction; in steady state, on\-policy staleness is bounded at 2 trainer updates\. Learning rates are set per stage in the range2×10−62\\times 10^\{\-6\}to1×10−51\\times 10^\{\-5\}, with the smallest values used during behavioral RL\.

#### IV\-B2Token efficiency

To encourage concise reasoning, we combine aspects from ALP\(Xianget al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib239)\)and ShortRL\(Yuanet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib238)\)to create a group\-relative, difficulty\-scaled length reward\.

Given a prompt with rollout group sizeGG, response rewardri∈\{0,1\}r\_\{i\}\\in\\\{0,1\\\}, and response lengthℓi\\ell\_\{i\}, we compute the group solve ratep=1G​∑i=1Grip=\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}r\_\{i\}and the shortest correct response lengthℓmin\\ell\_\{\\min\}\. Similar to ShortRL, we define a linear length interpolation, with the distinction thatℓmax\\ell\_\{\\max\}is a constant:

λi\\displaystyle\\lambda\_\{i\}=12−clamp⁡\(ℓi−ℓminℓmax−ℓmin,0,1\),\\displaystyle=\\frac\{1\}\{2\}\-\\operatorname\{clamp\}\\\!\\left\(\\frac\{\\ell\_\{i\}\-\\ell\_\{\\min\}\}\{\\ell\_\{\\max\}\-\\ell\_\{\\min\}\},\\;0,\\;1\\right\)\\,,\(8\)λ~i\\displaystyle\\tilde\{\\lambda\}\_\{i\}=\{12if​ℓi≤ℓmin\+Tℓ,λiotherwise\.\\displaystyle=
Letk=∑irik=\\sum\_\{i\}r\_\{i\}denote the number of correct responses in the group\. We apply the length reward only when at least two responses in the group are correct, so that there is a nontrivial comparison among correct response lengths\. We adopt the following correctness and difficulty gate:

mi=𝟙​\[p\>p∗−Tacc\]⋅𝟙​\[p\>1G\]⋅𝟙​\[ri=1\],m\_\{i\}=\\mathbbm\{1\}\\\!\\left\[p\>p^\{\*\}\-T\_\{\\text\{acc\}\}\\right\]\\cdot\\mathbbm\{1\}\\\!\\left\[p\>\\tfrac\{1\}\{G\}\\right\]\\cdot\\mathbbm\{1\}\\\!\\left\[r\_\{i\}=1\\right\]\\,,\(9\)
where the termp∗p^\{\*\}denotes a running maximum solve rate for the corresponding data source or environment, the conditionp\>1/Gp\>1/Gis equivalent tok≥2k\\geq 2for integer\-valued binary rewards, andTaccT\_\{\\mathrm\{acc\}\}is a tolerance that prevents the length reward from activating far below the current observed capability frontier\. We additionally scale the bonus by the solve ratepp, attenuating the length penalty on difficult problems and amplifying it on easier problems\. The final additive reward is:

Δ​ri=c⋅p⋅mi⋅λ~i,\\Delta r\_\{i\}=c\\cdot p\\cdot m\_\{i\}\\cdot\\tilde\{\\lambda\}\_\{i\}\\,,\(10\)whereccis a scaling coefficient\. This rewardΔ​ri\\Delta r\_\{i\}is added to the task reward, biasing the policy toward shorter correct solutions while preserving task accuracy in our production runs\.

#### IV\-B3Reasoning Warmup

The first RL stage is a 232\-step reasoning warmup on math, puzzle, and TTC reasoning prompts\. Its purpose is to adapt the SFT model to long verifiable rollouts before the broader RLVE and math\+code stages\. The warmup set contains 84,604 rows and is deliberately biased toward hard prompts: retained examples have prior pass rate at most 0\.75, with most examples at substantially lower pass rates\. Responses in this stage are long, with median replay response length around 17\.6K tokens and a p90 near 30K tokens\.

TABLE V:Coarse composition of the reasoning\-warmup RL data\. Percentages are computed over 84,604 warmup rows\.Rewards are verifiable and task\-specific\. For math problems, the reward is based on final\-answer correctness after normalization\. For puzzle environments, the reward is supplied by the environment verifier\. TTC prompts are formatted to match the Markovian RSA workflow described in Section[VI](https://arxiv.org/html/2605.05365#S6), so the model begins RL already seeing aggregation\-based reasoning prompts\.

#### IV\-B4RLVE\-Gym Difficulty Curriculum

The second RL stage trains for 400 steps on 400 adaptive and verifiable problem generators from RLVE\-Gym\(Zenget al\.,[2025b](https://arxiv.org/html/2605.05365#bib.bib234)\)\. We integrated RLVE as a dataset in VeRL\(Shenget al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib235)\)\. Although this stage has fewer optimizer steps than the later math\+code stage, the average step is roughly twice as long because responses are long, with reasoning lengths around 50K tokens\. We use this stage to expose the model to a broad distribution of puzzle\-like verifiable environments while keeping each environment near the model’s current difficulty boundary\.

During training, we used an online scheduler for problem difficulty, and we balanced environment selection using a weighted sampler for which the least sampled environments get the highest weight\. Our difficulty scheduler differs slightly from the authors’ in that it uses a tighter bound on the difficultyddand allows regressions\.

Lety^\\hat\{y\}denote either the optimal solution or a reasonable heuristic when optimal solutions are intractable\. We define

ri\\displaystyle r\_\{i\}=\{1if​\|y^−y\|<ϵ,0otherwise,\\displaystyle=\(11\)δi\\displaystyle\\delta\_\{i\}=\{\+1if​r¯\>0\.7​and​dgroup=d,−1if​r¯=0,0otherwise,\\displaystyle=d\\displaystyle d←d\+δi,\\displaystyle\\leftarrow d\+\\delta\_\{i\},
whererir\_\{i\}is reward per rollout in a group,r¯=1G​∑iri\\bar\{r\}=\\tfrac\{1\}\{G\}\\sum\_\{i\}r\_\{i\}is the group pass rate,ϵ\\epsilonis an environment\-specific numerical tolerance used to determine whether the rollout answeryyis close enough to the targety^\\hat\{y\}to receive reward 1,ddis the current difficulty setting, anddgroupd\_\{\\text\{group\}\}is the observed difficulty of the last computed group\. We constrain updates todgroup=dd\_\{\\text\{group\}\}=dto prevent stale difficulties from affecting the pass rate estimate\.

Crucially, we use an initial tuning step to avoid training on difficulties that are too easy for the model, and we aim to maximize the information content during training by initializing all environments to a difficulty that gives a 0\.5 pass rate for the model\. This tuning process is an adaptive search problem, and the search space is essentially unbounded\. Some environments are solvable into the range ofd\>100d\>100, while others are rarely solvable even at 0\. For this reason, we rely on Thompson Sampling\(Thompson,[1933](https://arxiv.org/html/2605.05365#bib.bib237)\)as a reasonably efficient method to determine the 0\.5 solve rate crossing point for every environment\. We model the pass rate using the complement of the logistic curve as is commonly done in Item\-Response\-Theory \(IRT\)\(Lord,[1980](https://arxiv.org/html/2605.05365#bib.bib236)\)and we sample from the midpoint with anε\\varepsilon\-greedy approach\. Each verified response group yields an estimate of the pass rate\. A parameter pool is maintained as with Thompson Sampling and a single Gaussian prior onμ\\muandssis used for all environments based on empirically observed ranges\.

psuccess\\displaystyle p\_\{\\text\{success\}\}=σ​\(−d−μs\)=11\+e\(d−μ\)/s,\\displaystyle=\\sigma\\\!\\left\(\-\\frac\{d\-\\mu\}\{s\}\\right\)=\\frac\{1\}\{1\+e^\{\(d\-\\mu\)/s\}\}\\,,\(12\)𝚯\\displaystyle\\boldsymbol\{\\Theta\}=\{\(μm,sm\)\}m=1M\.\\displaystyle=\\\{\(\\mu\_\{m\},s\_\{m\}\)\\\}\_\{m=1\}^\{M\}\\,\.
Given a parameter pool, we perform weighted sampling proportional to the posterior weight of the candidates \(initialized as uniform\) inΘ\\Thetaand for each iteration we sample a candidate and use it to compute a difficultyddatptargetp\_\{\\text\{target\}\}:

𝒘\\displaystyle\\boldsymbol\{w\}=\(w1,…,wM\),∑m=1Mwm=1,\\displaystyle=\(w\_\{1\},\\ldots,w\_\{M\}\),\\qquad\\sum\_\{m=1\}^\{M\}w\_\{m\}=1\\,,\(13\)j\\displaystyle j∼Categorical​\(𝒘\),\\displaystyle\\sim\\mathrm\{Categorical\}\(\\boldsymbol\{w\}\)\\,,dj\\displaystyle d\_\{j\}=μj\+sj⋅log⁡\(1−ptargetptarget\),\\displaystyle=\\mu\_\{j\}\+s\_\{j\}\\cdot\\log\\left\(\\frac\{1\-p\_\{\\text\{target\}\}\}\{p\_\{\\text\{target\}\}\}\\right\)\\,,where𝒘\\boldsymbol\{w\}is the normalized posterior\-weight vector over the candidate logistic\-curve parameters in𝚯\\boldsymbol\{\\Theta\}\.

We useptarget=0\.5p\_\{\\text\{target\}\}=0\.5, the maximum Fisher information point of the logistic model\(Lord,[1980](https://arxiv.org/html/2605.05365#bib.bib236)\), withε\\varepsilon\-greedy exploration to0\.5±0\.250\.5\\pm 0\.25\. We then perform rollouts and verification at the sampled difficulty and close the loop by updating and renormalizing the posterior:

wj\\displaystyle w\_\{j\}∝wj⋅Binomial​\(k;G,psuccess,j\),\\displaystyle\\propto w\_\{j\}\\cdot\\mathrm\{Binomial\}\\\!\\left\(k;\\,G,\\,p\_\{\\mathrm\{success\},j\}\\right\),\(14\)j∈\{1,…,M\}\.\\displaystyle\\hskip 20\.00003ptj\\in\\\{1,\\ldots,M\\\}\.
wherepsuccess,jp\_\{\\text\{success\},j\}is the current estimate for candidatejj\. Groups are generated asynchronously using vLLM with the previous phase’s frozen model weights\. If the effective sample size of the pool falls below a threshold, we resample with replacement fromΘ\\Theta, aggregate the observation history as a recency\-weighted sum of successes and failures, then re\-initialize likelihoods\.

This curriculum is intended to maximize useful verifier signal\. Environments that are too easy produce mostly positive groups and little policy\-gradient information; environments that are too hard produce mostly negative groups\. The initial calibration and online difficulty updates keep each environment near a solvable but non\-saturated regime, making the stage a bridge between the narrower reasoning warmup and the broader math\+code\+TTC RL stage\.

#### IV\-B5Math, Code, and Test\-Time Compute RL

The third RL stage is the main capability\-building stage of the cascade\. It combines olympiad\-level math, competitive\-programming code, Markovian RSA aggregation prompts, PaCoRe continuation prompts, and synthetic auxiliary code environments\. We run this stage in two phases: a 384\-step general math\+code\+TTC phase, followed by a 464\-step code\-focused phase\.

Table[VI](https://arxiv.org/html/2605.05365#S4.T6)summarizes the two data mixtures\. Phase 1 contains 18,656 rows and balances math and code while introducing TTC and PaCoRe variants\. Phase 2 contains 12,092 rows and increases the code share while retaining math TTC data\.

TABLE VI:Coarse composition of the math\+code\+TTC RL stage\. Phase 1 uses 18,656 rows; phase 2 uses 12,092 rows\. Percentages are grouped from source tags and rounded\.The auxiliary code environments are constructed by transforming competitive\-programming references into multiple verifiable tasks per source problem\. Each seed problem contains a problem statement, input/output specification, accepted reference implementations, rejected or incorrect implementations when available, and test cases\. From these seeds, we construct three auxiliary task families:

1. 1\.CodeI/O prediction\(Liet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib12)\)\. Given code and a set of inputs, the model predicts the outputs; in the reverse direction, given code and outputs, the model proposes inputs that produce them\. Output\-prediction rewards use exact normalized agreement with the reference execution\. Input\-prediction rewards execute the reference program on the generated input and check that the target output is produced while satisfying the input schema\.
2. 2\.CodeARC reconstruction\(Weiet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib15)\)\. Given a problem description, input/output specification, and example test cases, the model synthesizes code\. The verifier compiles or executes the generated solution and checks it against held\-out tests\.
3. 3\.Falsification\.Given a specification and a candidate implementation, the model must find an input that falsifies the implementation relative to the specification or a trusted correct implementation\. The verifier checks that the generated input is valid and that it induces a disagreement or specification violation\.

These tasks target algorithmic reasoning primitives rather than only end\-to\-end competitive\-programming solving\. CodeI/O emphasizes execution tracing and inverse reasoning over program behavior\. CodeARC emphasizes synthesis from sparse behavioral evidence\. Falsification emphasizes adversarial test construction and spec\-implementation comparison\. All three are binary\-verifiable and therefore fit the same RL objective as math and puzzle prompts\.

TTC prompts are included in the same RL stream\. For Markovian RSA examples, the prompt contains the original problem and a small set of candidate reasoning tails\. The policy generates a single aggregated solution and receives the standard verifiable reward for the final answer or produced code\. This lets the stage train both ordinary single\-rollout problem solving and the aggregation workflow used at inference time\.

#### IV\-B6Agentic task scope

ZAYA1\-8B does not include a dedicated multi\-turn agentic RL stage in this release\. We include some supervised agent, tool, and SWE traces during SFT, but the RL cascade is primarily optimized for verifiable reasoning, math, code, and instruction\-following behavior\. As a result, we expect agentic benchmarks such as BFCL\-v4 andτ2\\tau^\{2\}to lag models whose post\-training explicitly emphasizes multi\-turn tool use\. Scaling agentic data and agentic RL is left for future releases\.

#### IV\-B7Behavioral RL

The final RL stage tunes general chat behavior, style, and instruction following after the verifiable\-reasoning stages have established the model’s math and code capabilities\. Behavioral RL uses standard GRPO with reward standard\-deviation normalization rather than the MaxRL normalization used in the verifiable\-reasoning stages\. It also does not use the length reward from Section[IV\-B2](https://arxiv.org/html/2605.05365#S4.SS2.SSS2)\.

We first train for one epoch on 80K behavioral prompts\(Wanget al\.,[2024](https://arxiv.org/html/2605.05365#bib.bib241),[2025](https://arxiv.org/html/2605.05365#bib.bib242)\)\. This stage improves general response quality and chat behavior without changing the reasoning\-focused data distribution of the earlier stages\.

We then run two instruction\-following stages, each for one epoch\. The first uses simpler instruction\-following prompts; the second uses more difficult IFBench\-like prompts\. For these IF stages, the reward is gated by a binary instruction\-following checker\. If the completion fails the IF gate, its reward is set to zero\. If it passes the gate, the completion is scored by the reward model\. This prevents the reward model from assigning positive reward to fluent responses that fail the explicit instruction constraints\.

### IV\-CRL Infrastructure

##### Router replay

The single most important MoE\-specific change for RL stability is router replay: the trainer reuses the expert routing assignments produced by vLLM at rollout time during its own forward pass over the rollout, rather than recomputing routing decisions from scratch\. Even with the precision settings in Section[IV\-D](https://arxiv.org/html/2605.05365#S4.SS4), small numerical differences between the rollout engine and the trainer can produce different routing decisions for tokens near a router decision boundary\. In an MoE with top\-1 routing, a token routed to experteinferencee\_\{\\rm inference\}at rollout time but to a different expertetrain≠einferencee\_\{\\rm train\}\\neq e\_\{\\rm inference\}at gradient time produces different per\-token logits, which corrupts the on\-policy gradient\. Router replay eliminates this source of mismatch: by pinning the trainer’s expert selection to the rollout\-time decision, enforcingetrain≡einferencee\_\{\\rm train\}\\equiv e\_\{\\rm inference\}, the gradient is computed against the same expert sequence that produced the rollout\. We discuss the SNR view of this mismatch in Section[VII\-C](https://arxiv.org/html/2605.05365#S7.SS3)\.

In practice, vLLM writes per\-token and per\-layer expert assignment indices to a shared memory buffer during decode\. The write is overlapped with decode work to avoid slowing rollout generation\. Assignments are then packed alongside the rest of the rollout batch \(token IDs, masks, etc\.\) when the batch is shipped to the trainer, so router replay introduces no separate transport step\.

##### Memory and recompute strategy

For long\-rollout training the dominant memory pressure comes from activations\. We combine host\-side activation offloading with gradient checkpointing: the hidden state tensors from each layer that autograd must retain for backward are temporarily offloaded to CPU memory during the forward pass, while checkpointed layer interiors discard their forward activations and reconstruct them by rerunning the layer forward during backward\. This trades extra backward\-time compute and host\-to\-device traffic for substantially lower peak GPU activation memory\. In this configuration we use FSDP shard size 4 under the FSDP2 sharding strategy with sequence parallelism disabled; at this model size and per\-rank rollout length, the extra cross\-rank communication from sequence parallelism and ring attention is not worth the memory savings\.

##### Packing and dynamic batching

We use sequence packing and variable length attention for the trainer\. This allows the trainer to run with dynamic microbatch sizing: rather than fixing the number of rollouts per microbatch, we fix a token budget of 131,072 tokens per GPU per microbatch and pack rollouts into microbatches up to this budget\. This avoids paying for the longest rollout in a fixed\-rollout\-count microbatch when most rollouts are shorter, and keeps GPU memory utilization stable across batches even when rollout\-length distributions shift between training stages\. We additionally rebalance pack assignments across GPUs so that microbatches on different ranks contain comparable token counts; without this, the slowest rank gates the entire step, since synchronous gradient accumulation must wait for all ranks to finish\. With balancing, per\-step variance across ranks is small enough that no rank consistently bottlenecks training\.

##### Buffer management

The trainer pulls completed rollouts from a shared buffer with a maximum capacity bound and oldest\-sample eviction\. As described in Section[IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1), the trainer requires 2 batches of completed rollouts to be available before pulling\. This combination keeps the rollout pool from running ahead of the trainer \(which would inflate staleness and waste rollout compute\), while ensuring the trainer is never blocked waiting for fresh rollouts under our 2–5×\\timesrollout\-to\-trainer ratio\.

### IV\-DPrecision

![Refer to caption](https://arxiv.org/html/2605.05365v1/x4.png)Figure 6:Per\-token probability comparison \(log scaled frequency\): vLLM \(engine, used for rollout generation\) vs\. trainer prefill \(used for gradients\) with incremental precision improvements\.BF16: naive uniform BF16 implementation in inference and prefill\.BF16\+FP32: addition of selective upcasting of a subset of operations to FP32\.BF16\+FP32\+RR: additional improvement from implementing router replay on trainer prefill from cached indices of rollout\. Each point is a token from a 128\-prompt,G=16G=16evaluation batch with 4K\-token completions on ZAYA1\-8B\. Identity line shown \(dashes\)\. For BF16\+FP32\+RR, KL divergence=1\.3×10−4=1\.3\\times 10^\{\-4\}, Pearsonr\>0\.9996r\>0\.9996\.The default precision regime for ZAYA1\-8B RL is BF16 weights and activations, with a small set of operations promoted to FP32\. The subset of operations in FP32 is identical between the trainer and vLLM, which is necessary for engine\-trainer log\-prob agreement within the regime needed for stable PipelineRL training \(see Figure[6](https://arxiv.org/html/2605.05365#S4.F6)\)\.

##### FP32 operation set

The following operations run with FP32 numerics on both trainer and inference paths:

- •Loss/output:fused cross\-entropy accumulation and LM\-head matmul\.
- •Attention/normalization:CCA cache state, QK\-norm, QK\-mean, and RMSNorm; see Section[II](https://arxiv.org/html/2605.05365#S2)\.
- •Routing/residuals:router softmax and residual stream additions\.

The LM\-head FP32 promotion follows precedents in\(Khatriet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib163)\)and\(Chenet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib169)\)\. The remaining FP32 ops were added incrementally to close engine\-trainer log\-prob mismatch observed in early training runs; without them, mismatch produces grad\-norm spikes and stale\-policy artifacts under PipelineRL\.

##### FP16 detour

Recent work argues that training–inference mismatch in RL fine\-tuning can arise directly from floating\-point precision, and proposes using FP16 uniformly rather than BF16 as a simple way to reduce mismatch\(Qiet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib108)\)\. However, in our comparisons, we found that a hardened BF16 path with a small matched FP32 operation set on both the rollout engine and trainer achieved the engine–trainer agreement needed for stable PipelineRL training, while retaining BF16’s dynamic\-range advantages\. We therefore use BF16 weights and activations by default, promote only the operations listed above to FP32, as described previously\.

##### Rollout Engine\-trainer match

Figure[6](https://arxiv.org/html/2605.05365#S4.F6)compares per\-token log\-probabilities computed by vLLM \(used during rollout generation\) and by the trainer’s prefill \(used to compute gradients\)\. At our default precision setup, the two distributions are nearly identical: KL divergence=1\.3×10−4=1\.3\\times 10^\{\-4\}and Pearsonr\>0\.9996r\>0\.9996over a 128\-prompt,G=16G=16batch with 4K\-token completions\. This level of agreement is a precondition for stable PipelineRL training under our staleness regime; without the FP32 op set above, agreement degrades substantially and downstream training is unstable\.

### IV\-EOptimizer

Letℒt​\(W\)\\mathcal\{L\}\_\{t\}\(W\)denote the actor training loss for the parameter matrixWWon rollout batchtt, and letgt=∇Wℒt​\(W\)g\_\{t\}=\\nabla\_\{W\}\\mathcal\{L\}\_\{t\}\(W\)be the corresponding actor gradient\. Letmtm\_\{t\}denote Muon’s first moment buffer, and letℳ​\(⋅\)\\mathcal\{M\}\(\\cdot\)denote the Muon orthogonalization step via Newton\-Schulz\(Jordanet al\.,[2024](https://arxiv.org/html/2605.05365#bib.bib209)\)\. Standard Muon uses the update

mt\\displaystyle m\_\{t\}=μ​mt−1\+gt,\\displaystyle=\\mu m\_\{t\-1\}\+g\_\{t\}\\,,\(15\)Δ​Wt\\displaystyle\\Delta W\_\{t\}=−ηt​ℳ​\(mt\),\\displaystyle=\-\\eta\_\{t\}\\mathcal\{M\}\(m\_\{t\}\)\\,,whereηt\\eta\_\{t\}is the learning rate at optimizer steptt\. For actor updates, we setμ=0\\mu=0somt=gtm\_\{t\}=g\_\{t\}andΔ​Wt=−ηt​ℳ​\(gt\)\\Delta W\_\{t\}=\-\\eta\_\{t\}\\mathcal\{M\}\(g\_\{t\}\)\. Thus each actor update depends on the current rollout batch and does not carry first\-moment optimizer state across rollout batches\. For embedding and output\-head parameters, including the word embedding and LM head, we use AdamW rather than Muon\. For the remaining matrix\-valued actor weights, we use momentum\-free Muon\.

The motivation differs from pretraining\. Compared to AdamW, Muon stands as a more compute efficient optimizer that is well suited to the RL setting where updates to parameters are sparse\(Mukherjeeet al\.,[2026a](https://arxiv.org/html/2605.05365#bib.bib2)\)\. Furthermore, in next\-token pretraining, adjacent minibatches are drawn from a comparatively stationary data distribution, so momentum can average compatible gradient directions across steps\. In RL, each actor update is tied to a rollout batch whose prompts, sampled trajectories, rewards, and generating policy snapshot may differ from neighboring batches\. Following\(GLM\-5\-Teamet al\.,[2026](https://arxiv.org/html/2605.05365#bib.bib29)\), we view optimizer\-state reset as a useful stability heuristic for asynchronous RL\. Our setting extends this idea: instead of resetting the optimizer state only at rollout\-engine weight\-sync boundaries, we make every actor update momentum\-free\. This makes each update depend only on the current rollout batch while retaining Muon’s normalized matrix update,Δ​Wt=−ηt​ℳ​\(gt\)\\Delta W\_\{t\}=\-\\eta\_\{t\}\\mathcal\{M\}\(g\_\{t\}\), rather than a raw SGD step \(Δ​Wt=−ηt​gt\\Delta W\_\{t\}=\-\\eta\_\{t\}g\_\{t\}\)\. We treat this as a practical stability and memory choice, not as evidence that zero momentum is generally optimal for RL\.

This choice also avoids maintaining a persistent first\-moment buffer for the Muon\-updated actor weights during RL, reducing optimizer\-state memory relative to momentum Muon\. We did not include a controlled optimizer ablation in this report\. A direct comparison against momentum Muon, AdamW, and SGD updates is left for future work\.

### IV\-FMonitoring and maintaining stability

Reward and KL diagnostics describe the policy’s optimization dynamics but do not reflect the content of generated rollouts\. We monitor a small set of auxiliary rollout\-level statistics during RL training to fill this gap\. A subset of these statistics also act as reward gates, zeroing a rollout’s task reward when its content is flagged as degenerate\.

##### Streaming compressibility

Our primary canary is a sliding\-window LZ77 compressibility metric computed per chunk on the raw token\-ID bytes of each rollout\. Compression useszlibwith a210=10242^\{10\}=1024\-byte LZ77 window \(wbits=\-10\), level\-1 deflate, andZ\_SYNC\_FLUSHbetween chunks; the compressor is stateful, so each chunk’s compression ratio reflects compressibility relative to recent history bounded by the LZ77 window rather than whole\-sequence redundancy\. Each rollout is divided into fixed\-size chunks ofCCtokens \(with the final short chunk merged into its predecessor to avoidZ\_SYNC\_FLUSHoverhead inflating short\-tail ratios\), and the per\-chunk compression ratio

rc=compressed bytesc−flush overheadraw token\-ID bytescr\_\{c\}=\\frac\{\\text\{compressed bytes\}\_\{c\}\-\\text\{flush overhead\}\}\{\\text\{raw token\-ID bytes\}\_\{c\}\}\(16\)is computed for each chunkcc\.

A smallrcr\_\{c\}indicates a chunk that compresses well against its preceding context, which is the signature of degenerate repetition or copying: the model has emitted a span of tokens already present in the LZ77 window\. More generally, as is noted by\(Leeet al\.,[2026](https://arxiv.org/html/2605.05365#bib.bib245)\), an effective compression algorithm also serves as a computable upper bound on Kolmogorov Complexity\(Li and Vitányi,[2019](https://arxiv.org/html/2605.05365#bib.bib246)\), and both ends of the compressibility spectrum could arguably be filtered as either low information content or purely random\. We choose LZ77 in particular over simpler n\-gram or token counting methods because it takes into account sequence\-level matching within the window, whereas language and domain\-level n\-gram biases can complicate simpler presence/frequency metrics\. We flag a rollout if any chunk satisfiesrc<τrepeatr\_\{c\}<\\tau\_\{\\text\{repeat\}\}, with a conservativeτrepeat=0\.05\\tau\_\{\\text\{repeat\}\}=0\.05in production\. Flagged rollouts have their task reward zeroed before advantage computation, so the policy receives no positive learning signal for producing degenerate text even when the verifier accepts the \(technically correct\) final answer at the end of a long repetitive trace\. The per\-chunk granularity allows reward zeroing on rollouts where degenerate spans appear at any position rather than attempting to rely on coarser, full response compressibility\.

##### Rare\-token monitoring

As an independent signal, we track the fraction of tokens in each rollout whose token IDs fall in the topXX% of the tokenizer’s ID range\. This is a lightweight proxy for unusual or rarely used tokens in our tokenizer\. In production monitoring we track several cutoffs, including 10%, 5%, 2%, and 1%, and use the top\-10% token\-ID region for gibberish canaries\. A rising rare\-token fraction often precedes other failure indicators and is cheap to compute\.

##### Operational use

The low\-ratio repetition canary and rare\-token\-fraction statistics are computed per batch during RL training and visible alongside reward and KL in WandB\. The repetition canary additionally runs as a reward\-zeroing gate: rollouts that exceed the low\-ratio threshold have their rewards zeroed before advantage computation, regardless of verifier outcome\. Canary signals do not adjust learning rate or any other optimizer setting\.

##### Length bias from signed KL\-in\-reward under pipeline RL

Beyond rollout\-level canaries, we also monitored response\-length growth, which exposed an interaction between PipelineRL training and a sequence\-level signed log\-ratio reward penalty\. In early stress tests combining the two, we observed runaway response\-length growth: rollouts grew progressively longer over training without corresponding reward improvement\. Our working explanation is specific to this estimator and aggregation choice\. In pipeline RL, long completions can span multiple generator\-policy snapshots: early tokens may be sampled from a stale generator policyπgen,c\\pi\_\{\\text\{gen\},c\}that isΔc\\Delta\_\{c\}trainer updates behind the current actorπθ\\pi\_\{\\theta\}, while later tokens may be sampled from fresher snapshots with smallerΔc\\Delta\_\{c\}\.

The commonly usedK1K\_\{1\}\-estimator log\-ratio KL term is

lt=log⁡πθ​\(yt∣ht\)−log⁡πgen,c​\(t\)​\(yt∣ht\)\.l\_\{t\}=\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid h\_\{t\}\)\-\\log\\pi\_\{\\text\{gen\},c\(t\)\}\(y\_\{t\}\\mid h\_\{t\}\)\.\(17\)For tokens sampled fromπgen,c​\(t\)\\pi\_\{\\text\{gen\},c\(t\)\}, this signed log\-ratio is negative in expectation whenever the current policy differs from the generator policy:

𝔼yt∼πgen,c​\(t\)\[lt\]=−DKL\(πgen,c​\(t\)\(⋅∣ht\)∥πθ\(⋅∣ht\)\)≤0\.\\mathbb\{E\}\_\{y\_\{t\}\\sim\\pi\_\{\\text\{gen\},c\(t\)\}\}\[l\_\{t\}\]=\-D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\pi\_\{\\text\{gen\},c\(t\)\}\(\\cdot\\mid h\_\{t\}\)\\,\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid h\_\{t\}\)\\right\)\\leq 0\.\(18\)For fresh tokens with small policy lag,lt≈0l\_\{t\}\\approx 0\. If these terms are aggregated into a sequence\-level scalar,

Sseq=∑tlt,S\_\{\\text\{seq\}\}=\\sum\_\{t\}l\_\{t\},\(19\)and subtracted from reward as

A=r−βKL​Sseq,A=r\-\\beta\_\{\\mathrm\{KL\}\}S\_\{\\text\{seq\}\},\(20\)then stale off\-policy tokens can create a positive reward offset\. Longer completions can accumulate more negative signed log\-ratio terms, and when the resulting sequence\-level adjusted advantage is broadcast back to all tokens, stale\-prefix terms can affect the learning signal assigned to later suffix tokens\.

This produces a length\-dependent bias through two interacting effects\.

Stale\-prefix contamination\.Longer sequences can contain more stale prefix tokens contributing negativeltl\_\{t\}, makingSseqS\_\{\\text\{seq\}\}more negative\. Since the signed log\-ratio term enters as−βKL​Sseq\-\\beta\_\{\\mathrm\{KL\}\}S\_\{\\text\{seq\}\}, the negative sequence sum acts as a positive reward offset, inflating the advantage for longer sequences independent of task quality\.

Staleness\-dependent penalty scale\.The magnitude of the signed log\-ratio can also depend on chunk stalenessΔc\\Delta\_\{c\}\. Relatedly, Bartoldson derives a first\-order EMA\-reference approximation for asynchronous RL in which the log\-ratio between the current policy and aΔ\\Delta\-old inference policy can be interpreted as a surrogate for KL regularization against an EMA reference, under local linearity and first\-order Taylor assumptions\(Bartoldson,[2026](https://arxiv.org/html/2605.05365#bib.bib168)\)\. This suggests thatΔ\\Deltacan change the effective scale of a stale\-policy log\-ratio penalty\. In our setting, we use this only as intuition for lag\-dependent penalty strength; the length\-bias mechanism itself follows from applying a signed off\-policy log\-ratio at sequence level and subtracting it from reward\.

This mechanism should be distinguished from true KL regularization\. A KL divergence is non\-negative by construction, whereas the sampled signed log\-ratioltl\_\{t\}can be arbitrarily negative on individual samples and is negative in expectation under the generator distribution whenπgen,c​\(t\)≠πθ\\pi\_\{\\text\{gen\},c\(t\)\}\\neq\\pi\_\{\\theta\}\. The severity depends on both absolute staleness and within\-sequence policy heterogeneity\. In\-flight synchronization can create prefix–suffix heterogeneity by allowing one completion to span multiple generator snapshots; holding the generator fixed for the entire completion removes this specific coupling, but does not remove the off\-policy signed\-log\-ratio length offset if the fixed generator is stale relative to the trainer\.

##### Possible mitigations

Two practical mitigations target the specific stale\-prefix coupling described above\.Chunk\-local signed\-log\-ratio isolationaggregates the signed log\-ratio within each chunk rather than across the full sequence, so stale\-prefix terms do not directly contaminate the advantage assigned to fresher suffix chunks:

𝒜c=Areward−βKL​Sc,Sc=∑t∈clt\.\\mathcal\{A\}\_\{c\}=A\_\{\\text\{reward\}\}\-\\beta\_\{\\mathrm\{KL\}\}S\_\{c\},\\qquad S\_\{c\}=\\sum\_\{t\\in c\}l\_\{t\}\.\(21\)This localizes the bias but does not by itself turn the off\-policy signed log\-ratio into a true KL penalty\.

Staleness rescalingis an additional heuristic: divide the chunk term by an empirical staleness scaleg​\(Δc\)g\(\\Delta\_\{c\}\), withg​\(Δc\)\>0g\(\\Delta\_\{c\}\)\>0, to reduce variation in effective penalty strength across chunks generated at different lags:

𝒜c=Areward−βKL⋅1g​\(Δc\)​∑t∈clt\.\\mathcal\{A\}\_\{c\}=A\_\{\\text\{reward\}\}\-\\beta\_\{\\mathrm\{KL\}\}\\cdot\\frac\{1\}\{g\(\\Delta\_\{c\}\)\}\\sum\_\{t\\in c\}l\_\{t\}\.\(22\)A simple first\-order choice isg​\(Δc\)=max⁡\(1,Δc\)g\(\\Delta\_\{c\}\)=\\max\(1,\\Delta\_\{c\}\), motivated by the local\-linear lag dependence in Bartoldson’s EMA approximation\(Bartoldson,[2026](https://arxiv.org/html/2605.05365#bib.bib168)\), but the correct scale is implementation\- and dynamics\-dependent\.

For ZAYA1\-8B we did not implement either mitigation in production\. Instead, we removed KL\-in\-reward entirely and rely on the DPPO Binary\-TV trust region \(Section[IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1)\) for trust\-region enforcement\. This was sufficient for the training\-stability properties we required and avoided tracking chunk boundaries and per\-chunk generator staleness\. We document the mechanism here because it may arise in asynchronous or pipeline RL systems that combine stale or mixed\-policy rollouts, a signedK1K\_\{1\}\-estimator log\-ratio in the reward, sequence\-level aggregation, and broadcast of the resulting adjusted advantage\.

## VResults

![Refer to caption](https://arxiv.org/html/2605.05365v1/x5.png)Figure 7:Comparison of ZAYA1\-8B performance against open\-weight reasoning models on various evaluations\. The under\-bar plots model sizes in active and total parameters on a log scale to give a sense of the scale of the various models\.Results are organized into three tables\. Table[VII](https://arxiv.org/html/2605.05365#S5.T7)compares ZAYA1\-8B against open\-weight reasoning models at comparable scale\. Table[VIII](https://arxiv.org/html/2605.05365#S5.T8)extends to open\-weight models in the 26B–119B total\-parameter range\. Table[XI](https://arxiv.org/html/2605.05365#S6.T11)reports test\-time compute comparisons against open\-weight models in the 235B–671B range plus Gemini\-2\.5 Pro and GPT\-5\-High\.

### V\-AEvaluation Protocol

Unless otherwise noted, ZAYA1\-8B results are measured with the Zyphra evaluation harness\. In\-class comparator models are run in the same harness when feasible, using each model’s recommended sampling settings from its model card\. For ZAYA1\-8B single\-rollout reasoning evaluations, we use temperature 1\.0, top\-p 0\.95, top\-k \-1, and benchmark\-specific maximum generation lengths\. For thinking\-mode Qwen comparators, we mirror the recommended thinking\-mode settings from the corresponding model card\. Results reported from external release materials are marked with†\. TTC evaluations use the checkpoint immediately following the math\+code\+TTC RL stage and before the final behavioral\-RL polishing stage; the latter targets chat style, instruction following, and preference behavior rather than additional math/code/TTC capability\.

For pass\-rate evaluations in the Zyphra harness, we report averages over multiple samples per problem\. Math benchmarks, including AIME, HMMT, IMO\-AnswerBench, and APEX\-shortlist, are reported as avg@64\. Code benchmarks, including LiveCodeBench\-family tasks, are reported as avg@16\. GPQA\-Diamond andτ2\\tau^\{2\}are reported as avg@16 unless otherwise noted\. MMLU\-Pro, BFCL\-v4, HLE, IFEval, IFBench, EQBench, and Creative Writing are reported as mean@1 or as the benchmark’s standard single\-run score\. We use avg@k to mean the mean correctness over k independently sampled completions, estimating single\-sample pass rate under the stated sampler; it is not best\-of\-k/pass@k unless explicitly stated\. Markovian RSA results use the TTC protocol in Section[VI](https://arxiv.org/html/2605.05365#S6); its token counts are total newly generated decode tokens and exclude prompt/prefill tokens\. Results from external release materials may use different sampling and reporting protocols and are marked with†\.

### V\-BMain Results: In\-Class Comparison

∗Gemma4 includes 4B additional embedding parameters as a part of its total\.

†Qwen3\.5\-4B LiveCodeBench\-v6 scores taken from release materials\.

TABLE VII:In\-class comparison against models of comparable sizes\. ZAYA1\-8B used the following sampling settings:T=1\.0T\{=\}1\.0, top\-p=0\.95p\{=\}0\.95, top\-kkdisabled for math, knowledge, and instruction;T=0\.6T\{=\}0\.6, top\-p=0\.95p\{=\}0\.95, top\-k=20k\{=\}20for code, agentic, and style\. We used the recommended sampling settings in the model cards for the other models in this table\. EQBench and Creative Writing v3 use the official judge,anthropic/claude\-3\.7\-sonnet\.Table[VII](https://arxiv.org/html/2605.05365#S5.T7)compares ZAYA1\-8B against Qwen3\-4B\-Thinking\-2507, Qwen3\.5\-4B, and Gemma\-4\-E4B\-it\.

### V\-CScaling Comparison: Larger Open\-Weight Models

Table[VIII](https://arxiv.org/html/2605.05365#S5.T8)compares ZAYA1\-8B against larger open\-weight reasoning models: Arcee\-Trinity\-Mini, Nemotron\-3\-Nano, OLMo\-3\.1\-32B\-Think, Qwen3\-Next\-80B\-A3B\-Think, Intellect\-3, and Mistral\-Small\-4\-119B\-2603\.

TABLE VIII:Scaling comparison against larger open\-weight reasoning models, ordered by total parameter count\. All numbers are run on the Zyphra evaluation harness\.∗LCB\-v6 denotes the 2025\-02–2025\-05 LiveCodeBench\-v6 split\.

### V\-DTest\-Time Compute Scaling

Table[XI](https://arxiv.org/html/2605.05365#S6.T11)compares ZAYA1\-8B with Markovian RSA test\-time compute against substantially larger reasoning models\. With the headline Markovian RSA configuration \(β\\beta= 40K,τ\\tau= 4K,TT= 2,NN= 16,CC= 4\), ZAYA1\-8B reaches 91\.9 on AIME’25 and 89\.6 on HMMT’25 Feb\.

### V\-EEffect of Post\-Training

To quantify the effect of post\-training, we compare the 131K SFT checkpoint against the final ZAYA1\-8B checkpoint using the same evaluation harness and sampling settings in Table[IX](https://arxiv.org/html/2605.05365#S5.T9)\. This comparison measures the aggregate effect of the RL cascade rather than isolating the contribution of each individual stage\. We do not report per\-stage ablations in this release\.

TABLE IX:Aggregate effect of post\-training\. SFT and final ZAYA1\-8B checkpoints are evaluated with the same harness and benchmark\-specific sampling settings\. This table reports the aggregate effect of the post\-training recipe; it is not a per\-stage ablation\.

## VITest\-Time Compute

Test\-time compute \(TTC\) scaling — increasing inference compute per problem to improve answer quality — has become an important axis of capability scaling for reasoning models, alongside model scale and training compute\. Two recent lines of work motivate the design space considered here\.\(Venkatramanet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib157)\)introduce Recursive Self\-Aggregation \(RSA\), a TTC scheme that maintains a population of candidate reasoning chains and refines them through repeated aggregation: at each iteration, the model is shown a random subset of candidates and produces an improved candidate, which seeds the next iteration’s population\. Empirically, RSA allows smaller open\-weight models to approach the performance of larger reasoning models when given sufficient inference compute\.\(Aghajohariet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib158)\)introduce Markovian Thinker, a reformulation of the RL thinking environment in which the policy reasons in fixed\-size chunks with bounded carryover state between chunks, decoupling thinking length from context size\. Their key observation is that long\-context reasoning can be factorized in a Markovian way: with sufficient training, a model can sometimes carry forward only the information needed in a bounded textual state and continue reasoning indefinitely\.

We introduce*Markovian RSA*, a TTC method that combines RSA’s recursive candidate aggregation with the bounded\-workspace principle of Markovian Thinker\. We integrate it into ZAYA1\-8B’s training pipeline so the model is trained to use the same workflow at inference\. The method has three components: an algorithm that includes both RSA and Markovian\-Thinker for chunked reasoning as special cases \(Section[VI\-A](https://arxiv.org/html/2605.05365#S6.SS1)\), a training\-time integration that supplies verifier\-free aggregation examples for SFT and verifiable aggregation prompts for RL \(Section[VI\-B](https://arxiv.org/html/2605.05365#S6.SS2)\), and an inference\-time scaling profile with bounded per\-iteration aggregation context, capped attention costs, and predictable throughput \(Section[VI\-C](https://arxiv.org/html/2605.05365#S6.SS3)\)\.

### VI\-AMarkovian RSA

##### Algorithm

Given a problemqqand a base policyπ\\pi, Markovian RSA proceeds overTTaggregation rounds, indexedt=0,1,…,Tt=0,1,\\ldots,T\. Each round maintains a population ofNNcandidate reasoning traces\. At roundt=0t=0, the model generatesNNindependent rollouts directly fromqq, each with a per\-rollout thinking budgetβ\\beta\. Each rollout’s reasoning trace is then reduced to its lastτ\\tautokens, which we call the*tail*\. We writetailτ​\(y\)\\text\{tail\}\_\{\\tau\}\(y\)for the operation that returns the finalτ\\tautokens of reasoning traceyy, withτ≤β\\tau\\leq\\beta\.

For roundst≥1t\\geq 1, the algorithm operates on tails from the previous population\. To generate each new candidate, it samplesC≤NC\\leq Ntails uniformly at random, concatenates them into an aggregation prompt, and asks the model to reason over the candidate solutions and produce a single improved solution\. The model generates a new reasoning trace under the same per\-rollout budgetβ\\beta\. The trace is again reduced to its finalτ\\tautokens, and the resulting tail enters the population for roundtt\. This process repeats until roundTT, after which the final answer is extracted from the final round’s outputs using the standard answer\-extraction procedure\. The aggregation prompt simply asks the model to consider the candidates and produce the best solution; it does not require specialized parsing or verifier feedback\.

Both Markovian RSA and full\-chain RSA bound per\-rollout generation cost:β\\betacaps the number of tokens any single candidate generates\. The difference is what gets passed forward\. Full\-chain RSA passes the full reasoning chain111One can add a summarization step to full\-chain RSA to keep aggregation prompts short\. We focus on fixed\-tail forwarding because it gives a simple, bounded aggregation context without requiring an additional summarization model or parsing step\., so the aggregation prompt at roundt≥1t\\geq 1containsCCchains, each with length up toβ\\beta\. Markovian RSA passes only the finalτ\\tautokens of each chain, withτ≤β\\tau\\leq\\betachosen independently\. This decouples per\-rollout thinking depth from aggregation\-context size:β\\betacontrols how long each candidate may reason, whileτ\\taucontrols how much of that reasoning is carried into the next round\. Settingτ≪β\\tau\\ll\\betaallows larger per\-rollout thinking budgets while keeping aggregation prompts small\. As a result, decode\-attention cost, prefill\-attention cost, and KV\-cache footprint are bounded by configuration constants rather than by reasoning length\.

##### Default configuration

For ZAYA1\-8B, we use\(N,C,T\)=\(16,4,2\)\(N,C,T\)=\(16,4,2\)withβ\\betaset per workload andτ\\tauchosen as a fraction ofβ\\beta\(typicallyτ≤β/2\\tau\\leq\\beta/2\)\. Bothβ\\betaandτ\\taucan be tuned per deployment to trade off per\-round thinking depth against total inference budget\.

##### Inference profile

Markovian RSA changes the inference workload from a single long, position\-growing decode into a sequence of bounded\-context batched decoding stages\. At round0, the model generatesNNindependent candidates from the original problem, so decode runs at batch sizeNNrather than batch size11\. At each later aggregation round, the model again generatesNNcandidates, but each candidate conditions only on the problem andCCcarried\-forward tails of length at mostτ\\tau\. Thus the aggregation prefill length is bounded by

Lprefill≤\|q\|\+C​τ\+O​\(1\),L\_\{\\mathrm\{prefill\}\}\\leq\|q\|\+C\\tau\+O\(1\),\(23\)and the per\-candidate decode length is bounded byβ\\beta, independent of the total amount of reasoning generated across all rounds\. This gives a stable serving profile: prefill is short and predictable at every stage, decode uses high\-throughput batched generation, and no stage attends over the full reasoning history\.

This profile differs from both single\-rollout long\-CoT and full\-chain RSA\. A single long rollout has batch size11and a decode position that grows with the full reasoning length\. Full\-chain RSA supports batched candidate generation, but its aggregation prefill grows withC​βC\\betabecause it forwards full reasoning chains\. Markovian RSA keeps the batched candidate\-generation structure of RSA while replacing full\-chain forwarding with bounded tail forwarding, so increasingβ\\betaincreases per\-candidate thinking depth without increasing aggregation\-context length\.

TABLE X:Inference\-profile view of Markovian RSA\. Markovian RSA preserves the batched candidate\-generation structure of RSA while bounding the state forwarded between rounds\. SettingT=0T=0givesNNindependent responses with no aggregation; this becomes Best\-of\-NNonly if an external selector, verifier, or answer\-selection rule is applied\. SettingC=1C=1gives the Delethink bounded\-continuation regime, and settingτ=β\\tau=\\betarecovers full\-chain RSA\.
##### Special cases

Markovian RSA contains several common TTC regimes as special cases:

- •Parallel sampling /NNresponses\.SettingT=0T=0removes aggregation and producesNNindependent responses\. If a verifier, answer\-selection rule, or external scoring model is applied to these responses, this reduces to a Best\-of\-NNevaluation; otherwise it is simply parallel sampling\.
- •Full\-chain RSA\.Settingτ=β\\tau=\\betaforwards each full reasoning chain between rounds, recovering RSA\. In this limit, aggregation prefill grows with the full reasoning budget\.
- •Delethink bounded continuation\.SettingC=1C=1removes cross\-candidate aggregation while retaining bounded carryover\. Each candidate continues from its own tail, giving a parallel version of Markovian/Delethink chunked reasoning\. This isolates the effect of bounded continuation from the additional effect of cross\-candidate aggregation\.

##### Comparison with PaCoRe

\(Huet al\.,[2026](https://arxiv.org/html/2605.05365#bib.bib159)\)introduced PaCoRe, a related multi\-round parallel\-reasoning scheme that also bounds per\-round aggregation context\. PaCoRe compacts each trajectory by extracting its final\-answer or conclusion section and passing this extracted message forward between rounds\. Markovian RSA instead passes the finalτ\\tautokens of the reasoning trace itself as the carry\-forward state, regardless of whether the trajectory reached a final conclusion\. The two methods share the same goal of bounding aggregation context across rounds and differ in compaction mechanism: PaCoRe uses model\-structured final\-answer extraction, while Markovian RSA uses a fixed\-size suffix of generated reasoning\.

In practice, we also evaluate a PaCoRe hybrid compaction variant: when a candidate reaches a post\-think answer section, we pass that compact answer forward; otherwise, we fall back to passing the partial reasoning chain\. This hybrid keeps the compact\-message advantage of PaCoRe when candidates finish, while avoiding the need to setβ\\betalarge enough for every branch to reach a final answer\.

![Refer to caption](https://arxiv.org/html/2605.05365v1/figures/markovian_rsa_algo_trimmed.png)Figure 8:One round of Markovian RSA\. From a population ofNNcandidate reasoning traces \(left\), we extract the finalτ\\tautokens of each trace as its tail\. To produce each new candidate for the next round, we sampleCCtails uniformly at random and present them to the model as candidate solutions in an aggregation prompt\. The model produces a new reasoning trace, whose tail joins the next round’s population\. Aggregation context size and per\-round attention cost depend only onCCandτ\\tau, and are independent of the per\-rollout thinking budgetβ\\beta\.

### VI\-BTraining\-Time Integration

A TTC method may be more effective when the model is trained on the workflow it uses at inference\. Markovian RSA’s aggregation prompt presents the model with a problem and several candidate reasoning tails, then asks it to produce a single improved solution\. This behavior is rare in standard reasoning\-model training data, where each example typically consists of one problem and one solution\. To train ZAYA1\-8B for Markovian RSA scaling, we construct aggregation\-based examples from existing expert\-model reasoning data and include them in SFT and RL\.

##### SFT data construction from expert rollouts

Many open\-source reasoning datasets used during midtraining and SFT include multiple expert\-model rollouts per problem, often withn=8n=8rollouts \(e\.g\.,OpenMathReasoning,rStar\-Coder, internal reasoning gym and enigmata data\)\. For each problemqqwith rollouts\{y1,…,yn\}\\\{y\_\{1\},\\ldots,y\_\{n\}\\\}from a teacher model, we construct a round\-0\-to\-round\-1 aggregation example as follows: sampleCCrollouts from thennavailable; extract their tails\{tailτ​\(yi1\),…,tailτ​\(yiC\)\}\\\{\\text\{tail\}\_\{\\tau\}\(y\_\{i\_\{1\}\}\),\\ldots,\\text\{tail\}\_\{\\tau\}\(y\_\{i\_\{C\}\}\)\\\}; form an aggregation prompt containingqqand theCCtails; and condition the teacher to produce a new aggregated rollout under the same prompt\. The resulting aggregated rollout, including its reasoning trace and final answer, becomes the SFT target\.

This construction has two practical advantages\. It is offline and reuses existing rollout pools: no new expert\-model inference is needed for each round\-0 sample beyond the aggregation step\. It also does not require a verifier: the teacher’s aggregated rollout is used as the target regardless of whether the underlying answer is verifiable\. This makes the technique applicable to puzzle, code, and reasoning domains where the post\-think content is itself the answer and where final\-answer\-only aggregation strategies such as PaCoRe’s message compaction are not directly applicable\.

##### RL stage integration

During RL, Markovian RSA examples are folded into the standard prompt distribution and treated like other RL prompts\. Two variants are used during the math\+code\+TTC stage \(Section[IV\-B5](https://arxiv.org/html/2605.05365#S4.SS2.SSS5)\):

- •Expert\-aggregation\.Round\-1 prompts are constructed from expert\-model rollouts as described above\. The policy generates an aggregated rollout and is rewarded against the verifiable target\.
- •Self\-aggregation\.For prompts where rollouts from the current SFT checkpoint or a prior\-stage RL checkpoint are available, round\-1 prompts are constructed from those self\-rollouts\. The policy aggregates over its own reasoning traces, or over traces from its predecessor\.

In both variants, the aggregation example is a standard RL prompt: the policy generates a single rollout, and verifiable reward is applied to its final answer\. No special multi\-round RL machinery is required; the round structure is encoded in the prompt construction rather than in the gradient update\. We currently train on round\-1 self\-aggregation\. Round\-2\-and\-beyond self\-aggregation, where the policy aggregates rollouts from a prior\-stage version of itself in an online buffer, is a natural extension left for future work\.

##### Domain coverage

Aggregation\-based training data is included for math, code, reasoning gym, and enigmata puzzle problems\. Directly aggregating over reasoning tails is useful in domains where the post\-think content is the answer rather than a separate boxed result\. This allows the same approach to apply across domains regardless of answer format\.

### VI\-CInference\-Time Scaling

ModelActiveTotalAIME’25HMMT’25 Feb\.LCB\-v6\*ZAYA1\-8B \(single rollout\)0\.7B8\.0B88\.382\.765\.0ZAYA1\-8B \+ Markovian RSA \(40K/4K\)0\.7B8\.0B91\.989\.669\.2‡DeepSeek\-R1\-0528†37B671B87\.579\.468\.7Qwen3\-235B\-A22B\-Thinking\-2507†22B235B92\.383\.974\.1Gemini\-2\.5 Pro†––88\.082\.572\.5DeepSeek\-V3\.2†37B671B93\.192\.5–GPT\-5\-High†––94\.688\.3–
∗LCB\-v6 denotes the 2025\-02–2025\-05 LiveCodeBench\-v6 split\.

‡For LCB\-v6 on the same pre\-behavioral checkpoint after math\+code\+TTC RL, Markovian RSA improves ZAYA1\-8B from 65% single\-rollout to 69\.2%, while our PaCoRe hybrid compaction variant reaches 71\.1%\. This variant is not an exact implementation of PaCoRe: when a candidate reaches a post\-think answer section, we pass that compact answer forward; when it does not, we fall back to passing the partial reasoning chain rather than dropping the candidate\. Since the model was trained with both RSA and PaCoRe\-type aggregation examples, we do not attribute the gap to training exposure alone\.

TABLE XI:ZAYA1\-8B single\-rollout and TTC numbers in this table are evaluated on the pre\-behavioral checkpoint after math\+code\+TTC RL and before the final lightweight behavioral\-RL polishing stage, using the Zyphra evaluation harness\. The final behavioral stage targets chat style, instruction following, and preference behavior rather than math/code/TTC capability\. ZAYA1\-8B \+ Markovian RSA uses the 40K/4K configuration from Section[VI\-C](https://arxiv.org/html/2605.05365#S6.SS3)\. Numbers for comparator models marked†are taken from external sources\. DeepSeek\-R1\-0528, Qwen3\-235B\-A22B\-Thinking\-2507, and Gemini\-2\.5 Pro are from the Qwen3\-235B\-A22B\-Thinking\-2507 model card\(Qwen Team,[2025](https://arxiv.org/html/2605.05365#bib.bib100)\)\. DeepSeek\-V3\.2 is from the DeepSeek\-V3\.2 technical report\(DeepSeek\-AI,[2025c](https://arxiv.org/html/2605.05365#bib.bib9)\); the GPT\-5\-High row is reproduced from the comparison table in that report rather than from an OpenAI release table\.We evaluate Markovian RSA’s inference\-time scaling along two axes — per\-rollout reasoning budgetβ\\betaand tail sizeτ\\tau— with the configuration described in Section[VI\-A](https://arxiv.org/html/2605.05365#S6.SS1)\(N,C,T\)=\(16,4,2\)\(N,C,T\)=\(16,4,2\)\. The sampling settings are reported in Table[XII](https://arxiv.org/html/2605.05365#S6.T12)\. We also compare against full\-chain RSA\(Venkatramanet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib157)\), recovered as theτ=β\\tau=\\betalimit of our algorithm\. The final scores reported in this section are mean correctness over the final\-round candidate outputs, not best\-of\-NNunless explicitly stated\.

##### Headline result

With Markovian RSA at\(β,τ,T,N,C\)=\(40​K,4​K,2,16,4\)\(\\beta,\\tau,T,N,C\)=\(40\\mathrm\{K\},4\\mathrm\{K\},2,16,4\), ZAYA1\-8B reaches 91\.9% on AIME’25 and 89\.6% on HMMT’25 Feb\. These runs use temperature 1\.0, top\-p 1\.0, and a 40K\-token final\-response budget\. The result holds while carrying forward only a 4K\-token tail between aggregation rounds, approximately one\-tenth of the per\-rollout reasoning budget\.

##### Configuration sweep

Table[XII](https://arxiv.org/html/2605.05365#S6.T12)reports accuracy across four Markovian RSA configurations, alongside theC=1C=1Markovian Thinker baseline described in Section[VI\-A](https://arxiv.org/html/2605.05365#S6.SS1)\. AtT=2,N=16T=2,N=16, andC=4C=4, increasing the per\-rollout reasoning budgetβ\\betafrom 8K to 16K to 40K improves both benchmarks at fixed tail size: AIME’25 advances from 86\.5% to 88\.8% to 91\.9%, and HMMT’25 advances from 80\.8% to 87\.1% to 89\.6%\. HMMT’25 is especially responsive to longer per\-rollout reasoning, with a 6\.3\-point gain fromβ=8​K\\beta=8\\text\{K\}toβ=16​K\\beta=16\\text\{K\}\.

TABLE XII:Markovian RSA configuration sweep on AIME’25 and HMMT’25\.TTis the number of aggregation rounds,NNis the population size,CCis the number of candidates sampled per aggregation prompt,β\\betais the per\-rollout reasoning budget, andτ\\tauis the carry\-forward tail size\. Markovian RSA rows use temperature 1\.0 and top\-p 1\.0 with a 40K\-token final\-response budget\. The Markovian baseline row usesC=1C=1, removing cross\-candidate aggregation while retainingN=16N=16independent chunked rollouts\.
##### Evaluation scope

The algorithmic definition, training construction, and serving profile above describe the Markovian RSA method\. The remainder of this subsection reports empirical TTC scaling results, measuring how accuracy changes with the per\-rollout reasoning budgetβ\\beta, carried\-forward tail lengthτ\\tau, aggregation depthTT, and aggregation method under a fixed population size\.

##### Generated\-token accounting

We report realized total generated decode tokens separately from per\-worker trajectory lengths\. Markovian RSA generates multiple candidates in parallel at each non\-final stage, so a per\-worker or per\-stage average length is not the total decode cost of the method\. For a problemqq, letgs,j​\(q\)g\_\{s,j\}\(q\)be the number of newly generated tokens from workerjjin generation stagess, and letnsn\_\{s\}be the number of workers in that stage\. The realized generated\-token cost is

D​\(q\)=∑s∑j=1nsgs,j​\(q\)=∑sns​g¯s​\(q\),D\(q\)=\\sum\_\{s\}\\sum\_\{j=1\}^\{n\_\{s\}\}g\_\{s,j\}\(q\)=\\sum\_\{s\}n\_\{s\}\\bar\{g\}\_\{s\}\(q\),\(24\)whereg¯s​\(q\)\\bar\{g\}\_\{s\}\(q\)is the average generated length per worker in stagess\. This count includes newly generated candidate and aggregation tokens across all workers, but excludes the original problem prompt, aggregation\-prompt prefill, and copied carry\-forward tails\.

Under this accounting, with the final response budget of 40K, the reported AIME’25/HMMT’25 Markovian RSA evaluations use approximately 440K generated decode tokens per problem for the 16K/4K configuration and approximately 740K generated decode tokens per problem for the 40K/4K configuration\. These are realized averages from the evaluation runs, not worst\-case caps; they depend on early stopping, benchmark, sampling settings, and implementation details\. We include them to avoid conflating per\-worker trajectory length with total TTC cost\.

![Refer to caption](https://arxiv.org/html/2605.05365v1/figures/markovian_rsa_scaling_aime25_new.png)
![Refer to caption](https://arxiv.org/html/2605.05365v1/figures/markovian_rsa_scaling_hmmt25_new.png)

Figure 9:Accuracy vs\. realized total newly generated decode tokens per problem for Markovian RSA configurations on AIME’25 \(left\) and HMMT’25 \(right\)\. The token axis excludes the original problem prompt, aggregation\-prompt prefill, and copied carry\-forward tails, and corresponds to∑sg¯s​\(q\)\\sum\_\{s\}\\bar\{g\}\_\{s\}\(q\), using notations from \([24](https://arxiv.org/html/2605.05365#S6.E24)\), averaged over problems\. Curves correspond to the configurations in Table[XII](https://arxiv.org/html/2605.05365#S6.T12):β=8​K,τ=4​K\\beta=8\\text\{K\},\\tau=4\\text\{K\};β=16​K,τ=4​K\\beta=16\\text\{K\},\\tau=4\\text\{K\};β=40​K,τ=8​K\\beta=40\\text\{K\},\\tau=8\\text\{K\}; andβ=40​K,τ=4​K\\beta=40\\text\{K\},\\tau=4\\text\{K\}\. All curves useN=16N=16,C=4C=4,T=2T=2, temperature 1\.0, and top\-p 1\.0\. Allocating budget to longer per\-rollout reasoning yields more accuracy per generated token than allocating to longer tail carryover in this sweep\.
##### Tail size and iteration depth

Increasing the tail sizeτ\\taufrom 4K to 8K at fixedβ=40​K\\beta=40\\text\{K\}does not improve accuracy on AIME’25 or HMMT’25: the 4K\-tail configuration reaches 91\.9%/89\.6%, while the 8K\-tail configuration reaches 90\.8%/89\.2%\. Because these scores average over multiple generated candidates per problem, the comparison is less sensitive to a single unlucky rollout than a one\-sample evaluation\. We treat this as empirical evidence that, for these configurations and benchmarks, aggregation is not limited by retaining more than a 4K reasoning tail\.

This should not be read as a general claim that tail length never matters\. On harder benchmarks, the aggregation depth \(TT\), the diversity of the candidate responses \(NNandCC\),β\\beta, andτ\\taucan all contribute to saturating the model’s capacity\. We also evaluate higher\-compute Markovian RSA settings on APEX\-shortlist, a harder capacity\-ceiling benchmark\. Table[XIII](https://arxiv.org/html/2605.05365#S6.T13)reports the three APEX configurations used for the light, high, and extra\-high modes shown in Figure[10](https://arxiv.org/html/2605.05365#S6.F10)\. The strongest setting, withT=8T=8,N=32N=32,C=4C=4,β=32​K\\beta=32\\mathrm\{K\}, andτ=4​K\\tau=4\\mathrm\{K\}, reaches 51\.8% on APEX\-shortlist\. This setting uses approximately 5\.5M newly generated decode tokens per problem, so we treat it as a capability\-ceiling evaluation rather than a recommended deployment setting\.

TABLE XIII:APEX\-shortlist Markovian RSA configurations\. These three rows define the light, high, and extra\-high modes shown in Figure[10](https://arxiv.org/html/2605.05365#S6.F10)\.![Refer to caption](https://arxiv.org/html/2605.05365v1/x6.png)Figure 10:APEX\-shortlist performance under the light, high, and extra\-high Markovian RSA configurations from Table[XIII](https://arxiv.org/html/2605.05365#S6.T13)\. The extra\-high configuration reaches 51\.8% using approximately 5\.5M newly generated decode tokens per problem\. GPT\-OSS\-120B \(high\) and DeepSeek\-V3\.2 comparator scores are taken from the MathArena leaderboard\(Dekonincket al\.,[2026](https://arxiv.org/html/2605.05365#bib.bib99)\)\. These external comparator scores are shown for context only and may use different inference protocols\.
##### Markovian baseline and the value of aggregation

TheC=1C=1row of Table[XII](https://arxiv.org/html/2605.05365#S6.T12)runs the algorithm with a single previous tail conditioning each new candidate: each of theN=16N=16candidates carries forward only its own bounded textual state, with no cross\-candidate aggregation\. This is a parallel version of chunked Markovian\-Thinker rollouts\. Two observations follow\. First, theC=1C=1baseline reaches 82\.1% on AIME’25 and 75\.0% on HMMT’25, indicating that bounded carryover preserves much of the model’s long\-CoT reasoning capability without aggregation — consistent with\(Aghajohariet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib158)\)’s finding that off\-the\-shelf reasoning models support Markovian traces zero\-shot\. Second, the gap between theC=1C=1baseline and the corresponding Markovian RSA configuration \(4\.4 points on AIME’25 and 5\.8 points on HMMT’25, both atβ=8​K\\beta=8\\text\{K\},τ=4​K\\tau=4\\text\{K\}\) quantifies the additional gain from cross\-candidate aggregation on top of bounded continuation\. The two effects compose: bounded carryover establishes that the inference workload can be capped without losing reasoning capability, and recursive aggregation provides further gains from cross\-candidate refinement\.

##### Comparison to full\-chain RSA

Markovian RSA recovers full\-chain RSA in the limitτ=β\\tau=\\beta, but the bounded\-tail setting substantially reduces aggregation\-prompt length\. Atβ=40​K\\beta=40\\mathrm\{K\}andC=4C=4, full\-chain RSA would carry up to160​K160\\mathrm\{K\}candidate\-state tokens per aggregation item, whereas Markovian RSA withτ=4​K\\tau=4\\mathrm\{K\}carries only16​K16\\mathrm\{K\}\. We therefore use Markovian RSA for the reported high\-budget evaluations\. A fully matched empirical comparison against full\-chain RSA is left for future work\.

##### Cross\-model comparison

Training for TTC\. The benefits of training for a TTC workflow are not specific to Markovian RSA\. Figure[11](https://arxiv.org/html/2605.05365#S6.F11)compares ZAYA1\-8B against Qwen3\-4B\-Thinking\-2507 under Markovian RSA, with both models running the same TTC procedure\. This comparison is not intended as a method evaluation, since both models use the same procedure\. We interpret any gap as evidence of training\-design effects rather than method differences: ZAYA1\-8B is trained from midtraining onward with TTC aggregation traces in midtraining, SFT, and RL; Qwen3\-4B\-Thinking\-2507 is not\.

![Refer to caption](https://arxiv.org/html/2605.05365v1/figures/markovian_rsa_qwen_compare_dejavu_serif.png)Figure 11:Test\-time compute scaling for ZAYA1\-8B and Qwen3\-4B\-Thinking\-2507\. Both models use the same TTC procedure \(Markovian RSA withT=2T=2,β=16​K\\beta=16\\mathrm\{K\}, andτ=4​K\\tau=4\\mathrm\{K\}\) with the final response budget of 40K for both the TTC and non\-TTC runs \(the recommended sampling parameters in the model card were used for the Qwen model\), so performance differences reflect model\-level differences in the ability to exploit aggregation\. ZAYA1\-8B saw long\-CoT reasoning data during midtraining and aggregation\-based data during SFT and RL; Qwen3\-4B\-Thinking\-2507 was not trained for this specific aggregation workflow\. We interpret the gap as evidence of training\-design effects rather than method differences\.
##### Serving profile and compute efficiency

Markovian RSA has a favorable serving profile because every stage is both batched and bounded\. Round\-0 generation runsNNindependent candidates in parallel\. Later aggregation rounds also run at batch sizeNN, with each item prefilling only the original problem plusCCcarried\-forward tails\. With the deployment settingN=16N=16,C=4C=4, andτ=4\\tau=4K, the candidate\-state portion of each aggregation prompt is bounded byC​τ=16C\\tau=16K tokens, plus the original problem and formatting overhead\. Prefill is therefore bounded and predictable at every round, while decode proceeds as batched generation with per\-candidate length capped byβ\\beta\.

This differs from both single long\-CoT and full\-chain RSA\. A single long rollout has batch size11, with decode position and KV\-cache length growing continuously with the full trace\. Full\-chain RSA batches candidates, but forwards full reasoning chains into aggregation prompts, so the carried candidate state grows withC​βC\\beta\. Atβ=40\\beta=40K andC=4C=4, full\-chain RSA would carry up to160160K candidate\-state tokens per aggregation item, whereas Markovian RSA withτ=4\\tau=4K carries only1616K, a90%90\\%reduction in carried candidate state before accounting for the shared problem prompt\. Markovian RSA therefore lets us increase per\-candidate reasoning depth throughβ\\betawithout increasing aggregation\-prefill length\.

This bounded\-context profile should be distinguished from total generated\-token cost\. Markovian RSA still spends a large aggregate decode budget because it generates many candidates across parallel workers and aggregation rounds\. For the 40K/4K configuration, the realized total is approximately 740K newly generated decode tokens per problem across all workers on the reported AIME’25/HMMT’25 evaluation runs\. We therefore compare TTC configurations using both active\-parameter×\\timestotal generated\-token cost and the serving\-side context profile: per\-candidate decode capped byβ\\beta, aggregation prefill bounded by\|q\|\+C​τ\|q\|\+C\\tau, and no stage attending over the full generated reasoning history\.

##### Recommended deployment configuration

For deployment, we recommend the 16K/4K configuration from Table[XII](https://arxiv.org/html/2605.05365#S6.T12)as a lower\-cost default\. It provides a strong accuracy–cost tradeoff while using substantially fewer realized decode tokens than 40K/4K: approximately 440K vs\. 740K generated tokens per problem on the reported AIME’25/HMMT’25 evaluations, excluding prompt, prefill, and copied\-tail tokens\. In our current serving setup, we also observed the lightweight Markovian RSA configuration \(T=2,N=8,C=4,β=8​K,τ=4​KT=2,N=8,C=4,\\beta=8\\mathrm\{K\},\\tau=4\\mathrm\{K\}, and final response budget 40K\) completing in roughly0\.4×0\.4\\timesthe wall\-clock time of our standardN=1N=1long\-reasoning baseline on the same evaluation harness\. We attribute this to serving the workload as bounded\-context batched decoding rather than as one long position\-growing trace\. We report this wall\-clock ratio as an implementation\-specific observation, not as a hardware\-independent throughput benchmark\. The 40K/4K configuration is reserved for capability\-ceiling evaluation\.

## VIIDiscussion

In this technical report, we presented ZAYA1\-8B, the first and smallest model in the ZAYA\-1 family\. ZAYA1\-8B is designed to maximize reasoning performance per active parameter, with a particular focus on reasoning\-intensive mathematics and coding\. In this target regime, the model is strongly competitive with systems that use far more active parameters, and reaches or exceeds the level of earlier frontier\-scale reasoning models such as DeepSeek\-R1\-0528 and Gemini\-2\.5 Pro on several challenging math and code benchmarks\.

With its native Markovian RSA TTC mode, ZAYA1\-8B approaches the mathematical performance of much larger frontier reasoning models such as Gemini\-2\.5 Pro, DeepSeek\-V3\.2, and GPT\-5\-High\. ZAYA1\-8B is also competitive with substantially larger open\-weight models including OLMo\-3\.1\-32B\-Think, Nemotron\-3\-Nano\-30B\-A3B, Intellect\-3, and Mistral\-Small\-4\-119B, with its clearest advantage on math/code reasoning density\.

We attribute ZAYA1\-8B’s performance to a combination of its architecture, cascaded RL pipeline, reasoning\-heavy training data, and an AMD training stack that supported long\-context pretraining, midtraining, and SFT\.

Finally, we introduced Markovian RSA, a test\-time scaling method that combines recursive self\-aggregation with bounded carry\-forward state\. The model generates and aggregates batches of candidate responses in parallel, while each aggregation round conditions only on a fixed number of bounded\-length reasoning tails\. This preserves RSA’s cross\-candidate refinement benefits while keeping aggregation context bounded, avoiding attention over the full generated reasoning history\.

We believe TTC is an especially promising avenue for smaller reasoning\-focused models and can make them competitive with substantially larger models in active\-parameter×\\timesgenerated\-token cost for some reasoning workloads\. If TTC methods reliably convert additional generated tokens into accuracy, then active\-parameter count and inference\-time reasoning tokens become complementary axes of scaling\.

Below, we discuss observations and lessons learned from the ZAYA1\-8B training process\.

### VII\-ARL Sample Efficiency

The reasoning RL portion of ZAYA1\-8B post\-training is short relative to the pretraining and midtraining compute that precedes it\. The main verifiable\-reasoning cascade uses 232 reasoning\-warmup steps, 400 RLVE\-Gym steps, 384 math\+code\+TTC phase\-1 steps, and 464 math\+code\+TTC phase\-2 steps, for 1,480 total reasoning\-RL update steps before behavioral RL\. Despite this small number of optimizer steps, the aggregate post\-training gain is large\. Relative to the SFT checkpoint, we observe roughly a 20–30 point gain on AIME\-like math evaluations and roughly a 10\-point gain on LiveCodeBench\-v6, with smaller but positive changes on many other benchmarks\. Achieving gains of this size with ordinary midtraining or SFT would likely require substantially more data and optimization compute, and identifying the right supervised distribution would itself be nontrivial\.

One interpretation is that RL takes a KL\-minimal path to optimality during training\(Shenfeldet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib248)\), keeping the parameters relatively close to their initial values, compared to SFT, which may move the model arbitrarily in parameter space\. Thus, the pretraining and midtraining stages appear to install most of the latent capabilities needed for reasoning; RL then changes the policy’s sampling distribution so that these capabilities are expressed more reliably under long generation budgets\. This view is consistent with our pass@k observations: in our previous base\-model report, the reasoning checkpoint showed strong pass@64 behavior at a 30K generation budget, while the post\-RL model moves much of that capability into average sampled performance at longer generation lengths\. However, unlike prior work\(Cuiet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib160)\)that argued RL simply uses up the inherent entropy of the midtraining checkpoint, we observed pass@k staying stable or even increasing during RL training where it did not hit the performance ceiling\.

The sample efficiency of RL remains an open problem\. One plausible contributor is that the optimization problem in verifiable RL is qualitatively different from dense cross\-entropy training\. In SFT, every target token supplies a supervised gradient\. In verifiable RL, useful signal is concentrated in trajectory\-level outcomes, group\-relative comparisons, verifier acceptance, and trust\-region filtering: only a subset of prompts and sampled rollouts produces informative contrast for a given update\. This makes the effective learning signal sparse over prompts and trajectories, before considering how the optimizer transforms that signal into parameter updates\. We discuss optimizer\-dependent sparsity in the resulting parameter deltas in Section[VII\-B](https://arxiv.org/html/2605.05365#S7.SS2)\.

We do not claim here that SFT on the same data would fail to recover the RL gains; we have not run this experiment in a controlled form for this release\. The safer interpretation is that short verifiable\-RL runs can move a strong pretrained/SFT model into a nearby region of policy space that is difficult to identify from supervised next\-token prediction alone\.

### VII\-BMomentum\-Free RL Optimization

ZAYA1\-8B uses Muon with momentum set to zero for matrix\-valued actor weights during RL\. This is unconventional from the perspective of pretraining, where momentum and adaptive optimizer state are standard tools for stabilizing dense next\-token\-prediction training\. In our RL setting, however, momentum\-free Muon worked well and was compatible with the stability requirements of PipelineRL\. We did not run a controlled optimizer ablation for this report, so we present this as an empirical recipe choice rather than as a general optimizer recommendation\.

Our motivation is that asynchronous verifiable RL differs from pretraining in both signal structure and stationarity\. Each actor update is tied to a rollout batch with sampled trajectories, verifier outcomes, group\-relative advantages, and a generating policy snapshot that may differ from neighboring batches\. In this setting, carrying first\-moment state across batches can average gradient information collected under different prompt sets, sampled solutions, reward outcomes, and policy lags\. This may be useful when the persistent component of the gradient dominates minibatch noise, but it may be less useful when the stationarity horizon of the RL signal is short relative to the optimizer’s momentum horizon\. We therefore use momentum\-free Muon as a simple optimizer\-state\-reset variant: each actor update depends only on the current rollout batch while retaining Muon’s normalized matrix update\.

This choice is related to recent observations that RL finetuning of LLMs can update a relatively small subset of parameters and that memory\-light optimizers can remain competitive in RL settings\(Mukherjeeet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib161),[2026b](https://arxiv.org/html/2605.05365#bib.bib166)\)\. In an informal matched\-step RL diagnostic, SGD updates left 99\.51% of parameters exactly unchanged and 99\.94% below10−510^\{\-5\}, while AdamW left 92\.82% exactly unchanged and 96\.40% below10−510^\{\-5\}\. We do not use this diagnostic to claim that SGD, momentum\-free Muon, or sparse updates are generally preferable, nor to claim that RL is universally sparse relative to SFT\. Rather, it suggests that in our setting the effective update can be highly concentrated, making optimizer\-state design a relevant practical consideration\.

Momentum\-free Muon also reduces optimizer\-state memory because no persistent Muon first\-moment buffer is maintained for the actor weights\. For matrix\-valued actor parameters, the update remains Muon’s normalized matrix update rather than a raw SGD step, while embedding and output\-head parameters are optimized with AdamW under the standard matrix\-parameter split\. A direct comparison against momentum Muon, AdamW, and SGD actor updates is left for future work\.

### VII\-CMoE Logit Mismatch and Router Replay

PipelineRL requires the gradient step to be computed against the same policy distribution that generated the rollout, up to bounded staleness\. In practice, this is an SNR problem rather than a binary correctness problem\. Small engine\-trainer logit differences add noise to the policy\-gradient estimate; when the rest of the recipe is stable, this noise tends to appear as slower learning, unstable high\-learning\-rate behavior, or plateaus rather than immediate collapse\.

MoE models introduce an additional source of mismatch beyond ordinary numerical differences between the rollout engine and trainer\. In a dense model, a small numeric perturbation usually causes a small logit perturbation\. In a top\-1 MoE, the same perturbation can flip a token’s expert assignment, producing a discontinuous change in the computation path\. A token generated by experterolloute\_\{\\mathrm\{rollout\}\}but trained through expertetrain≠erolloute\_\{\\mathrm\{train\}\}\\neq e\_\{\\mathrm\{rollout\}\}gives the actor gradient the wrong local model for that token\. This mismatch is especially harmful in long\-rollout RL, where many such token\-level routing errors can accumulate across a sequence\.

Router replay\(Maet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib139)\)addresses this MoE\-specific mismatch by recording the per\-token, per\-layer expert choices made by vLLM during rollout generation and replaying those choices during all trainer forward passes over the rollout\. In ZAYA1\-8B training, router replay was a major stability improvement: high learning rates that were unstable without replay became usable with replay\. Router replay does not remove all sources of mismatch, but it removes the discontinuous top\-1 routing component\.

The remaining engine\-trainer mismatch is handled by the FP32 operation set described in Section[IV\-D](https://arxiv.org/html/2605.05365#S4.SS4)\. We use engine\-vs\-trainer probability scatter plots as the main diagnostic: before hardening and router replay, the scatter broadens and token probabilities deviate from the identity line; after hardening, the distributions align closely\. In the final configuration, the engine\-trainer comparison reaches KL divergence approximately1\.3×10−41\.3\\times 10^\{\-4\}and Pearson correlation above0\.99960\.9996on a 128\-prompt,G=16G=16, 4K\-completion diagnostic batch\. We found substantial benefits in terms of training stability and final performance from being extremely careful about reducing sources of numerical error, even if they initially seemed small\.

### VII\-DData and Verifier SNR

The sample efficiency of verifiable RL depends strongly on data and verifier SNR\. A verifier can provide binary reward at scale, but a binary reward is only useful when the prompt distribution produces informative variation across sampled rollouts\. If most groups are solved by every rollout, the gradient has little contrast\. If no rollout solves the prompt, the gradient is also weak\. If the verifier accepts shortcuts or rewards a skewed answer distribution, the model can learn the shortcut rather than the intended reasoning behavior which is often described as ‘reward hacking’\.

We therefore treat difficulty curation as a fundamental part of the RL algorithm rather than as a preprocessing detail\. The reasoning warmup and math\+code\+TTC stages use pass\-rate filtering to select hard but not fully saturated prompts\. The RLVE\-Gym stage performs this filtering online through an adaptive difficulty scheduler targeting the high\-information region of each environment\. Between major stages, we re\-filter our datasets using the current RL policy: first with an instruction\-mode filtering pass using more aggressive sampling settings, and then with a thinking\-mode pass closer to the RL rollout setup but at a lower response\-length limit\. These filtering passes remove the high end of the distribution, progressively raising the difficulty floor as the model improves\.

Verifier quality becomes more important as the model approaches the ceiling of a benchmark or training distribution\. At low capability, even a coarse verifier can provide useful signal because the model is far from saturation\. Near saturation, small verifier errors, spurious binary ground\-truth patterns, incomplete code tests, or skewed answer distributions can dominate the remaining gradient signal, often causing learning to plateau\. In practice, we typically could trace back a plateau to one of three causes: prompts that were too easy, prompts that were effectively impossible under the current rollout budget, or verifier/data artifacts that made a shortcut easier than genuine reasoning\.

This observation also helps explain why short RL runs can be effective\. When the prompt distribution is centered near the model’s current capability boundary and the verifier is high precision, each rollout batch contains useful contrast\. When data or verifier SNR degrades, increasing RL steps alone is inefficient: the optimizer repeatedly sees low\-information or misleading groups which ultimately diminish the signal to the point where learning plateaus at a noise floor\. For this reason, the ZAYA1\-8B cascade alternates capability\-building stages with data curation and difficulty filtering, rather than treating the RL dataset as fixed\.

### VII\-EWhy ZAYA1\-8B Benefits from Test\-Time Compute

ZAYA1\-8B is a small\-active\-parameter MoE: each generated token uses roughly 700M active parameters while drawing on a larger 8B\-parameter expert pool across tokens\. This makes test\-time compute especially attractive\. Generating many candidate traces is cheap in active\-parameter compute, while the total expert pool still gives the model more specialization capacity than a similarly active dense model\. Inference compute is therefore better measured by active parameters multiplied by generated tokens than by total parameters alone\.

Markovian RSA exploits this regime\. The method spends additional compute on many relatively cheap candidate rollouts and aggregation passes, while bounding aggregation context through the carried\-forward tail\. This lets ZAYA1\-8B trade more decode tokens for higher accuracy without paying the per\-token cost of a much larger dense or high\-active\-parameter model\. The relevant deployment question is not just whether accuracy increases with more tokens, but whether the active\-parameter×\\timesgenerated\-token product is favorable relative to larger alternatives\.

Training also matters\. ZAYA1\-8B sees TTC aggregation traces before inference: long\-CoT data appears during midtraining, aggregation\-based examples appear during SFT, and Markovian RSA prompts are included in the math\+code\+TTC RL stage\. As a result, the model is not asked to discover aggregation behavior only at inference time\. It has learned a strong prior for reading several candidate tails, reconciling partial reasoning paths, and producing an improved solution\.

This combination helps explain why Markovian RSA scales well on ZAYA1\-8B\. The model has low per\-token active compute, enough total expert capacity to support diverse candidate trajectories, and explicit training exposure to the aggregation workflow\. We have observed weaker TTC scaling from models not trained for this workflow under the same RSA, Markovian RSA, and PaCoRe procedures, but those comparisons are not fully controlled because the models differ in architecture, training data, and post\-training recipe\. We therefore treat the main claim as specific to ZAYA1\-8B: its architecture and training recipe make test\-time compute a particularly effective way to convert low\-active\-parameter rollouts into higher reasoning accuracy\.

### VII\-FKL\-in\-Reward and Length Bias under PipelineRL

Section[IV\-F](https://arxiv.org/html/2605.05365#S4.SS6)describes a length\-bias failure mode we observed when combining PipelineRL with a sampled signedK1K\_\{1\}\-estimator log\-ratio term in the reward\. In short, stale or mixed\-policy rollouts can make the signed sequence\-level log\-ratio negative, so subtracting it from reward can create a positive length\-dependent offset\. When one completion spans multiple generator\-policy snapshots, stale\-prefix terms can also affect the sequence\-level advantage assigned to fresher suffix tokens\. For ZAYA1\-8B, we avoided this configuration by removing KL\-in\-reward and relying on DPPO Binary\-TV for trust\-region control\. More principled chunk\-local signed\-log\-ratio handling and staleness rescaling are left for future work\.

### VII\-GOpen Questions and Limitations

Several open questions remain\. We provide evidence for the viability of the AMD hardware and networking stack for pretraining at the 8B scale, which is larger than prior public pretraining runs on combined AMD GPU and networking hardware that we are aware of\. However, training ZAYA1\-8B required only data parallelism plus context parallelism during context\-extension phases\. Although we have stress\-tested other parallelism strategies, including cross\-node parallelism, further scaling is needed to validate the stack for substantially larger models\.

It was also unclear at the outset whether our architectural changes would support effective reasoning and long\-context behavior\. This was a particular concern for CCA, which we had not previously tested at long contexts\. ZAYA1\-8B’s performance on long\-context and reasoning benchmarks suggests that CCA’s advantage over attention variants such as MLA and GQA can be maintained in these settings\. However, 8B total parameters is still modest relative to frontier\-scale models, and architectural behavior at larger scales remains to be tested\.

The evaluation profile of ZAYA1\-8B is uneven in a useful way\. The model is strongest on reasoning\-heavy mathematics and code, where it is competitive with much larger models and, under TTC, approaches frontier mathematical performance\. On knowledge\-heavy and broad factual evaluations such as MMLU\-Pro and GPQA\-D, ZAYA1\-8B remains strong for its active\-parameter scale but does not fully close the gap to substantially larger models\. This pattern is consistent with the intuition that reasoning performance and factual storage scale differently: a small\-active model can express strong algorithmic reasoning while still having less capacity for broad memorized knowledge than much larger models\.

This motivates a useful direction for future systems: small\-active reasoning models may be especially effective when paired with test\-time compute and external retrieval\. Rather than requiring all capability to be stored in parameters, such systems can combine a compact reasoning core, cheap parallel inference, and external knowledge sources\. ZAYA1\-8B is one example of this tradeoff, but establishing the generality of this pattern requires further scaling and controlled comparisons\.

Some of ZAYA1\-8B’s reasoning strength may come from its relatively deep architecture\. ZAYA1\-8B has 40 layers, compared with 36 layers for Qwen3\-4B and 16 layers for OLMoE at a similar total\-parameter scale\. We hypothesize that this depth helps the model represent more serial computation within a single forward pass, which may be useful for reasoning\. The residual\-scaling mechanism may also help preserve the contribution of later layers by controlling residual\-norm growth, while the ZAYA1 router improves routing stability and expert specialization\. We treat these as architectural hypotheses supported by our training experience rather than as isolated causal claims; controlled ablations at larger scale are left for future work\.

On agentic tasks, ZAYA1\-8B trails models whose post\-training emphasizes multi\-turn tool use, especially on benchmarks such as BFCL\-v4 andτ2\\tau^\{2\}\. This is an expected consequence of the release scope: our midtraining, SFT, and RL budgets prioritize math, code, TTC aggregation, and instruction following rather than dedicated multi\-turn agentic RL\. We view this as a data\-and\-training emphasis gap rather than an architectural limitation\. Scaling agentic data and RL remains a priority for future releases\.

## Acknowledgements

We would like to thank Paul White, Danny Martinelli, Steven Brook, Kristina Zhao, and Krithik Puthalath for help with the release\. We would also like to thank Yuankai Chen and Yao Fu from AMD for their support and close technical collaboration\.

## References

- The markovian thinker: architecture\-agnostic linear scaling of reasoning\.arXiv preprint arXiv:2510\.06557\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p7.4),[§VI\-C](https://arxiv.org/html/2605.05365#S6.SS3.SSS0.Px6.p1.6),[§VI](https://arxiv.org/html/2605.05365#S6.p1.1)\.
- J\. Ainslie, J\. Lee\-Thorp, M\. De Jong, Y\. Zemlyanskiy, F\. Lebrón, and S\. Sanghai \(2023\)Gqa: training generalized multi\-query transformer models from multi\-head checkpoints\.arXiv preprint arXiv:2305\.13245\.Cited by:[§II\-A1](https://arxiv.org/html/2605.05365#S2.SS1.SSS1.p1.1)\.
- S\. N\. Akter, S\. Prabhumoye, E\. Nyberg, M\. Patwary, M\. Shoeybi, Y\. Choi, and B\. Catanzaro \(2025\)Front\-loading reasoning: the synergy between pretraining and post\-training data\.arXiv preprint arXiv:2510\.03264\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p5.1),[§III\-A](https://arxiv.org/html/2605.05365#S3.SS1.SSS0.Px2.p1.1),[§III\-A](https://arxiv.org/html/2605.05365#S3.SS1.p1.1)\.
- AMD Pensando \(2024\)AMD Pollara 400 Card\.Note:[https://www\.amd\.com/content/dam/amd/en/documents/pensando\-technical\-docs/product\-briefs/pollara\-product\-brief\.pdf](https://www.amd.com/content/dam/amd/en/documents/pensando-technical-docs/product-briefs/pollara-product-brief.pdf)Cited by:[TABLE XIV](https://arxiv.org/html/2605.05365#A1.T14.4.4.1.1.1)\.
- AMD \(2024\)The AMD CDNA™ 3 architecture\.White PaperAMD\.External Links:[Link](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/white-papers/amd-cdna-3-white-paper.pdf)Cited by:[TABLE XIV](https://arxiv.org/html/2605.05365#A1.T14.1.1.1.1.1)\.
- Q\. Anthony, Y\. Tokpanov, S\. Szot, S\. Rajagopal, P\. Medepalli, A\. Golubeva, V\. Shyam, R\. Washbourne, R\. Iyer, A\. Chaurasia,et al\.\(2025\)Training foundation models on a full\-stack amd platform: compute, networking, and system design\.arXiv preprint arXiv:2511\.17127\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p4.1),[§I](https://arxiv.org/html/2605.05365#S1.p8.1),[TABLE II](https://arxiv.org/html/2605.05365#S3.T2),[§III](https://arxiv.org/html/2605.05365#S3.p1.1)\.
- K\. J\. Åström and T\. Hägglund \(2006\)PID control\.IEEE Control Systems Magazine1066,pp\. 30–31\.Cited by:[§II\-A2](https://arxiv.org/html/2605.05365#S2.SS1.SSS2.p2.16)\.
- B\. Bartoldson \(2026\)Cheaply approximating kl against an ema: an async rl hack\.https://brianbartoldson\.wordpress\.com/2026/05/04/cheaply\-approximating\-kl\-against\-an\-ema\-an\-async\-rl\-hack/\.Cited by:[§IV\-F](https://arxiv.org/html/2605.05365#S4.SS6.SSS0.Px4.p5.3),[§IV\-F](https://arxiv.org/html/2605.05365#S4.SS6.SSS0.Px5.p2.3)\.
- A\. Chen, A\. Li, B\. Gong, B\. Jiang, B\. Fei, B\. Yang, B\. Shan, C\. Yu, C\. Wang, C\. Zhu,et al\.\(2025\)Minimax\-m1: scaling test\-time compute efficiently with lightning attention\.arXiv preprint arXiv:2506\.13585\.Cited by:[§IV\-D](https://arxiv.org/html/2605.05365#S4.SS4.SSS0.Px1.p2.1)\.
- G\. Comanici, E\. Bieber, M\. Schaekermann, I\. Pasupat, N\. Sachdeva, I\. Dhillon, M\. Blistein, O\. Ram, D\. Zhang, E\. Rosen,et al\.\(2025\)Gemini 2\.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.arXiv preprint arXiv:2507\.06261\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p2.1)\.
- G\. Cui, Y\. Zhang, J\. Chen, L\. Yuan, Z\. Wang, Y\. Zuo, H\. Li, Y\. Fan, H\. Chen, W\. Chen,et al\.\(2025\)The entropy mechanism of reinforcement learning for reasoning language models\.arXiv preprint arXiv:2505\.22617\.Cited by:[§VII\-A](https://arxiv.org/html/2605.05365#S7.SS1.p2.1)\.
- D\. Dai, C\. Deng, C\. Zhao, R\. X\. Xu, H\. Gao, D\. Chen, J\. Li, W\. Zeng, X\. Yu, Y\. Wu, Z\. Xie, Y\. K\. Li, P\. Huang, F\. Luo, C\. Ruan, Z\. Sui, and W\. Liang \(2024\)DeepSeekMoE: towards ultimate expert specialization in mixture\-of\-experts language models\.External Links:2401\.06066,[Link](https://arxiv.org/abs/2401.06066)Cited by:[§II\-A3](https://arxiv.org/html/2605.05365#S2.SS1.SSS3.p4.1),[§II\-A](https://arxiv.org/html/2605.05365#S2.SS1.p1.1)\.
- DeepSeek\-AI \(2025a\)DeepSeek\-v3 technical report\.External Links:2412\.19437,[Link](https://arxiv.org/abs/2412.19437)Cited by:[§II\-A1](https://arxiv.org/html/2605.05365#S2.SS1.SSS1.p1.1),[§II\-A2](https://arxiv.org/html/2605.05365#S2.SS1.SSS2.p2.16),[§II\-A3](https://arxiv.org/html/2605.05365#S2.SS1.SSS3.p5.5)\.
- DeepSeek\-AI \(2025b\)DeepSeek\-v3\.2\-exp: boosting long\-context efficiency with deepseek sparse attention\.Cited by:[§II\-A3](https://arxiv.org/html/2605.05365#S2.SS1.SSS3.p4.1)\.
- DeepSeek\-AI \(2025c\)DeepSeek\-v3\.2: pushing the frontier of open large language models\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p2.1),[TABLE XI](https://arxiv.org/html/2605.05365#S6.T11)\.
- J\. Dekoninck, N\. Jovanović, T\. Gehrunger, K\. Rögnvalddson, I\. Petrov, C\. Sun, and M\. Vechev \(2026\)Beyond benchmarks: matharena as an evaluation platform for mathematics with llms\.External Links:2605\.00674,[Link](https://arxiv.org/abs/2605.00674)Cited by:[Figure 10](https://arxiv.org/html/2605.05365#S6.F10),[Figure 10](https://arxiv.org/html/2605.05365#S6.F10.3.2)\.
- H\. Ding, Z\. Wang, G\. Paolini, V\. Kumar, A\. Deoras, D\. Roth, and S\. Soatto \(2024\)Fewer truncations improve language modeling\.External Links:2404\.10830,[Link](https://arxiv.org/abs/2404.10830)Cited by:[§IV\-A](https://arxiv.org/html/2605.05365#S4.SS1.p2.1)\.
- W\. Fedus, B\. Zoph, and N\. Shazeer \(2022\)Switch transformers: scaling to trillion parameter models with simple and efficient sparsity\.Journal of Machine Learning Research23\(120\),pp\. 1–39\.Cited by:[§II\-A](https://arxiv.org/html/2605.05365#S2.SS1.p1.1)\.
- T\. Figliolia, N\. Alonso, R\. Iyer, Q\. Anthony, and B\. Millidge \(2025\)Compressed convolutional attention: efficient attention in a compressed latent space\.External Links:2510\.04476,[Link](https://arxiv.org/abs/2510.04476)Cited by:[Figure 12](https://arxiv.org/html/2605.05365#A3.F12),[Figure 12](https://arxiv.org/html/2605.05365#A3.F12.3.2),[§I](https://arxiv.org/html/2605.05365#S1.p4.1)\.
- GLM\-5\-Team, :, A\. Zeng, X\. Lv, Z\. Hou, Z\. Du, Q\. Zheng, B\. Chen, D\. Yin, C\. Ge, C\. Huang, C\. Xie, C\. Zhu, C\. Yin, C\. Wang, G\. Pan, H\. Zeng, H\. Zhang, H\. Wang, H\. Chen, J\. Zhang, J\. Jiao, J\. Guo, J\. Wang, J\. Du, J\. Wu, K\. Wang, L\. Li, L\. Fan, L\. Zhong, M\. Liu, M\. Zhao, P\. Du, Q\. Dong, R\. Lu, Shuang\-Li, S\. Cao, S\. Liu, T\. Jiang, X\. Chen, X\. Zhang, X\. Huang, X\. Dong, Y\. Xu, Y\. Wei, Y\. An, Y\. Niu, Y\. Zhu, Y\. Wen, Y\. Cen, Y\. Bai, Z\. Qiao, Z\. Wang, Z\. Wang, Z\. Zhu, Z\. Liu, Z\. Li, B\. Wang, B\. Wen, C\. Huang, C\. Cai, C\. Yu, C\. Li, C\. Hu, C\. Zhang, D\. Zhang, D\. Lin, D\. Yang, D\. Wang, D\. Ai, E\. Zhu, F\. Yi, F\. Chen, G\. Wen, H\. Sun, H\. Zhao, H\. Hu, H\. Zhang, H\. Liu, H\. Zhang, H\. Peng, H\. Tai, H\. Zhang, H\. Liu, H\. Wang, H\. Yan, H\. Ge, H\. Liu, H\. Chu, J\. Zhao, J\. Wang, J\. Zhao, J\. Ren, J\. Wang, J\. Zhang, J\. Gui, J\. Zhao, J\. Li, J\. An, J\. Li, J\. Yuan, J\. Du, J\. Liu, J\. Zhi, J\. Duan, K\. Zhou, K\. Wei, K\. Wang, K\. Luo, L\. Zhang, L\. Sha, L\. Xu, L\. Wu, L\. Ding, L\. Chen, M\. Li, N\. Lin, P\. Ta, Q\. Zou, R\. Song, R\. Yang, S\. Tu, S\. Yang, S\. Wu, S\. Zhang, S\. Li, S\. Li, S\. Fan, W\. Qin, W\. Tian, W\. Zhang, W\. Yu, W\. Liang, X\. Kuang, X\. Cheng, X\. Li, X\. Yan, X\. Hu, X\. Ling, X\. Fan, X\. Xia, X\. Zhang, X\. Zhang, X\. Pan, X\. Zou, X\. Zhang, Y\. Liu, Y\. Wu, Y\. Li, Y\. Wang, Y\. Zhu, Y\. Tan, Y\. Zhou, Y\. Pan, Y\. Zhang, Y\. Su, Y\. Geng, Y\. Yan, Y\. Tan, Y\. Bi, Y\. Shen, Y\. Yang, Y\. Li, Y\. Liu, Y\. Wang, Y\. Li, Y\. Wu, Y\. Zhang, Y\. Duan, Y\. Zhang, Z\. Liu, Z\. Jiang, Z\. Yan, Z\. Zhang, Z\. Wei, Z\. Chen, Z\. Feng, Z\. Yao, Z\. Chai, Z\. Wang, Z\. Zhang, B\. Xu, M\. Huang, H\. Wang, J\. Li, Y\. Dong, and J\. Tang \(2026\)GLM\-5: from vibe coding to agentic engineering\.External Links:2602\.15763,[Link](https://arxiv.org/abs/2602.15763)Cited by:[§IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1.Px7.p1.1),[§IV\-E](https://arxiv.org/html/2605.05365#S4.SS5.p2.2)\.
- J\. Hu, Y\. Zhang, S\. Shang, X\. Yang, Y\. Peng, Z\. Huang, H\. Zhou, X\. Wu, J\. Cheng, F\. Wan,et al\.\(2026\)PaCoRe: learning to scale test\-time compute with parallel coordinated reasoning\.arXiv preprint arXiv:2601\.05593\.Cited by:[§VI\-A](https://arxiv.org/html/2605.05365#S6.SS1.SSS0.Px5.p1.1)\.
- K\. Jordan, Y\. Jin, V\. Boza, Y\. Jiacheng, F\. Cecista, L\. Newhouse, and J\. Bernstein \(2024\)Muon: an optimizer for hidden layers in neural networks, 2024\.URL https://kellerjordan\. github\. io/posts/muon6\.Cited by:[§III](https://arxiv.org/html/2605.05365#S3.p4.1),[§IV\-E](https://arxiv.org/html/2605.05365#S4.SS5.p1.6)\.
- D\. Khatri, L\. Madaan, R\. Tiwari, R\. Bansal, S\. S\. Duvvuri, M\. Zaheer, I\. S\. Dhillon, D\. Brandfonbrener, and R\. Agarwal \(2025\)The art of scaling reinforcement learning compute for llms\.arXiv preprint arXiv:2510\.13786\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p5.1),[§I](https://arxiv.org/html/2605.05365#S1.p6.1),[§III\-A](https://arxiv.org/html/2605.05365#S3.SS1.SSS0.Px2.p1.1),[§IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1.Px1.p1.1),[§IV\-D](https://arxiv.org/html/2605.05365#S4.SS4.SSS0.Px1.p2.1)\.
- D\. Lee, S\. Han, A\. Kumar, and P\. Agrawal \(2026\)Training language models via neural cellular automata\.arXiv preprint arXiv:2603\.10055\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2603.10055),[Link](https://arxiv.org/abs/2603.10055)Cited by:[§IV\-F](https://arxiv.org/html/2605.05365#S4.SS6.SSS0.Px1.p2.3)\.
- J\. Li, D\. Guo, D\. Yang, R\. Xu, Y\. Wu, and J\. He \(2025\)CodeI/o: condensing reasoning patterns via code input\-output prediction\.External Links:2502\.07316,[Link](https://arxiv.org/abs/2502.07316)Cited by:[item 1](https://arxiv.org/html/2605.05365#S4.I1.i1.p1.1)\.
- M\. Li and P\. Vitányi \(2019\)An introduction to kolmogorov complexity and its applications\.4 edition,Texts in Computer Science,Springer,Cham\.External Links:ISBN 978\-3\-030\-11297\-4,[Document](https://dx.doi.org/10.1007/978-3-030-11298-1),[Link](https://doi.org/10.1007/978-3-030-11298-1)Cited by:[§IV\-F](https://arxiv.org/html/2605.05365#S4.SS6.SSS0.Px1.p2.3)\.
- J\. Liu, J\. Su, X\. Yao, Z\. Jiang, G\. Lai, Y\. Du, Y\. Qin, W\. Xu, E\. Lu, J\. Yan,et al\.\(2025\)Muon is scalable for llm training\.arXiv preprint arXiv:2502\.16982\.Cited by:[§III](https://arxiv.org/html/2605.05365#S3.p4.1)\.
- Z\. Liu, C\. Chen, W\. Li, P\. Qi, T\. Pang, C\. Du, W\. S\. Lee, and M\. Lin \(2024\)Understanding r1\-zero\-like training: a critical perspective, 2025\.URL https://arxiv\. org/abs/2503\.20783\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p6.1),[§IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1.Px3.p1.1)\.
- F\. M\. Lord \(1980\)Applications of item response theory to practical testing problems\.Lawrence Erlbaum Associates,Hillsdale, NJ\.External Links:ISBN 089859006XCited by:[§IV\-B4](https://arxiv.org/html/2605.05365#S4.SS2.SSS4.p5.4),[§IV\-B4](https://arxiv.org/html/2605.05365#S4.SS2.SSS4.p9.3)\.
- W\. Ma, H\. Zhang, L\. Zhao, Y\. Song, Y\. Wang, Z\. Sui, and F\. Luo \(2025\)Stabilizing moe reinforcement learning by aligning training and inference routers\.External Links:2510\.11370,[Link](https://arxiv.org/abs/2510.11370)Cited by:[§VII\-C](https://arxiv.org/html/2605.05365#S7.SS3.p3.1)\.
- Mistral AI \(2026\)Introducing Mistral Small 4\.Note:[https://mistral\.ai/news/mistral\-small\-4](https://mistral.ai/news/mistral-small-4)Accessed: 2026\-05\-06Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p1.1)\.
- S\. Mukherjee, L\. Yuan, D\. Hakkani\-Tur, and H\. Peng \(2025\)Reinforcement learning finetunes small subnetworks in large language models\.arXiv preprint arXiv:2505\.11711\.Cited by:[§VII\-B](https://arxiv.org/html/2605.05365#S7.SS2.p3.2)\.
- S\. Mukherjee, L\. Yuan, P\. Jayasinha, D\. Hakkani\-Tür, and H\. Peng \(2026a\)Do we need adam? surprisingly strong and sparse reinforcement learning with sgd in llms\.External Links:2602\.07729,[Link](https://arxiv.org/abs/2602.07729)Cited by:[§IV\-E](https://arxiv.org/html/2605.05365#S4.SS5.p2.2)\.
- S\. Mukherjee, L\. Yuan, P\. Jayasinha, D\. Hakkani\-Tür, and H\. Peng \(2026b\)Do we need adam? surprisingly strong and sparse reinforcement learning with sgd in llms\.arXiv preprint arXiv:2602\.07729\.Cited by:[§VII\-B](https://arxiv.org/html/2605.05365#S7.SS2.p3.2)\.
- M\. N\. Nguyen, A\. Baker, C\. Neo, A\. Roush, A\. Kirsch, and R\. Shwartz\-Ziv \(2024\)Turning up the heat: min\-p sampling for creative and coherent llm outputs\.arXiv preprint arXiv:2407\.01082\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2407.01082),[Link](https://arxiv.org/abs/2407.01082)Cited by:[Appendix E](https://arxiv.org/html/2605.05365#A5.p6.5)\.
- NVIDIA \(2025\)Nemotron 3 Nano: open, efficient mixture\-of\-experts hybrid Mamba\-Transformer model for Agentic reasoning\.Note:Technical reportExternal Links:[Link](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Nano-Technical-Report.pdf)Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p1.1)\.
- OpenAI \(2025\)GPT\-5 System Card\.Note:[https://openai\.com/index/gpt\-5\-system\-card/](https://openai.com/index/gpt-5-system-card/)Accessed: 2026\-05\-06Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p2.1)\.
- M\. Pagliardini, A\. Mohtashami, F\. Fleuret, and M\. Jaggi \(2024\)Denseformer: enhancing information flow in transformers via depth weighted averaging\.Advances in neural information processing systems37,pp\. 136479–136508\.Cited by:[§II\-A2](https://arxiv.org/html/2605.05365#S2.SS1.SSS2.p1.1)\.
- A\. Piché, E\. Kamalloo, R\. Pardinas, X\. Chen, and D\. Bahdanau \(2025\)Pipelinerl: faster on\-policy reinforcement learning for long sequence generation\.arXiv preprint arXiv:2509\.19128\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p6.1),[§IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1.Px1.p1.1)\.
- P\. Qi, Z\. Liu, X\. Zhou, T\. Pang, C\. Du, W\. S\. Lee, and M\. Lin \(2025\)Defeating the training\-inference mismatch via fp16\.External Links:2510\.26788,[Link](https://arxiv.org/abs/2510.26788)Cited by:[§IV\-D](https://arxiv.org/html/2605.05365#S4.SS4.SSS0.Px2.p1.1)\.
- P\. Qi, X\. Zhou, Z\. Liu, T\. Pang, C\. Du, M\. Lin, and W\. S\. Lee \(2026\)Rethinking the trust region in llm reinforcement learning\.arXiv preprint arXiv:2602\.04879\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p6.1),[§IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1.Px2.p1.3)\.
- Z\. Qiu, Z\. Wang, B\. Zheng, Z\. Huang, K\. Wen, S\. Yang, R\. Men, L\. Yu, F\. Huang, S\. Huang,et al\.\(2025\)Gated attention for large language models: non\-linearity, sparsity, and attention\-sink\-free\.arXiv preprint arXiv:2505\.06708\.Cited by:[§II\-A3](https://arxiv.org/html/2605.05365#S2.SS1.SSS3.p3.3)\.
- Qwen Team \(2025\)Qwen3\-235B\-A22B\-Thinking\-2507\.Note:Hugging Face model cardAccessed: 2026\-05\-06External Links:[Link](https://huggingface.co/Qwen/Qwen3-235B-A22B-Thinking-2507)Cited by:[TABLE XI](https://arxiv.org/html/2605.05365#S6.T11)\.
- S\. Rajbhandari, C\. Li, Z\. Yao, M\. Zhang, R\. Y\. Aminabadi, A\. A\. Awan, J\. Rasley, and Y\. He \(2022\)Deepspeed\-moe: advancing mixture\-of\-experts inference and training to power next\-generation ai scale\.InInternational conference on machine learning,pp\. 18332–18346\.Cited by:[§II\-A3](https://arxiv.org/html/2605.05365#S2.SS1.SSS3.p5.5)\.
- N\. Shazeer, A\. Mirhoseini, K\. Maziarz, A\. Davis, Q\. Le, G\. Hinton, and J\. Dean \(2016\)Outrageously large neural networks: the sparsely\-gated mixture\-of\-experts layer\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/1701.06538)Cited by:[§II\-A](https://arxiv.org/html/2605.05365#S2.SS1.p1.1)\.
- I\. Shenfeld, J\. Pari, and P\. Agrawal \(2025\)RL’s razor: why online reinforcement learning forgets less\.arXiv preprint arXiv:2509\.04259\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2509.04259),[Link](https://arxiv.org/abs/2509.04259)Cited by:[§VII\-A](https://arxiv.org/html/2605.05365#S7.SS1.p2.1)\.
- G\. Sheng, C\. Zhang, Z\. Ye, X\. Wu, W\. Zhang, R\. Zhang, Y\. Peng, H\. Lin, and C\. Wu \(2025\)HybridFlow: a flexible and efficient rlhf framework\.InProceedings of the Twentieth European Conference on Computer Systems \(EuroSys 2025\),pp\. 1279–1297\.Note:arXiv preprint arXiv:2409\.19256External Links:[Document](https://dx.doi.org/10.1145/3689031.3696075)Cited by:[§IV\-B4](https://arxiv.org/html/2605.05365#S4.SS2.SSS4.p1.1)\.
- J\. Su, Y\. Lu, S\. Pan, A\. Murtadha, B\. Wen, and Y\. Liu \(2023\)RoFormer: enhanced transformer with rotary position embedding\.External Links:2104\.09864,[Link](https://arxiv.org/abs/2104.09864)Cited by:[§II\-A3](https://arxiv.org/html/2605.05365#S2.SS1.SSS3.p5.5)\.
- F\. Tajwar, G\. Zeng, Y\. Zhou, Y\. Song, D\. Arora, Y\. Jiang, J\. Schneider, R\. Salakhutdinov, H\. Feng, and A\. Zanette \(2026\)Maximum likelihood reinforcement learning\.External Links:2602\.02710,[Link](https://arxiv.org/abs/2602.02710)Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p6.1),[§IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1.Px4.p1.2),[§IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1.Px4.p1.3)\.
- F\. C\. Team, J\. Copet, Q\. Carbonneaux, G\. Cohen, J\. Gehring, J\. Kahn, J\. Kossen, F\. Kreuk, E\. McMilin, M\. Meyer, Y\. Wei, D\. Zhang, K\. Zheng, J\. Armengol\-Estapé, P\. Bashiri, M\. Beck, P\. Chambon, A\. Charnalia, C\. Cummins, J\. Decugis, Z\. V\. Fisches, F\. Fleuret, F\. Gloeckle, A\. Gu, M\. Hassid, D\. Haziza, B\. Y\. Idrissi, C\. Keller, R\. Kindi, H\. Leather, G\. Maimon, A\. Markosyan, F\. Massa, P\. Mazaré, V\. Mella, N\. Murray, K\. Muzumdar, P\. O’Hearn, M\. Pagliardini, D\. Pedchenko, T\. Remez, V\. Seeker, M\. Selvi, O\. Sultan, S\. Wang, L\. Wehrstedt, O\. Yoran, L\. Zhang, T\. Cohen, Y\. Adi, and G\. Synnaeve \(2025a\)CWM: an open\-weights llm for research on code generation with world models\.arXiv preprint arXiv:2510\.02387\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2510.02387),[Link](https://arxiv.org/abs/2510.02387)Cited by:[Appendix E](https://arxiv.org/html/2605.05365#A5.p3.1)\.
- K\. Team, Y\. Bai, Y\. Bao, Y\. Charles, C\. Chen, G\. Chen, H\. Chen, H\. Chen, J\. Chen, N\. Chen, R\. Chen, Y\. Chen, Y\. Chen, Y\. Chen, Z\. Chen, J\. Cui, H\. Ding, M\. Dong, A\. Du, C\. Du, D\. Du, Y\. Du, Y\. Fan, Y\. Feng, K\. Fu, B\. Gao, C\. Gao, H\. Gao, P\. Gao, T\. Gao, Y\. Ge, S\. Geng, Q\. Gu, X\. Gu, L\. Guan, H\. Guo, J\. Guo, X\. Hao, T\. He, W\. He, W\. He, Y\. He, C\. Hong, H\. Hu, Y\. Hu, Z\. Hu, W\. Huang, Z\. Huang, Z\. Huang, T\. Jiang, Z\. Jiang, X\. Jin, Y\. Kang, G\. Lai, C\. Li, F\. Li, H\. Li, M\. Li, W\. Li, Y\. Li, Y\. Li, Y\. Li, Z\. Li, Z\. Li, H\. Lin, X\. Lin, Z\. Lin, C\. Liu, C\. Liu, H\. Liu, J\. Liu, J\. Liu, L\. Liu, S\. Liu, T\. Y\. Liu, T\. Liu, W\. Liu, Y\. Liu, Y\. Liu, Y\. Liu, Y\. Liu, Z\. Liu, E\. Lu, H\. Lu, L\. Lu, Y\. Luo, S\. Ma, X\. Ma, Y\. Ma, S\. Mao, J\. Mei, X\. Men, Y\. Miao, S\. Pan, Y\. Peng, R\. Qin, Z\. Qin, B\. Qu, Z\. Shang, L\. Shi, S\. Shi, F\. Song, J\. Su, Z\. Su, L\. Sui, X\. Sun, F\. Sung, Y\. Tai, H\. Tang, J\. Tao, Q\. Teng, C\. Tian, C\. Wang, D\. Wang, F\. Wang, H\. Wang, H\. Wang, J\. Wang, J\. Wang, J\. Wang, S\. Wang, S\. Wang, S\. Wang, X\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Y\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, Z\. Wang, C\. Wei, Q\. Wei, H\. Wu, W\. Wu, X\. Wu, Y\. Wu, C\. Xiao, J\. Xie, X\. Xie, W\. Xiong, B\. Xu, J\. Xu, L\. H\. Xu, L\. Xu, S\. Xu, W\. Xu, X\. Xu, Y\. Xu, Z\. Xu, J\. Xu, J\. Xu, J\. Yan, Y\. Yan, H\. Yang, X\. Yang, Y\. Yang, Y\. Yang, Z\. Yang, Z\. Yang, Z\. Yang, H\. Yao, X\. Yao, W\. Ye, Z\. Ye, B\. Yin, L\. Yu, E\. Yuan, H\. Yuan, M\. Yuan, S\. Yuan, H\. Zhan, D\. Zhang, H\. Zhang, W\. Zhang, X\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, Z\. Zhang, H\. Zhao, Y\. Zhao, Z\. Zhao, H\. Zheng, S\. Zheng, L\. Zhong, J\. Zhou, X\. Zhou, Z\. Zhou, J\. Zhu, Z\. Zhu, W\. Zhuang, and X\. Zu \(2026\)Kimi k2: open agentic intelligence\.External Links:2507\.20534,[Link](https://arxiv.org/abs/2507.20534)Cited by:[4th item](https://arxiv.org/html/2605.05365#A3.I1.i4.p1.3)\.
- K\. Team, Y\. Bai, Y\. Bao, G\. Chen, J\. Chen, N\. Chen, R\. Chen, Y\. Chen, Y\. Chen, Y\. Chen,et al\.\(2025b\)Kimi k2: open agentic intelligence\.arXiv preprint arXiv:2507\.20534\.Cited by:[§II\-A3](https://arxiv.org/html/2605.05365#S2.SS1.SSS3.p4.1)\.
- O\. Team, A\. Ettinger, A\. Bertsch, B\. Kuehl, D\. Graham, D\. Heineman, D\. Groeneveld, F\. Brahman, F\. Timbers, H\. Ivison,et al\.\(2025c\)Olmo 3\.arXiv preprint arXiv:2512\.13961,pp\. 23–25\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p1.1)\.
- P\. I\. Team \(2025a\)INTELLECT\-3: technical report\.External Links:[Link](https://huggingface.co/PrimeIntellect/INTELLECT-3)Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p1.1)\.
- Q\. Team \(2025b\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p2.1)\.
- W\. R\. Thompson \(1933\)On the likelihood that one unknown probability exceeds another in view of the evidence of two samples\.Biometrika25\(3–4\),pp\. 285–294\.External Links:[Document](https://dx.doi.org/10.1093/biomet/25.3-4.285)Cited by:[§IV\-B4](https://arxiv.org/html/2605.05365#S4.SS2.SSS4.p5.4)\.
- C\. Tian, K\. Chen, J\. Liu, Z\. Liu, Z\. Zhang, and J\. Zhou \(2025\)Towards greater leverage: scaling laws for efficient mixture\-of\-experts language models\.arXiv preprint arXiv:2507\.17702\.Cited by:[§II\-A3](https://arxiv.org/html/2605.05365#S2.SS1.SSS3.p4.1)\.
- S\. Venkatraman, V\. Jain, S\. Mittal, V\. Shah, J\. Obando\-Ceron, Y\. Bengio, B\. R\. Bartoldson, B\. Kailkhura, G\. Lajoie, G\. Berseth,et al\.\(2025\)Recursive self\-aggregation unlocks deep thinking in large language models\.arXiv preprint arXiv:2509\.26626\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p7.4),[§VI\-C](https://arxiv.org/html/2605.05365#S6.SS3.p1.5),[§VI](https://arxiv.org/html/2605.05365#S6.p1.1)\.
- Z\. Wang, Y\. Dong, O\. Delalleau, J\. Zeng, G\. Shen, D\. Egert, J\. J\. Zhang, M\. N\. Sreedhar, and O\. Kuchaiev \(2024\)HelpSteer2: open\-source dataset for training top\-performing reward models\.External Links:2406\.08673Cited by:[§IV\-B7](https://arxiv.org/html/2605.05365#S4.SS2.SSS7.p2.1)\.
- Z\. Wang, J\. Zeng, O\. Delalleau, H\. Shin, F\. Soares, A\. Bukharin, E\. Evans, Y\. Dong, and O\. Kuchaiev \(2025\)HelpSteer3\-preference: open human\-annotated preference data across diverse tasks and languages\.External Links:2505\.11475,[Link](https://arxiv.org/abs/2505.11475)Cited by:[§IV\-B7](https://arxiv.org/html/2605.05365#S4.SS2.SSS7.p2.1)\.
- A\. Wei, T\. Suresh, J\. Cao, N\. Kannan, Y\. Wu, K\. Yan, T\. S\. F\. X\. Teixeira, K\. Wang, and A\. Aiken \(2025\)CodeARC: benchmarking reasoning capabilities of llm agents for inductive program synthesis\.External Links:2503\.23145,[Link](https://arxiv.org/abs/2503.23145)Cited by:[item 2](https://arxiv.org/html/2605.05365#S4.I1.i2.p1.1)\.
- V\. Xiang, C\. Blagden, R\. Rafailov, N\. Lile, S\. Truong, C\. Finn, and N\. Haber \(2025\)Just enough thinking: efficient reasoning with adaptive length penalties reinforcement learning\.arXiv preprint arXiv:2506\.05256\.Cited by:[§IV\-B2](https://arxiv.org/html/2605.05365#S4.SS2.SSS2.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p5.1),[§III\-A](https://arxiv.org/html/2605.05365#S3.SS1.SSS0.Px2.p1.1)\.
- Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, M\. Qiao, Y\. Wu, and M\. Wang \(2025\)DAPO: an open\-source llm reinforcement learning system at scale\.External Links:2503\.14476,[Link](https://arxiv.org/abs/2503.14476)Cited by:[§IV\-B1](https://arxiv.org/html/2605.05365#S4.SS2.SSS1.Px4.p1.2)\.
- D\. Yuan, T\. Xie, S\. Huang, Z\. Gong, H\. Zhang, C\. Luo, F\. Wei, and D\. Zhao \(2025\)Shorten after you’re right: lazy length penalties for reasoning rl\.arXiv preprint arXiv:2505\.12284v3\.Cited by:[§IV\-B2](https://arxiv.org/html/2605.05365#S4.SS2.SSS2.p1.1)\.
- Z\. Zeng, H\. Ivison, Y\. Wang, L\. Yuan, S\. S\. Li, Z\. Ye, S\. Li, J\. He, R\. Zhou, T\. Chen,et al\.\(2025a\)Rlve: scaling up reinforcement learning for language models with adaptive verifiable environments\.arXiv preprint arXiv:2511\.07317\.Cited by:[§I](https://arxiv.org/html/2605.05365#S1.p6.1)\.
- Z\. Zeng, H\. Ivison, Y\. Wang, L\. Yuan, S\. S\. Li, Z\. Ye, S\. Li, J\. He, R\. Zhou, T\. Chen, C\. Zhao, Y\. Tsvetkov, S\. S\. Du, N\. Jaques, H\. Peng, P\. W\. Koh, and H\. Hajishirzi \(2025b\)RLVE: scaling up reinforcement learning for language models with adaptive verifiable environments\.arXiv preprint arXiv:2511\.07317\.Cited by:[§IV\-B4](https://arxiv.org/html/2605.05365#S4.SS2.SSS4.p1.1)\.

## Appendix ACluster Details

Table[XIV](https://arxiv.org/html/2605.05365#A1.T14)summarizes the hardware configuration of the compute, storage, and login nodes\. All nodes also include separate local drives for the operating system\.

TABLE XIV:Hardware configuration of the compute, storage, and login nodes\.
## Appendix BStorage Node Sizing and I/O Calculations

We analyze shared\-storage requirements for dataset reads during training, assuming Megatron\-style pretokenized corpora accessed throughmmapor buffered reads on a dedicated storage fabric\. Large sequential checkpoint writes are throughput\-bound rather than IOPS\-bound and are not considered here\.

LetGGbe the global batch size,ssthe sequence length,bbbytes per token,PPthe storage page size,ttthe iteration time, andImaxI\_\{\\max\}the sustainable IOPS capacity\. We introduce a*scatter factor*σ≥1\\sigma\\geq 1to capture how much real dataset access patterns, including metadata touches, index probes, page\-cache misses, and small random seeks, deviate from perfectly contiguous reads\. Each iteration readsG⋅s⋅bG\\cdot s\\cdot bbytes, requiring

NIO/iter=σ⋅⌈G⋅s⋅bP⌉\\displaystyle N\_\{\\text\{IO/iter\}\}\\;=\\;\\sigma\\cdot\\left\\lceil\\frac\{G\\cdot s\\cdot b\}\{P\}\\right\\rceil\(25\)effective I/O operations\. The sustained IOPS requirement is therefore

IOPSneeded=NIO/itert=σt⋅⌈G⋅s⋅bP⌉,\\displaystyle\\text\{IOPS\}\_\{\\text\{needed\}\}\\;=\\;\\frac\{N\_\{\\text\{IO/iter\}\}\}\{t\}\\;=\\;\\frac\{\\sigma\}\{t\}\\cdot\\left\\lceil\\frac\{G\\cdot s\\cdot b\}\{P\}\\right\\rceil,\(26\)and the break\-even iteration time under budgetImaxI\_\{\\max\}is

tbreak=σImax⋅⌈G⋅s⋅bP⌉\.\\displaystyle t\_\{\\text\{break\}\}\\;=\\;\\frac\{\\sigma\}\{I\_\{\\max\}\}\\cdot\\left\\lceil\\frac\{G\\cdot s\\cdot b\}\{P\}\\right\\rceil\.\(27\)
We can estimate the scatter factorσ\\sigmafrom the average number of additional page faults per sample\. Letmmdenote the average count of additional page faults from metadata,\*\.idxprobes, document\-boundary straddles, or cold reads\. If the ideal pages per sample are\(s⋅b\)/P\(s\\cdot b\)/P, a practical approximation is

σ≈1\+m⋅Ps⋅b\.\\displaystyle\\sigma\\;\\approx\\;1\\;\+\\;\\frac\{m\\cdot P\}\{s\\cdot b\}\.\(28\)This interpolates between contiguous, warm\-cache access \(σ→1\\sigma\\to 1\) and fragmented, small\-document regimes \(σ\>1\\sigma\>1\)\. In our experience with well\-packed Megatron datasets,σ∈\[1,2\]\\sigma\\in\[1,2\]is typical; heavily fragmented or multi\-shard random\-seek workloads can reachσ∈\[2,8\]\\sigma\\in\[2,8\]\.

For the ZAYA1 training run withG=4096G\{=\}4096,s=4096s\{=\}4096,b=4b\{=\}4B,P=4096P\{=\}4096B,t=2\.5t\{=\}2\.5s, andImax=70,000I\_\{\\max\}\{=\}70\{,\}000IOPS, each iteration reads 64 MiB across 16,384 pages\. This requires approximately6,554⋅σ6\{,\}554\\cdot\\sigmaIOPS, with break\-even timetbreak≈0\.234⋅σt\_\{\\text\{break\}\}\\approx 0\.234\\cdot\\sigmas\. At the observedt=2\.5t\{=\}2\.5s, the run remains above the break\-even time even forσ=8\\sigma\{=\}8, indicating that the 70K IOPS storage budget is sufficient\.

## Appendix CCompressed Convolutional Attention

![Refer to caption](https://arxiv.org/html/2605.05365v1/figures/cca_pretty.png)Figure 12:We briefly review the Compressed Convolutional Attention block\(Figlioliaet al\.,[2025](https://arxiv.org/html/2605.05365#bib.bib193)\)\.CCA \(shown in Figure[12](https://arxiv.org/html/2605.05365#A3.F12)\) modifies the attention block so that attention is performed in a compressed latent space\. This reduces both memory and FLOP costs\. In prior work, CCA outperformed alternatives such as GQA and MLA on perplexity and training/inference FLOPs while enabling high KV\-cache compression, which is important for fast decoding\.

CCA has several core components:

- •Low\-Rank Projections:Low\-rank down\-projections reduce compute and memory\.
- •Sequence\-Mixing Convolutions:A short convolution and grouped head\-wise convolution act as lightweight preconditioners before attention\.
- •Value Head Time\-Delay:A time delay of one token is applied to half of the value heads\.
- •Skip Connections and Normalization:The architecture utilizes query/key mean skip connections to enforce representational similarity, coupled with an RMSNorm layer that applies a head\-wise temperature strictly to the keys\. To stabilize training, we modify the standard CCA mechanism by scaling the keys by a learned temperatureTTinstead ofexp⁡\(T\)\\exp\(T\)\. Becauseexp⁡\(T\)\\exp\(T\)can easily grow excessively large, the query\-key inner product becomes susceptible to unbounded growth\. Our linear parameterization successfully mitigates the resulting maximum attention logit instability, drawing parallels to similar stabilization efforts in MLA\(Teamet al\.,[2026](https://arxiv.org/html/2605.05365#bib.bib249)\)\. The scaled query\-key inner product is then upper bounded \(assuming QK norm\) to: T⋅dhdh=T​dh,\\displaystyle\\frac\{T\\cdot d\_\{h\}\}\{\\sqrt\{d\_\{h\}\}\}=T\\sqrt\{d\_\{h\}\}\\,,\(29\)wheredhd\_\{h\}is the head dimension\.

These additions to precondition query and key, especially the convolutions, provide expressivity and nonlinearity that allow CCA to match or exceed full attention while requiring less compute and memory\.

The CCA block can also operate in ‘GQA\-mode’, where multiple KV heads are shared across query heads\. This further reduces decoding cost through additional KV\-cache compression\. We call the combined method CCGQA\. For ZAYA1\-8B, we use CCGQA with 2 KV heads for 8 query heads, on top of2×2\\timesquery compression, for an8×8\\timesKV\-cache compression ratio relative to full multi\-head attention\.

## Appendix DExpert Redundancy Diagnostic

We include a small diagnostic for expert redundancy in the MoE feed\-forward blocks\. This diagnostic is not an evaluation benchmark and should not be interpreted as a complete measure of expert specialization\. Its purpose is narrower: to check whether ZAYA1\-8B’s experts appear unusually collapsed or redundant relative to other public MoE checkpoints\.

For each MoE layerlland expertee, letWl,eW\_\{l,e\}denote the expert projection under study\. We compute the top\-ddsingular subspaceQl,e∈ℝD×dQ\_\{l,e\}\\in\\mathbb\{R\}^\{D\\times d\}withd=128d=128and orthonormal columns\. For the input\-side metric,Ql,eQ\_\{l,e\}is the right singular subspace of the full first FFN projection\. For gated FFNs with separate branches, we concatenate the gate and up projections along the FFN dimension before computing the right singular subspace; for ZAYA1\-8B, this corresponds to the fusedlinear\_fc1projection\. For the output\-side metric,Ql,eQ\_\{l,e\}is the left singular subspace of the expert output projection, corresponding tolinear\_fc2,down\_proj, orw2\.

For two expertsiiandjjin the same layer, we define

sl,i,j=1d​‖Ql,i⊤​Ql,j‖F2,s\_\{l,i,j\}=\\frac\{1\}\{d\}\\left\\\|Q\_\{l,i\}^\{\\top\}Q\_\{l,j\}\\right\\\|\_\{F\}^\{2\}\\,,\(30\)where∥⋅∥F\\\|\\cdot\\\|\_\{F\}is the Frobenius norm\. The reported score averagessl,i,js\_\{l,i,j\}over all off\-diagonal expert pairs and all MoE layers\. Larger values indicate that experts share more of the same input or output directions; smaller values indicate more distinct expert subspaces under this particular projection\-space diagnostic\.

TABLE XV:Mean within\-layer expert subspace overlap withd=128d=128\. Input overlap uses the right singular subspace of the full first FFN projection; output overlap uses the left singular subspace of the expert output projection\. Standard deviations are over all off\-diagonal expert pairs across analyzed MoE layers\. All rows use a 2048\-dimensional comparison space, giving a random\-subspace baseline ofd/D=0\.0625d/D=0\.0625\. The Qwen row uses Qwen3\-30B\-A3B\-Thinking\-2507\.![Refer to caption](https://arxiv.org/html/2605.05365v1/x7.png)Figure 13:Expert redundancy diagnostic\. Panels \(a\) and \(b\) show global mean overlap for input and output projections; bar labels report the random\-normalized ratioρ\\rho\. Panels \(c\) and \(d\) show raw overlap averaged over depth quartiles\. The dashed horizontal line is the random\-subspace baselined/D=0\.0625d/D=0\.0625\. ZAYA1\-8B is not an outlier toward higher expert overlap: its first\-projection input overlap is close to Qwen and below LFM2 and OLMoE, while its output\-projection overlap is intermediate between LFM2 and the lower\-overlap OLMoE/Qwen group\.For independent randomdd\-dimensional subspaces inℝD\\mathbb\{R\}^\{D\}, the expected value of this metric isd/Dd/D\. All models in Table[XV](https://arxiv.org/html/2605.05365#A4.T15)haveD=2048D=2048for both the input and output comparison spaces, so the random\-subspace floor is128/2048=0\.0625128/2048=0\.0625\. We therefore report both the raw overlap and the random\-normalized ratio

ρ=sd/D\.\\rho=\\frac\{s\}\{d/D\}\.\(31\)The “Var\.” columns report the mean fraction of projection Frobenius energy captured by the top\-128 singular directions; they are included as spectral context, not as a normalization of the overlap score\.

Overall, this diagnostic does not indicate unusual expert collapse in ZAYA1\-8B\. On the input\-side first projection, ZAYA1\-8B is1\.45×1\.45\\timesthe random\-subspace baseline, close to Qwen’s1\.48×1\.48\\timesand below LFM2 and OLMoE\. On the output projection, ZAYA1\-8B is intermediate at1\.58×1\.58\\timesrandom: higher than OLMoE and Qwen, but below LFM2\. ZAYA1\-8B’s input\-side overlap also rises with depth, from0\.0750\.075in the first depth quartile to0\.1050\.105in the last, indicating more shared input directions in later MoE layers\. We therefore use this appendix only as evidence against obvious expert collapse, not as a claim that ZAYA1\-8B has stronger output\-side specialization than all baselines\.

## Appendix ERare\-token sampling artifacts

We define gibberish as brief, recoverable episodes of seemingly random tokens that interrupt the CoT, and we argue that it is distinct from other forms of degeneration, which may involve cache corruption, numeric issues, repeats, etc\. because these other issues are generally non\-recoverable or at least persistent in the CoT\.

In this set of examples \(Table[XVI](https://arxiv.org/html/2605.05365#A5.T16)\), it is clear that the gibberish token has no relationship to the prefix, and interestingly, the suffix seems to mostly ignore it\. However, this does not mean it is benign, because we found that it is likely to reinforce itself as a result of successful \(rewarded\) traces containing gibberish tokens\.

TABLE XVI:Traces at the gibberish token position\.As a heuristic for gibberish detection we rely on the method described in\(Teamet al\.,[2025a](https://arxiv.org/html/2605.05365#bib.bib244)\)which combines a low logprob mask \(2 nats below uniform\) with token ID mask \(a proxy for token rarity\)\. While this is a good indicator/metric in practice, it is a somewhat arbitrary hard threshold and in our experience it isn’t precise or sensitive enough to be used in training for reward zeroing or loss masking\.

![Refer to caption](https://arxiv.org/html/2605.05365v1/x8.png)Figure 14:Aggregated tokenwise metrics for gibberish responses compared to normal responses\.We collected traces and their top\-3 token probabilities using the vLLM streaming API and aborted the traces around any gibberish\-flagged tokens as they arose using the heuristic mask\. While entropy is spiking in those cases, the most probable tokens still constitute a significant portion of the total probability\. Looking into the top\-kktokens themselves, we found that the top token was a coherent continuation of the prefix, with the most common case being simply "the" as with examples 0, 1, 4, 6 in the table above\. In another more context\-dependent case, any of the top 3 most likely tokens would have been a coherent alternative \(Figure[15](https://arxiv.org/html/2605.05365#A5.F15)\)\.

In general, the baseline/prefix entropy of traces aborted with gibberish tokens tends to be higher \(Figure[14](https://arxiv.org/html/2605.05365#A5.F14)\)\. This could be because the model is challenged by a difficult problem and not confident as a result\. Around the flagged token, entropy spikes, and while the probability mass in the top 3 tokens is median 0\.7, a significant mass remains in the tail of the distribution\. For this reason, we believe that despite the sampled token having an extremely low probability, it was chosen because it was one of many possible low\-probability tokens\.

![Refer to caption](https://arxiv.org/html/2605.05365v1/x9.png)Figure 15:Top token choices for an otherwise coherent trace that sampled a random Tibetan token\. In this example the model was attempting to reason about a challenging isometric ASCII drawing problem from the RLVE gym known as BlockImage\.A natural solution is to use min\-pp\(Nguyenet al\.,[2024](https://arxiv.org/html/2605.05365#bib.bib243)\)sampling to eliminate extremely low probability choices from the sampling distribution\. In our experiments, we find that even in the most degraded test checkpoint available to us, with a baseline rate of 19\.9% of flagged responses, min\-ppsampling with a threshold of10−510^\{\-5\}reduces this to 0\.1%\. In training, to prevent engine/trainer mismatch, we recommend implementing min\-ppreplay, which exposes the kept/omitted token IDs to the trainer for consistency in the min\-pprenormalization\.