Capability-Stratified Degradation in Ternary Language Models

# 1 Introduction
Source: [https://arxiv.org/html/2608.28809](https://arxiv.org/html/2608.28809)
Capability\-Stratified Degradation in Ternary Language Models CLOE V1\.1 \- OneBit AI

Qwen3\.5\-0\.8B→\\rightarrowQAT→\\rightarrowternary CloeA capability, representation, learnability, and deployment study of a752M752\\text\{\\,\}\\mathrm\{M\}\-parameter pretrained language model under ternary conversion\.

Anirudh Malik \| Poojith Devan \| M Sparsh Mehra

Executive takeaway\.The central result is not that ternarisation produces a uniformly weaker language model\. Instead, degradation iscapability\-stratified: specialist factual knowledge and language\-modelling quality degrade sharply, while several forms of commonsense judgement, representational capacity, and downstream task learnability remain measurable\. Across ten benchmarks where Cloe demonstrably discriminates between candidate answers, it retains a mean77\.1%of the full\-precision teacher\. After task\-specific fine\-tuning, it reaches95\.6%of an identically fine\-tuned teacher on SST\-2 and79\.4%on XSum\. The engineering study further shows a634\.7 MB packed artifactand higher prefill/decode throughput than the measured latent representation, while runtime memory remains a separate constraint\.

Keywords:ternary quantisation, 1\.58\-bit models, quantisation\-aware training, small language models, capability retention, representation learning, model compression, edge AI, efficient inference

###### Abstract

Extreme low\-bit inference offers a route toward smaller model artifacts and more constrained deployment\. Ternary language models restrict weights to\{−1,0,\+1\}\\\{\-1,0,\+1\\\}, approaching the information\-theoretic limit oflog2⁡3≈1\.585\\log\_\{2\}3\\approx 1\.585bits per weight\. The practical question for an existing pretrained model, however, is not simply whether the weights can be quantised, but*which capabilities survive the transformation*and whether the resulting model remains useful as a substrate for downstream adaptation\.

We study this question by converting Qwen3\.5\-0\.8B, a752M752\\text\{\\,\}\\mathrm\{M\}\-parameter pretrained model, to ternary weights using 72\.4M tokens of quantisation\-aware training \(QAT\)\. The resulting model, Cloe, is evaluated across 29 benchmarks and through representation\-level diagnostics, matched downstream fine\-tuning, and deployment measurements\. The evidence shows a non\-uniform degradation pattern\. A linear probe recovers 43\.76% of MMLU answers from the full\-precision teacher’s final\-layer representations but only 26\.19% from Cloe, close to the 25% chance level, supporting the interpretation that specialist factual information is substantially lost rather than merely hidden behind an impaired output channel\. At the same time, Cloe retains measurable performance on ten tasks, averaging 77\.1% of teacher performance on that interpretable subset\. More importantly for application\-specific deployment, fine\-tuning raises Cloe from 61\.9% to 89\.8% on SST\-2, corresponding to 95\.6% of the matched fine\-tuned teacher, while XSum reaches 79\.4% teacher retention\. Because the available QAT budget was limited to 72\.4M tokens, we attribute the observed degradation conservatively to a combination of quantization\-induced information loss and incomplete recovery, rather than to ternarisation alone

We also identify an evaluation pitfall: standard answer\-letter scoring initially reported 22\.97% MMLU accuracy because Cloe emitted “A” on 98\.6% of questions\. Continuation scoring and majority\-class guards are therefore used throughout the interpretable evaluation\. The study consequently argues for a capability\-aware view of extreme quantisation: a ternary conversion may be unsuitable as a drop\-in replacement for a general\-purpose pretrained model, yet remain valuable as a compact, adaptable substrate for application\-specific models\.

The deployment economics of language models are increasingly shaped by the cost of moving, storing, and executing model weights\. Reducing numerical precision is one of the most direct ways to reduce the information carried by a parameter tensor\. Ternary models take this idea to an extreme by constraining weights to\{−1,0,\+1\}\\\{\-1,0,\+1\\\}, for which the theoretical information content islog2⁡3≈1\.585\\log\_\{2\}3\\approx 1\.585bits per weight\[[3](https://arxiv.org/html/2608.28809#bib.bib3)\]\.

The research landscape has already established that models can be trained natively under such low\-bit constraints\[[2](https://arxiv.org/html/2608.28809#bib.bib2),[3](https://arxiv.org/html/2608.28809#bib.bib3)\]\. The more operationally difficult question is what happens when an*already pretrained*model is pushed into this regime\. This distinction matters for practitioners because pretrained models contain a large amount of accumulated factual, linguistic, and task\-relevant structure\. If that structure cannot survive conversion, a smaller checkpoint is not necessarily a more useful model\.

This work therefore takes a deliberately diagnostic perspective\. Rather than reducing the outcome to a single aggregate benchmark score, we ask four connected questions:

1. 1\.Capability:Which behaviours survive ternarisation?
2. 2\.Representation:Is poor task performance caused by representational collapse, or by loss of specific stored information?
3. 3\.Learnability:Can the compressed model still acquire a new task through supervised adaptation?
4. 4\.Deployment:What does the packed representation actually buy in storage and inference measurements?

This produces a more useful engineering picture than a single “accuracy after quantisation” number\.

Why this matters for an AI product\.A general\-purpose model and an application\-specific model have different requirements\. A general\-purpose model must preserve broad knowledge\. An application\-specific model can instead be judged by whether it can efficiently represent and execute the capability required by a particular product\. The distinction between*knowledge retention*and*task learnability*is therefore commercially relevant: it determines whether extreme compression should be viewed only as destructive compression or also as a possible route toward compact specialised models\.

## 2 Research Positioning and Related Work

Prior work on ternary language models can be broadly separated into native low\-bit training and conversion of pretrained models\[[4](https://arxiv.org/html/2608.28809#bib.bib4),[7](https://arxiv.org/html/2608.28809#bib.bib7),[8](https://arxiv.org/html/2608.28809#bib.bib8),[9](https://arxiv.org/html/2608.28809#bib.bib9),[10](https://arxiv.org/html/2608.28809#bib.bib10)\]\. BitNet b1\.58 established the feasibility of training language models with ternary weights\. More recent approaches such as CAT\-Q, PTQTP, ScaleQ\-1\.58, and Ternary Mamba investigate how pretrained models can be converted or adapted into low\-bit regimes\. Falcon\-Edge represents another direction: native 1\.58\-bit pretrained models rather than conversion of an existing full\-precision checkpoint\[[11](https://arxiv.org/html/2608.28809#bib.bib11),[12](https://arxiv.org/html/2608.28809#bib.bib12)\]\.

The distinction is important for positioning this study\. The contribution here is*not*claimed to be a new state\-of\-the\-art ternary quantiser\. Instead, the work uses QAT to create a controlled ternary conversion and then asks what information, behaviour, and adaptability survive\.

Table 1:Positioning against major directions represented in the project literature\.Work / directionModel regimeCore approachPrimary emphasisBitNet b1\.58Native ternaryTrain directly at low precisionDemonstrates viability of ternary training and inference\.CAT\-QPretrained conversionCalibration / post\-training conversionDemonstrates that pretrained models can be brought toward ternary operation\.PTQTPPretrained conversionDual trit\-plane representationImproves the representation of ternary weights in PTQ settings\.ScaleQ\-1\.58Pretrained conversionCalibration with additional generated signalsExplores stronger post\-training ternary conversion\.Ternary MambaPretrained conversionGrouped quantisation \+ distillation / QATClosest precedent in spirit for converting an existing model and reporting retention\.Falcon\-EdgeNative low\-bitNative 1\.58\-bit pretrainingShows the alternative strategy of avoiding conversion altogether\.This workPretrained conversionQAT ternarisation \+ capability diagnosticsCharacterises what is lost, what survives, and whether downstream learnability remains after conversion\.The comparison in[table1](https://arxiv.org/html/2608.28809#S2.T1)defines the paper’s central distinction\. A conversion method is normally evaluated by asking whether the resulting model retains benchmark performance\. Here we additionally ask whether a failed benchmark result reflects missing information, an impaired output channel, representational collapse, or a failure that can be repaired through task\-specific training\. This makes the study closer to a*capability characterization*than a conventional quantisation leaderboard entry\.

## 3 Model and Ternary Conversion

### 3\.1 Base model

Cloe is derived from Qwen3\.5\-0\.8B, with approximately752M752\\text\{\\,\}\\mathrm\{M\}parameters\[[13](https://arxiv.org/html/2608.28809#bib.bib13)\]\. The model contains 24 transformer layers, with 18 linear\-attention layers and 6 full\-attention layers\. Its hidden size is 1024 and its head dimension is 256\. The vocabulary contains 248,320 entries, with tied embeddings and language\-model head\.

Table 2:Core configuration of the Cloe conversion\.PropertyValueBase modelQwen3\.5\-0\.8BParameters752M752\\text\{\\,\}\\mathrm\{M\}Transformer layers24Attention structure18 linear\-attention \+ 6 full\-attentionHidden size1024Head dimension256Vocabulary248,320Ternary linear layers186 / 186Distinct weight values per ternary layerExactly 3Measured ternary entropy1\.5821 bits/weightIdeal ternary entropy1\.5850 bits/weightCumulative QAT tokens72\.4MThe values in[table2](https://arxiv.org/html/2608.28809#S3.T2)establish that the conversion is not merely a quantisation\-inspired approximation\. Every targeted linear layer is verified to contain exactly three distinct weight values, and the measured entropy of 1\.5821 bits/weight is very close to the theoretical ternary limit\. This distinction is important because a model can otherwise be described as “ternary” while retaining continuous\-valued weights in parts of its computation\.

### 3\.2 Ternary parameterisation

Each targeted linear layer uses abs\-mean scaling and a straight\-through estimator\[[14](https://arxiv.org/html/2608.28809#bib.bib14)\]\. The effective quantisation can be represented as

s\\displaystyle s=mean⁡\(\|W\|\),\\displaystyle=\\operatorname\{mean\}\(\|W\|\),\(1\)Q⁡\(W\)\\displaystyle Q\(W\)=round⁡\(clamp⁡\(Ws,−1,1\)\)​s,\\displaystyle=\\operatorname\{round\}\\left\(\\operatorname\{clamp\}\\left\(\\frac\{W\}\{s\},\-1,1\\right\)\\right\)s,\(2\)Weff\\displaystyle W\_\{\\mathrm\{eff\}\}=W\+λ​\(Q⁡\(W\)−W\)detach\.\\displaystyle=W\+\\lambda\\left\(Q\(W\)\-W\\right\)\_\{\\mathrm\{detach\}\}\.\(3\)
At inference,λ=1\\lambda=1\. An RMSNorm is applied to activations before the low\-precision matrix multiplication\[[15](https://arxiv.org/html/2608.28809#bib.bib15)\]\.

PretrainedBF16 weightsAbsmeanscalingTernaryprojectionQAT \+ STECloe\{−1,0,\+1\}\\\{\-1,0,\+1\\\}Weight grid is enforced during the forward pass186/186 targeted linear layers verifiedFigure 1:Overview of the conversion pipeline used to obtain Cloe\.
### 3\.3 Verification of the ternary execution path

A potential implementation failure in low\-bit systems is that a checkpoint may contain a quantisation mechanism while the actual forward pass silently uses latent full\-precision weights\. The project therefore verifies the loaded quantisation state across all 186 targeted layers\. The reconstructed weights differ from the exact ternary grid only by floating\-point rounding at a relative scale of approximately1\.8×10−61\.8\\times 10^\{\-6\}\. Settingλ=0\\lambda=0and running the latent weights produces gibberish, providing a direct sanity check that the evaluated model depends on the ternary path\.

Interpretation\.This verification closes a common reproducibility gap in quantisation experiments: the reported behaviour is attributable to the quantised computation rather than an accidentally retained full\-precision path\.

### 3\.4 Important accounting distinction

The phrase “1\.58\-bit model” refers to the ternary weight representation itself, not necessarily to the complete checkpoint\. The 248,320\-token embedding table remains BF16 and is tied to the language\-model head\. Consequently, the overall checkpoint is reported in the project as approximately 6\.47 bits/parameter on disk, rather than a literal 1\.58 bits/parameter for the entire model\.

This distinction is preserved throughout the paper because it separates a*weight\-representation result*from a*whole\-model storage result*\.

## 4 Evaluation Framework

The evaluation is designed around a simple principle: a low\-bit model should not be declared capable merely because a raw benchmark number is numerically high\. Conversely, a low number should not automatically be interpreted as proof that the underlying capability has disappeared\.

Table 3:Evaluation instruments and the question each instrument answers\.InstrumentWhat it measuresWhy it is includedContinuation scoringCandidate\-answer likelihoodAvoids relying on an answer\-letter output channel\.Majority baselineTrivial class\-selection performanceDetects cases where raw accuracy is inflated by label imbalance\.Balanced accuracyClass\-balanced discriminationPrevents constant predictions from being interpreted as capability\.Linear probeRecoverable task information in hidden statesSeparates inaccessible/removed information from output\-head problems\.Effective rankDiversity of representation directionsTests whether quantisation causes broad representational collapse\.Matched fine\-tuningDownstream task learnabilityTests whether useful task structure can still be acquired after conversion\.PerplexityLanguage\-modelling qualityMeasures the cost to token\-level modelling rather than task classification\.The evaluation therefore moves from*behaviour*to*representation*to*learning*\. This progression is important: a benchmark tells us that a model failed a task, while a representation probe and a matched fine\-tuning experiment help explain whether the failure reflects lost information or a loss of adaptability\.

### 4\.1 The answer\-letter failure

The original multiple\-choice evaluation produced 22\.97% MMLU accuracy, which initially suggested catastrophic failure\[[16](https://arxiv.org/html/2608.28809#bib.bib16)\]\. Inspection of the generated answer distribution revealed that Cloe emitted “A” for 13,852 of 14,042 questions, or 98\.6% of the evaluation set\.

Table 4:Answer\-letter distribution that exposed the degenerate MMLU output channel\.ModelABCDCloe, pre\-SFT13,8521131463Cloe, post\-SFT6,8012,5051,2483,488BF16 teacher3012,9142,5248,303The implication is methodological rather than merely negative\. A score produced by answer\-letter elicitation is only meaningful if the model distributes its outputs in a way that permits genuine discrimination\. The project therefore replaces this instrument with length\-normalised continuation log\-probability over the candidate answer text\. This also makes the reported benchmark results more comparable to the ternary literature represented by BitNet, Ternary Mamba, and CAT\-Q\[[2](https://arxiv.org/html/2608.28809#bib.bib2),[3](https://arxiv.org/html/2608.28809#bib.bib3),[7](https://arxiv.org/html/2608.28809#bib.bib7),[11](https://arxiv.org/html/2608.28809#bib.bib11)\]\.

### 4\.2 False\-positive guards

A task is treated as a demonstrated capability only when it exceeds both a chance/majority threshold and a balanced\-accuracy threshold by a two\-standard\-error margin:

accn​o​r​m\\displaystyle\\mathrm\{acc\}\_\{norm\}\>max⁡\(chance,majority\)\+2​S​E,\\displaystyle\>\\max\(\\mathrm\{chance\},\\mathrm\{majority\}\)\+2\\,SE,\(4\)balanced\\displaystyle\\mathrm\{balanced\}\>chance\+2​S​E\.\\displaystyle\>\\mathrm\{chance\}\+2\\,SE\.\(5\)
This guard is important because CoLA, MRPC, and BoolQ produced superficially strong raw accuracies while emitting a constant label\. Balanced accuracy exposed these as degenerate outcomes\.

How failures are treated in this paper\.Invalid or degenerate measurements are reported where necessary to establish measurement validity, but they are not treated as headline capability results\. The main results use the guarded evaluation protocol\.

## 5 Capability Retention

The central behavioural result is that ternarisation does not degrade every capability equally\. On the subset of ten benchmarks for which Cloe demonstrably discriminates between candidate answers, mean performance is 77\.1% of the full\-precision teacher\.

### 5\.1 Teacher–Cloe comparison

To obtain a capability\-retention estimate that is not confounded by tasks on which the evaluation fails to demonstrate meaningful discrimination, we first apply the discrimination guard\. Only tasks satisfying both the majority\-class and chance\-level balanced\-accuracy criteria are included in the primary retention aggregate\. This procedure identifies ten tasks in the present evaluation\. The reported mean retention therefore summarizes these ten discriminative tasks only; the remaining tasks are retained in the analysis and reported separately in table[6](https://arxiv.org/html/2608.28809#S5.T6)\.

Table 5:Benchmark performance and capability retention relative to the full\-precision teacher\. “Bar” denotes the baseline ternary model, while “Over bar” reports the absolute Cloe improvement over that baseline\. Tasks shown are those satisfying both the majority\-baseline and chance\-level balanced\-accuracy criteria\. Retention is computed relative to the full\-precision teacher\.TaskCapabilityCloeTeacherRetentionBarOver barraceReading34\.0%38\.3%89%28\.0%\+6\.0piqaPhysical commonsense60\.7%69\.0%88%50\.7%\+10\.0commonsenseqaCommonsense34\.0%39\.7%86%23\.7%\+10\.3qascKnowledge\-easy31\.3%37\.3%84%15\.0%\+16\.3sciqKnowledge\-easy54\.7%66\.0%83%27\.0%\+27\.7hellaswagCommonsense37\.3%47\.3%79%28\.0%\+9\.3swagCommonsense42\.0%59\.3%71%28\.0%\+14\.0arc\_easyKnowledge\-easy38\.7%55\.7%69%30\.3%\+8\.3sst2Sentiment59\.3%88\.0%67%51\.7%\+7\.7ag\_newsTopic36\.0%66\.0%55%29\.3%\+6\.7MeanTen\-task benchmark mean43\.8%56\.3%77\.1%31\.2%\+11\.6The most important feature of[table5](https://arxiv.org/html/2608.28809#S5.T5)is the spread rather than any single score\. RACE, PIQA, and CommonsenseQA retain 86–89% of teacher performance, while QASC and SciQ remain above 80%\. These results suggest that several forms of plausibility, commonsense judgement, and relatively accessible knowledge remain usable after conversion\[[22](https://arxiv.org/html/2608.28809#bib.bib22),[23](https://arxiv.org/html/2608.28809#bib.bib23)\]\.

At the same time, the 77\.1% figure must be read as a*selected interpretable subset mean*, not as a claim that Cloe retains 77\.1% of all general intelligence\. Tasks for which both models collapse or for which the measurement instrument is degenerate do not provide a defensible retention denominator\.

00101020203030404050506060707080809090100100RACEPIQACommonsenseQAQASCSciQHellaSwag898988888686848483837979Cloe performance as % of teacherFigure 2:Representative retained capabilities\. The main visual message is the non\-uniform degradation profile: several commonsense and plausibility\-oriented tasks remain substantially above chance after ternarisation\.Table 6:Capability boundaries and evaluation\-limited tasks\. “Bar” denotes the baseline ternary model, while “Over bar” reports the absolute Cloe improvement over that baseline\. Retention is reported relative to the full\-precision teacher; unusually large retention values are shown without positive highlighting when the teacher score is itself very low\.TaskCapabilityCloeTeacherRetentionBarOver barwinograndeCoreference53\.7%55\.3%97%50\.0%\+3\.7gsm8kArithmetic reasoning2\.0%4\.0%50%0\.0%\+2\.0copaCausal commonsense57\.0%64\.0%89%55\.0%\+2\.0rteEntailment53\.4%52\.7%101%52\.7%\+0\.7boolqReading62\.7%62\.7%100%62\.7%\+0\.0mrpcParaphrase69\.7%69\.7%100%69\.7%\+0\.0colaGrammaticality65\.7%65\.7%100%65\.7%\+0\.0arc\_challengeKnowledge\-hard27\.1%38\.5%70%27\.8%\-0\.7openbookqaKnowledge\-easy26\.7%30\.7%87%28\.0%\-1\.3cbEntailment48\.2%8\.9%540%50\.0%\-1\.8qnliEntailment49\.0%49\.0%100%51\.0%\-2\.0wicWord sense48\.7%48\.7%100%51\.3%\-2\.7mmluKnowledge\-hard27\.3%27\.0%101%31\.7%\-4\.3medmcqaKnowledge\-expert25\.7%27\.0%95%31\.3%\-5\.7wnliEntailment43\.7%43\.7%100%56\.3%\-12\.7wscCoreference36\.5%36\.5%100%63\.5%\-26\.9truthfulqaTruthfulness28\.3%26\.3%108%100\.0%\-71\.7

## 6 Locating the Lost Information

The behavioural results raise a deeper question\. If some benchmarks fail, is the underlying information still encoded in the hidden states but inaccessible to the standard output head, or has the conversion actually removed the information?

### 6\.1 Linear\-probe diagnostic

A multinomial logistic\-regression probe is trained on final\-layer hidden states from 8,000 MMLU auxiliary\-training examples and evaluated on all 14,042 test questions\. The probe was itself validated on synthetic representations before being applied to Cloe: it recovered 100% accuracy when the answer was explicitly linearly encoded and approximately chance when the signal was absent\.

Table 7:Representation\-level probe results\. Chance for four\-way MMLU classification is 25%\.RepresentationProbe accuracyOwn answer accuracyEffective rankBF16 teacher43\.76%44\.72%557\.5Cloe, post\-SFT26\.19%24\.31%602\.1Cloe, pre\-SFT26\.68%22\.97%538\.9Chance25\.00%25\.00%–The probe result is one of the strongest mechanistic observations in the study\. Cloe’s 26\.19% probe accuracy is only about 1\.2 percentage points above chance, whereas the teacher reaches 43\.76%\. Since the probe provides a relatively generous linear readout of the hidden state, the result supports the interpretation that specialist MMLU information is substantially absent from Cloe’s representation, rather than simply hidden behind a poorly calibrated output head\.

TeacherCloe002020404043\.7643\.7626\.1926\.1925252525Linear\-probe accuracy \(%\)Probe accuracyChanceFigure 3:MMLU information recoverable from final\-layer representations\. Cloe approaches the 25% four\-way chance level while the teacher retains substantial recoverable signal\.

## 7 Representation Capacity Survives

The probe result establishes that a particular class of specialist information is lost\. It does not, however, imply that the entire representation has collapsed\. To test this distinction, we measure effective rank of the hidden\-state representations\.

Table 8:Effective\-rank comparison\.ModelEffective rankRelative to teacherBF16 teacher557\.5100\.0%Cloe, pre\-SFT538\.996\.7%Cloe, post\-SFT602\.1108\.0%The result is notable because the post\-SFT Cloe representation has an effective rank of 602\.1 compared with 557\.5 for the teacher\. Thus the observed knowledge loss cannot be reduced to a simple “all information collapsed into a few directions” explanation\. Cloe has substantial representational diversity even while specific specialist information is poorly recoverable\.

Key distinction\.The evidence supports a separation between*representational capacity*and*stored knowledge*\. Ternarisation can preserve a rich activation geometry without preserving all of the fine\-grained factual information that was encoded in the original pretrained weights\.

## 8 Downstream Learnability

The representation result becomes substantially more useful when combined with a learning experiment\. If the compressed model has lost broad factual knowledge but retains useful representational structure, can it still acquire a new application through fine\-tuning?

We answer this using matched teacher controls: the teacher is fine\-tuned under the same downstream setup, so Cloe is compared with the teacher after both have been adapted to the task\.

### 8\.1 SST\-2 sentiment classification

Cloe begins at 61\.9% and reaches 89\.8% after fine\-tuning\. Relative to the matched fine\-tuned teacher, this corresponds to 95\.6% retention\. The absolute gap is reported alongside the retention number because a retention ratio without the teacher’s absolute score can exaggerate the apparent quality of a weak baseline\.

The important observation is not simply that the score increased\. It is that a model whose specialist knowledge is substantially degraded can still reorganise its parameters sufficiently to acquire a downstream discriminative task\. The measured learning gap is approximately 4\.13 percentage points with an estimated standard error of±2\.39\\pm 2\.39points in the reported experiment\.

### 8\.2 XSum summarisation

The same pattern is observed on a generative task\. Cloe improves from 11\.1 to 19\.0 ROUGE\-L and reaches 79\.4% of the matched teacher\[[28](https://arxiv.org/html/2608.28809#bib.bib28)\]\. It also exceeds the LEAD\-1 copy/input\-order baseline by approximately 2\.4×\\times, supporting the interpretation that the improvement reflects actual summarisation learning rather than simple copying\[[29](https://arxiv.org/html/2608.28809#bib.bib29)\]\.

Table 9:Matched downstream learning results\.TaskCloe beforeCloe afterTeacher retentionSST\-261\.9%89\.8%95\.6%XSum11\.1 ROUGE\-L19\.0 ROUGE\-L79\.4%SST\-2XSum00505010010095\.695\.679\.479\.4Cloe performance relative to matched teacher \(%\)Figure 4:Downstream learnability expressed relative to identically fine\-tuned teacher controls\. The absolute task scores remain important and are reported in[table9](https://arxiv.org/html/2608.28809#S8.T9)\.Why this matters\.The combination of the probe and fine\-tuning results gives the paper its strongest conceptual separation: Cloe does not preserve all of the teacher’s pretrained knowledge, but it retains enough representational structure to learn useful application\-specific behaviour\. This is the basis for viewing extreme quantisation as potentially useful for specialised deployment rather than only as an attempt to reproduce a general\-purpose teacher\.

## 9 Capability Profile: What Changes and What Survives?

The results can be condensed into a capability map\. This table is intentionally more important than a long narrative: the reader should be able to understand the operating envelope of the model before reading the interpretation below it\.

Table 10:Capability\-stratified outcome profile of ternarisation\.PropertyDirectionEvidenceInterpretationSpecialist factual knowledge↓↓↓\\downarrow\\downarrow\\downarrowMMLU \+ linear probeMuch of the teacher’s recoverable specialist information is lost\.Arithmetic reasoning↓↓↓\\downarrow\\downarrow\\downarrowGSM8KNot a reliable general arithmetic model after conversion\.Language modelling↓⁣↓\\downarrow\\downarrowLAMBADA / WikiText\-2Token\-level modelling quality degrades substantially\.Commonsense judgement→\\rightarrow10\-task benchmark subsetSeveral plausibility\-oriented capabilities remain\.Representation diversity→\\rightarrowEffective rankNo evidence of broad representational collapse\.Downstream learnability→\\rightarrowSST\-2 / XSumTask\-specific adaptation remains effective\.Packed storage↑⁣↑\\uparrow\\uparrowDeployment benchmarkPacked artifact is substantially smaller than latent representation\.Inference throughput↑\\uparrowPacked benchmarkPrefill and decode are faster in the measured packed\-vs\-latent comparison\.The table suggests that the conversion should not be interpreted as a single scalar loss of intelligence\. Different properties occupy different layers of the system\. The most severe losses appear in specialist factual retrieval, arithmetic, and language modelling\. In contrast, the model retains useful performance on a subset of commonsense and plausibility tasks, shows substantial hidden\-state diversity, and can be adapted to new downstream objectives\.

This stratification is the central empirical result of the study\. It also changes how a compressed model should be evaluated\. If the intended product only requires a narrow task after supervised adaptation, the relevant question is not whether the compressed model retains all of the teacher’s world knowledge\. The relevant question is whether the compressed substrate retains enough capacity to learn the required task at the target deployment cost\.

## 10 Deployment and Efficiency Characterisation

All deployment measurements were collected on a fixed workstation equipped with an AMD Ryzen 9 7900X CPU, 32 GB system memory, and an NVIDIA GeForce RTX 5070 GPU with 11\.94 GiB of VRAM\. The experiments were conducted under Windows 10 Pro using Python 3\.10\.11, PyTorch 2\.12\.0\.dev20260408\+cu128, CUDA 12\.8, and Transformers 4\.51\.0\. PyTorch was configured with 12 CPU threads, and no explicitCUDA\_VISIBLE\_DEVICES,OMP\_NUM\_THREADS, orMKL\_NUM\_THREADSoverrides were present in the environment\. Both the latent and packed representations were evaluated under the same software and hardware configuration\.

Table 11:Hardware and software environment used for deployment measurements\.ComponentConfigurationCPUAMD Ryzen 9 7900X \(12 cores, 24 threads\)System memory32 GB RAMGPUNVIDIA GeForce RTX 5070GPU VRAM12,227 MiB \( 11\.94 GiB\)Operating systemWindows 10 Pro \(64\-bit\)Python3\.10\.11PyTorch2\.12\.0\.dev20260408\+cu128CUDA12\.8Transformers4\.51\.0PyTorch CPU threads12The capability study establishes what the model retains\. We now examine the engineering consequence of packing the ternary representation\.

The current deployment comparison is between a*packed*representation and the measured*latent*representation\. Because the exact runtime semantics of the latent representation differ from those of a conventional FP/BF16 baseline, we do not use this comparison to claim a universal speedup over all full\-precision implementations\. Instead, it is reported as a representation\-level deployment measurement\.

Table 12:Packed versus latent deployment measurements from the current benchmark\.MetricPackedLatentRelative changeInterpretationDisk footprint634\.65 MB1505\.60 MB2\.37×\\timessmallerMajor storage reductionWeight memory1578\.36 MiB1460\.59 MiB\+8\.1%Runtime memory overheadPeak memory1708\.98 MiB1591\.22 MiB\+7\.4%Modest peak\-memory increaseLoad time2\.858 s1\.570 s1\.82×\\timesslowerInitialization penaltyPrefill1806\.76 tok/s1477\.03 tok/s\+22\.3%Faster steady\-state processingDecode7\.634 tok/s6\.145 tok/s\+24\.2%Faster token generationThree observations stand out from[table12](https://arxiv.org/html/2608.28809#S10.T12)\. First, the packed artifact is dramatically smaller on disk: approximately 634\.7 MB versus 1\.50 GB for the latent representation\. This is the clearest storage benefit in the current measurement\. Second, the packed path shows approximately 22\.3% higher prefill throughput and 24\.2% higher decode throughput in this particular comparison\. Third, storage compression does not translate one\-to\-one into runtime memory: peak memory is approximately 1\.67 GiB for the packed path and slightly higher than the latent measurement\.

The deployment measurements reveal a trade\-off rather than a uniformly dominant representation\. The packed model substantially reduces persistent storage requirements and provides higher steady\-state inference throughput, while incurring modest runtime\-memory overhead and a longer initialization time\. Importantly, the storage, loading, and inference measurements correspond to distinct stages of deployment and therefore need not move in the same direction\. This last point is important for product engineering\. A compact model file and a low\-memory runtime are related but distinct optimisation targets\. The deployment stack may need additional buffers, unpacking structures, kernels, or other runtime state\. Consequently, the paper reports both disk footprint and peak memory rather than using one as a proxy for the other\.

PackedLatent001,0001\{,\}0002,0002\{,\}0003,0003\{,\}000634\.65634\.653,037\.443\{,\}037\.44MB\(a\)Disk footprintPackedLatent00224466887\.637\.636\.156\.15tokens/s\(b\)Decode throughputPackedLatent005005001,0001\{,\}0001,5001\{,\}5002,0002\{,\}0001,806\.761\{,\}806\.761,477\.031\{,\}477\.03tokens/s\(c\)Prefill throughputPackedLatent001122332\.862\.861\.571\.57seconds\(d\)Load time
Figure 5:Current packed\-versus\-latent deployment measurements\. The strongest observed benefit is storage footprint, while throughput also improves in the measured execution path\. Runtime memory and loading remain separate engineering considerations\.
## 11 Failure Modes as Operating\-Boundary Conditions

The study includes negative results because a commercially useful technical paper must identify where the system should*not*be used\. These results are therefore presented as operating boundaries rather than as the central narrative\.

Table 13:Selected limitations and what they imply for deployment\.ConsiderationCurrent evidenceEngineering implicationSpecialist knowledgeMMLU / ARC\-C / MedMCQA near chanceDo not treat Cloe as a drop\-in replacement for a broad knowledge model\.ArithmeticGSM8K≈\\approx2\.0%Applications requiring reliable symbolic arithmetic need dedicated mechanisms\.TruthfulnessTruthfulQA 28\.3%; high unanswerable\-question confabulationSafety\-sensitive factual deployment requires task\-specific safeguards\.Language modellingLAMBADA perplexity 12\.1×\\timesworse; WikiText\-2 3\.1×\\timesworseGeneral free\-form generation quality is materially degraded\.Whole\-checkpoint compressionEmbeddings remain BF16; overall checkpoint≈\\approx6\.47 bits/parameterThe 1\.58\-bit figure should be reserved for the ternary weight representation\.Runtime memoryPacked peak≈\\approx1\.67 GiBDisk compression should not be interpreted as equal runtime\-memory compression\.Fine\-tuning uncertaintySingle reported seedAdditional seeds are needed for production\-grade variance estimates\.Quantiser comparisonNo PTQ control in the current studyThe study does not claim QAT is superior to every alternative conversion recipe\.The limitations do not invalidate the main capability\-stratification result\. Instead, they define its scope\. The current Cloe checkpoint should not be presented as a general\-purpose replacement for the teacher\. The stronger and more defensible interpretation is that extreme ternarisation produces a model with a distinctive capability profile: broad pretrained knowledge is substantially damaged, but representational diversity and task\-specific learnability remain useful\.

## 12 Application and Product Implications

The technical findings suggest a deployment paradigm different from simply trying to preserve every capability of a large general\-purpose model\.

### 12\.1 From knowledge preservation to task\-specific adaptation

The combined results can be viewed as a three\-stage transformation:

General\-purposepretrained modelTernarycompressed substrateTask\-specificadapted modelExtreme compressionFine\-tuningBroad knowledge is availableSome knowledge is lost,capacity remainsApplication capability canbe recovered / learnedFigure 6:Conceptual deployment thesis suggested by the combined experiments\.The proposed interpretation is therefore not that every application should replace a general\-purpose model with Cloe\. Instead, applications with narrow or well\-defined objectives may value a compact substrate whose task\-specific capability can be learned after compression\.

Table 14:Potential application mapping\. These are deployment hypotheses, not claims that every scenario has been experimentally validated in this study\.ScenarioWhy compression mattersEvidence / conditionEdge / constrained inferenceSmaller model artifact can reduce storage and transfer burdenCurrent packed artifact is 634\.7 MB\.Private or local assistantsSmaller local model can reduce dependence on remote inferenceRequires further end\-to\-end device validation\.Task\-specific classifiersGeneral knowledge is less important than adaptationSST\-2 reaches 95\.6% matched\-teacher retention\.Specialised summarisationTask learnability matters more than broad pretrained knowledgeXSum reaches 79\.4% teacher retention after adaptation\.Cost\-sensitive inferenceThroughput and representation size can affect serving economicsCurrent packed path shows \+22\.3% prefill and \+24\.2% decode versus latent measurement\.General\-purpose factual assistantBroad pretrained knowledge is essentialNot recommended from current evidence\.The market\-facing implication should therefore be framed around*deployment economics and specialization*, not around an unsupported claim that ternary conversion preserves general intelligence\. The current results provide evidence for compact storage, measurable inference benefits in the tested packed path, and strong downstream adaptation on selected tasks\. They do not establish a universal advantage across hardware, model scales, or workloads\.

## 13 Discussion

### 13\.1 A capability\-stratified view of quantisation

The most useful conceptual outcome of the study is a separation of three properties that are often conflated:

1. 1\.Stored knowledge:substantially reduced, particularly for specialist factual information\.
2. 2\.Representational capacity:not obviously destroyed, as indicated by effective rank\.
3. 3\.Learnability:remains substantial on the tested downstream tasks\.

This separation provides a more informative description than calling the conversion simply “successful” or “failed\.” It explains why a model can simultaneously perform poorly on broad knowledge benchmarks and still reach near\-teacher performance after supervised adaptation on a targeted task\.

### 13\.2 Why the evaluation methodology is part of the contribution

The initial MMLU result demonstrates that low\-bit models can expose failure modes that conventional evaluation pipelines are not designed to distinguish\. Answer\-letter bias can convert a degenerate output policy into an apparently meaningful accuracy number\. Likewise, retention ratios can become misleading when the teacher itself collapses on a task\.

The practical lesson is that low\-bit model evaluation should combine behavioural benchmarks with representation\-level and learning\-level diagnostics\. A useful evaluation stack therefore contains:

Benchmark→Guard→Representation probe→Matched adaptation\\boxed\{\\text\{Benchmark\}\\rightarrow\\text\{Guard\}\\rightarrow\\text\{Representation probe\}\\rightarrow\\text\{Matched adaptation\}\}
This is particularly important when the research goal is to understand the*deployment envelope*rather than merely to rank checkpoints\.

### 13\.3 What the results do not establish

The study does not establish that QAT conversion is universally superior to PTQ, that Cloe is a general\-purpose replacement for Qwen3\.5\-0\.8B, or that the current packed benchmark translates directly into lower cost on every target device\. Similarly, the 1\.5821 bits/weight entropy describes the ternary weight tensors, whereas the complete checkpoint remains larger because the embedding/LM\-head structure is not ternarised\.

The present study does not include a recovery\-budget ablation\. Consequently, the observed capability gap cannot be uniquely decomposed into representational loss caused by ternarisation and capability that remains unrecovered because of limited QAT\. A controlled sweep over QAT budgets \(e\.g\., 20M, 72\.4M, and 200M tokens\) would establish the capability–compute recovery curve and provide a stronger estimate of the irreducible effect of ternarisation\.

## 14 Conclusion

This study investigates what happens when a pretrained small language model is converted into a ternary representation using quantisation\-aware training\. The answer is not uniform degradation\. Instead, the conversion produces a structured capability profile\.

The most significant loss is specialist pretrained knowledge: the linear probe falls from 43\.76% for the BF16 teacher to 26\.19% for Cloe, close to the 25% chance level\. At the same time, Cloe retains meaningful performance across a subset of commonsense and plausibility\-oriented benchmarks, with a mean 77\.1% teacher retention on ten demonstrably interpretable tasks\. Effective rank remains high, and downstream adaptation is particularly encouraging: Cloe reaches 95\.6% of the matched teacher on SST\-2 and 79\.4% on XSum\. Because all reported post\-quantization results correspond to a single 72\.4M token recovery budget, these measurements characterize the outcome of this recovery regime rather than an irreducible capability limit of the ternary representation\.

The engineering measurements add a second dimension\. The current packed artifact occupies approximately 634\.7 MB on disk and shows higher prefill and decode throughput than the measured latent representation\. Runtime memory, however, does not fall in direct proportion to disk footprint, demonstrating why storage, runtime memory, and throughput should be measured separately\.

Taken together, the results support a practical thesis:

Extreme quantisation can destroy broad pretrained knowledge without destroying the ability to represent and learn useful application\-specific behaviour\.

## References

- \[1\]A\. Yang et al\.Qwen3 Technical Report\.arXiv:2505\.09388, 2025\.[https://arxiv\.org/abs/2505\.09388](https://arxiv.org/abs/2505.09388)\.
- \[2\]H\. Wang, S\. Ma, L\. Dong, S\. Huang, H\. Wang, L\. Ma, F\. Yang, R\. Wang, Y\. Wu, and F\. Wei\.BitNet: Scaling 1\-bit Transformers for Large Language Models\.arXiv:2310\.11453, 2023\.[https://arxiv\.org/abs/2310\.11453](https://arxiv.org/abs/2310.11453)\.
- \[3\]S\. Ma, H\. Wang, L\. Ma, L\. Wang, W\. Wang, S\. Huang, L\. Dong, R\. Wang, J\. Xue, and F\. Wei\.The Era of 1\-bit LLMs: All Large Language Models are in 1\.58 Bits\.arXiv:2402\.17764, 2024\.[https://arxiv\.org/abs/2402\.17764](https://arxiv.org/abs/2402.17764)\.
- \[4\]E\. Frantar, S\. Ashkboos, T\. Hoefler, and D\. Alistarh\.GPTQ: Accurate Post\-Training Quantization for Generative Pre\-trained Transformers\.In*International Conference on Learning Representations \(ICLR\)*, 2023\.arXiv:2210\.17323\.[https://arxiv\.org/abs/2210\.17323](https://arxiv.org/abs/2210.17323)\.
- \[5\]X\. Ouyang, T\. Ge, T\. Hartvigsen, Z\. Zhang, H\. Mi, and D\. Yu\.Low\-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics*, 2025\.arXiv:2411\.17691\.[https://arxiv\.org/abs/2411\.17691](https://arxiv.org/abs/2411.17691)\.
- \[6\]G\. Hinton, O\. Vinyals, and J\. Dean\.Distilling the Knowledge in a Neural Network\.arXiv:1503\.02531, 2015\.[https://arxiv\.org/abs/1503\.02531](https://arxiv.org/abs/1503.02531)\.
- \[7\]S\. Wang, C\. Li, Y\. Kang, J\. Fan, and A\. Yao\.CAT\-Q: Cost\-efficient and Accurate Ternary Quantization for LLMs\.In*International Conference on Machine Learning \(ICML\)*, 2026\.arXiv:2606\.26650\.[https://arxiv\.org/abs/2606\.26650](https://arxiv.org/abs/2606.26650)\.
- \[8\]Z\. Zhao, Z\. Xu, Z\. Chen, X\. Hu, Z\. Jiang, and D\. Yang\.TWLA: Achieving Ternary Weights and Low\-Bit Activations for LLMs via Post\-Training Quantization\.arXiv:2606\.13054, 2026\.[https://arxiv\.org/abs/2606\.13054](https://arxiv.org/abs/2606.13054)\.
- \[9\]H\. Xiao, R\. Yang, Q\. Yang, W\. Xu, Z\. Li, Y\. Su, Z\. Liu, H\. Yang, and N\. Wong\.PTQTP: Post\-Training Quantization to Trit\-Planes for Large Language Models\.arXiv:2509\.16989, 2025\.[https://arxiv\.org/abs/2509\.16989](https://arxiv.org/abs/2509.16989)\.
- \[10\]S\. Wang, C\. Li, Y\. Kang, J\. Fan, and A\. Yao\.Attend to Your Own Thoughts: Breaking the Barrier for Post\-Training Quantization of Reasoning LLMs through the Lens of 1\.58\-Bit Quantization\.arXiv:2608\.01078, 2026\.[https://arxiv\.org/abs/2608\.01078](https://arxiv.org/abs/2608.01078)\.
- \[11\]R\. Ganesaraja, S\. D\. Panse, and S\. N\.Ternary Mamba: Grouped Quantization\-Aware Training of W1\.58A16 State Space Models\.arXiv:2606\.18114, 2026\.[https://arxiv\.org/abs/2606\.18114](https://arxiv.org/abs/2606.18114)\.
- \[12\]Falcon\-LLM Team\.Falcon\-E, a Series of Powerful, Universal and Fine\-tunable 1\.58bit Language Models\.Technical report / project publication, 2025\.[https://falcon\-lm\.github\.io/blog/falcon\-edge/](https://falcon-lm.github.io/blog/falcon-edge/)\.
- \[13\]Qwen Team\.Qwen3\.5: Towards Native Multimodal Agents\.2026\.[https://qwen\.ai/blog?id=qwen3\.5](https://qwen.ai/blog?id=qwen3.5)\.
- \[14\]Y\. Bengio, N\. Léonard, and A\. Courville\.Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation\.arXiv:1308\.3432, 2013\.[https://arxiv\.org/abs/1308\.3432](https://arxiv.org/abs/1308.3432)\.
- \[15\]C\. Steinmetz, G\. Childress, A\. Herbst, G\. Jones, J\. Singh, E\. Vang, and K\. Weinstock\.An Extra RMSNorm is All You Need for Fine Tuning to 1\.58 Bits\.arXiv:2505\.08823, 2025\.[https://arxiv\.org/abs/2505\.08823](https://arxiv.org/abs/2505.08823)\.
- \[16\]D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt\.Measuring Massive Multitask Language Understanding\.In*International Conference on Learning Representations \(ICLR\)*, 2021\.arXiv:2009\.03300\.[https://arxiv\.org/abs/2009\.03300](https://arxiv.org/abs/2009.03300)\.
- \[17\]P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord\.Think You Have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge\.arXiv:1803\.05457, 2018\.[https://arxiv\.org/abs/1803\.05457](https://arxiv.org/abs/1803.05457)\.
- \[18\]T\. Mihaylov, P\. Clark, T\. Khot, and A\. Sabharwal\.Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering\.In*Proceedings of EMNLP*, 2018\.[https://aclanthology\.org/D18\-1260/](https://aclanthology.org/D18-1260/)\.
- \[19\]J\. Welbl, N\. F\. Liu, and M\. Gardner\.Crowdsourcing Multiple Choice Science Questions\.arXiv:1707\.06209, 2017\.[https://arxiv\.org/abs/1707\.06209](https://arxiv.org/abs/1707.06209)\.
- \[20\]T\. Khot, P\. Clark, M\. Guerquin, P\. Jansen, and A\. Sabharwal\.QASC: A Dataset for Question Answering via Sentence Composition\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, 2020\.arXiv:1910\.11473\.[https://arxiv\.org/abs/1910\.11473](https://arxiv.org/abs/1910.11473)\.
- \[21\]R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. Choi\.HellaSwag: Can a Machine Really Finish Your Sentence?In*ACL*, 2019\.arXiv:1905\.07830\.[https://arxiv\.org/abs/1905\.07830](https://arxiv.org/abs/1905.07830)\.
- \[22\]Y\. Bisk, R\. Zellers, R\. Le Bras, J\. Gao, and Y\. Choi\.PIQA: Reasoning about Physical Commonsense in Natural Language\.In*AAAI*, 2020\.arXiv:1911\.11641\.[https://arxiv\.org/abs/1911\.11641](https://arxiv.org/abs/1911.11641)\.
- \[23\]A\. Talmor, J\. Herzig, N\. Lourie, and J\. Berant\.CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge\.In*NAACL*, 2019\.arXiv:1811\.00937\.[https://arxiv\.org/abs/1811\.00937](https://arxiv.org/abs/1811.00937)\.
- \[24\]K\. Sakaguchi, R\. Le Bras, C\. Bhagavatula, and Y\. Choi\.WinoGrande: An Adversarial Winograd Schema Challenge at Scale\.In*AAAI*, 2020\.arXiv:1907\.10641\.[https://arxiv\.org/abs/1907\.10641](https://arxiv.org/abs/1907.10641)\.
- \[25\]S\. Lin, J\. Hilton, and O\. Evans\.TruthfulQA: Measuring How Models Mimic Human Falsehoods\.In*ACL*, 2022\.arXiv:2109\.07958\.[https://arxiv\.org/abs/2109\.07958](https://arxiv.org/abs/2109.07958)\.
- \[26\]A\. Wang, A\. Singh, J\. Michael, F\. Hill, O\. Levy, and S\. R\. Bowman\.GLUE: A Multi\-Task Benchmark and Analysis Platform for Natural Language Understanding\.In*ICLR*, 2019\.arXiv:1804\.07461\.[https://arxiv\.org/abs/1804\.07461](https://arxiv.org/abs/1804.07461)\.
- \[27\]A\. Wang, Y\. Pruksachatkun, N\. Nangia, A\. Singh, J\. Michael, F\. Hill, O\. Levy, and S\. R\. Bowman\.SuperGLUE: A Stickier Benchmark for General\-Purpose Language Understanding Systems\.In*NeurIPS*, 2019\.arXiv:1905\.00537\.[https://arxiv\.org/abs/1905\.00537](https://arxiv.org/abs/1905.00537)\.
- \[28\]C\.\-Y\. Lin\.ROUGE: A Package for Automatic Evaluation of Summaries\.In*Text Summarization Branches Out*, ACL, 2004\.
- \[29\]S\. Narayan, S\. B\. Cohen, and M\. Lapata\.Don’t Give Me the Details, Just the Summary\! Topic\-Aware Convolutional Neural Networks for Extreme Summarization\.In*EMNLP*, 2018\.arXiv:1808\.08745\.[https://arxiv\.org/abs/1808\.08745](https://arxiv.org/abs/1808.08745)\.