RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment

# RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment
Source: [https://arxiv.org/html/2605.28827](https://arxiv.org/html/2605.28827)
Jaber Jaber RightNow AI jaber@rightnowai\.co &Osama Jaber RightNow AI osama@rightnowai\.co

###### Abstract

Open Arabic large language models split into two classes: sub\-1B multilingual models that treat Arabic as an afterthought \(Qwen2\.5\-0\.5B, Falcon\-H1\-0\.5B\), and 7B–70B Arabic\-specialized models that require a server to run \(Jais, AceGPT, ALLaM, SILMA\)\. The one published attempt at a sub\-2B Arabic\-specialized model, Kuwain\-1\.5B, never released its weights\. We presentRightNow\-Arabic\-0\.5B\-Turbo, a 518M\-parameter Arabic\-specialized decoder LLM built on Qwen2\.5\-0\.5B\. The pipeline adds 27,032 Arabic tokens via mean\-subtoken initialization, continues pretraining on 504M Arabic tokens on8×8\\timesH100 with FSDP, FlashAttention varlen packing, and Liger fused kernels, then applies supervised fine\-tuning on 129,116 Arabic instruction pairs with response\-only loss masking, direct preference optimization on 6,750 Arabic preference pairs, and weight soup merging across three checkpoints\. On three lm\-evaluation\-harness Arabic benchmarks \(COPA\-ar, Arabic HellaSwag, ArabicMMLU\) the merged model reaches 35\.9% mean accuracy, beats every same\-class open model, ties Falcon\-H1\-1\.5B on COPA\-ar \(58\.4%\) at one\-third the size, and recovers 67% of SILMA\-9B’s mean at 1/18 the parameters\. The edge build quantizes to 398 MB \(q4\_k\_m\) and delivers 635 tokens/s at batch size 1 on a single H100 via llama\.cpp\. All code \(5,555 lines across 25 scripts\), weights \(bf16, int8, and four GGUF quantizations\), and benchmark scripts are released openly at[https://huggingface\.co/RightNowAI/RightNow\-Arabic\-0\.5B\-Turbo](https://huggingface.co/RightNowAI/RightNow-Arabic-0.5B-Turbo)\.

![[Uncaptioned image]](https://arxiv.org/html/2605.28827v1/rightnow_logo.png)

## 1Introduction

Arabic is spoken by more than 400 million people and is the official language of 25 countries, yet open Arabic language model weights trail English models by an uncomfortable margin\. The strongest open Arabic models, Jais\(Sengupta et al\.,[2023](https://arxiv.org/html/2605.28827#bib.bib16)\), AceGPT\(Huang et al\.,[2023](https://arxiv.org/html/2605.28827#bib.bib7)\), ALLaM\(Bari et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib1)\), and SILMA, all live in the 7B–70B parameter range\. They achieve strong benchmark scores but demand 16–140 GB of memory and are out of reach for phone, laptop\-CPU, and embedded deployment\. At the other end of the spectrum, the sub\-1B open models that can actually run on commodity hardware, Qwen2\.5\-0\.5B\-Instruct\(Qwen Team et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib14)\), Falcon\-H1\-0\.5B, treat Arabic as one of a hundred languages and do not allocate Arabic\-specialized vocabulary or training data\.

The one published effort that targeted this gap, Kuwain\-1\.5B\(Hennara et al\.,[2025](https://arxiv.org/html/2605.28827#bib.bib4)\), proposed “language injection” on TinyLlama\(Zhang et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib18)\)and reported an 8% average Arabic improvement over the base\. Misraj\.ai \(Khobar, Saudi Arabia\) released the paper but did not release the weights\. This leaves practical Arabic deployment on edge devices without a real option: the smallest downloadable Arabic\-specialized decoder LLM on HuggingFace is nothing\.

We presentRightNow\-Arabic\-0\.5B\-Turbo, a 518M\-parameter Arabic\-specialized decoder LLM\. The model is built by injecting 27,032 new Arabic tokens into Qwen2\.5\-0\.5B\(Qwen Team et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib14)\), continuing pretraining on 504M Arabic tokens, supervised fine\-tuning with response\-only loss masking on 129,116 instruction pairs, applying direct preference optimization\(Rafailov et al\.,[2023](https://arxiv.org/html/2605.28827#bib.bib15)\), and merging three checkpoints via weight averaging\(Wortsman et al\.,[2022](https://arxiv.org/html/2605.28827#bib.bib17)\)\. The result is the smallest open Arabic\-specialized decoder LLM released to date\. On COPA\-ar it matches Falcon\-H1\-1\.5B at one\-third the parameter count; quantized to 4 bits it fits in 398 MB and reaches 635 tokens/s at bs=1 on a single H100 via llama\.cpp\.

#### Key insight\.

Sub\-1B Arabic models do not need new training techniques, every component of our pipeline exists in prior work\. What they need is careful orchestration: vocabulary expansion that actually lowers Arabic token fertility, a data loader that avoids multi\-rank HuggingFace Hub stalls, SFT that masks prompt tokens so the loss signal is concentrated on the assistant response, and an export path that produces artifacts small enough for the device class the model is targeting\. Every mistake costs the accuracy budget of a model already near its scale ceiling\.

#### Contributions\.

1. 1\.The smallest open Arabic\-specialized decoder LLM on HuggingFace as of the submission date, at 518M parameters and 398 MB on disk \(q4\_k\_m\)\.
2. 2\.A reproducible vocabulary\-injection pipeline that mean\-subtoken\-initializes 27,032 Arabic tokens into Qwen2\.5\-0\.5B and cuts Arabic tokenizer fertility from 2\.18 to 1\.80 tokens per word \(17\.3% reduction\)\.
3. 3\.Direct head\-to\-head benchmarks against 6 competing models on three Arabic lm\-evaluation\-harness tasks under identical methodology, showing where a 0\.5B model wins and where it loses to models 3–18×\\timeslarger\.
4. 4\.A weight\-soup merge ablation over 7 variants \(SLERP and LERP across the DPO/SFT/pretrain checkpoints\), selecting a configuration that improves mean accuracy by 0\.44 absolute points over the DPO endpoint alone\.
5. 5\.llama\.cpp GGUF conversion producing 4 quantization levels \(f16, q8\_0, q5\_k\_m, q4\_k\_m\) with measured throughput of 582–646 tokens/s at bs=1 on H100\.
6. 6\.Full open release: 5,555 lines of Python across 25 scripts and 13 runtime modules, all training configs, all benchmark scripts, all intermediate checkpoints, and the final weights\.

## 2Related Work

#### Arabic\-specialized LLMs\.

Jais\(Sengupta et al\.,[2023](https://arxiv.org/html/2605.28827#bib.bib16)\)from Inception/MBZUAI set the initial standard with a GPT\-3\-style 13B/30B decoder pretrained on a mixture of Arabic, English, and code\. AceGPT\(Huang et al\.,[2023](https://arxiv.org/html/2605.28827#bib.bib7)\)from FreedomIntelligence targeted cultural alignment via RLAIF and released 7B/13B chat variants\. ALLaM\(Bari et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib1)\)from SDAIA \(Saudi Arabia\) scaled to 7B/13B/34B/70B using vocabulary expansion and Arabic\-English mixed pretraining\. SILMA is a 9B Arabic\-specialized model released by SILMA AI without an accompanying paper\. All four operate in the 7B\+ parameter range and target server deployment\.

#### Sub\-2B Arabic models\.

The closest prior work to ours is Kuwain\-1\.5B\(Hennara et al\.,[2025](https://arxiv.org/html/2605.28827#bib.bib4)\), published by Misraj\.ai in April 2025\. Kuwain injects Arabic vocabulary into TinyLlama\-1\.1B\(Zhang et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib18)\)and reports an average 8% improvement on Arabic benchmarks over the base\. Mutarjim\(Misraj\.ai,[2025](https://arxiv.org/html/2605.28827#bib.bib10)\)builds on Kuwain for Arabic\-English translation\. Critically, neither Kuwain nor Mutarjim weights are published on HuggingFace at the time of this submission, so neither can be directly compared or deployed\. RightNow\-Arabic\-0\.5B\-Turbo is strictly smaller \(518M vs 1\.5B\) and fully open\.

#### Vocabulary expansion for language adaptation\.

Extending a pretrained tokenizer with new vocabulary is a standard adaptation technique\. ALLaM\(Bari et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib1)\)argues that vocabulary expansion paired with English anchoring prevents catastrophic forgetting\. The specific initialization scheme we use, averaging the old embeddings of a new token’s sub\-piece decomposition, was formalized by WECHSEL\(Minixhofer et al\.,[2022](https://arxiv.org/html/2605.28827#bib.bib11)\)and subsequently adopted by Kuwain and ALLaM\.

#### Small\-model training stack\.

Training on 8×\\timesH100 SXM5 demands specific infrastructure\. FlashAttention\(Dao et al\.,[2022](https://arxiv.org/html/2605.28827#bib.bib2)\)and its varlen variant remove the memory and kernel\-launch overhead of dense attention\. PyTorch FSDP\(Zhao et al\.,[2023](https://arxiv.org/html/2605.28827#bib.bib19)\)shards optimizer and gradient state; we use the\_HYBRID\_SHARD\_ZERO2strategy\. Liger Kernel\(Hsu et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib6)\)replaces RMSNorm, RoPE, SwiGLU, and fused linear cross\-entropy with Triton implementations, saving memory and time\. Our vocabulary expansion increases the output projection to 178,697 rows, which makes fused linear cross\-entropy essential: materialized float32 logits at batch 16×\\times4096×\\times178697 would require 44 GiB\.

#### Post\-training\.

Direct preference optimization\(Rafailov et al\.,[2023](https://arxiv.org/html/2605.28827#bib.bib15)\)reformulates RLHF as a classification loss over preference pairs, eliminating the reward model\. Model souping\(Wortsman et al\.,[2022](https://arxiv.org/html/2605.28827#bib.bib17)\)averages weights across multiple fine\-tuned checkpoints and has been shown to improve out\-of\-distribution generalization\. Our pipeline applies both: DPO on 6,750 Arabic preference pairs from argilla\-dpo\-mix\-7k\-arabic, and a linear\-weight soup across the DPO, SFT, and pretrain checkpoints\.

#### Data\.

Pretraining uses Arabic Wikipedia \(504M tokens after merging with the new tokenizer\) via the wikimedia/wikipedia corpus\. We originally planned to mix in FineWeb\-2\-ar\(Penedo et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib13)\)but encountered persistent HuggingFace Hub 504 stalls during multi\-rank streaming; we resolved this by pre\-tokenizing the corpus to a flatint32memmap and sampling windows per rank\. Instruction tuning uses a merge of five Arabic instruction datasets; preference tuning uses the argilla Arabic DPO mix\. All data is publicly available\. Evaluation uses three tasks from lm\-evaluation\-harness\(Gao et al\.,[2021](https://arxiv.org/html/2605.28827#bib.bib3)\): COPA\-ar, Arabic MT HellaSwag, and the Arabic Leaderboard ArabicMMLU\(Koto et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib8)\)\. Table[1](https://arxiv.org/html/2605.28827#S2.T1)summarizes the open Arabic LLM landscape and shows where our model sits\.

Table 1:The open Arabic LLM landscape\. Our model is the only sub\-1B Arabic\-specialized decoder with publicly available weights\.

## 3Method

Figure[1](https://arxiv.org/html/2605.28827#S3.F1)shows the full pipeline\. The model starts as Qwen2\.5\-0\.5B \(base, not Instruct\), gains 27,032 new Arabic tokens, sees 504M pretraining tokens, 129,116 SFT examples, and 6,750 DPO pairs, and ends as a merged checkpoint exported in bf16, int8, and four GGUF quantizations\.

Qwen2\.5\-0\.5B base \(494M, 151,665 vocab\)Tokenizer surgery\+27,032 Arabic tokens \(mean\-subtoken init\)→\\to178,697 vocabContinued pretraining504M arwiki tokens, 2500 steps, 8×\\timesH100 FSDP \+ FA\-varlen \+ LigerSupervised fine\-tuning129,116 instructions, 5 epochs, response\-only loss maskingDirect preference optimization6,750 Arabic preference pairs, 2 epochs, single\-GPUWeight souplinear\(DPO 0\.5, SFT 0\.25, pretrain 0\.25\)Exportbf16 \(1\.04 GB\), int8 \(664 MB\), GGUF \{f16, q8\_0, q5\_k\_m, q4\_k\_m\}Figure 1:Training and deployment pipeline\. Each stage produces a checkpoint that feeds the next; the soup stage also consumes the pretrain and SFT checkpoints directly\.### 3\.1Tokenizer surgery

Qwen2\.5\-0\.5B’s original byte\-level BPE tokenizer has 151,665 tokens and encodes Arabic text at 2\.18 tokens per word on a held\-out sample, roughly1\.4×1\.4\\timesthe English rate\. To lower this, we train a SentencePiece unigram model\(Kudo and Richardson,[2018](https://arxiv.org/html/2605.28827#bib.bib9)\)with 32,000 tokens on a 12\.54 GB Arabic corpus composed of 21\.5M lines of Arabic Wikipedia \(via wikimedia/wikipedia 20231101\.ar\) and 5\.4 GB of filtered Arabic web text\. We apply standard Arabic text normalization before training: NFKC, alif\-variant normalization \(hamza\-above, hamza\-below, madda→\\tobare alif\), tatweel stripping, and ya\-variant normalization\.

The resulting SentencePiece model is merged into the Qwen tokenizer with a deduplication pass: any SentencePiece piece whose surface string already round\-trips to a single existing Qwen token is discarded, leaving 27,032 net\-new Arabic tokens\. We then callmodel\.resize\_token\_embeddings\(178697\)and initialize the 27,032 new embedding rows as in Algorithm[1](https://arxiv.org/html/2605.28827#alg1)\.

Algorithm 1Mean\-subtoken embedding initialization0:old tokenizer

ToldT\_\{\\text\{old\}\}, new token strings

𝒩\\mathcal\{N\}, embedding matrix

E∈ℝVnew×dE\\in\\mathbb\{R\}^\{V\_\{\\text\{new\}\}\\times d\}, original vocabulary size

VoldV\_\{\\text\{old\}\}
1:

Eold←E\[:Vold\]\.clone\(\)E\_\{\\text\{old\}\}\\leftarrow E\[:V\_\{\\text\{old\}\}\]\.\\text\{clone\}\(\)// freeze original rows

2:for allnew token id

nnwith surface string

ssin

𝒩\\mathcal\{N\}do

3:

I←Told\.encode​\(s,add\_special\_tokens=False\)I\\leftarrow T\_\{\\text\{old\}\}\.\\text\{encode\}\(s,\\text\{add\\\_special\\\_tokens\}=\\text\{False\}\)
4:if

\|I\|=0\|I\|=0then

5:

I←\[Told\.unk\_id\]I\\leftarrow\[T\_\{\\text\{old\}\}\.\\text\{unk\\\_id\}\]
6:endif

7:

E​\[n\]←1\|I\|​∑i∈IEold​\[i\]E\[n\]\\leftarrow\\frac\{1\}\{\|I\|\}\\sum\_\{i\\in I\}E\_\{\\text\{old\}\}\[i\]
8:endfor

9:iflm\_head\.weight\.data\_ptr

==embed\.weight\.data\_ptrthen

10:// tied embeddings: lm\_head rows update automatically

11:else

12:

Elm\_head\[Vold:Vnew\]←E\[Vold:Vnew\]E\_\{\\text\{lm\\\_head\}\}\[V\_\{\\text\{old\}\}:V\_\{\\text\{new\}\}\]\\leftarrow E\[V\_\{\\text\{old\}\}:V\_\{\\text\{new\}\}\]
13:endif

After merging, we verify thatmodel\.lm\_head\.weight\.data\_ptr\(\)still equalsmodel\.get\_input\_embeddings\(\)\.weight\.data\_ptr\(\), Qwen2\.5\-0\.5B ties these, so that both the input embeddings and output projection receive the mean\-init rows simultaneously\. On a 1000\-word Arabic sample, fertility drops from 2\.08 to 1\.77 tokens per word \(14\.7% reduction\)\. On a larger held\-out 368\-word sample used for the final measurement, fertility drops from 2\.18 to 1\.80 \(17\.3% reduction\)\.

### 3\.2Continued pretraining

The merged model is trained for 2,500 optimizer steps on a pre\-tokenized flatint32memmap containing 504,318,692 tokens of Arabic Wikipedia encoded with the new tokenizer\. We use a per\-GPU micro\-batch of 16 sequences of 4096 tokens with gradient accumulation of 8, giving an effective batch of16×4096×8×8=4\.1916\\times 4096\\times 8\\times 8=4\.19M tokens per step and 10\.5B total training tokens\. FSDP\(Zhao et al\.,[2023](https://arxiv.org/html/2605.28827#bib.bib19)\)wraps each Qwen2DecoderLayer withShardingStrategy\.\_HYBRID\_SHARD\_ZERO2and bf16 mixed precision; FlashAttention varlen\(Dao et al\.,[2022](https://arxiv.org/html/2605.28827#bib.bib2)\)takes packedcu\_seqlensdirectly from the memmap iterator so document boundaries are respected without padding; Liger fused kernels\(Hsu et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib6)\)cover RMSNorm, RoPE, SwiGLU, and fused linear cross\-entropy\.

The optimizer is fused AdamW,β=\(0\.9,0\.95\)\\beta=\(0\.9,0\.95\),ϵ=10−8\\epsilon=10^\{\-8\}, weight decay 0\.1, peak learning rate2×10−42\\times 10^\{\-4\}with 500\-step linear warmup and cosine decay to2×10−52\\times 10^\{\-5\}\. Training runs at 415,000 tokens/s aggregate on 8×\\timesH100 SXM5 with peak per\-GPU memory of 24 GB\.

### 3\.3Supervised fine\-tuning

We merge five Arabic instruction datasets into a single deduplicated pool: FreedomIntelligence/evol\-instruct\-arabic \(59,022 rows\), FreedomIntelligence/alpaca\-gpt4\-arabic \(49,969\), FreedomIntelligence/sharegpt\-arabic \(5,231\), arbml/CIDAR \(10,000\), and the Arabic subset of CohereForAI/aya\_dataset \(4,947 after filtering onlanguage\_code=="arb"\)\. MD5 deduplication on the rendered ChatML string yields 129,116 unique examples\.

Each example is rendered into ChatML with the system prompt“you are a smart assistant that answers in formal Arabic”and pre\-tokenized into two parallelint32memmaps:sft\_tokens\.bincontains the full input id sequence,sft\_labels\.bincontains−100\-100on prompt positions and the actual token id on assistant\-response positions\. Training is 5 epochs at peak learning rate2×10−52\\times 10^\{\-5\}, grad accumulation 2, 30 warmup steps, and computes loss*only on response tokens*\(Ouyang et al\.,[2022](https://arxiv.org/html/2605.28827#bib.bib12)\)\. Of the 43,918,266 total pretokenized tokens, 31,678,374 \(72\.1%\) carry non\-negative\-100 labels; masking the prompt saves capacity for the distribution we actually care about\.

### 3\.4Direct preference optimization

The SFT checkpoint undergoes DPO\(Rafailov et al\.,[2023](https://arxiv.org/html/2605.28827#bib.bib15)\)on 6,750 Arabic preference pairs from 2A2I/argilla\-dpo\-mix\-7k\-arabic\. Each pair provides a user prompt, a chosen assistant response, and a rejected assistant response\. Training runs on a single H100 with per\-device batch size 2, gradient accumulation 8,β=0\.1\\beta=0\.1, peak learning rate5×10−75\\times 10^\{\-7\}, 2 epochs, and an explicit frozen reference model \(not LoRA\-implicit\)\. Total training time is 34 minutes for 844 optimizer steps\.

### 3\.5Weight soup merging

The DPO run converges to a loss nearln⁡2≈0\.693\\ln 2\\approx 0\.693with reward\-accuracy 0\.48 and margin−6\.9×10−4\-6\.9\\times 10^\{\-4\}, indicating the preference dataset supplies weak signal at this scale\. We mitigate this by averaging the DPO checkpoint with the earlier pretrain and SFT checkpoints, all three share the same 178,697\-vocab architecture, which makes direct weight averaging safe\. We produce 7 merge variants: linear interpolations of DPO and pretrain att∈\{0\.3,0\.5,0\.7\}t\\in\\\{0\.3,0\.5,0\.7\\\}, SLERP att∈\{0\.3,0\.5\}t\\in\\\{0\.3,0\.5\\\}, linear DPO/SFT att=0\.5t=0\.5, and a 50/25/25 soup of DPO/SFT/pretrain\. Each merge is benchmarked on the same 3 Arabic tasks \(Section[5\.1](https://arxiv.org/html/2605.28827#S5.SS1)\); the 50/25/25 soup wins and becomes the final checkpoint\.

### 3\.6Edge deployment

For edge deployment we convert the merged model to llama\.cpp GGUF format and produce four quantization levels\. The llama\.cpp converter does not recognize our tokenizer hash \(because we added 27k tokens\), so we patchedconvert\_hf\_to\_gguf\.pyto map our hash to the existingqwen2pre\-tokenizer type\. After conversion,llama\-quantizeproduces q8\_0 \(525 MB\), q5\_k\_m \(419 MB\), and q4\_k\_m \(398 MB\)\. Note that q5\_k\_m and q4\_k\_m fall back to higher\-bit quantization for 144 of the 290 tensors, specifically the expanded Arabic embedding rows, because k\-quants require tile sizes that the added vocabulary does not align with\. Effective bits\-per\-weight is therefore 6\.79 \(q5\_k\_m\) and 6\.45 \(q4\_k\_m\) rather than the nominal 5 and 4\.

## 4Implementation

Table[2](https://arxiv.org/html/2605.28827#S4.T2)summarizes the codebase\. Total Python source is 5,555 lines across 25 scripts and 13 runtime modules\. The full pipeline runs on a single Nebius gpu\-h100\-sxm instance \(8×\\timesH100 80GB SXM5, Ubuntu 24\.04, CUDA 13\.0, PyTorch 2\.11\.0\+cu130, flash\-attn 2\.8\.3, transformers 5\.5\.0, trl 1\.0\.0, liger\-kernel 0\.7\.0\)\.

Table 2:Codebase metrics\. Every training, benchmark, merge, and export step is reproducible from the released scripts\.#### Memmap data loader\.

Early runs attempted HuggingFace Hub streaming for the pretraining corpus\. Under 8\-rank torchrun, one or more ranks would stall for 15\+ minutes at the first parquet fetch, causing NCCL collective deadlocks\. We resolved this by pre\-tokenizing the entire corpus to a flat int32 memmap \(arwiki\_tokens\.bin, 2\.0 GB\) and sampling fixed\-length windows per rank\. The loader is 162 lines and serves data at the speed of sequential memory reads, effectively zero overhead compared to the GPU forward pass\.

#### Packed attention with document boundaries\.

The loader also producescu\_seqlenspointers for every batch: each window is scanned for EOS positions and boundaries are inserted so FlashAttention varlen correctly masks cross\-document attention\. We monkey\-patchQwen2Attention\.forwardto readcu\_seqlensfrom a process\-global \(not thread\-local\) dictionary, because PyTorch gradient checkpointing recomputes forwards on a different thread during backward, and a thread\-local would be empty at that time\.

#### FSDP hybrid sharding\.

For 0\.5B parameters on 8×\\times80 GB H100, full parameter sharding is unnecessary\. We useShardingStrategy\.\_HYBRID\_SHARD\_ZERO2, which replicates parameters across ranks but shards optimizer state and gradients within each rank group\. This keeps the training compute local and all\-gather overhead minimal\.

## 5Experimental Evaluation

#### Hardware\.

All experiments run on a single Nebiusgpu\-h100\-sxmnode: 8×\\timesNVIDIA H100 80 GB SXM5 HBM3, NVLink4, 128 vCPUs, 1\.5 TiB RAM, 1\.28 TiB SSD, driver 580\.126\.09, CUDA 13\.0\.88\. NCCL allreduce busbw was measured at 466\.9 GB/s on a 1 GiB tensor across 8 ranks at session start\.

#### Evaluation methodology\.

We use lm\-evaluation\-harness\(Gao et al\.,[2021](https://arxiv.org/html/2605.28827#bib.bib3)\)v0\.4\.11 with three tasks:copa\_ar,arabic\_mt\_hellaswag, andarabic\_leaderboard\_arabic\_mmlu\(our model gets evaluated on the full 14,575\-question ArabicMMLU\(Koto et al\.,[2024](https://arxiv.org/html/2605.28827#bib.bib8)\)\)\. Every model is scored withapply\_chat\_template=True\(with a fallback to raw prompts for models whose tokenizer has nochat\_templateattribute, specifically AceGPT\-7B\-chat\), batch size 2,max\_length=1536, per\-tasklimit=200, andacc\_normpreferred overaccwhere available\. The same methodology is applied to all baselines\. Evaluating each model on all three tasks takes 10–25 minutes depending on model size\.

### 5\.1Main results

Table[3](https://arxiv.org/html/2605.28827#S5.T3)shows the head\-to\-head comparison\. Figure[2](https://arxiv.org/html/2605.28827#S5.F2)plots the same data on a Pareto scatter of accuracy versus parameters\.

Table 3:Arabic benchmark results\. Ours is the only Arabic\-specialized open model at 0\.5B class\. Methodology: lm\-eval\-harness v0\.4\.11,apply\_chat\_template=True\(fallback for AceGPT\-7B\-chat which has no template\), limit=200 per task,acc\_normpreferred\. Bold indicates the best in column; the last column is the unweighted mean\.0\.5B1B2B5B10B303035354040454550505555edge\-deployableOurs \(0\.5B\)Qwen2\.5\-0\.5BFalcon\-H1\-0\.5BFalcon\-H1\-1\.5BAceGPT\-7BALLaM\-7BSILMA\-9B1/14 size of AceGPT\-7B
ties Falcon\-H1\-1\.5B on COPA
398 MB quantized \(q4\_k\_m\)ParametersMean Arabic accuracy \(%\)RightNow\-Arabic \(ours\)Multilingual 0\.5–1\.5BArabic\-specialized 7–9BFigure 2:Mean Arabic benchmark accuracy \(COPA\-ar, HellaSwag\-ar, ArabicMMLU\) versus parameter count\. Our model is the only Arabic\-specialized entry in the sub\-1B edge\-deployable zone\. Dashed line shows the Pareto frontier\.#### Same\-class wins\.

Among models in the 0\.5B class, we are first on COPA\-ar \(\+4\.5 vs Qwen2\.5\-0\.5B\-Instruct, \+13\.5 vs Falcon\-H1\-0\.5B\), first on HellaSwag\-ar \(\+3\.5 vs Qwen, \+3\.0 vs Falcon\), and first on mean \(\+1\.8 vs Qwen, \+5\.2 vs Falcon\)\. We lose ArabicMMLU by 2\.8 points to Qwen, the one task where the base multilingual model’s broader world knowledge still beats our Arabic\-specialized continued pretraining\.

#### Scaling gap\.

Against the 7–9B Arabic\-specialized models, the gap is exactly where theory predicts it should be: knowledge\-intensive ArabicMMLU separates by 12–30 points, HellaSwag by 1–12 points, and COPA\-ar by 10–11 points\. On the mean, the 0\.5B model recovers35\.9/53\.5=67\.1%35\.9/53\.5=67\.1\\%of SILMA\-9B’s score at518​M/9​B=5\.8%518\\text\{M\}/9\\text\{B\}=5\.8\\%of the parameters\.

#### A specific tie\.

COPA\-ar is the one task where our 0\.5B exactly matches Falcon\-H1\-1\.5B \(both at 58\.4%\) despite a 3×\\timesparameter gap\. COPA is a commonsense reasoning task with short inputs where the Arabic tokenizer efficiency directly helps, fewer tokens per premise means more of the model’s context is real content\. Figure[3](https://arxiv.org/html/2605.28827#S5.F3)breaks down the per\-task scores across all models\.

01010202030304040505060607070SILMA\-9BALLaM\-7BAceGPT\-7BFalcon\-H1\-1\.5BOurs \(0\.5B\)Falcon\-H1\-0\.5BQwen2\.5\-0\.5B58\.426\.023\.2Accuracy \(%\)COPA\-arHellaSwag\-arArabicMMLUFigure 3:Per\-task accuracy breakdown across all evaluated models\. Our 0\.5B model \(dashed box\) wins COPA\-ar and HellaSwag\-ar among same\-class models\. The ArabicMMLU gap to 7B\+ models reflects the knowledge ceiling at sub\-1B scale\.

### 5\.2Merge ablation

Table[4](https://arxiv.org/html/2605.28827#S5.T4)shows the 7 merge variants and the original DPO checkpoint, each benchmarked on the same 3 tasks with the same methodology\.

Table 4:Weight soup merge ablation\. The 50/25/25 soup of DPO, SFT, and pretrain checkpoints wins on mean\. Differences are small \(0\.44 points spread\) but reliable: the best merge is chosen by downstream metric, not training loss\. Small differences from Table[3](https://arxiv.org/html/2605.28827#S5.T3)are due to a separate evaluation pass; Table[3](https://arxiv.org/html/2605.28827#S5.T3)reports the final unified run\.The soup gains \+0\.44 absolute points over the DPO endpoint alone\. The gain comes almost entirely from HellaSwag\-ar \(\+1\.33\)\. The\(DPO,Pretrain\)\(\\text\{DPO\},\\text\{Pretrain\}\)lerp family clusters in a tight 0\.13\-point band around 35\.35%, and the SLERP variants are indistinguishable from linear at these ratios\. The\(DPO,SFT\)\(\\text\{DPO\},\\text\{SFT\}\)lerp is the only merge that*underperforms*the DPO baseline\.

### 5\.3Tokenizer efficiency

Table 5:Arabic tokenizer fertility \(tokens per word\) on a 368\-word held\-out sample\. Lower is better\. Our merged tokenizer uses 17\.3% fewer tokens per Arabic word than the Qwen2\.5 baseline tokenizer\.Table[5](https://arxiv.org/html/2605.28827#S5.T5)reports the result\. The 17\.3% fertility reduction translates directly into a 17\.3% speedup on Arabic\-only workloads at the same parameter count, because inference cost is linear in token count\. Combined with the 0\.5B parameter count, this compounds: on a single Arabic user query, RightNow\-Arabic\-0\.5B\-Turbo emits the same semantic content as Qwen2\.5\-0\.5B\-Instruct using 17\.3% fewer forward passes, each of which is already small because the model is small\.

### 5\.4Inference speed

We measure inference throughput via llama\.cpp’sllama\-benchon H100 SXM5 \(CUDA backend\) for each GGUF quantization at batch size 1, prompt length 128, generation length 128\. Numbers are reported in Table[6](https://arxiv.org/html/2605.28827#S5.T6)\.

Table 6:llama\.cpp CUDA inference speed per GGUF quantization on a single H100 SXM5\. All numbers fromllama\-bench \-n 128 \-p 128 \-b 1 \-ngl 99\. q8\_0 is the best throughput while q4\_k\_m is the smallest footprint\.All four quantizations clear 580 tokens/s at batch size 1\. HuggingFacemodel\.generate\(\)on the same hardware tops out at 82 tokens/s at bs=1 due to Python and per\-token kernel\-launch overhead; llama\.cpp’s CUDA graph capture and optimized C\+\+ sampling loop removes that ceiling and delivers an 8×\\timesspeedup at identical model weights\.

### 5\.5Training dynamics

Table[7](https://arxiv.org/html/2605.28827#S5.T7)lists the three major training phases, their step count, and their loss trajectory\.

Table 7:Training phase summary\. Pretraining step counts start at 0 and the loss column shows first\-logged vs final\-logged batch loss \(bf16, accumulated across micro\-batches and averaged across ranks via all\-reduce\)\.Figure[4](https://arxiv.org/html/2605.28827#S5.F4)visualizes the pretraining loss trajectory\. Pretraining reduces perplexity from roughlye14\.21=1\.48​Me^\{14\.21\}=1\.48\\text\{M\}toe1\.69=5\.42e^\{1\.69\}=5\.42, a 270,000×\\timesimprovement\.

05005001,0001\{,\}0001,5001\{,\}5002,0002\{,\}0002,5002\{,\}500022446688101012121414warmup1\.6914\.21ckptckptckptckptckptStepLossPretrain \(arwiki\)Figure 4:Pretraining loss over 2,500 steps on 504M Arabic Wikipedia tokens\. The steep initial drop \(steps 0–200\) corresponds to the model learning the 27,032 new Arabic token embeddings\. Loss plateaus near 1\.69 \(perplexity 5\.42\) by step 2,000\.SFT, which computes loss only on the roughly 72% of tokens that are assistant responses, drops response\-only loss from 1\.95 at step 70 to 1\.81 at step 415\. DPO, on weak Arabic preference data, moves essentially nothing: the training loss stays atln⁡2\\ln 2and reward margin stays near zero, confirming that 6,750 machine\-translated preference pairs are insufficient signal for a 0\.5B model\. The soup merge in Section[5\.2](https://arxiv.org/html/2605.28827#S5.SS2)recovers some capability not by reweighting via a learned objective but simply by averaging with the earlier checkpoints that the SFT stage partially forgot\.

## 6Discussion and Limitations

#### Knowledge ceiling is a parameter ceiling\.

The ArabicMMLU gap to the 7B\+ models \(29\+ points\) is the clearest evidence of a fundamental limit: knowledge benchmarks scale with parameter count, period\. No amount of tokenizer efficiency, merge tuning, or training\-stack optimization can close 29 points at 0\.5B\. If the target deployment environment can afford 14 GB of model weights, AceGPT\-7B or ALLaM\-7B are the correct choice\. We target the niche where 14 GB is not available, phones, edge CPUs, browsers, and knowledge is a secondary concern\.

#### DPO was weak at this scale\.

The DPO stage did not move the model\. We attribute this to two causes: \(1\) the 6,750\-pair preference dataset was machine\-translated from English DPO data rather than written by native Arabic speakers, so the preference signal is noisy, and \(2\) 0\.5B parameters may be too small to benefit from preference tuning beyond what SFT already provides\. A better dataset and a stronger reference model might change this; we report what we observed\. The soup merge was the more productive post\-SFT intervention\.

#### Single Arabic variety\.

The pretraining corpus is Modern Standard Arabic \(via Wikipedia\)\. The model handles MSA well and dialects poorly: a query in Egyptian, Gulf, or Levantine Arabic will receive an MSA response\. For dialect coverage, the pretraining corpus would need explicit dialect data, which was outside the scope of the initial release\.

#### Tokenizer fertility short of target\.

Our 17\.3% fertility reduction is real but below the 30% reduction that a larger\-vocab, Arabic\-only SentencePiece model would achieve\. The main constraint is that we merged into the existing Qwen2\.5 BPE rather than replacing the tokenizer entirely; full replacement would invalidate the base model’s pre\-trained embedding geometry and demand a full retrain\. A future version could split the difference by adding a second wave of common multi\-word Arabic phrases\.

#### Pretraining tokens per parameter\.

At 504M pretraining tokens for 518M parameters, the ratio is≈1\\approx 1, two orders of magnitude below the Chinchilla\-optimal ratio\(Hoffmann et al\.,[2022](https://arxiv.org/html/2605.28827#bib.bib5)\)\. Our training is continued pretraining on top of an already\-trained base, so the absolute ratio is misleading, but additional Arabic pretraining data would benefit the model directly on ArabicMMLU\-style knowledge tasks\. Gating issues prevented the use of FineWeb\-2\-ar and CulturaX\-ar during this release; we plan to address this in v2\.

#### GGUF quantization tile alignment\.

q4\_k\_m and q5\_k\_m fall back to higher\-bit quantization for the 144 tensors that interact with the added vocabulary, because k\-quant tile sizes do not align with our 178,697\-row embedding matrix\. Effective bits\-per\-weight is 6\.45 and 6\.79 rather than 4 and 5\. A proper fix would require aligning the vocabulary padding to a llama\.cpp\-friendly block size; we leave this to future work\.

## 7Conclusion

RightNow\-Arabic\-0\.5B\-Turbois a 518M\-parameter Arabic\-specialized decoder LLM built by injecting 27,032 Arabic tokens into Qwen2\.5\-0\.5B, continuing pretraining on 504M Arabic tokens, supervised fine\-tuning with response\-only loss masking, direct preference optimization, weight\-soup merging, and exporting to GGUF for edge deployment\. At the same parameter count, the model beats Qwen2\.5\-0\.5B\-Instruct and Falcon\-H1\-0\.5B on Arabic benchmarks; at 398 MB quantized it delivers 635 tokens/s at bs=1 on a single H100 via llama\.cpp and fits comfortably on a phone\. It is the smallest open Arabic\-specialized decoder LLM released to date\. The full pipeline, weights, training code, benchmark scripts, and seven merge variants, is available at[https://huggingface\.co/RightNowAI/RightNow\-Arabic\-0\.5B\-Turbo](https://huggingface.co/RightNowAI/RightNow-Arabic-0.5B-Turbo)\.

## References

- Bari et al\. \(2024\)M\. Saiful Bari, Yazeed Alnumay, Norah A\. Alzahrani, Nouf M\. Alotaibi, Hisham A\. Alyahya, Sultan AlRashed, et al\.ALLaM: Large Language Models for Arabic and English\.*arXiv preprint arXiv:2407\.15390*, 2024\.
- Dao et al\. \(2022\)Tri Dao, Daniel Y\. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré\.FlashAttention: Fast and memory\-efficient exact attention with IO\-awareness\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2022\.
- Gao et al\. \(2021\)Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, and Eric Tang\.A framework for few\-shot language model evaluation\.*EleutherAI open\-source software*, 2021\.[https://github\.com/EleutherAI/lm\-evaluation\-harness](https://github.com/EleutherAI/lm-evaluation-harness)\.
- Hennara et al\. \(2025\)Khalil Hennara, Sara Chrouf, Mohamed Motaism Hamed, Zeina Aldallal, Omar Hadid, and Safwan AlModhayan\.Kuwain 1\.5B: An Arabic SLM via language injection\.*arXiv preprint arXiv:2504\.15120*, 2025\.
- Hoffmann et al\. \(2022\)Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W\. Rae, Oriol Vinyals, and Laurent Sifre\.Training compute\-optimal large language models\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2022\.
- Hsu et al\. \(2024\)Pin\-Lun Hsu, Yun Dai, Vignesh Kothapalli, Qingquan Song, Shao Tang, Siyu Zhu, Steven Shimizu, Shivam Sahni, Haowen Ning, and Yanning Chen\.Liger Kernel: Efficient Triton kernels for LLM training\.*arXiv preprint arXiv:2410\.10989*, 2024\.
- Huang et al\. \(2023\)Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Abdulmohsen Alharthi, Bang An, Juncai He, Ziche Liu, Zhiyi Zhang, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, and Jinchao Xu\.AceGPT, Localizing Large Language Models in Arabic\.*arXiv preprint arXiv:2309\.12053*, 2023\.
- Koto et al\. \(2024\)Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Boda Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, and Timothy Baldwin\.ArabicMMLU: Assessing massive multitask language understanding in Arabic\.*arXiv preprint arXiv:2402\.12840*, 2024\.
- Kudo and Richardson \(2018\)Taku Kudo and John Richardson\.SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing\.In*Proceedings of EMNLP: System Demonstrations*, 2018\.
- Misraj\.ai \(2025\)Khalil Hennara, Muhammad Hreden, Mohamed Motaism Hamed, Zeina Aldallal, Sara Chrouf, and Safwan AlModhayan\.Mutarjim: Advancing bidirectional Arabic\-English translation with a small language model\.*arXiv preprint arXiv:2505\.17894*, 2025\.
- Minixhofer et al\. \(2022\)Benjamin Minixhofer, Fabian Paischer, and Navid Rekabsaz\.WECHSEL: Effective initialization of subword embeddings for cross\-lingual transfer of monolingual language models\.In*Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics \(NAACL\)*, 2022\.
- Ouyang et al\. \(2022\)Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L\. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe\.Training language models to follow instructions with human feedback\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2022\.
- Penedo et al\. \(2024\)Guilherme Penedo, Hynek Kydlíček, Loubna Ben Allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro von Werra, and Thomas Wolf\.The FineWeb datasets: Decanting the web for the finest text data at scale\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2024\.
- Qwen Team et al\. \(2024\)Qwen Team: An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, et al\.Qwen2\.5 technical report\.*arXiv preprint arXiv:2412\.15115*, 2024\.
- Rafailov et al\. \(2023\)Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D\. Manning, and Chelsea Finn\.Direct preference optimization: Your language model is secretly a reward model\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2023\.
- Sengupta et al\. \(2023\)Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondos Mahmoud Bsharat, Alham Fikri Aji, Zhiqiang Shen, Zhengzhong Liu, Natalia Vassilieva, Joel Hestness, Andy Hock, Andrew Feldman, Jonathan Lee, Andrew Jackson, Hector Xuguang Ren, Preslav Nakov, Timothy Baldwin, and Eric Xing\.Jais and Jais\-chat: Arabic\-centric foundation and instruction\-tuned open generative large language models\.*arXiv preprint arXiv:2308\.16149*, 2023\.
- Wortsman et al\. \(2022\)Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo\-Lopes, Ari S\. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt\.Model soups: Averaging weights of multiple fine\-tuned models improves accuracy without increasing inference time\.In*International Conference on Machine Learning \(ICML\)*, 2022\.
- Zhang et al\. \(2024\)Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu\.TinyLlama: An open\-source small language model\.*arXiv preprint arXiv:2401\.02385*, 2024\.
- Zhao et al\. \(2023\)Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien\-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Pritam Damania, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Ajit Mathews, and Shen Li\.PyTorch FSDP: Experiences on scaling fully sharded data parallel\.*Proceedings of the VLDB Endowment*, 16\(12\):3848–3860, 2023\.