适配沃洛夫语的工业级开放权重语言模型

# Wolof-Adapted Industry-Ready Open-Weight Language Models
Source: [https://arxiv.org/html/2610.20824](https://arxiv.org/html/2610.20824)
Delphine CharletPierre AdamFloriane BehanzinAffiliation:Lionel Delphin\-Poulat,Johannes Heinecke,Géraldine Damnati,Affiliation:Pelin Kaytmaz,Nourredine Khaous,Charbel Kindji,Meriam Maadi,Affiliation:Abdoulaye Mbaye,Claire Perroux,Ismaël Rousseau,Papa Séga Wadé,Affiliation:Gilles Xiberras,Olivier Zhang,Malek ZitouniAffiliation:Orange Research, FranceAffiliation:Corresponding author:gwenole\.lecorve@orange\.com

###### Abstract

Wolof is spoken by 23M people, mostly in Senegal, but remains weakly supported by open\-weight language models, while API models raise cost, governance, and sovereignty constraints for local deployments\. In this paper, we present 0\.6B\-14B open\-weight models adapted to Wolof through instruction tuning, preference alignment, and model merging\. After presenting the pipeline and data, extensive experiments on 63 public and in\-house tasks show good performance on Wolof understanding and generation while remaining effective on other languages and cross\-lingual skills, with the 8B and 14B models being particularly competitive with substantially larger commercial systems\. The models are currently being evaluated in internal customer\-care pilots and are publicly available111[https://huggingface\.co/collections/Orange/wolof\-lm](https://huggingface.co/collections/Orange/wolof-lm)\.

## 1Introduction

Wolof—a West\-Atlantic language of the Niger\-Congo family—is the lingua franca of Senegal with 23M speakers, and the native language for about 40% of the population[Mbaye et al\. \(2025\)](https://arxiv.org/html/2610.20824#bib.bib8)\. However, it remains under\-served in current Large Language Models \(LLMs\) compared to English, French, and better\-covered African languages\([Hussen et al\., 2026](https://arxiv.org/html/2610.20824#bib.bib7);[Adebara et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib11)\)\. This limits applications in banking, telecommunications, and public services where users expect Wolof support\. Commercial API\-based LLMs show some Wolof capability but are often too costly or incompatible with local data\-governance and sovereignty requirements\. This paper addresses this gap by adapting open\-weight small language models to Wolof and evaluating them on industrially realistic use cases\. A representative application is customer\-relationship automation, spanning offline conversation analysis, FAQ answering, transactional support, and interactive chat\. The models are currently being integrated into internal Wolof customer\-care pilots, with a target customer base of several millions and the goal of reducing dependence on commercial systems\. Adaptation pipeline mostly relies on Wolof\-centric instruction tuning, which, depending on the model size, is complemented with preference alignment and model merging to balance the acquired Wolof capabilities with the pre\-existing ones in the base models\. Although these adaptation stages are well known in the literature, applying them to a severely under\-resourced language requires substantial data engineering, empirical tuning, and a rigorous targeted evaluation methodology\. Hence, the main contributions are: \(*i*\) we provide a suite of Wolof\-adapted Qwen models \(from 0\.6 to 14B\) for deployment\-constrained settings; \(*ii*\) we propose a broad evaluation covering public and in\-house tasks, with skill\-based aggregations and French/English regression checks for multilingual operations; and \(*iii*\) we release associated Wolof data: 222K instructions and some of the in\-house test sets from our evaluation\.

The paper is organised as follows: Section[2](https://arxiv.org/html/2610.20824#S2)reviews African LLMs and Wolof/Senegalese Natural Language Processing \(NLP\); Section[3](https://arxiv.org/html/2610.20824#S3)details the adaptation method; Section[4](https://arxiv.org/html/2610.20824#S4)introduces our evaluation benchmark while Section[5](https://arxiv.org/html/2610.20824#S5)reports automatic and human evaluation results\.

## 2Related Work

Recent African LLM surveys and benchmark studies report persistent coverage, tokenization, and evaluation gaps, with Wolof still rarely evaluated systematically\([Hussen et al\., 2026](https://arxiv.org/html/2610.20824#bib.bib7);[Adebara et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib11);[Alhanai et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib10)\)\. Recent Wolof and Senegalese\-language resources remain mostly task\-specific, including banking\-domain intent classification\([Kandji et al\., 2026](https://arxiv.org/html/2610.20824#bib.bib12)\)and resource surveys\([Mbaye et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib8)\)\. This leaves a gap for openly documented Wolof models designed and evaluated for general\-purpose industrial use\.

Regarding language modeling, existing work explores both training compact models from scratch and adapting pretrained models\. InkubaLM follows the former approach for five African languages\([Tonja et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib40)\)\. Aya uses multilingual instruction tuning and studies the composition of instruction data\([Singh et al\., 2024b](https://arxiv.org/html/2610.20824#bib.bib18);[Üstün et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib17)\), while AfriqueLLM and Lugha\-Llama investigate continued pre\-training and the effect of data composition\([Yu et al\., 2026](https://arxiv.org/html/2610.20824#bib.bib6);[Buzaaba et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib41)\)\. DIPLomA instead combines target\-language continued pre\-training with delta\-based merging\([Sarasua et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib9)\)\. Given the scarcity of raw Wolof text, we rely primarily on Wolof\-centric instruction tuning using translated public datasets, bilingual and native Wolof data, complemented by preference alignment and model merging\.

Regarding evaluation, public Wolof evaluation resources remain sparse but are increasing: MasakhaNER 2\.0 provides curated named\-entity recognition data for Wolof\([Adelani et al\., 2022](https://arxiv.org/html/2610.20824#bib.bib14)\), while AfrIFact covers evidence\-grounded verification\([Azime et al\., 2026](https://arxiv.org/html/2610.20824#bib.bib15)\)\. Broader multilingual benchmarks such as IrokoBench and Belebele further extend task coverage, although translated or English\-pivoted data may not fully reflect in\-language quality\([Adelani et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib38);[Bandarkar et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib39)\)\. For instance,[Smart \(2026\)](https://arxiv.org/html/2610.20824#bib.bib16)presents a question answering dataset with samples generated in Wolof by an LM that we benchmarked as non\-Wolof\-capable\. We therefore complement automatic evaluation with manually checked in\-house test sets, and blind human assessment by Wolof speakers\. For model baselines, Aya\-101 is multilingual but does not list Wolof\([Üstün et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib17)\), whereas Tiny Aya explicitly includes it\([Salamanca et al\., 2026](https://arxiv.org/html/2610.20824#bib.bib19)\)\. Oolel\-v0\.1 is a Wolof\-dedicated 7B Qwen\-2\.5 model advertised for RAG, translation, generation, and math[Soynade Research \(2025\)](https://arxiv.org/html/2610.20824#bib.bib24), but lacks public training data or a detailed evaluation, motivating our controlled comparison\.

## 3Adaptation Method

The target adapted models are intended for public non\-commercial release, with training data released under open licenses where possible\. The main objective is strong Wolof understanding and generation while preserving French and English, with multi\-turn conversation for customer\-service use\. We also track non\-linguistic base model skills, especially instruction following, knowledge or safety for data processing or user interaction, while treating function calling, long contexts, and code as secondary checks for future agentic use\.

Our adaptation process first performs instruction tuning through Supervised Fine\-Tuning \(SFT\) to acquire Wolof capabilities while seeking to preserve general behaviors through a mixed training corpus\. We then evaluate preference alignment and model merging as complementary ways to improve this balance without adding new Wolof supervision\. The contribution of these extra stages depends on model capacity: with larger positive effect as the model is large\. These are usual post\-training techniques, but implementing successful recipes in always a practical challenge\. Finally, let one note that all adaptations rely on full fine\-tuning, rather than parameter\-efficient approaches such as LoRA since the latter can be effective for domain adaptation but are less suited to adding a language\.

### 3\.1Instruction Tuning

Because native digital Wolof is scarce, our instruction tuning combines native bilingual resources with machine\-translated public instructions to obtain both scale and task diversity\. After comparing translation systems, Google Translate was selected for Wolof instruction generation\. The quality of the translations have been assessed by native speakers\. First, random instructed were selected and checked\. Then, more rigorously, we conducted a human annotation of Google Translate outputs with two native speakers, who evaluated 50 sentence translations w\.r\.t\. the fidelity of the translated sentence’s meaning \(from 1 to 5\) and the linguistic quality of the Wolof \(from 1 to 5\)\. The scores obtained—correct meaning: 4\.59/5 and Wolof quality: 4\.78/5—indicate that Google Translate produces translations that are both semantically faithful and linguistically correct\. Details are given App\.[D\.1](https://arxiv.org/html/2610.20824#A4.SS1)\. We therefore obviously do not treat translated data as a substitute for fully native corpora, but consider it as a scalable imperfect\-but\-reliable supervision and further counterbalance it by native/bilingual sources and Wolof\-specific evaluation\.

Data mixture was guided by iterative studies of skills: Wolof understanding and generation, conversational coherence, multi\-turn interaction, and function calling\. The resulting corpus is organized into four blocks, detailed in Table[1](https://arxiv.org/html/2610.20824#S3.T1)\. The bilingual French\-Wolof block uses Alwaly translation pairs to anchor the model in authentic Wolof text\([Alwaly Ndiaye, 2026](https://arxiv.org/html/2610.20824#bib.bib21)\)\. The translated general block mixes instruction following, dialogue summarization, QA, and masked\-turn dialogue completion from Alpaca\([Taori et al\., 2023](https://arxiv.org/html/2610.20824#bib.bib22)\), Dolly\-15k\([Conover et al\., 2023](https://arxiv.org/html/2610.20824#bib.bib23)\), SAMSum\([Gliwa et al\., 2019](https://arxiv.org/html/2610.20824#bib.bib25)\), Helpful Instructions\([HuggingFaceH4, 2023](https://arxiv.org/html/2610.20824#bib.bib26)\), Hermes\-217k\([lodrick\-the\-lafted, 2026](https://arxiv.org/html/2610.20824#bib.bib28)\), Natural Instructions\([Wang et al\., 2022](https://arxiv.org/html/2610.20824#bib.bib29)\), and Claire Dialogue\([Hunter et al\., 2023](https://arxiv.org/html/2610.20824#bib.bib13)\)\. Multi\-turn Wolof dialogue is injected through the Claire one\-to\-one split\([Hunter et al\., 2023](https://arxiv.org/html/2610.20824#bib.bib13)\)\. Preserving of structured tool calls is aimed via English function calling data\([Kandah, 2024](https://arxiv.org/html/2610.20824#bib.bib30);[Zhang et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib31);[Liu et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib32)\)\. Let one note that the massively multilingual dataset Aya \(with a few thousands Wolof instructions\) was voluntarily excluded from our training and reserved for evaluation to avoid contaminating one of the few public Wolof benchmarks\. The mixture contains 446K samples and 109\.4M tokens\. All details of our training corpus, along with the associated validations sets used for monitoring and development are given in App\.[B\.1](https://arxiv.org/html/2610.20824#A2.SS1)\.

For completeness, we also investigated the use of a Continued Pre\-Training \(CPT\) stage prior to SFT\. However, only a raw corpus of 13M Wolof tokens could be collected, which was insufficient\. The corresponding experiments are reported in App\.[E\.2](https://arxiv.org/html/2610.20824#A5.SS2)\. Tokenization adaptation has also been experimented using the CPT data\. Interesting results are reported in App\.[E\.3](https://arxiv.org/html/2610.20824#A5.SS3), but this is neither included in main recipe of the paper\.

Table 1:Training data\. WO, FR, and EN stand for Wolof, French, and English, resp\.\. Turns are user\-assistant pairs of messages \(\+ optional system message\)\.
### 3\.2Preference Alignment and Model Merging

Preference alignment and model merging were tested to balance after the SFT step\.

Regarding preference alignment, we applied*Direct Preference Optimization \(DPO\)*[Rafailov et al\. \(2023\)](https://arxiv.org/html/2610.20824#bib.bib5)using the DOLCI preference dataset from OLMo 3\([Team OLMo et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib58)\)\. The corpus contains 233,804 preference pairs in its training set, covering instruction\-following, general knowledge, coding, mathematics, and safety\.98,%98,\\%of the samples are in English, and none in Wolof: the purpose here is to reinforce general assistant behavior after language adaptation while retaining the Wolof capabilities acquired through SFT\. Appendix[C](https://arxiv.org/html/2610.20824#A3)provides the data composition and training configuration\.

Regarding model merging, the idea is to avoid the adapted model to move too far from the original one to decrease the risk of catastrophic forgetting\. In practice, we also use the simple*linear interpolation*technique, also often referred to as*model soup*[Wortsman et al\. \(2022\)](https://arxiv.org/html/2610.20824#bib.bib4)\. In the experiments, model merging is experimented either after SFT or after DPO\. In the first case, a sweep over interpolation coefficients selected weights of 0\.4 for the base model and 0\.6 for the adapted model\. In the second case, the merging mixes the base, post\-SFT and post\-DPO model\. The best coefficients selected are 1/3\. More advanced recent techniques[Yang et al\. \(2024\)](https://arxiv.org/html/2610.20824#bib.bib2);[Zhou et al\. \(2024\)](https://arxiv.org/html/2610.20824#bib.bib3);[Touayouch et al\. \(2026\)](https://arxiv.org/html/2610.20824#bib.bib1)could be investigated to optimize these coefficients but this is left for future work since this is not the central scope of the paper\.

### 3\.3Training Setup

We target open\-weight small models to support on\-premise inference on reasonable infrastructure, an important constraint when access to high\-capability cloud services can be legally restricted in Senegal\. After testing several families \(Llama, Gemma, Phi, Mistral, GPT\-OSS\), Qwen proved the best tradeoff between performance and stability\. Development then used Qwen\-2\.5 7B models, while final experiments are applied on 0\.6\-14B Qwen3 models\.

SFT and DPO trainings were conducted on internal single nodes with 2 NVIDIA H100 80GB or 1 H200 140GB GPUs\. SFT models were trained for 3 epochs, peak learning rate of2×10−52\\times 10^\{\-5\}, and an effective batch size of 1024\. The maximum sequence length was 2048 tokens\. DPO models were trained for 2 epochs with an effective batch size of 512, a maximum sequence length of 1024 tokens, with regularization coefficientβ=0\.2\\beta=0\.2and a peak learning rate of5×10−65\\times 10^\{\-6\}\. Final checkpoints for SFT and DPO were selected by on loss on validation sets\. All hyperparameters for both stages are provided in App\.[B\.2](https://arxiv.org/html/2610.20824#A2.SS2)and[C](https://arxiv.org/html/2610.20824#A3)\.

TaskInputOutputWolofIn\-houseLang\. underst\.Lang\. gen\.KnowledgeChatReason\./MathCodeFormatingInstr\. followingFunc\. callingLong contextSafety/ethicsSamplesMetricKnowledge & QA \(6 tasks\)mmlu\_mcqa\_ENQuestion \(EN\)ChoiceUKFo1,140Accuracymmlu\_mcqa\_FRQuestion \(FR\)ChoiceUKFo1,140Accuracyirokobench\_mmluText \(WO\)Choice✓UKFo500Accuracyaya\_WO\_culture\_qaQuestion \(WO\)Answer \(WO\)✓UGK865LLM scorecorporate\_faq\_french\_mrqaQuestion \+ document \(FR\)Answer \(FR\)✓UGK122LLM scorecorporate\_faq\_wolof\_mrqaQuestion \+ document \(WO\)Answer \(WO\)✓✓UGK122METEORTranslation & Generation \(16 tasks\)aya\_WO\_translation\_from\_WOSentence \(WO\)Transl\. \(FR/EN\)✓UG16METEORaya\_WO\_translation\_to\_WOSentence \(FR/EN\)Transl\. \(WO\)✓UG31METEORaya\_WO\_orthographic\_correctionText \(WO\)Corrected text \(WO\)✓UG244Edit distanceflores\_\*\_to\_\*\(6 tasks\)Sentence \(EN/FR/WO\)Transl\. \(EN/FR/WO\)✓\(4/6\)UG1,012METEORwolbanking77\_text\_\*\_to\_\*\(6 tasks\)Sentence \(EN/FR/WO\)Transl\. \(EN/FR/WO\)✓\(4/6\)UG500METEORwolbanking77\_audio\_WO\_ASR\_correctionASR transcript \(WO\)Corrected text \(WO\)✓UG641WER changeClassification \(9 tasks\)irokobench\_xnliText pair \(WO\)Class✓UFo600Accuracywolbanking77\_text\_\*\_intent\(3 tasks\)Sentence \(EN/FR/WO\)Class✓\(1/3\)UFo500Macro F1wolbanking77\_audio\_WO\_manual\_intentManual transcript \(WO\)Class✓UFo641Macro F1wolbanking77\_audio\_WO\_ASR\_intentASR transcript \(WO\)Class✓UFo641Macro F1corporate\_dialog\_act\_labellingConv\. \(WO\)Labels✓✓UFo150Macro F1corporate\_resolution\_classificationConv\. \(WO\)Class✓✓UFo150Macro F1corporate\_sentiment\_classificationConv\. \(WO\)Labels✓✓UFo150Macro F1Information Extraction \(7 tasks\)aya\_WO\_pos\_taggingText \(WO\)POS tags✓UFo159Macro F1masakha\_nerText \(WO\)BIO labels✓UFo536Exact F1afrifact\_claim\_classificationText \(WO\) \+ claim \(WO\)Class✓UFo1,658Macro F1afrifact\_evidence\_extractionText \(WO\) \+ claim \(WO\)Span✓U1,658Partial F1corporate\_gdpr\_labellingConv\. \(WO\)Entity list✓✓UFo150Exact F1corporate\_nerConv\. \(WO\)Entity list✓✓UFo150Exact F1corporate\_full\_analysisConv\. \(WO\)Composite \(JSON\)✓✓UGFo150CompositeConversation \(8 tasks\)claire\_WO\_conversation\_coherenceConv\. \(WO\)Response \(WO\)✓UGCh292LLM scoredialogsum\_conversation\_summarizationConv\. \(EN\)Summary \(EN\)UG200ROUGE\-1personachat\_conversation\_coherenceConv\. \+ persona \(EN\)Response \(EN\)UGCh500LLM scorewebnlg\_qa\_conversational\_mrqaQuestion \+ conv\. \(EN\)Response \(EN\)UCh232LLM scorecorporate\_faq\_french\_conv\_mrqaQuestion \+ conv\. \(FR\)Response \(FR\)✓UGCh122LLM scorecorporate\_faq\_wolof\_conv\_mrqaQuestion \+ conv\. \(WO\)Response \(WO\)✓✓UGCh122METEORcorporate\_conversation\_coherenceConv\. \(WO\)Response \(WO\)✓✓UGCh287LLM scorecorporate\_summarizationConv\. \(WO\)Summary \(WO\)✓✓UG150LLM scoreReasoning, Math & Code \(6 tasks\)arc\_challenge\_reasoningScience question \(EN\)ChoiceUKR2,600Accuracygpqa\_diamond\_EN\_mcqaScience question \(EN\)ChoiceUKRFo198Accuracygpqa\_diamond\_FR\_mcqaScience question \(FR\)ChoiceUKRFo197Accuracygsm8k\_reasoningMath problem \(EN\)IntegerUR1,319Accuracyirokobench\_mgsmMath problem \(WO\)Integer✓UR250Exact matchhumaneval\_code\_generationCode problem \(EN\)Code \(Python\)UGCoFo164pass@1Safety & Ethics \(4 tasks\)bbq\_demographic\_bias\_mcqaQuestion \(EN\)LabelUFoS500Accuracyhalueval\_qa\_hallucination\_detectionQuestion \(EN\)LabelUFoS500Accuracytoxigen\_implicit\_hate\_detectionText \(EN\)LabelUFoS500Accuracywildjailbreak\_harmful\_request\_refusalText \(EN\)Refusal indicatorUS500Refusal rateInstruction Following & Function Calling \(5 tasks\)multi\_if\_instruction\_followingInstruction \(EN/FR\)TextUI808Accuracyifbench\_instruction\_followingInstruction \(EN\)TextUI300Accuracymifeval\_EN\_instruction\_followingInstruction \(EN\)TextUI541Accuracymifeval\_FR\_instruction\_followingInstruction \(FR\)TextUI235Accuracybfcl\_function\_callingInstruction \(EN\)Tool callUFoFu3,641Micro accuracyLong\-Context QA \(2 tasks\)ruler\_8k\_context\_mrqaQuestion \+ documents \(EN\)AnswerUL60Exact matchcorporate\_faq\_wolof\_long\_contx\_mrqaQuestion \+ documents \(WO\)Answer✓✓UL99METEOR

Table 2:Evaluation suite grouped by task family with contribution to target skills\. Detailed task descriptions and dataset references are provided in App\.[H](https://arxiv.org/html/2610.20824#A8)\.

## 4Benchmarking Tasks and Skills

Table 3:Automatic results by language and skill: normalised scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.We designed an evaluation suite of 63 tasks around the paper’s main deployment objective: improving Wolof understanding and generation while preserving English and French capability\. The suite combines public academic benchmarks with in\-house customer\-service tasks\. In total, 35 tasks include Wolof and 13 use in\-house data\. Table[2](https://arxiv.org/html/2610.20824#S3.T2)summarises each task \(input, output, samples, metric\), Wolof and in\-house coverage, and contribution to skills\. Details are provided in App\.[H](https://arxiv.org/html/2610.20824#A8)\. The in\-house tasks use internal documents and customer conversations annotated by Wolof\-speaking professionals, and cover FAQ retrieval, intent routing, privacy\-sensitive entity extraction, summarization, and response generation\.

Wolof understanding and generation are the primary adaptation target skills\. English and French monitor multilingual behavior, especially for Senegal\. Knowledge, chat, formatting, instruction following, and safety capture assistant requirements for interaction with users, while function calling and reasoning target agentic AI\. Finally, code and long context serve as broader regression checks\.

We use deterministic metrics for outputs that can be matched reliably and LLM\-as\-a\-judge for free\-form generation: GPT\-4o\-mini for English/French and Gemini\-2\.5\-flash for Wolof\. For the latter, on 450 Wolof responses, native speakers accepted 95% of the scores assigned by Gemini\-2\.5\-flash \(99% for Gemini\-2\.5\-pro\), as detailed in App\.[D\.2](https://arxiv.org/html/2610.20824#A4.SS2)\. These LLM scores should still be treated as comparative signals rather than used as standalone evidence\. This is why automatic evaluation of Sec\.[5\.1](https://arxiv.org/html/2610.20824#S5.SS1)is complemented by human evaluation in Sec\.[5\.2](https://arxiv.org/html/2610.20824#S5.SS2)\.

Finally, all evaluations use multiple prompt templates \(listed in App\.[H](https://arxiv.org/html/2610.20824#A8)\) sampled with fixed seed for reproducibility\. Decoding uses temperature zero, and on\-premise models run in 16\-bit precision\.

## 5Results

This section reports final automatic and human results\. Whereas post\-SFT gains vary by model size, this section only reports the best setup for each size \(the final released model\) due to space constraints\. Stage\-by\-stage comparisons are provided in App\.[F](https://arxiv.org/html/2610.20824#A6)\.

### 5\.1Automatic evaluation

Table[3](https://arxiv.org/html/2610.20824#S4.T3)reports the overall and skill\-aggregated results for four model families: unadapted open\-weight Qwen3 baseline models, our Wolof\-adapted Qwen3 models, commercial API systems used as upper reference points, and Wolof\-aware open\-weight models from prior releases \(Oolel and tiny\-aya\-earth\)\. Scores are normalised per task based on their \[min,max\] range, such thatminis the score of a random predictor \(random classification or random text ; not reported here\) andmaxdepends on the metric \(e\.g\., 1 for METEOR, 5 for LLM\-as\-a\-judge, etc\.\)\. All scores are the higher, the better\. Normalized values below zero indicate performance below the lower \(random\) score on that skill\. Explicit<think\>traces are removed from Qwen3 outputs before scoring\. A graphical summary of Table[3](https://arxiv.org/html/2610.20824#S4.T3)is also provided in App\.[A](https://arxiv.org/html/2610.20824#A1), showing overall scores as a function of model size\.

First, the clearest result is that adaptation produces a genuine acquisition of Wolof\. Across sizes, the unadapted Qwen3 models remain close to the random baseline both in Wolof understanding and generation, whereas adaptation significantly raises scores for these skills\. Our models fall within the range of the commercial APIs\. As compared to prior Wolof\-aware open models, Oolel demonstrates similar linguistic performance in Wolof but ranks lower in most other skills\. Aya\-tiny’s results show that it is not dedicated to Wolof and thus performs poorly\.

Then, at 4B\-14B scales, these gains coexist with good preservation of the base models’ broader capabilities like English, French, and most other skills like safety or reasoning, or even long context and coding which were yet not part of the originally targeted skills for our models\. The gains are nevertheless uneven at the smallest sizes\. In particular, the 0\.6B model improves on Wolof but degrades on several skills\. Likewise, whereas the targeted applications are not agentic ones, the function calling skill declines\. After manual investigations, this is mainly due to false positives, i\.e\., the models predict tool calls too frequently, even when none is needed\.

Deeper investigations lead to other interesting conclusions\. First, task\-by\-task results \(detailed in App\.[G](https://arxiv.org/html/2610.20824#A7)\) show that improvements on Wolof capabilities span all types of tasks: classification, labelling, generation, etc\. Second, the stage\-by\-stage analysis \(see App\.[F](https://arxiv.org/html/2610.20824#A6)\) identifies SFT as the main driver for adaptation, while the effect of DPO and merging is more variable and depending on the model size\. A possible explanation is that the hyperparameters—which have been tuned on validation data only for the 8B size—should be set differently according to the size\.

### 5\.2Human Evaluation via Arena

Automatic metrics are complemented with an arena\-style blind human evaluation on Wolof\-facing tasks\. Twelve Wolof\-speaking annotators compared outputs from 5 models: Gemini\-2\.5\-pro, Gemini\-2\.5\-flash, our final Wolof\-Qwen3\-8B model, its unadapted Qwen3\-8B base model, and Oolel\-v0\.1\. The annotators were not informed of the actual model identities during evaluation\. The evaluation used 100 samples from each of 9 tasks\.222claire\_WO\_conversation\_coherence,corporate\_faq\_WO\_mrqa,flores\_translation\_EN\_to\_WO,flores\_translation\_WO\_to\_FR,corporate\_summarization,wolbanking77\_WO\_intent\_classification,wolbanking77\_FR\_to\_WO\_translation,wolbanking77\_WO\_to\_EN\_translation, andwolbanking77\_audio\_transcription\_correctionThe four criteria were correctness, formatting, Wolof quality, and overall preference\. Annotators could select either system, mark both outputs as good or bad, or skip the comparison\. Pairwise outcomes were converted to Elo scores, splitting ties equally, as well as win probabilities for a better intuition \(0\.5 indicates equal chances to win or lose\)\. For the Wolof quality, samples whose expected output was English or French or a classification label were excluded\.

Table 4:Human evaluation Elo scores and win probability by criterion\. The Wolof quality column excludes tasks whose expected output is English or French or a classification label\.Table[4](https://arxiv.org/html/2610.20824#S5.T4)provides the results\. Overall, Gemini models are slightly preferred but very close to our 8B model, while Oolel is slightly further behind and the non\-adapted model is clearly worse, which confirms the results of automatic evaluations\. In terms of pure Wolof quality \(i\.e\., , independent of correctness\), our model is clearly comparable to closed models\. Consequently, these results are a positive indication of our models’ deployability\.

## 6Conclusion and Further Improvements

In this paper, we introduced open\-weight models adapted to Wolof, ranging from 0\.6B to 14B parameters, along with detailed descriptions of the training pipeline, datasets, and evaluation tasks\. Automatic and human evaluations show that our models have developed strong Wolof capabilities while retaining their multilingual and operational abilities, especially for customer service applications\. The 8B and 14B models, in particular, narrow the gap with commercial APIs\. These models, training datasets, and selected in\-house test sets are now ready to be publicly released for free non\-commercial use\.

Our results also provide practical insights into the effects of model size, the role of SFT, and the variability of preference alignment and model merging, leaving room for further improvements\. In parallel, targeting agentic AI capabilities represents an important next step, requiring a particular focus on function calling\. Work on Group Relative Policy Optimization \(GRPO\)\([Shao et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib60)\)with verifiable rewards has begun in pursuit of this new objective\.

## Limitations

Our final experiments use the Qwen family\. Altough preliminary experiments had been conducted to select the Qwen familiy, cross\-family generalisation should be tested again now that full recipes have been built\.

Then, as shown in the paper, Wolof adaptation can produce uneven changes across English, French, instruction\-following, and tool\-use tasks, particularly at the smallest model sizes\. Multilingual or task\-specific deployments therefore still require task\-specific validation\.

Third, the evaluation suite could still be augmented with industrial failure modes \(adversarial inputs, privacy leakage, ASR noise, low\-literacy users\), agentic scenarios, and larger user studies\. Regarding safety, specific tests related to the socio\-cultural environment in Senegal should complement the standard safety tasks used so far\. We hope that the internal deployment of these models in our service will help discover such new quality indicators\.

While LLM\-as\-a\-judge is still a research question for high\-resource languages, it should even more be cautiously interpreted for Wolof\. In spite of our human assessments of LLM judges, this is an area of improvement\.

Finally, computational constraints \(single node trainings on 1 or 2 GPUs\) limit ablation scope and multi\-seed runs\. Nonetheless, we think that the diversity of evaluation tasks counterbalances this aspect to provide a reliable view of our models’ performance\.

## Ethics and Societal Impact

LLMs were used in this paper to automate the generation of repetitiveLaTeXstructures \(tasks details, skill tables, etc\.\), as well as improve English quality wherever required\.

Annotators involved in this work are also employees of the company and were thus paid for the work that they accomplished\.

Publicly sourced datasets use permissive licenses \(CC\-BY, MIT, Apache\-2\.0\)\.

Translated and web\-scraped data risk reinforcing gender, ethnic, or regional biases; systematic Wolof\-specific fairness audits \(in collaboration with local linguists\) should be conducted, as well as alignment work to mitigate these biases\. Likewise, safety evaluation includes English\-centric checks \(WildJailbreak, HaluEval\); Wolof\-specific red\-teaming \(phishing, culturally inappropriate responses\) should be conducted\.

Deployment may displace customer\-service labor; we advocate human\-in\-the\-loop workflows where models handle routine queries and escalate complex cases to preserve employment\.

Fine\-tuning and inference for the benchmark evolves a large quantity of carbon emission\. These emissions are measured and taken into consideration when planning experiments to limit them\.

## Acknowledgments

The authors would like to thank the human annotators involved in the numerous stages of this work\.

## Appendix Contents

## References

- Adebaraet al\.\(2025\)I\. Adebara, H\. O\. Toyin, N\. T\. Ghebremichael, A\. A\. Elmadany, and M\. Abdul\-MageedWhere Are We? Evaluating LLM Performance on African Languages\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 32704–32731\.External Links:[Link](https://aclanthology.org/2025.acl-long.1572/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1572),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2610.20824#S1.p1.1),[§2](https://arxiv.org/html/2610.20824#S2.p1.1)\.
- Adelaniet al\.\(2022\)D\. I\. Adelani, G\. Neubig, S\. Ruder, S\. Rijhwani, M\. Beukman, C\. Palen\-Michel, C\. Lignos, J\. O\. Alabi, S\. H\. Muhammad, P\. Nabende, C\. M\. B\. Dione, A\. Bukula, R\. Mabuya, B\. F\. P\. Dossou, B\. Sibanda, H\. Buzaaba, J\. Mukiibi, G\. Kalipe, D\. Mbaye, A\. Taylor, F\. Kabore, C\. C\. Emezue, A\. Aremu, P\. Ogayo, C\. Gitau, E\. Munkoh\-Buabeng, V\. Memdjokam Koagne, A\. A\. Tapo, T\. Macucwa, V\. Marivate, E\. Mboning, T\. Gwadabe, T\. Adewumi, O\. Ahia, J\. Nakatumba\-Nabende, N\. L\. Mokono, I\. Ezeani, C\. Chukwuneke, M\. Adeyemi, G\. Q\. Hacheme, I\. Abdulmumin, O\. Ogundepo, O\. Yousuf, T\. Moteu Ngoli, and D\. KlakowMasakhaNER 2\.0: Africa\-centric Transfer Learning for Named Entity Recognition\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),Abu Dhabi, United Arab Emirates,pp\. 4488–4508\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.298/),[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.298)Cited by:[§H\.40](https://arxiv.org/html/2610.20824#A8.SS40.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.20824#S2.p3.1)\.
- Adelaniet al\.\(2025\)D\. I\. Adelani, J\. Ojo, I\. A\. Azime, J\. Y\. Zhuang, J\. Alabi, X\. He, M\. Ochieng, S\. Hooker, A\. Bukula, E\. A\. Lee,et al\.Irokobench: a new benchmark for african languages in the age of large language models\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 2732–2757\.Cited by:[§H\.37](https://arxiv.org/html/2610.20824#A8.SS37.SSS0.Px2.p1.1),[§H\.38](https://arxiv.org/html/2610.20824#A8.SS38.SSS0.Px2.p1.1),[§H\.39](https://arxiv.org/html/2610.20824#A8.SS39.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.20824#S2.p3.1)\.
- Alhanaiet al\.\(2025\)T\. Alhanai, A\. Kasumovic, M\. Ghassemi, A\. Zitzelberger, J\. Lundin, and G\. Chabot\-CoutureBridging the Gap: Enhancing LLM Performance for African Languages\.InProceedings of the AAAI Conference on Artificial Intelligence,External Links:[Document](https://dx.doi.org/10.1609/aaai.v39i27.34996)Cited by:[§2](https://arxiv.org/html/2610.20824#S2.p1.1)\.
- Alwaly Ndiaye \(2026\)A\. Alwaly NdiayeWolof French\.Note:Hugging Face datasetAccessed: 2026\-06\-17External Links:[Link](https://huggingface.co/datasets/Alwaly/wolof_french)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- Azimeet al\.\(2026\)I\. A\. Azime, J\. O\. Alabi, C\. Zhang, I\. Maab, A\. L\. Tonja, T\. D\. Belay, F\. P\. Alabi, S\. Osei, S\. M\. Aliyu, N\. F\. Aguobi, B\. F\. Balcha, B\. K\. Sibanda, D\. David, M\. Mboup, D\. Abolade, N\. Putini, P\. Slusallek, D\. I\. Adelani, and D\. KlakowAfrIFact: Cultural Information Retrieval, Evidence Extraction and Fact Checking for African Languages\.External Links:2604\.00706,[Link](https://arxiv.org/abs/2604.00706)Cited by:[§H\.1](https://arxiv.org/html/2610.20824#A8.SS1.SSS0.Px2.p1.1),[§H\.2](https://arxiv.org/html/2610.20824#A8.SS2.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.20824#S2.p3.1)\.
- Baiet al\.\(2022\)Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon, C\. Chen, C\. Olsson, C\. Olah, D\. Hernandez, D\. Drain, D\. Ganguli, D\. Li, E\. Tran\-Johnson, E\. Perez, J\. Kerr, J\. Mueller, J\. Ladish, J\. Landau, K\. Ndousse, K\. Lukosuite, L\. Lovitt, M\. Sellitto, N\. Elhage, N\. Schiefer, N\. Mercado, N\. DasSarma, R\. Lasenby, R\. Larson, S\. Ringer, S\. Johnston, S\. Kravec, S\. E\. Showk, S\. Fort, T\. Lanham, T\. Telleen\-Lawton, T\. Conerly, T\. Henighan, T\. Hume, S\. R\. Bowman, Z\. Hatfield\-Dodds, B\. Mann, D\. Amodei, N\. Joseph, S\. McCandlish, T\. Brown, and J\. KaplanTraining a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback\.External Links:2204\.05862,[Link](https://arxiv.org/abs/2204.05862)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px5.p1.1)\.
- Bandarkaret al\.\(2024\)L\. Bandarkar, D\. Liang, B\. Muller, M\. Artetxe, S\. N\. Shukla, D\. Husa, N\. Goyal, A\. Krishnan, L\. Zettlemoyer, and M\. KhabsaThe Belebele Benchmark: a parallel reading comprehension dataset in 122 language variants\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.44),[Link](https://aclanthology.org/2024.acl-long.44/)Cited by:[§2](https://arxiv.org/html/2610.20824#S2.p3.1)\.
- Berkeley Gorilla Team \(2024\)Berkeley Gorilla TeamBerkeley Function Calling Leaderboard\.Note:Benchmark and leaderboardExternal Links:[Link](https://gorilla.cs.berkeley.edu/leaderboard.html)Cited by:[§H\.10](https://arxiv.org/html/2610.20824#A8.SS10.SSS0.Px2.p1.1)\.
- Buzaabaet al\.\(2025\)H\. Buzaaba, A\. Wettig, D\. I\. Adelani, and C\. FellbaumLugha\-Llama: adapting large language models for african languages\.External Links:2504\.06536,[Link](https://arxiv.org/abs/2504.06536)Cited by:[§2](https://arxiv.org/html/2610.20824#S2.p2.1)\.
- Chenet al\.\(2021a\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, A\. Paino, N\. Tezak, J\. Tang, I\. Babuschkin, S\. Balaji, S\. Jain, W\. Saunders, C\. Hesse, A\. N\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. ZarembaEvaluating Large Language Models Trained on Code\.External Links:2107\.03374,[Link](https://arxiv.org/abs/2107.03374)Cited by:[§H\.35](https://arxiv.org/html/2610.20824#A8.SS35.SSS0.Px2.p1.1)\.
- Chenet al\.\(2021b\)Y\. Chen, Y\. Liu, L\. Chen, and Y\. ZhangDialogSum: a real\-life scenario dialogue summarization dataset\.InFindings of the Association for Computational Linguistics \(ACL\-IJCNLP\),pp\. 5062–5074\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.findings-acl.449),[Link](https://aclanthology.org/2021.findings-acl.449/)Cited by:[§H\.12](https://arxiv.org/html/2610.20824#A8.SS12.SSS0.Px2.p1.1)\.
- Clarket al\.\(2018\)P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. TafjordThink you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge\.InarXiv preprint arXiv:1803\.05457,External Links:[Link](https://arxiv.org/abs/1803.05457)Cited by:[§H\.3](https://arxiv.org/html/2610.20824#A8.SS3.SSS0.Px2.p1.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining Verifiers to Solve Math Word Problems\.External Links:2110\.14168,[Link](https://arxiv.org/abs/2110.14168)Cited by:[§H\.33](https://arxiv.org/html/2610.20824#A8.SS33.SSS0.Px2.p1.1)\.
- Conoveret al\.\(2023\)M\. Conover, M\. Hayes, A\. Mathur, J\. Xie, J\. Wan, S\. Shah, A\. Ghodsi, P\. Wendell, M\. Zaharia, and R\. XinFree Dolly: Introducing the World’s First Truly Open Instruction\-Tuned LLM\.Note:Hugging Face datasetExternal Links:[Link](https://huggingface.co/datasets/databricks/databricks-dolly-15k)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px3.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- Costa\-jussàet al\.\(2022\)M\. R\. Costa\-jussà, J\. Cross, O\. Çelebi, M\. Elbayad, K\. Heafield, K\. Heffernan, E\. Kalbassi, J\. Lam, D\. Licht, J\. Maillard,et al\.No Language Left Behind: Scaling Human\-Centered Machine Translation\.Transactions of the Association for Computational Linguistics10,pp\. 1335–1351\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00504),[Link](https://aclanthology.org/2022.tacl-1.80/)Cited by:[§H\.26](https://arxiv.org/html/2610.20824#A8.SS26.SSS0.Px2.p1.1),[§H\.27](https://arxiv.org/html/2610.20824#A8.SS27.SSS0.Px2.p1.1),[§H\.28](https://arxiv.org/html/2610.20824#A8.SS28.SSS0.Px2.p1.1),[§H\.29](https://arxiv.org/html/2610.20824#A8.SS29.SSS0.Px2.p1.1),[§H\.30](https://arxiv.org/html/2610.20824#A8.SS30.SSS0.Px2.p1.1)\.
- Cuiet al\.\(2024\)G\. Cui, L\. Yuan, N\. Ding, G\. Yao, B\. He, W\. Zhu, Y\. Ni, G\. Xie, R\. Xie, Y\. Lin, Z\. Liu, and M\. SunUltraFeedback: boosting language models with scaled ai feedback\.External Links:2310\.01377,[Link](https://arxiv.org/abs/2310.01377)Cited by:[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px2.p1.1)\.
- Dussolleet al\.\(2025\)A\. Dussolle, A\. Cardeña Díaz, S\. Sato, and P\. DevineM\-IFEval: Multilingual Instruction\-Following Evaluation\.External Links:2502\.04688,[Link](https://arxiv.org/abs/2502.04688)Cited by:[§H\.41](https://arxiv.org/html/2610.20824#A8.SS41.SSS0.Px2.p1.1)\.
- Geet al\.\(2024\)T\. Ge, X\. Chan, X\. Wang, D\. Yu, H\. Mi, and D\. YuScaling Synthetic Data Creation with 1,000,000,000 Personas\.External Links:2406\.20094,[Link](https://arxiv.org/abs/2406.20094)Cited by:[§B\.1\.2](https://arxiv.org/html/2610.20824#A2.SS1.SSS2.p1.1)\.
- Gliwaet al\.\(2019\)B\. Gliwa, I\. Mochol, M\. Biesek, and A\. WawerSAMSum Corpus: a human\-annotated dialogue dataset for abstractive summarization\.InProceedings of the 2nd Workshop on New Frontiers in Summarization,Hong Kong, China,pp\. 70–79\.External Links:[Document](https://dx.doi.org/10.18653/v1/D19-5409),[Link](https://aclanthology.org/D19-5409/)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px4.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- Hartvigsenet al\.\(2022\)T\. Hartvigsen, S\. Gabriel, H\. Palangi, M\. Sap, D\. Ray, and E\. KamarToxiGen: A Large\-Scale Machine\-Generated Dataset for Adversarial and Implicit Hate Speech Detection\.External Links:2203\.09509,[Link](https://arxiv.org/abs/2203.09509)Cited by:[§H\.47](https://arxiv.org/html/2610.20824#A8.SS47.SSS0.Px2.p1.1)\.
- Heet al\.\(2024\)Y\. He, D\. Jin, C\. Wang, C\. Bi, K\. Mandyam, H\. Zhang, C\. Zhu, N\. Li, T\. Xu, H\. Lv, S\. Bhosale, C\. Zhu, K\. A\. Sankararaman, E\. Helenowski, M\. Kambadur, A\. Tayade, H\. Ma, H\. Fang, and S\. WangMulti\-IF: Benchmarking LLMs on Multi\-Turn and Multilingual Instructions Following\.External Links:2410\.15553,[Link](https://arxiv.org/abs/2410.15553)Cited by:[§H\.44](https://arxiv.org/html/2610.20824#A8.SS44.SSS0.Px2.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. SteinhardtMeasuring Massive Multitask Language Understanding\.External Links:2009\.03300,[Link](https://arxiv.org/abs/2009.03300)Cited by:[§H\.42](https://arxiv.org/html/2610.20824#A8.SS42.SSS0.Px2.p1.1),[§H\.43](https://arxiv.org/html/2610.20824#A8.SS43.SSS0.Px2.p1.1)\.
- Hsiehet al\.\(2024\)C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. GinsburgRULER: What’s the Real Context Size of Your Long\-Context Language Models?\.External Links:2404\.06654,[Link](https://arxiv.org/abs/2404.06654)Cited by:[§H\.46](https://arxiv.org/html/2610.20824#A8.SS46.SSS0.Px2.p1.1)\.
- HuggingFaceH4 \(2023\)HuggingFaceH4Helpful Instructions\.Note:Hugging Face datasetExternal Links:[Link](https://huggingface.co/datasets/HuggingFaceH4/helpful-instructions)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px5.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- Hunteret al\.\(2023\)J\. Hunter, J\. Louradour, V\. Rennard, I\. Harrando, G\. Shang, and J\. LorréThe Claire French Dialogue Dataset\.arXiv preprint\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2311.16840)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px8.p1.1),[§H\.11](https://arxiv.org/html/2610.20824#A8.SS11.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- Hussenet al\.\(2026\)K\. Hussen, W\. Sewunetie, A\. Ayele, S\. Imam, E\. Alemu, S\. Muhammad, and S\. YimamThe State of Large Language Models for African Languages: Progress and Challenges\.InDLI 2025 Research Track,H\. Haddad, A\. N\. Kahira, S\. Bourhim, I\. E\. Olatunji, L\. Makhafola, and C\. Mwase \(Eds\.\),Proceedings of Machine Learning Research, Vol\.302,pp\. 1–27\.External Links:[Link](https://proceedings.mlr.press/v302/hussen26a.html)Cited by:[§1](https://arxiv.org/html/2610.20824#S1.p1.1),[§2](https://arxiv.org/html/2610.20824#S2.p1.1)\.
- Ivisonet al\.\(2024\)H\. Ivison, Y\. Wang, J\. Liu, Z\. Wu, V\. Pyatkin, N\. Lambert, N\. A\. Smith, Y\. Choi, and H\. HajishirziUnpacking dpo and ppo: disentangling best practices for learning from preference feedback\.External Links:2406\.09279,[Link](https://arxiv.org/abs/2406.09279)Cited by:[§C\.2](https://arxiv.org/html/2610.20824#A3.SS2.p2.1)\.
- Jianget al\.\(2024\)L\. Jiang, K\. Rao, S\. Han, A\. Ettinger, F\. Brahman, S\. Kumar, N\. Mireshghallah, X\. Lu, M\. Sap, Y\. Choi, and N\. DziriWildTeaming at Scale: From In\-the\-Wild Jailbreaks to \(Adversarially\) Safer Language Models\.External Links:2406\.18510,[Link](https://arxiv.org/abs/2406.18510)Cited by:[§H\.49](https://arxiv.org/html/2610.20824#A8.SS49.SSS0.Px2.p1.1)\.
- Kandah \(2024\)Y\. Kandahtool\-calling\-mix\.Note:Hugging Face datasetExternal Links:[Link](https://huggingface.co/datasets/younissk/tool-calling-mix)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px11.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- Kandjiet al\.\(2026\)A\. K\. Kandji, F\. Precioso, C\. Ba, S\. Ndiaye, and A\. NdioneWolBanking77: Wolof Banking Speech Intent Classification Dataset\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems \(NeurIPS\), Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=7k0JBDeHAv)Cited by:[§H\.50](https://arxiv.org/html/2610.20824#A8.SS50.SSS0.Px2.p1.1),[§H\.51](https://arxiv.org/html/2610.20824#A8.SS51.SSS0.Px2.p1.1),[§H\.52](https://arxiv.org/html/2610.20824#A8.SS52.SSS0.Px2.p1.1),[§H\.53](https://arxiv.org/html/2610.20824#A8.SS53.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.20824#S2.p1.1)\.
- Köpfet al\.\(2023a\)A\. Köpf, Y\. Kilcher, D\. von Rütte, S\. Anagnostidis, Z\. Tam, K\. Stevens, A\. Barhoum, N\. M\. Duc, O\. Stanley, R\. Nagyfi, S\. ES, S\. Suri, D\. Glushkov, A\. Dantuluri, A\. Maguire, C\. Schuhmann, H\. Nguyen, and A\. MattickOpenAssistant conversations – democratizing large language model alignment\.External Links:2304\.07327,[Link](https://arxiv.org/abs/2304.07327)Cited by:[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px1.p1.1)\.
- Köpfet al\.\(2023b\)A\. Köpf, Y\. Kilcher, D\. von Rütte, S\. Anagnostidis, Z\. Tam, K\. Stevens, A\. Barhoum, N\. M\. Duc, O\. Stanley, R\. Nagyfi, S\. ES, S\. Suri, D\. Glushkov, A\. Dantuluri, A\. Maguire, C\. Schuhmann, H\. Nguyen, and A\. MattickOpenAssistant Conversations: democratizing large language model alignment\.External Links:2304\.07327,[Link](https://arxiv.org/abs/2304.07327)Cited by:[§B\.1\.2](https://arxiv.org/html/2610.20824#A2.SS1.SSS2.p1.1)\.
- Lambertet al\.\(2025\)N\. Lambert, J\. Morrison, V\. Pyatkin, S\. Huang, H\. Ivison, F\. Brahman, L\. J\. V\. Miranda, A\. Liu, N\. Dziri, S\. Lyu, Y\. Gu, S\. Malik, V\. Graf, J\. D\. Hwang, J\. Yang, R\. L\. Bras, O\. Tafjord, C\. Wilhelm, L\. Soldaini, N\. A\. Smith, Y\. Wang, P\. Dasigi, and H\. HajishirziTulu 3: pushing frontiers in open language model post\-training\.External Links:2411\.15124,[Link](https://arxiv.org/abs/2411.15124)Cited by:[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px3.p1.1),[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px4.p1.1)\.
- Lecorvéet al\.\(2022\)G\. Lecorvé, M\. Veyret, Q\. Brabant, and L\. M\. Rojas BarahonaSPARQL\-to\-text question generation for knowledge\-based conversational applications\.InProceedings of the 2nd Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),Y\. He, H\. Ji, S\. Li, Y\. Liu, and C\. Chang \(Eds\.\),Online only,pp\. 131–147\.External Links:[Link](https://aclanthology.org/2022.aacl-main.11/),[Document](https://dx.doi.org/10.18653/v1/2022.aacl-main.11)Cited by:[§H\.48](https://arxiv.org/html/2610.20824#A8.SS48.SSS0.Px2.p1.1)\.
- Liet al\.\(2023\)J\. Li, X\. Cheng, W\. X\. Zhao, J\. Nie, and J\. WenHaluEval: A Large\-Scale Hallucination Evaluation Benchmark for Large Language Models\.External Links:2305\.11747,[Link](https://arxiv.org/abs/2305.11747)Cited by:[§H\.34](https://arxiv.org/html/2610.20824#A8.SS34.SSS0.Px2.p1.1)\.
- Liuet al\.\(2024\)Z\. Liu, T\. Hoang, J\. Zhang, M\. Zhu, T\. Lan, S\. Kokane, J\. Tan, W\. Yao, Z\. Liu, Y\. Feng,et al\.APIGen: automated pipeline for generating verifiable and diverse function\-calling datasets\.External Links:2406\.18518,[Link](https://arxiv.org/abs/2406.18518)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px11.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- lodrick\-the\-lafted \(2026\)lodrick\-the\-laftedHermes\-217K\.Note:Hugging Face datasetAccessed: 2026\-06\-17External Links:[Link](https://huggingface.co/datasets/lodrick-the-lafted/Hermes-217K)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px6.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- Longpreet al\.\(2023\)S\. Longpre, L\. Hou, T\. Vu, A\. Webson, H\. W\. Chung, Y\. Tay, D\. Zhou, Q\. V\. Le, B\. Zoph, J\. Wei, and A\. RobertsThe flan collection: designing data and methods for effective instruction tuning\.External Links:2301\.13688,[Link](https://arxiv.org/abs/2301.13688)Cited by:[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px8.p1.1)\.
- Luoet al\.\(2025\)Z\. Luo, C\. Xu, P\. Zhao, Q\. Sun, X\. Geng, W\. Hu, C\. Tao, J\. Ma, Q\. Lin, and D\. JiangWizardCoder: empowering code large language models with evol\-instruct\.External Links:2306\.08568,[Link](https://arxiv.org/abs/2306.08568)Cited by:[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px3.p1.1)\.
- Mbayeet al\.\(2025\)D\. Mbaye, T\. Mbengue, M\. Seye, M\. Diallo, M\. Ndiaye, D\. Adjanohoun, C\. S\. Wade, D\. Sow, J\. Munyaka Baraka, and J\. ChenalOpportunities and Challenges of Natural Language Processing for Low\-Resource Senegalese Languages in Social Science Research\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2601.09716)Cited by:[§1](https://arxiv.org/html/2610.20824#S1.p1.1),[§2](https://arxiv.org/html/2610.20824#S2.p1.1)\.
- OpenAI \(2024\)OpenAIMMMLU\.Note:Hugging Face datasetExternal Links:[Link](https://huggingface.co/datasets/openai/MMMLU)Cited by:[§H\.42](https://arxiv.org/html/2610.20824#A8.SS42.SSS0.Px2.p1.1),[§H\.43](https://arxiv.org/html/2610.20824#A8.SS43.SSS0.Px2.p1.1)\.
- Parrishet al\.\(2022\)A\. Parrish, A\. Chen, N\. Nangia, V\. Padmakumar, J\. Phang, J\. Thompson, P\. M\. Htut, and S\. R\. BowmanBBQ: A Hand\-Built Bias Benchmark for Question Answering\.InFindings of the Association for Computational Linguistics: ACL 2022,Dublin, Ireland,pp\. 2086–2105\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.findings-acl.165),[Link](https://aclanthology.org/2022.findings-acl.165/)Cited by:[§H\.9](https://arxiv.org/html/2610.20824#A8.SS9.SSS0.Px2.p1.1)\.
- Pyatkinet al\.\(2025\)V\. Pyatkin, S\. Malik, V\. Graf, H\. Ivison, S\. Huang, P\. Dasigi, N\. Lambert, and H\. HajishirziGeneralizing Verifiable Instruction Following\.External Links:2507\.02833,[Link](https://arxiv.org/abs/2507.02833)Cited by:[§H\.36](https://arxiv.org/html/2610.20824#A8.SS36.SSS0.Px2.p1.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.Advances in neural information processing systems36,pp\. 53728–53741\.Cited by:[§3\.2](https://arxiv.org/html/2610.20824#S3.SS2.p2.1)\.
- Reinet al\.\(2023\)D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. BowmanGPQA: A Graduate\-Level Google\-Proof Q&A Benchmark\.External Links:2311\.12022,[Link](https://arxiv.org/abs/2311.12022)Cited by:[§H\.31](https://arxiv.org/html/2610.20824#A8.SS31.SSS0.Px2.p1.1),[§H\.32](https://arxiv.org/html/2610.20824#A8.SS32.SSS0.Px2.p1.1)\.
- Salamancaet al\.\(2026\)A\. R\. Salamanca, D\. Abagyan, D\. D’souza, A\. Khairi, D\. Mora, S\. Dash, V\. Aryabumi, S\. Rajaee, M\. Mofakhami, A\. Sahu, T\. Euyang, B\. Prince, M\. Smith, H\. Lin, A\. Locatelli, S\. Hooker, T\. Kocmi, A\. Gomez, I\. Zhang, P\. Blunsom, N\. Frosst, J\. Pineau, B\. Ermis, A\. Üstün, J\. Kreutzer, and M\. FadaeeTiny Aya: Bridging Scale and Multilingual Depth\.External Links:2603\.11510,[Link](https://arxiv.org/abs/2603.11510)Cited by:[§2](https://arxiv.org/html/2610.20824#S2.p3.1)\.
- Sarasuaet al\.\(2025\)I\. Sarasua, A\. Corral, and X\. SaralegiDIPLomA: efficient adaptation of instructed LLMs to low\-resource languages via post\-training delta merging\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 24898–24912\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.1355/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.1355),ISBN 979\-8\-89176\-335\-7Cited by:[§2](https://arxiv.org/html/2610.20824#S2.p2.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§6](https://arxiv.org/html/2610.20824#S6.p2.1)\.
- Singhet al\.\(2024a\)S\. Singh, F\. Vargus, D\. Dsouza, B\. F\. Karlsson, A\. Mahendiran, W\. Ko, H\. Shandilya, J\. Patel, D\. Mataciunas, L\. OMahony, M\. Zhang, R\. Hettiarachchi, J\. Wilson, M\. Machado, L\. S\. Moura, D\. Krzemiński, H\. Fadaei, I\. Ergün, I\. Okoh, A\. Alaagib, O\. Mudannayake, Z\. Alyafeai, V\. M\. Chien, S\. Ruder, S\. Guthikonda, E\. A\. Alghamdi, S\. Gehrmann, N\. Muennighoff, M\. Bartolo, J\. Kreutzer, A\. Üstün, M\. Fadaee, and S\. HookerAya dataset: an open\-access collection for multilingual instruction tuning\.External Links:2402\.06619,[Link](https://arxiv.org/abs/2402.06619)Cited by:[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px6.p1.1)\.
- Singhet al\.\(2024b\)S\. Singh, F\. Vargus, D\. D’souza, B\. F\. Karlsson, A\. Mahendiran, W\. Ko, H\. Shandilya, J\. Patel, D\. Mataciunas, L\. O’Mahony, M\. Zhang, R\. Hettiarachchi, J\. Wilson, M\. Machado, L\. Moura, D\. Krzemiński, H\. Fadaei, I\. Ergun, I\. Okoh, A\. Alaagib, O\. Mudannayake, Z\. Alyafeai, V\. Chien, S\. Ruder, S\. Guthikonda, E\. Alghamdi, S\. Gehrmann, N\. Muennighoff, M\. Bartolo, J\. Kreutzer, A\. Üstün, M\. Fadaee, and S\. HookerAya Dataset: An Open\-Access Collection for Multilingual Instruction Tuning\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 11521–11567\.External Links:[Link](https://aclanthology.org/2024.acl-long.620/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.620)Cited by:[§H\.4](https://arxiv.org/html/2610.20824#A8.SS4.SSS0.Px2.p1.1),[§H\.5](https://arxiv.org/html/2610.20824#A8.SS5.SSS0.Px2.p1.1),[§H\.6](https://arxiv.org/html/2610.20824#A8.SS6.SSS0.Px2.p1.1),[§H\.7](https://arxiv.org/html/2610.20824#A8.SS7.SSS0.Px2.p1.1),[§H\.8](https://arxiv.org/html/2610.20824#A8.SS8.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.20824#S2.p2.1)\.
- Smart \(2026\)D\. S\. SmartMultiWikiQA: A Reading Comprehension Benchmark in 300\+ Languages\.InProceedings of the Fifteenth Language Resources and Evaluation Conference \(LREC 2026\),S\. Piperidis, N\. Bel, H\. van den Heuvel, N\. Ide, S\. Krek, and A\. Toral \(Eds\.\),Palma, Mallorca, Spain,pp\. 6298–6311\.External Links:[Document](https://dx.doi.org/10.63317/2msrgsu9isrx)Cited by:[§2](https://arxiv.org/html/2610.20824#S2.p3.1)\.
- Soynade Research \(2025\)Soynade ResearchOolel\-v0\.1\.Note:Hugging Face modelExternal Links:[Link](https://huggingface.co/soynade-research/Oolel-v0.1)Cited by:[§2](https://arxiv.org/html/2610.20824#S2.p3.1)\.
- Taoriet al\.\(2023\)R\. Taori, I\. Gulrajani, T\. Zhang, Y\. Dubois, X\. Li, C\. Guestrin, P\. Liang, and T\. B\. HashimotoAlpaca: 52K Instruction\-Following Demonstrations\.Note:Hugging Face datasetExternal Links:[Link](https://huggingface.co/datasets/tatsu-lab/alpaca)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px2.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- Team OLMoet al\.\(2025\)Team OLMo, A\. Ettinger, A\. Bertsch, B\. Kuehl, D\. Groeneveld, F\. Brahman, H\. Ivison, N\. Lambert, N\. A\. Smith, and H\. HajishirziOLMo 3\.External Links:2512\.13961,[Link](https://arxiv.org/abs/2512.13961)Cited by:[§3\.2](https://arxiv.org/html/2610.20824#S3.SS2.p2.1)\.
- Tonjaet al\.\(2024\)A\. L\. Tonja, B\. F\. P\. Dossou, J\. Ojo, J\. Rajab, F\. Thior, E\. P\. Wairagala, A\. Aremu, P\. Moiloa, J\. Abbott, V\. Marivate, and B\. RosmanInkubaLM: a small language model for low\-resource african languages\.External Links:2408\.17024,[Link](https://arxiv.org/abs/2408.17024)Cited by:[§2](https://arxiv.org/html/2610.20824#S2.p2.1)\.
- Toshniwalet al\.\(2024\)S\. Toshniwal, I\. Moshkov, S\. Narenthiran, D\. Gitman, F\. Jia, and I\. GitmanOpenMathInstruct\-1: a 1\.8 million math instruction tuning dataset\.External Links:2402\.10176,[Link](https://arxiv.org/abs/2402.10176)Cited by:[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px4.p1.1)\.
- Touayouchet al\.\(2026\)B\. Touayouch, L\. Fosse, G\. Damnati, and G\. LecorvéDivMerge: a divergence\-based model merging method for multi\-tasking\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 7157–7180\.Cited by:[§3\.2](https://arxiv.org/html/2610.20824#S3.SS2.p3.1)\.
- Üstünet al\.\(2024\)A\. Üstün, V\. Aryabumi, Z\. Yong, W\. Ko, D\. D’souza, G\. Onilude, N\. Bhandari, S\. Singh, H\. Ooi, A\. Kayid, F\. Vargus, P\. Blunsom, S\. Longpre, N\. Muennighoff, M\. Fadaee, J\. Kreutzer, and S\. HookerAya Model: An Instruction Finetuned Open\-Access Multilingual Language Model\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15894–15939\.External Links:[Link](https://aclanthology.org/2024.acl-long.845/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.845)Cited by:[§2](https://arxiv.org/html/2610.20824#S2.p2.1),[§2](https://arxiv.org/html/2610.20824#S2.p3.1)\.
- Waddenet al\.\(2025\)D\. Wadden, K\. Shi, J\. Morrison, A\. Li, A\. Naik, S\. Singh, N\. Barzilay, K\. Lo, T\. Hope, L\. Soldaini, S\. Z\. Shen, D\. Downey, H\. Hajishirzi, and A\. CohanSciRIFF: a resource to enhance language model instruction\-following over scientific literature\.External Links:2406\.07835,[Link](https://arxiv.org/abs/2406.07835)Cited by:[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px2.p1.1)\.
- Wanget al\.\(2022\)Y\. Wang, S\. Mishra, P\. Alipoormolabashi, Y\. Kordi, A\. Mirzaei, A\. Arunkumar, A\. Ashok, A\. S\. Dhanasekaran, A\. Naik, D\. Stap, E\. Pathak, G\. Karamanolakis, H\. G\. Lai, I\. Purohit, I\. Mondal, J\. Anderson, K\. Kuznia, K\. Doshi, M\. Patel, K\. K\. Pal, M\. Moradshahi, M\. Parmar, M\. Purohit, N\. Varshney, P\. R\. Kaza, P\. Verma, R\. S\. Puri, R\. Karia, S\. K\. Sampat, S\. Doshi, S\. Mishra, S\. Reddy, S\. Patro, T\. Dixit, X\. Shen, C\. Baral, Y\. Choi, N\. A\. Smith, H\. Hajishirzi, and D\. KhashabiSuper\-NaturalInstructions: Generalization via Declarative Instructions on 1600\+ NLP Tasks\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Abu Dhabi, United Arab Emirates,pp\. 5085–5109\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.340),[Link](https://aclanthology.org/2022.emnlp-main.340/)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px7.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- Wortsmanet al\.\(2022\)M\. Wortsman, G\. Ilharco, S\. Y\. Gadre, R\. Roelofs, R\. Gontijo\-Lopes, A\. S\. Morcos, H\. Namkoong, A\. Farhadi, Y\. Carmon, S\. Kornblith,et al\.Model soups: averaging weights of multiple fine\-tuned models improves accuracy without increasing inference time\.InInternational conference on machine learning,pp\. 23965–23998\.Cited by:[§3\.2](https://arxiv.org/html/2610.20824#S3.SS2.p3.1)\.
- Yanget al\.\(2024\)E\. Yang, Z\. Wang, L\. Shen, S\. Liu, G\. Guo, X\. Wang, and D\. TaoAdamerging: adaptive model merging for multi\-task learning\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 22743–22763\.Cited by:[§3\.2](https://arxiv.org/html/2610.20824#S3.SS2.p3.1)\.
- Yuet al\.\(2026\)H\. Yu, T\. Xu, M\. A\. Hedderich, W\. Hamidouche, S\. W\. Zamir, and D\. I\. AdelaniAfriqueLLM: how data mixing and model architecture impact continued pre\-training for African languages\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 5909–5928\.External Links:[Link](https://aclanthology.org/2026.acl-long.267/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.267),ISBN 979\-8\-89176\-390\-6Cited by:[§2](https://arxiv.org/html/2610.20824#S2.p2.1)\.
- Zhaet al\.\(2023\)L\. Zha, J\. Zhou, L\. Li, R\. Wang, Q\. Huang, S\. Yang, J\. Yuan, C\. Su, X\. Li, A\. Su, T\. Zhang, C\. Zhou, K\. Shou, M\. Wang, W\. Zhu, G\. Lu, C\. Ye, Y\. Ye, W\. Ye, Y\. Zhang, X\. Deng, J\. Xu, H\. Wang, G\. Chen, and J\. ZhaoTableGPT: towards unifying tables, nature language and commands into one gpt\.External Links:2307\.08674,[Link](https://arxiv.org/abs/2307.08674)Cited by:[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px8.p1.1)\.
- Zhanget al\.\(2024\)J\. Zhang, T\. Lan, M\. Zhu, Z\. Liu, T\. Hoang, S\. Kokane, W\. Yao, J\. Tan, A\. Prabhakar, H\. Chen, Z\. Liu, Y\. Feng, T\. Awalgaonkar, R\. Murthy, E\. Hu, Z\. Chen, R\. Xu, J\. C\. Niebles, S\. Heinecke, H\. Wang, S\. Savarese, and C\. XiongxLAM: A Family of Large Action Models to Empower AI Agent Systems\.External Links:2409\.03215,[Link](https://arxiv.org/abs/2409.03215)Cited by:[§B\.1\.1](https://arxiv.org/html/2610.20824#A2.SS1.SSS1.Px11.p1.1),[§3\.1](https://arxiv.org/html/2610.20824#S3.SS1.p2.1)\.
- Zhanget al\.\(2018\)S\. Zhang, E\. Dinan, J\. Urbanek, A\. Szlam, D\. Kiela, and J\. WestonPersonalizing Dialogue Agents: I have a dog, do you have pets too?\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Melbourne, Australia,pp\. 2204–2213\.External Links:[Document](https://dx.doi.org/10.18653/v1/P18-1205),[Link](https://aclanthology.org/P18-1205/)Cited by:[§H\.45](https://arxiv.org/html/2610.20824#A8.SS45.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2024\)W\. Zhao, X\. Ren, J\. Hessel, C\. Cardie, Y\. Choi, and Y\. DengWildChat: 1m chatgpt interaction logs in the wild\.External Links:2405\.01470,[Link](https://arxiv.org/abs/2405.01470)Cited by:[§C\.1](https://arxiv.org/html/2610.20824#A3.SS1.SSS0.Px1.p1.1)\.
- Zhouet al\.\(2023\)J\. Zhou, T\. Lu, S\. Mishra, S\. Brahma, S\. Basu, Y\. Luan, D\. Zhou, and L\. HouInstruction\-Following Evaluation for Large Language Models\.External Links:2311\.07911,[Link](https://arxiv.org/abs/2311.07911)Cited by:[§B\.1\.2](https://arxiv.org/html/2610.20824#A2.SS1.SSS2.p1.1)\.
- Zhouet al\.\(2024\)Y\. Zhou, L\. Song, B\. Wang, and W\. ChenMetagpt: merging large language models using model exclusive task arithmetic\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 1711–1724\.Cited by:[§3\.2](https://arxiv.org/html/2610.20824#S3.SS2.p3.1)\.

## Appendix AOverall Score as a Function of Model Size

Figure[1](https://arxiv.org/html/2610.20824#A1.F1)plots the overall scores against model size\. For commercial API systems, the horizontal segment represents a prospectively estimated parameter\-count range, not a disclosed or verified model size\. These purely indicative estimates are derived from unofficial, publicly accessible online information and should not be interpreted as vendor\-confirmed specifications\.

Figure 1:Aggregate scores as a function of model size\. Orange squares denote our Wolof\-adapted models\. Circles denote unadapted baselines, and diamonds denote Wolof\-aware models from the literature\. Commercial API systems are shown using purely indicative, prospectively estimated parameter ranges based on unofficial online information, rather than vendor\-confirmed specifications\.
## Appendix BInstruction tuning details

### B\.1Data

Table[5](https://arxiv.org/html/2610.20824#A2.T5)provides complete statistics for training, validation, and test splits across all datasets, broken down by language and task type\.

DatasetSplitExamplesTokensAvg LenLanguageType*Translated General Instruction \(Wolof\)*Alpaca \(Wolof\)Train51,93910,420,068200\.6WolofSingle turnValidation306,456215\.2WolofSingle turnDolly\-15k \(Wolof\)Train14,9854,352,286290\.4WolofSingle turnValidation309,723324\.1WolofSingle turnSAMSum \(Wolof\)Train14,6964,006,869272\.7WolofSingle turnValidation307,718257\.3WolofSingle turnAnthropic Helpful \(Wolof\)Train65,39915,257,134233\.3WolofSingle turnValidation10015,467154\.7WolofSingle turnHermes\-217k \(33k subset\)Train33,00017,083,410517\.7WolofSingle turnValidation10051,751517\.5WolofSingle turnWiseNUT \(Wolof\)Train19,7005,315,879269\.8WolofSingle turnValidation10011,428114\.3WolofSingle turnClaire multilogueTrain15,31722,485,0291,468\.0WolofSingle turnValidation100202,8622,028\.6WolofSingle turn*Bilingual French\-Wolof*Alwaly French→\\rightarrowWolofTrain90,9564,998,52855\.0French/WolofSingle turnValidation1005,24252\.4French/WolofSingle turnAlwaly Wolof→\\rightarrowFrenchTrain90,9564,999,31655\.0Wolof/FrenchSingle turnValidation1005,25852\.6Wolof/FrenchSingle turn*Native Conversational \(Wolof\)*Claire one\-to\-oneTrain6,7575,692,874842\.5WolofMultiple turnValidation100102,5991,026\.0WolofMultiple turnIn\-house conversationsValidation5033,883677\.7WolofMultiple turnTest14999,657668\.8WolofMultiple turn*Function Calling \(English\)*English function callingTrain42,15014,866,461352\.7EnglishMultiple turnValidation2,218791,172356\.7EnglishMultiple turn*Validation Only \(English/French\)*OASST1\-EnglishValidation18881,840435\.3EnglishMultiple turnOASST1\-FrenchValidation167,490468\.1FrenchMultiple turnSynthetic\-FrenchValidation17991,310510\.1FrenchSingle turnTulu\-3\-SFT\-personas\-IFValidation26,982567,69321\.0EnglishSingle turn*Aggregates by Language*Total Wolof \(mono\)222,58285,155,093382\.6WolofTotal bilingual Wolof182,11210,008,34455\.0French/Wolof, Wolof/FrenchTotal English71,53816,307,166228\.0EnglishTotal French19598,800506\.7FrenchTotal Training445,855109,477,854245\.5Total Validation30,4231,991,89265\.5Total Test14999,657668\.8Table 5:Dataset statistics by split\.Avg Lenis the average number of tokens per example\. Wolof\-containing data \(monolingual and bilingual\) accounts for approximately 90% of training examples\.The qualitative descriptions below follow the same source families as Table[1](https://arxiv.org/html/2610.20824#S3.T1), while Table[5](https://arxiv.org/html/2610.20824#A2.T5)reports the corresponding split\-level statistics\.

#### B\.1\.1Datasets

##### Alwaly\.

Alwaly provides native bilingual French\-Wolof text derived from educational and religious sources\([Alwaly Ndiaye, 2026](https://arxiv.org/html/2610.20824#bib.bib21)\)\. We use the corpus in both directions: French inputs with Wolof outputs, and Wolof inputs with French outputs\. These examples provide authentic Wolof text and help preserve translation ability between the two languages most relevant to the target deployment setting\.

##### Alpaca\.

Alpaca is a general\-purpose instruction\-following dataset covering question answering, classification, creative generation, and factual knowledge queries\([Taori et al\., 2023](https://arxiv.org/html/2610.20824#bib.bib22)\)\. We machine\-translated the instruction and response fields into Wolof and used the resulting examples to broaden single\-turn instruction coverage beyond translation\-style supervision\.

##### Dolly\-15k\.

Dolly\-15k is a human\-written instruction corpus released by Databricks\([Conover et al\., 2023](https://arxiv.org/html/2610.20824#bib.bib23)\)\. Its categories include brainstorming, classification, closed question answering, generation, information extraction, open question answering, and summarization\. After translation into Wolof, it contributes diverse single\-turn supervision with relatively natural human\-authored prompts\.

##### SAMSum\.

SAMSum contains messenger\-like conversations paired with human\-written summaries\([Gliwa et al\., 2019](https://arxiv.org/html/2610.20824#bib.bib25)\)\. We translated the data into Wolof and formatted it as instruction\-style dialogue summarization, which adds supervision for compressing conversational content while still fitting the single\-turn training format used in the main mixture\.

##### Helpful instructions\.

The Helpful instructions subset is derived from the HuggingFaceH4 helpfulness data\([HuggingFaceH4, 2023](https://arxiv.org/html/2610.20824#bib.bib26)\), including examples from Anthropic helpfulness data\([Bai et al\., 2022](https://arxiv.org/html/2610.20824#bib.bib27)\)\. It contains assistant\-style interactions that were translated into Wolof and converted into instruction examples\. This source complements Alpaca and Dolly by emphasizing helpful response behavior and natural instruction following\.

##### Hermes\-217k\.

Hermes\-217k is a large synthetic instruction collection with coverage of coding, mathematics, reasoning, knowledge, and general QA\-style tasks\([lodrick\-the\-lafted, 2026](https://arxiv.org/html/2610.20824#bib.bib28)\)\. We use a curated translated subset to add task diversity and reasoning\-oriented supervision to the Wolof instruction mixture\.

##### WiseNUT\.

WiseNUT contributes Natural\-Instructions\-style examples translated into Wolof\([Wang et al\., 2022](https://arxiv.org/html/2610.20824#bib.bib29)\)\. The retained subset provides QA\-oriented and speaker\-identification prompts, adding factual and structured instruction patterns that complement the broader general\-purpose sources\.

##### Claire multilogue\.

Claire multilogue is built from the OpenLLM\-France Claire\-Dialogue\-French\-0\.1 dataset\([Hunter et al\., 2023](https://arxiv.org/html/2610.20824#bib.bib13)\)by extracting dialogues with at least three speakers\. For each dialogue, one speaker is randomly selected, one of that speaker’s turns is masked, and the model is trained to reconstruct the masked turn from the full dialogue context\. The resulting examples are dialogue\-derived but are used as a single\-turn masked\-turn completion objective\.

##### Claire one\-to\-one\.

Claire one\-to\-one is built from the same Claire source by extracting dialogues with exactly two speakers\. These samples are formatted as standard multi\-turn conversations, with a system message providing short speaker profiles and dialogue context, followed by alternating user and assistant turns\. The profiles, context, and messages were machine\-translated into Wolof, giving the model direct supervision for turn\-taking and persona\-consistent Wolof dialogue\.

##### In\-house conversations\.

The in\-house conversation split contains de\-identified Wolof customer\-care dialogues\. It is used to monitor deployment\-relevant conversational behavior, including service terminology, turn\-taking, and customer\-intent patterns\. Raw logs are not released, but the split informs validation and test coverage for the industrial customer\-service setting\.

##### Function calling instructions\.

The English function\-calling block aggregates examples fromtool\-calling\-mix\([Kandah, 2024](https://arxiv.org/html/2610.20824#bib.bib30)\),xLAM k60k\([Zhang et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib31);[Liu et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib32)\), and synthetic examples generated with Gemini 2\.5 Pro\. We deduplicated and filtered the public sources to keep examples where at least one tool is actually invoked and to exclude BFCL\-derived examples\. The synthetic portion targets Python, Java, and JavaScript function signatures with 1–20 available tools per example, following the Berkeley Function Calling Leaderboard distribution of tool\-count complexity\. This block is kept in English to preserve structured tool\-use behavior and support cross\-lingual transfer to Wolof\-facing assistant use\.

#### B\.1\.2Validation data

First, we included Bilingual French\-Wolof and Translated General validation sets\. Then, we added stratified validation sets by language and task type\. For English and French conversations, we use established conversational benchmarks including OASST1\([Köpf et al\., 2023b](https://arxiv.org/html/2610.20824#bib.bib33)\), synthetic French dialogues generated by ourselves, and Tulu\-3\-SFT\-personas\([Ge et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib34)\)to assess conversational quality and instruction\-following in these languages\. Wolof conversational validation relies on the held\-out split ofclaire\_one\_to\_one\_dialogueand an in\-house customer\-care conversation dataset to evaluate dialogue coherence and turn\-taking in Wolof\. Validation for structured output and tool use employs the held\-out split ofen\_function\_callingandtulu\-3\-sft\-personas\-if\([Zhou et al\., 2023](https://arxiv.org/html/2610.20824#bib.bib35)\)\(instruction\-following variant\)\. A composite validation score aggregating performance across these dimensions guided model checkpoint selection\.

### B\.2Training Setup

As detailed in Table[6](https://arxiv.org/html/2610.20824#A2.T6), training was conducted on a single\-node setup with 2 NVIDIA H100 80GB and H200 140GB GPUs\. Main experiments ran for 3 epochs and typically lasted from a few hours to several days, depending on model size and corpus scale\. Optimization used paged AdamW \(paged\_adamw\_32bit\) with a cosine learning\-rate schedule, 5 % warmup, and a peak learning rate of2×10−52\\times 10^\{\-5\}\. Regularization included gradient clipping at 1\.0 and weight decay of 0\.1\.

Models were trained in bfloat16 with DeepSpeed ZeRO\-3 and gradient checkpointing\. The batch size was 16 per device with gradient accumulation over 32 steps, yielding an effective batch size of 1024\. The maximum sequence length was set to 2048 tokens, and attention was accelerated with FlashAttention2 \(flash\_attention\_2\)\. Data loading used 4 workers per GPU, with persistent workers, pinned memory, and a prefetch factor of 2\.

Checkpoints were saved at the end of each epoch, keeping only the most recent one and selecting models based on validation loss\. Validation was performed every 10 steps on EN/FR/WO and function\-calling splits using a composite score for model selection\. For reproducibility, training relied on YAML configurations, fixed seeds \(42\), and TensorBoard logging\. Carbon emissions were tracked with CodeCarbon, reporting 1\.6 kg CO2eq for the main run\.

Table 6:Instruction tuning \(SFT\) training setup\.

## Appendix CPreference Alignment Details

### C\.1Data

Table 7:Composition and Qwen3 token\-length statistics for Dolci\-Instruct\-DPO\. Token averages are computed using Qwen3 tokenizer\.Table[7](https://arxiv.org/html/2610.20824#A3.T7)details the composition, split sizes, and sequence\-length profiles across the main categories of the Dolci\-Instruct\-DPO dataset\. The corpus contains 259,784 total pairwise comparisons partitioned into a 90/10 train/validation split \(233,804 training and 25,980 validation pairs\) which are stratified consistently across all individual sources\.

##### Instruction\-Following\.

The dataset counts for 72,296 preference pairs \(27\.8% of the corpus\), it combines real\-world user queries with strict constraint\-following tasks with natural user interactions from Chat and WildChat\([Zhao et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib61)\)alongside volunteer\-annotated dialog trees from OpenAssistant Conversations \(OASST1;[Köpf et al\. 2023a](https://arxiv.org/html/2610.20824#bib.bib62)\)\. These interactions are complemented by synthetic instances from Dolci Instruct Precise IF and Persona Precise IF, which use verifiable constraints \(e\.g\., specific word counts, negative constraints, and exact formatting tags\)\.

##### General and Scientific Knowledge\.

The corpus covers 57,319 pairs \(22\.1%\), this partition provides supervision for factual correctness, multi\-step explanations, and domain\-specific knowledge\. It incorporates SciRiff\([Wadden et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib63)\), an instruction benchmark covering biomedical and scientific information extraction, as well as Dolci Instruct OpenThought3, with detailed reasoning traces\. These examples are combined with UltraFeedback\([Cui et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib64)\), where outputs from different models scored across instruction\-following, truthfulness, and honesty metrics using GPT\-4 as a judge\.

##### Coding\.

The coding subset contains 41,732 pairs \(16\.1%\) focused on program summaries, algorithmic problem\-solving, and code execution correctness\. It integrates Evol\-CodeAlpaca\([Luo et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib65)\), produced via iterative prompting \(e\.g\., adding constraints, improving complexity\), with Tülu 3 Persona Python\([Lambert et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib66)\)and Dolci Instruct Python Algorithms\. Tülu 3 Persona tasks present modular problems across different programmer skills\.

##### Mathematics\.

This category contains 27,045 pairs \(10\.4%\) designed to strengthen quantitative reasoning\. It uses data from OpenMathInstruct\-1\([Toshniwal et al\., 2024](https://arxiv.org/html/2610.20824#bib.bib67)\)and the math parts of Tülu 3 Persona\([Lambert et al\., 2025](https://arxiv.org/html/2610.20824#bib.bib66)\), like GSM8K, MATH, and algebra problems\. Pairs are typically formed by ranking correct final answers and valid intermediate steps over hallucinated reasoning paths\.

##### Safety and Alignment\.

The dataset presents 26,706 pairs \(10\.3%\), The pairs teach the model to clearly say no to dangerous requests \(such as cyberattacks\), while still giving helpful answers to safe questions that only contain sensitive words\.

##### Multilingual

This part has 6,500 pairs \(2\.5% of the data\) taken from the Aya collection\([Singh et al\., 2024a](https://arxiv.org/html/2610.20824#bib.bib68)\), covering many different languages\. Even though Wolof is not included here, this data helps the model remember how to work in multiple languages and stop it from forgetting other languages after training\.

##### Long\-Context Reasoning\.

This category counts 9,975 pairs \(3\.8% of the data\) made from the Dolci Instruct Self\-Talk and Synthetic Context datasets\. The inputs are quite long \(averaging 773\.7 tokens and over 850 in validation\) and include long chat histories, text snippets, and complex instructions\. These pairs teach the model not to get lost or ignore parts of long texts\. Instead, it learns to find key details in long prompts and give short, direct answers \(averaging 177\.9 chosen tokens\)\.

##### Others\.

This last group contains 18,211 pairs \(7\.0% of the data\) covering tables, classic NLP tasks, and general questions\. It includes table data from TableGPT\([Zha et al\., 2023](https://arxiv.org/html/2610.20824#bib.bib69)\), standard tasks from the FLAN collection\([Longpre et al\., 2023](https://arxiv.org/html/2610.20824#bib.bib70)\), and multi\-turn conversations from NVIDIA’s DaringAnteater\. Good answers here follow special formats and extract data cleanly, which helps the model stay reliable across many different kinds of everyday tasks\.

### C\.2Training Setup

As detailed in Table[8](https://arxiv.org/html/2610.20824#A3.T8), experiments were conducted on a single node with either2×2\\timesNVIDIA H100 \(80GB\) GPUs for the Qwen3\-14B model, or1×1\\timesH200 \(140GB\) GPU for the 8B models\. Training durations ranged from approximately one day for smaller models \(0\.6B\-1\.7B\) to eight days for the largest models\. All trainings targeted an effective batch size of 512, with the same learning\-rate and scheduler\. Evaluations as well as checkpointing were done each 40 steps, and the best model was selected using validation loss \(eval\_validation\_loss\)\.

To save energy and time, hyper\-parameters tuning was performed on Qwen3\-0\.6B model, iterating over learning rates \(5×10−5,5×10−6,5×10−7,5×10−85\\times 10^\{\-5\},5\\times 10^\{\-6\},5\\times 10^\{\-7\},5\\times 10^\{\-8\}\) andβ\\betavalues \(0\.01,0\.02,0\.05,0\.1,0\.2,0\.30\.01,0\.02,0\.05,0\.1,0\.2,0\.3\), extending recommendations from[Ivison et al\. \(2024\)](https://arxiv.org/html/2610.20824#bib.bib59)\. The two best\-performing configurations were then validated on an 8B model\. Finally, the optimal configuration \(β=0\.2\\beta=0\.2and a peak learning rate of5×10−65\\times 10^\{\-6\}\) was selected for all final training runs\.

Table 8:Preference alignment \(DPO\) training setup\.

## Appendix DHuman Assessment of Translation and LLM Judging

### D\.1Translation Quality

To assess the quality of the automatically translated Wolof data, we conducted a manual evaluation on a randomly selected, domain\-stratified sample of 50 English\-Wolof sentence pairs from FLORES\-200\. The translations were produced by Google Translate and evaluated by two native Wolof speakers using two five\-point scales: semantic adequacy \(“correctness of meaning”\) and linguistic quality of the Wolof output\. The translations received an average score of 4\.59/5 for semantic adequacy and 4\.78/5 for linguistic quality, indicating that they were generally faithful to the source sentences and grammatically well formed\. The inter\-annotator agreement for semantic adequacy and linguistic quality are64%64\\%and68%68\\%, respectively, rising to88%88\\%and100%100\\%when a∓1\\mp\{\}1margin is tolerated\.

On the same samples, automatic evaluation yielded a METEOR score of 0\.24\. The Pearson correlation between sentence\-level METEOR scores and human semantic\-adequacy ratings wasr=0\.12r=0\.12, suggesting little relationship between METEOR and perceived translation quality in this setting\. This discrepancy is due to specific aspects of Wolof, that penalize lexical\-based scores such as METEOR, whereas the translation quality is judged acceptable by human:

- •Syntactic flexibility: Wolof may allow more than one valid syntactic realization of the same proposition\. A translation can therefore preserve the meaning while using a word order that differs from the FLORES\-200 reference\.
- •Orthographic variation: Wolof orthography is not always completely standardized across resources and speakers\. Variants such as ñoo and ñu, or bi and bu, may be acceptable depending on the grammatical and contextual usage, while still producing limited character or word overlap with the reference\.
- •Accepted lexical borrowings and alternatives: The system may use lexical borrowings or alternative terms that are readily understood and accepted by Wolof speakers\. For example, the translation may use pied for “feet” or tournoi for “tournament,” whereas the FLORES\-200 reference uses different Wolof equivalents\.

These results motivate the use of human assessment alongside automatic metrics when evaluating machine\-translated Wolof data; accordingly, we retain the automatically generated data as silver data and explicitly document its provenance\.

### D\.2LLM\-as\-a\-Judge Reliability

While LLM\-as\-a\-judge is rather a common practice for the most popular languages, the question of using an LLM to judge free text in Wolof is less documented\. Hence, gemini\-2\.5 models have been validated with 3 Wolof speakers on held\-out data\.

In this experiment, users had to on agree or disagree with the scores provided by the LLM judge for 450 predictions of various models on Aya tasks\. The agreement rate was 95% for gemini\-2\.5\-flash and 99 % for gemini\-2\.5\-pro\.

## Appendix EDevelopmental results

This appendix reports the intermediate development experiments using the same normalized skill aggregates as the main automatic evaluation\.

### E\.1Instruction Ablation

Table[9](https://arxiv.org/html/2610.20824#A5.T9)compares the Qwen2\.5\-7B\-Instruct base model with staged Wolof\-adapted variants\. The comparison isolates the contribution of instruction tuning, function\-calling data, conversational data, and the final combined recipe before transferring the setup to Qwen3\.

Table 9:Training\-data ablation comparison on Qwen2\.5\-7B\-Instruct: normalised scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.
### E\.2Continued Pre\-training

Table[10](https://arxiv.org/html/2610.20824#A5.T10)compares the Qwen2\.5\-7B baseline, the SFT\-only recipe, and a continued\-pre\-training plus SFT variant\. This experiment evaluates whether the available raw Wolof text is sufficient to justify a CPT stage before supervised adaptation\.

Table 10:Continued\-pretraining comparison on Qwen2\.5\-7B\-Instruct: normalised scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.
### E\.3Tokenization

Table[11](https://arxiv.org/html/2610.20824#A5.T11)compares the current Qwen2\.5\-7B instruction\-tuned setup with a tokenizer\-extension variant\. This experiment tests whether vocabulary adaptation improves Wolof coverage while preserving the broader skill profile\.

Table 11:Vocabulary\-augmentation comparison on Qwen2\.5\-7B\-Instruct: normalised scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.

## Appendix FStage\-by\-Stage Results

This appendix isolates the contribution of the successive adaptation stages\. Table[12](https://arxiv.org/html/2610.20824#A6.T12)compares the Qwen3 baselines, the models after Wolof\-centric SFT, the models after DPO, and the merged models\. The SFT stage is the only one that uses Wolof data\. DPO and model merging aim to recover general capabilities while preserving the Wolof skills acquired during SFT\.

Table 12:Post\-SFT comparison of Qwen3 baselines, Wolof\-adapted Qwen3 models, and DPO/model\-merging variants\. DPO uses the DOLCI preference dataset; merging denotes a 40/60 interpolation between the unadapted and adapted checkpoints: normalised scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.The recovery mechanism changes with model capacity\. At 14B, merging contributes substantially more after SFT than DPO, and adding DPO before merging leaves the overall score unchanged while slightly reshaping the skill profile\. At 8B, DPO and merging independently reach the same overall score and their combination adds a smaller further gain, indicating complementarity\. DPO is instead the strongest post\-SFT option at 4B and 1\.7B, where merging becomes unstable; at 0\.6B, neither stage improves on SFT\. This transition motivates the size\-specific final recipes reported in the main table\.

## Appendix GDetailed Final Results

This appendix reports every task for the models in Table[3](https://arxiv.org/html/2610.20824#S4.T3), grouped by skill\. The adapted 4B and 8B outperform their respective bases on all 24 public Wolof tasks, and the 14B on 23, showing that the aggregate Wolof gains are not driven by a small subset of benchmarks\.

ModelAvg\. scoreAvg\. rankAfrIFact / Claim ClassificationAfrIFact / Evidence ExtractionAya Wolof / Culture QAAya Wolof / Orthographic CorrectionAya Wolof / POS TaggingAya Wolof / Translation / FROM WolofClaire Wolof / Conversation CoherenceCorporate FAQ Wolof / Conversational MRQACorporate FAQ Wolof / Long Context MRQACorporate FAQ Wolof / MRQAFLORES / Translation / Wolof → EnglishFLORES / Translation / Wolof → FrenchIrokoBench / MGSMIrokoBench / MMLUIrokoBench / XNLIMasakhaNER / NER \(News Domain\)Corporate / Conversation CoherenceCorporate / Dialog Act LabellingCorporate / Full AnalysisCorporate / GDPR/PII ExtractionCorporate / NER \(Public Entities\)Corporate / Resolution ClassificationCorporate / Sentiment ClassificationCorporate / SummarizationWolBanking77 / Intent Classification / WolofWolBanking77 / Translation / Wolof → EnglishWolBanking77 / Translation / Wolof → FrenchWolBanking77 Audio / Intent Classification / ASRWolBanking77 Audio / Intent Classification / ReferenceWolBanking77 Audio / Transcription CorrectionBaseline open\-weight modelsQwen3\-14B\.1013\.4\.24 \(14\)\.29 \(10\)\-\.47 \(15\)\.05 \(16\)\.30 \(13\)\.03 \(17\)\-\.60 \(13\)\.13 \(15\)\.09 \(15\)\.11 \(17\)\.24 \(14\)\.18 \(14\)\.05 \(13\)\.18 \(7\)\.02 \(16\)\.23 \(13\)\.15 \(19\)\.22 \(5\)\.11 \(12\)\.15 \(10\)\.04 \(15\)\.18 \(8\)\.02 \(11\)\-\.33 \(16\)\.03 \(14\)\.12 \(15\)\.14 \(15\)\.22 \(14\)\.26 \(15\)\.76 \(11\)Qwen3\-8B\.0714\.9\.27 \(12\)\.23 \(14\)\-\.56 \(17\)\.03 \(19\)\.29 \(14\)\.04 \(16\)\-\.76 \(16\)\.16 \(12\)\.12 \(12\)\.12 \(15\)\.21 \(15\)\.14 \(16\)\.02 \(17\)\.05 \(16\)\-\.02 \(19\)\.27 \(12\)\.16 \(17\)\.11 \(10\)\.09 \(13\)\.08 \(14\)\.01 \(17\)\.15 \(13\)\.04 \(9\)\-\.49 \(18\)\.03 \(14\)\.11 \(16\)\.14 \(15\)\.19 \(16\)\.26 \(15\)\.68 \(19\)Qwen3\-4B\.0715\.1\.24 \(14\)\.25 \(13\)\-\.11 \(12\)\.05 \(16\)\.24 \(15\)\.07 \(12\)\-1\.01 \(19\)\.14 \(14\)\.10 \(14\)\.12 \(15\)\.20 \(16\)\.15 \(15\)\.04 \(14\)\.06 \(14\)\.04 \(13\)\.14 \(15\)\.19 \(16\)\.19 \(7\)\.02 \(18\)\.01 \(18\)\.01 \(17\)\.18 \(8\)\-\.04 \(17\)\-\.49 \(18\)\.01 \(17\)\.10 \(17\)\.13 \(17\)\.16 \(17\)\.19 \(17\)\.70 \(17\)Qwen3\-1\.7B\.0416\.2\.25 \(13\)\.19 \(15\)\-\.64 \(19\)\.05 \(16\)\.21 \(16\)\.03 \(17\)\-\.88 \(18\)\.09 \(18\)\.03 \(18\)\.09 \(19\)\.17 \(17\)\.11 \(18\)\.01 \(18\)\-\.04 \(19\)\.05 \(12\)\.08 \(17\)\.16 \(17\)\.06 \(12\)\.02 \(18\)\.02 \(17\)\.03 \(16\)\.13 \(15\)\-\.02 \(16\)\-\.46 \(17\)\.02 \(16\)\.08 \(18\)\.10 \(18\)\.15 \(18\)\.17 \(18\)\.90 \(1\)Qwen3\-0\.6B\.0616\.2\.15 \(18\)\.17 \(16\)\-\.61 \(18\)\.47 \(8\)\.13 \(18\)\.03 \(17\)\-\.61 \(14\)\.08 \(19\)\.02 \(19\)\.11 \(17\)\.14 \(18\)\.07 \(19\)\.00 \(19\)\.03 \(18\)\.04 \(13\)\.04 \(19\)\.28 \(14\)\.06 \(12\)\.05 \(15\)\.06 \(15\)\.08 \(12\)\.08 \(18\)\-\.10 \(19\)\-\.24 \(15\)\.01 \(17\)\.04 \(19\)\.06 \(19\)\.10 \(19\)\.15 \(19\)\.85 \(4\)Commercial API modelsClaude\-sonnet\-4\-5\.317\.1\.47 \(8\)\.41 \(3\)\.27 \(5\)\.19 \(11\)\.70 \(3\)\.12 \(10\)\.31 \(3\)\.23 \(5\)\.14 \(11\)\.18 \(7\)\.38 \(6\)\.31 \(6\)\.26 \(4\)\.20 \(6\)\.20 \(8\)\.66 \(3\)\.45 \(2\)\.03 \(18\)\.27 \(4\)\.16 \(8\)\.08 \(12\)\.28 \(4\)\.07 \(4\)\.16 \(8\)\.22 \(7\)\.41 \(9\)\.41 \(7\)\.52 \(9\)\.63 \(10\)\.75 \(12\)Claude\-haiku\-4\-5\.239\.7\.34 \(10\)\.35 \(7\)\-\.06 \(10\)\.17 \(12\)\.50 \(9\)\.07 \(12\)\.05 \(5\)\.22 \(7\)\.06 \(17\)\.16 \(8\)\.28 \(12\)\.22 \(12\)\.12 \(7\)\.04 \(17\)\.07 \(10\)\.52 \(6\)\.43 \(4\)\.08 \(11\)\.23 \(5\)\.09 \(13\)\.15 \(3\)\.21 \(6\)\.04 \(9\)\.02 \(11\)\.08 \(12\)\.30 \(12\)\.30 \(12\)\.43 \(13\)\.52 \(11\)\.82 \(7\)Gemini\-2\.5\-pro\.452\.8\.72 \(1\)\.40 \(4\)\.48 \(2\)\.16 \(14\)\.62 \(5\)\.32 \(2\)\.46 \(1\)\.28 \(1\)\.25 \(1\)\.23 \(1\)\.56 \(1\)\.47 \(1\)\.61 \(1\)\.57 \(1\)\.47 \(1\)\.63 \(4\)\.45 \(2\)\.16 \(9\)\.33 \(2\)\.34 \(2\)\.11 \(6\)\.31 \(3\)\.08 \(3\)\.40 \(1\)\.48 \(1\)\.54 \(1\)\.54 \(1\)\.88 \(1\)\.90 \(1\)\.77 \(9\)Gemini\-2\.5\-flash\.443\.3\.70 \(2\)\.44 \(2\)\.56 \(1\)\.22 \(10\)\.70 \(3\)\.31 \(3\)\.42 \(2\)\.25 \(4\)\.23 \(2\)\.20 \(4\)\.52 \(2\)\.45 \(2\)\.57 \(2\)\.36 \(3\)\.41 \(2\)\.68 \(1\)\.41 \(5\)\.06 \(12\)\.30 \(3\)\.34 \(2\)\.09 \(10\)\.32 \(2\)\.07 \(4\)\.37 \(2\)\.41 \(3\)\.53 \(2\)\.53 \(2\)\.87 \(2\)\.88 \(3\)\.87 \(3\)GPT\-5\.413\.7\.69 \(3\)\.38 \(5\)\.43 \(3\)\.52 \(7\)\.76 \(2\)\.50 \(1\)\-\.76 \(16\)\.26 \(2\)\.20 \(4\)\.20 \(4\)\.49 \(3\)\.42 \(3\)\.55 \(3\)\.52 \(2\)\.25 \(5\)\.68 \(1\)\.46 \(1\)\.36 \(1\)\.35 \(1\)\.36 \(1\)\.11 \(6\)\.35 \(1\)\.16 \(1\)\.32 \(3\)\.45 \(2\)\.50 \(4\)\.49 \(3\)\.87 \(2\)\.86 \(5\)\.73 \(15\)GPT\-5\-nano\.229\.3\.34 \(10\)\.31 \(9\)\-\.05 \(9\)\.17 \(12\)\.50 \(9\)\.14 \(8\)\-\.32 \(11\)\.17 \(10\)\.15 \(10\)\.14 \(11\)\.29 \(11\)\.23 \(10\)\.08 \(11\)\.16 \(8\)\.07 \(10\)\.55 \(5\)\.28 \(14\)\.23 \(4\)\.17 \(9\)\.25 \(5\)\.12 \(4\)\.17 \(10\)\.06 \(7\)\-\.12 \(13\)\.10 \(10\)\.32 \(11\)\.33 \(9\)\.48 \(11\)\.45 \(13\)\.85 \(4\)Mistral\-medium\-3\.1711\.3\.21 \(16\)\.28 \(11\)\-\.47 \(15\)\.11 \(15\)\.34 \(12\)\.06 \(14\)\-\.57 \(12\)\.15 \(13\)\.16 \(7\)\.16 \(8\)\.30 \(10\)\.23 \(10\)\.03 \(16\)\.21 \(5\)\.13 \(9\)\.47 \(7\)\.34 \(8\)\.06 \(12\)\.20 \(6\)\.16 \(8\)\.09 \(10\)\.20 \(7\)\-\.01 \(14\)\-\.12 \(13\)\.06 \(13\)\.27 \(13\)\.27 \(14\)\.44 \(12\)\.51 \(12\)\.70 \(17\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.364\.7\.58 \(4\)\.46 \(1\)\.24 \(6\)\.72 \(1\)\.56 \(7\)\.24 \(4\)\-\.08 \(6\)\.26 \(2\)\.21 \(3\)\.21 \(3\)\.44 \(4\)\.35 \(4\)\.25 \(5\)\.23 \(4\)\.21 \(7\)\.45 \(8\)\.32 \(10\)\.27 \(2\)\.20 \(6\)\.27 \(4\)\.18 \(1\)\.22 \(5\)\.02 \(11\)\.26 \(4\)\.31 \(4\)\.52 \(3\)\.49 \(3\)\.85 \(4\)\.89 \(2\)\.75 \(12\)Wolof\-Qwen3\-8B\.336\.1\.56 \(5\)\.32 \(8\)\.05 \(7\)\.69 \(3\)\.56 \(7\)\.22 \(5\)\-\.13 \(9\)\.23 \(5\)\.20 \(4\)\.22 \(2\)\.38 \(6\)\.29 \(7\)\.18 \(6\)\.15 \(10\)\.27 \(4\)\.45 \(8\)\.32 \(10\)\.25 \(3\)\.18 \(8\)\.25 \(5\)\.12 \(4\)\.15 \(13\)\.02 \(11\)\.24 \(5\)\.23 \(5\)\.48 \(5\)\.48 \(5\)\.78 \(6\)\.84 \(6\)\.89 \(2\)Wolof\-Qwen3\-4B\.307\.0\.50 \(7\)\.38 \(5\)\-\.06 \(10\)\.71 \(2\)\.43 \(11\)\.13 \(9\)\-\.09 \(7\)\.20 \(9\)\.20 \(4\)\.19 \(6\)\.32 \(8\)\.32 \(5\)\.12 \(7\)\.12 \(12\)\.24 \(6\)\.44 \(10\)\.34 \(8\)\.18 \(8\)\.13 \(10\)\.20 \(7\)\.11 \(6\)\.17 \(10\)\.10 \(2\)\.20 \(6\)\.23 \(5\)\.45 \(7\)\.43 \(6\)\.79 \(5\)\.87 \(4\)\.77 \(9\)Wolof\-Qwen3\-1\.7B\.2210\.6\.43 \(9\)\.27 \(12\)\-\.22 \(13\)\.69 \(3\)\.20 \(17\)\.08 \(11\)\-\.09 \(7\)\.17 \(10\)\.16 \(7\)\.14 \(11\)\.32 \(8\)\.29 \(7\)\.06 \(12\)\.06 \(14\)\.03 \(15\)\.09 \(16\)\.35 \(7\)\.04 \(16\)\.05 \(15\)\.13 \(12\)\.16 \(2\)\.10 \(16\)\-\.01 \(14\)\.12 \(9\)\.15 \(9\)\.44 \(8\)\.37 \(8\)\.60 \(7\)\.72 \(7\)\.71 \(16\)Wolof\-Qwen3\-0\.6B\.1214\.1\-\.02 \(19\)\.08 \(19\)\-\.34 \(14\)\.56 \(6\)\.03 \(19\)\.06 \(14\)\-\.22 \(10\)\.13 \(15\)\.08 \(16\)\.13 \(14\)\.13 \(19\)\.13 \(17\)\.04 \(14\)\.08 \(13\)\-\.01 \(17\)\.06 \(18\)\.32 \(10\)\.04 \(16\)\.09 \(13\)\.04 \(16\)\.08 \(12\)\.08 \(18\)\.05 \(8\)\.04 \(10\)\.01 \(17\)\.27 \(13\)\.28 \(13\)\.21 \(15\)\.33 \(14\)\.85 \(4\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.317\.6\.53 \(6\)\.13 \(17\)\.39 \(4\)\.68 \(5\)\.78 \(1\)\.16 \(7\)\.06 \(4\)\.21 \(8\)\.16 \(7\)\.16 \(8\)\.43 \(5\)\.28 \(9\)\.09 \(10\)\.16 \(8\)\.40 \(3\)\.35 \(11\)\.41 \(5\)\.21 \(6\)\.13 \(10\)\.15 \(10\)\.10 \(9\)\.17 \(10\)\.07 \(4\)\.19 \(7\)\.21 \(8\)\.46 \(6\)\.33 \(9\)\.57 \(8\)\.64 \(9\)\.74 \(14\)tiny\-aya\-earth \(3B\)\.1712\.6\.19 \(17\)\.12 \(18\)\-\.01 \(8\)\.30 \(9\)\.60 \(6\)\.19 \(6\)\-\.72 \(15\)\.13 \(15\)\.11 \(13\)\.14 \(11\)\.28 \(12\)\.22 \(12\)\.12 \(7\)\.15 \(10\)\-\.01 \(17\)\.19 \(14\)\.29 \(13\)\-\.01 \(19\)\.03 \(17\)\.00 \(19\)\.00 \(19\)\.10 \(16\)\-\.06 \(18\)\.00 \(12\)\.09 \(11\)\.33 \(10\)\.33 \(9\)\.50 \(10\)\.70 \(8\)\.79 \(8\)

Table 13:Detailed Wolof understanding results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.
ModelAvg\. scoreAvg\. rankAya Wolof / Culture QAAya Wolof / Orthographic CorrectionAya Wolof / Translation / TO WolofClaire Wolof / Conversation CoherenceCorporate FAQ Wolof / Conversational MRQACorporate FAQ Wolof / MRQAFLORES / Translation / English → WolofFLORES / Translation / French → WolofCorporate / Conversation CoherenceCorporate / Full AnalysisCorporate / SummarizationWolBanking77 / Translation / English → WolofWolBanking77 / Translation / French → WolofWolBanking77 Audio / Transcription CorrectionBaseline open\-weight modelsQwen3\-14B\.0014\.9\-\.47 \(15\)\.05 \(16\)\.03 \(12\)\-\.60 \(13\)\.13 \(15\)\.11 \(17\)\.06 \(16\)\.05 \(17\)\.15 \(19\)\.11 \(12\)\-\.33 \(16\)\-\.00 \(15\)\-\.01 \(15\)\.76 \(11\)Qwen3\-8B\-\.0316\.1\-\.56 \(17\)\.03 \(19\)\.02 \(17\)\-\.76 \(16\)\.16 \(12\)\.12 \(15\)\.06 \(16\)\.06 \(15\)\.16 \(17\)\.09 \(13\)\-\.49 \(18\)\-\.01 \(16\)\-\.01 \(15\)\.68 \(19\)Qwen3\-4B\-\.0215\.9\-\.11 \(12\)\.05 \(16\)\.02 \(17\)\-1\.01 \(19\)\.14 \(14\)\.12 \(15\)\.07 \(15\)\.06 \(15\)\.19 \(16\)\.02 \(18\)\-\.49 \(18\)\-\.01 \(16\)\-\.01 \(15\)\.70 \(17\)Qwen3\-1\.7B\-\.0415\.6\-\.64 \(19\)\.05 \(16\)\.07 \(2\)\-\.88 \(18\)\.09 \(18\)\.09 \(19\)\.03 \(18\)\.03 \(18\)\.16 \(17\)\.02 \(18\)\-\.46 \(17\)\-\.04 \(18\)\-\.04 \(19\)\.90 \(1\)Qwen3\-0\.6B\.0315\.1\-\.61 \(18\)\.47 \(8\)\.03 \(12\)\-\.61 \(14\)\.08 \(19\)\.11 \(17\)\.02 \(19\)\.02 \(19\)\.28 \(14\)\.05 \(15\)\-\.24 \(15\)\-\.05 \(19\)\-\.03 \(18\)\.85 \(4\)Commercial API modelsClaude\-sonnet\-4\-5\.277\.5\.27 \(5\)\.19 \(11\)\.03 \(12\)\.31 \(3\)\.23 \(5\)\.18 \(7\)\.24 \(9\)\.23 \(9\)\.45 \(2\)\.27 \(4\)\.16 \(8\)\.27 \(9\)\.26 \(9\)\.75 \(12\)Claude\-haiku\-4\-5\.189\.5\-\.06 \(10\)\.17 \(12\)\.03 \(12\)\.05 \(5\)\.22 \(7\)\.16 \(8\)\.15 \(13\)\.15 \(13\)\.43 \(4\)\.23 \(5\)\.02 \(11\)\.07 \(13\)\.07 \(13\)\.82 \(7\)Gemini\-2\.5\-pro\.354\.8\.48 \(2\)\.16 \(14\)\.02 \(17\)\.46 \(1\)\.28 \(1\)\.23 \(1\)\.30 \(4\)\.28 \(5\)\.45 \(2\)\.33 \(2\)\.40 \(1\)\.34 \(4\)\.34 \(4\)\.77 \(9\)Gemini\-2\.5\-flash\.354\.5\.56 \(1\)\.22 \(10\)\.06 \(3\)\.42 \(2\)\.25 \(4\)\.20 \(4\)\.29 \(6\)\.28 \(5\)\.41 \(5\)\.30 \(3\)\.37 \(2\)\.31 \(7\)\.30 \(8\)\.87 \(3\)GPT\-5\.275\.7\.43 \(3\)\.52 \(7\)\.05 \(4\)\-\.76 \(16\)\.26 \(2\)\.20 \(4\)\.28 \(7\)\.27 \(7\)\.46 \(1\)\.35 \(1\)\.32 \(3\)\.32 \(5\)\.33 \(5\)\.73 \(15\)GPT\-5\-nano\.1410\.4\-\.05 \(9\)\.17 \(12\)\.04 \(6\)\-\.32 \(11\)\.17 \(10\)\.14 \(11\)\.17 \(12\)\.17 \(11\)\.28 \(14\)\.17 \(9\)\-\.12 \(13\)\.15 \(12\)\.15 \(11\)\.85 \(4\)Mistral\-medium\-3\.0612\.0\-\.47 \(15\)\.11 \(15\)\.04 \(6\)\-\.57 \(12\)\.15 \(13\)\.16 \(8\)\.15 \(13\)\.14 \(14\)\.34 \(8\)\.20 \(6\)\-\.12 \(13\)\.04 \(14\)\.05 \(14\)\.70 \(17\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.314\.4\.24 \(6\)\.72 \(1\)\.04 \(6\)\-\.08 \(6\)\.26 \(2\)\.21 \(3\)\.32 \(1\)\.30 \(1\)\.32 \(10\)\.20 \(6\)\.26 \(4\)\.36 \(2\)\.37 \(1\)\.75 \(12\)Wolof\-Qwen3\-8B\.285\.1\.05 \(7\)\.69 \(3\)\.05 \(4\)\-\.13 \(9\)\.23 \(5\)\.22 \(2\)\.30 \(4\)\.29 \(2\)\.32 \(10\)\.18 \(8\)\.24 \(5\)\.32 \(5\)\.33 \(5\)\.89 \(2\)Wolof\-Qwen3\-4B\.275\.8\-\.06 \(10\)\.71 \(2\)\.04 \(6\)\-\.09 \(7\)\.20 \(9\)\.19 \(6\)\.31 \(2\)\.29 \(2\)\.34 \(8\)\.13 \(10\)\.20 \(6\)\.35 \(3\)\.37 \(1\)\.77 \(9\)Wolof\-Qwen3\-1\.7B\.229\.4\-\.22 \(13\)\.69 \(3\)\.03 \(12\)\-\.09 \(7\)\.17 \(10\)\.14 \(11\)\.28 \(7\)\.27 \(7\)\.35 \(7\)\.05 \(15\)\.12 \(9\)\.28 \(8\)\.31 \(7\)\.71 \(16\)Wolof\-Qwen3\-0\.6B\.1710\.1\-\.34 \(14\)\.56 \(6\)\.04 \(6\)\-\.22 \(10\)\.13 \(15\)\.13 \(14\)\.22 \(10\)\.22 \(10\)\.32 \(10\)\.09 \(13\)\.04 \(10\)\.18 \(10\)\.22 \(10\)\.85 \(4\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.315\.1\.39 \(4\)\.68 \(5\)\.08 \(1\)\.06 \(4\)\.21 \(8\)\.16 \(8\)\.31 \(2\)\.29 \(2\)\.41 \(5\)\.13 \(10\)\.19 \(7\)\.37 \(1\)\.37 \(1\)\.74 \(14\)tiny\-aya\-earth \(3B\)\.1211\.3\-\.01 \(8\)\.30 \(9\)\.04 \(6\)\-\.72 \(15\)\.13 \(15\)\.14 \(11\)\.20 \(11\)\.17 \(11\)\.29 \(13\)\.03 \(17\)\.00 \(12\)\.16 \(11\)\.15 \(11\)\.79 \(8\)

Table 14:Detailed Wolof generation results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.ModelAvg\. scoreAvg\. rankAya Wolof / Translation / TO WolofCorporate FAQ French / Conversational MRQACorporate FAQ French / MRQAFLORES / Translation / French → EnglishFLORES / Translation / French → WolofGPQA Diamond / MCQA / FrenchMIFEval / Instruction Following / FrenchMMLU / MCQA / FrenchMulti\-IF / Instruction FollowingCorporate / Conversation CoherenceCorporate / Full AnalysisCorporate / Resolution ClassificationCorporate / Sentiment ClassificationWolBanking77 / Intent Classification / FrenchWolBanking77 / Translation / French → EnglishWolBanking77 / Translation / French → WolofBaseline open\-weight modelsQwen3\-14B\.409\.7\.03 \(12\)\.72 \(4\)\.66 \(6\)\.74 \(6\)\.05 \(17\)\.14 \(9\)\.75 \(7\)\.71 \(7\)\.80 \(6\)\.15 \(19\)\.11 \(12\)\.18 \(8\)\.02 \(11\)\.55 \(7\)\.75 \(9\)\-\.01 \(15\)Qwen3\-8B\.3811\.2\.02 \(17\)\.70 \(8\)\.65 \(9\)\.73 \(8\)\.06 \(15\)\.01 \(15\)\.74 \(9\)\.74 \(4\)\.79 \(7\)\.16 \(17\)\.09 \(13\)\.15 \(13\)\.04 \(9\)\.50 \(9\)\.74 \(11\)\-\.01 \(15\)Qwen3\-4B\.3712\.2\.02 \(17\)\.69 \(10\)\.65 \(9\)\.73 \(8\)\.06 \(15\)\.12 \(10\)\.71 \(10\)\.66 \(11\)\.72 \(10\)\.19 \(16\)\.02 \(18\)\.18 \(8\)\-\.04 \(17\)\.49 \(10\)\.74 \(11\)\-\.01 \(15\)Qwen3\-1\.7B\.3214\.4\.07 \(2\)\.63 \(13\)\.59 \(14\)\.71 \(11\)\.03 \(18\)\-\.01 \(16\)\.63 \(13\)\.58 \(14\)\.61 \(13\)\.16 \(17\)\.02 \(18\)\.13 \(15\)\-\.02 \(16\)\.36 \(15\)\.70 \(16\)\-\.04 \(19\)Qwen3\-0\.6B\.2716\.6\.03 \(12\)\.58 \(16\)\.57 \(15\)\.64 \(17\)\.02 \(19\)\-\.03 \(18\)\.41 \(16\)\.48 \(17\)\.48 \(16\)\.28 \(14\)\.05 \(15\)\.08 \(18\)\-\.10 \(19\)\.19 \(18\)\.66 \(18\)\-\.03 \(18\)Commercial API modelsClaude\-sonnet\-4\-5\.504\.6\.03 \(12\)\.74 \(1\)\.68 \(3\)\.77 \(1\)\.23 \(9\)\.51 \(4\)\.87 \(3\)\.74 \(4\)\.81 \(5\)\.45 \(2\)\.27 \(4\)\.28 \(4\)\.07 \(4\)\.54 \(8\)\.79 \(1\)\.26 \(9\)Claude\-haiku\-4\-5\.467\.0\.03 \(12\)\.72 \(4\)\.67 \(4\)\.73 \(8\)\.15 \(13\)\.42 \(6\)\.81 \(6\)\.79 \(3\)\.76 \(9\)\.43 \(4\)\.23 \(5\)\.21 \(6\)\.04 \(9\)\.57 \(6\)\.77 \(4\)\.07 \(13\)Gemini\-2\.5\-pro\.525\.1\.02 \(17\)\.71 \(6\)\.65 \(9\)\.76 \(2\)\.28 \(5\)\.66 \(1\)\.87 \(3\)\.56 \(15\)\.83 \(4\)\.45 \(2\)\.33 \(2\)\.31 \(3\)\.08 \(3\)\.62 \(2\)\.78 \(3\)\.34 \(4\)Gemini\-2\.5\-flash\.524\.1\.06 \(3\)\.70 \(8\)\.66 \(6\)\.76 \(2\)\.28 \(5\)\.65 \(2\)\.85 \(5\)\.71 \(7\)\.84 \(3\)\.41 \(5\)\.30 \(3\)\.32 \(2\)\.07 \(4\)\.62 \(2\)\.79 \(1\)\.30 \(8\)GPT\-5\.562\.6\.05 \(4\)\.74 \(1\)\.69 \(2\)\.75 \(5\)\.27 \(7\)\.60 \(3\)\.90 \(1\)\.91 \(1\)\.92 \(1\)\.46 \(1\)\.35 \(1\)\.35 \(1\)\.16 \(1\)\.66 \(1\)\.76 \(6\)\.33 \(5\)GPT\-5\-nano\.476\.5\.04 \(6\)\.74 \(1\)\.70 \(1\)\.74 \(6\)\.17 \(11\)\.49 \(5\)\.88 \(2\)\.70 \(9\)\.86 \(2\)\.28 \(14\)\.17 \(9\)\.17 \(10\)\.06 \(7\)\.61 \(4\)\.76 \(6\)\.15 \(11\)Mistral\-medium\-3\.457\.0\.04 \(6\)\.71 \(6\)\.67 \(4\)\.76 \(2\)\.14 \(14\)\.38 \(7\)\.75 \(7\)\.80 \(2\)\.79 \(7\)\.34 \(8\)\.20 \(6\)\.20 \(7\)\-\.01 \(14\)\.61 \(4\)\.77 \(4\)\.05 \(14\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.447\.6\.04 \(6\)\.66 \(12\)\.66 \(6\)\.70 \(12\)\.30 \(1\)\.12 \(10\)\.71 \(10\)\.74 \(4\)\.68 \(12\)\.32 \(10\)\.20 \(6\)\.22 \(5\)\.02 \(11\)\.49 \(10\)\.76 \(6\)\.37 \(1\)Wolof\-Qwen3\-8B\.429\.3\.05 \(4\)\.67 \(11\)\.65 \(9\)\.69 \(13\)\.29 \(2\)\.18 \(8\)\.66 \(12\)\.68 \(10\)\.69 \(11\)\.32 \(10\)\.18 \(8\)\.15 \(13\)\.02 \(11\)\.43 \(13\)\.75 \(9\)\.33 \(5\)Wolof\-Qwen3\-4B\.3910\.3\.04 \(6\)\.60 \(14\)\.60 \(13\)\.59 \(18\)\.29 \(2\)\.05 \(12\)\.54 \(15\)\.63 \(12\)\.59 \(14\)\.34 \(8\)\.13 \(10\)\.17 \(10\)\.10 \(2\)\.40 \(14\)\.73 \(14\)\.37 \(1\)Wolof\-Qwen3\-1\.7B\.3114\.2\.03 \(12\)\.57 \(17\)\.56 \(17\)\.68 \(14\)\.27 \(7\)\-\.03 \(18\)\.24 \(17\)\.52 \(16\)\.32 \(17\)\.35 \(7\)\.05 \(15\)\.10 \(16\)\-\.01 \(14\)\.30 \(17\)\.70 \(16\)\.31 \(7\)Wolof\-Qwen3\-0\.6B\.2415\.1\.04 \(6\)\.48 \(19\)\.48 \(19\)\.49 \(19\)\.22 \(10\)\.03 \(14\)\.11 \(19\)\.37 \(19\)\.18 \(19\)\.32 \(10\)\.09 \(13\)\.08 \(18\)\.05 \(8\)\.05 \(19\)\.62 \(19\)\.22 \(10\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.3510\.4\.08 \(1\)\.60 \(14\)\.49 \(18\)\.68 \(14\)\.29 \(2\)\.05 \(12\)\.13 \(18\)\.62 \(13\)\.29 \(18\)\.41 \(5\)\.13 \(10\)\.17 \(10\)\.07 \(4\)\.44 \(12\)\.73 \(14\)\.37 \(1\)tiny\-aya\-earth \(3B\)\.3314\.3\.04 \(6\)\.57 \(17\)\.57 \(15\)\.66 \(16\)\.17 \(11\)\-\.01 \(16\)\.60 \(14\)\.48 \(17\)\.56 \(15\)\.29 \(13\)\.03 \(17\)\.10 \(16\)\-\.06 \(18\)\.35 \(16\)\.74 \(11\)\.15 \(11\)

Table 15:Detailed French understanding results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.
ModelAvg\. scoreAvg\. rankAya Wolof / Translation / FROM WolofCorporate FAQ French / Conversational MRQACorporate FAQ French / MRQAFLORES / Translation / English → FrenchFLORES / Translation / Wolof → FrenchWolBanking77 / Translation / English → FrenchWolBanking77 / Translation / Wolof → FrenchBaseline open\-weight modelsQwen3\-14B\.459\.9\.03 \(17\)\.72 \(4\)\.66 \(6\)\.69 \(7\)\.18 \(14\)\.76 \(6\)\.14 \(15\)Qwen3\-8B\.4411\.9\.04 \(16\)\.70 \(8\)\.65 \(9\)\.67 \(9\)\.14 \(16\)\.74 \(10\)\.14 \(15\)Qwen3\-4B\.4412\.6\.07 \(12\)\.69 \(10\)\.65 \(9\)\.66 \(13\)\.15 \(15\)\.71 \(12\)\.13 \(17\)Qwen3\-1\.7B\.3916\.0\.03 \(17\)\.63 \(13\)\.59 \(14\)\.60 \(16\)\.11 \(18\)\.64 \(16\)\.10 \(18\)Qwen3\-0\.6B\.3217\.4\.03 \(17\)\.58 \(16\)\.57 \(15\)\.47 \(18\)\.07 \(19\)\.48 \(18\)\.06 \(19\)Commercial API modelsClaude\-sonnet\-4\-5\.554\.1\.12 \(10\)\.74 \(1\)\.68 \(3\)\.74 \(1\)\.31 \(6\)\.82 \(1\)\.41 \(7\)Claude\-haiku\-4\-5\.507\.4\.07 \(12\)\.72 \(4\)\.67 \(4\)\.69 \(7\)\.22 \(12\)\.82 \(1\)\.30 \(12\)Gemini\-2\.5\-pro\.594\.3\.32 \(2\)\.71 \(6\)\.65 \(9\)\.70 \(5\)\.47 \(1\)\.76 \(6\)\.54 \(1\)Gemini\-2\.5\-flash\.603\.9\.31 \(3\)\.70 \(8\)\.66 \(6\)\.71 \(3\)\.45 \(2\)\.80 \(3\)\.53 \(2\)GPT\-5\.614\.0\.50 \(1\)\.74 \(1\)\.69 \(2\)\.70 \(5\)\.42 \(3\)\.70 \(13\)\.49 \(3\)GPT\-5\-nano\.525\.3\.14 \(8\)\.74 \(1\)\.70 \(1\)\.71 \(3\)\.23 \(10\)\.78 \(5\)\.33 \(9\)Mistral\-medium\-3\.507\.6\.06 \(14\)\.71 \(6\)\.67 \(4\)\.72 \(2\)\.23 \(10\)\.80 \(3\)\.27 \(14\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.556\.6\.24 \(4\)\.66 \(12\)\.66 \(6\)\.67 \(9\)\.35 \(4\)\.75 \(8\)\.49 \(3\)Wolof\-Qwen3\-8B\.537\.7\.22 \(5\)\.67 \(11\)\.65 \(9\)\.67 \(9\)\.29 \(7\)\.75 \(8\)\.48 \(5\)Wolof\-Qwen3\-4B\.4810\.7\.13 \(9\)\.60 \(14\)\.60 \(13\)\.62 \(14\)\.32 \(5\)\.67 \(14\)\.43 \(6\)Wolof\-Qwen3\-1\.7B\.4313\.4\.08 \(11\)\.57 \(17\)\.56 \(17\)\.56 \(17\)\.29 \(7\)\.58 \(17\)\.37 \(8\)Wolof\-Qwen3\-0\.6B\.3317\.1\.06 \(14\)\.48 \(19\)\.48 \(19\)\.45 \(19\)\.13 \(17\)\.44 \(19\)\.28 \(13\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.4412\.4\.16 \(7\)\.60 \(14\)\.49 \(18\)\.61 \(15\)\.28 \(9\)\.65 \(15\)\.33 \(9\)tiny\-aya\-earth \(3B\)\.4711\.3\.19 \(6\)\.57 \(17\)\.57 \(15\)\.67 \(9\)\.22 \(12\)\.72 \(11\)\.33 \(9\)

Table 16:Detailed French generation results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.ModelAvg\. scoreAvg\. rankARC\-Challenge / Scientific ReasoningAya Wolof / Translation / TO WolofBBQ / Demographic Bias MCQABFCL / Function CallingDialogSum / Conversation SummarizationFLORES / Translation / English → FrenchFLORES / Translation / English → WolofGPQA Diamond / MCQA/ EnglishGSM8K / Mathematical ReasoningHaluEval / QA Hallucination DetectionIFBench / Instruction FollowingMIFEval / Instruction Following / EnglishMMLU / MCQA / EnglishMulti\-IF / Instruction FollowingPersonaChat / Conversation CoherenceRULER / MRQAToxiGen / Implicit Hate DetectionWebNLG QA / Conversational MRQAWildJailbreak / Harmful Request RefusalWolBanking77 / Intent Classification / EnglishWolBanking77 / Translation / English → FrenchWolBanking77 / Translation / English → WolofBaseline open\-weight modelsQwen3\-14B\.528\.3\.12 \(12\)\.03 \(12\)\.36 \(8\)\.70 \(2\)\.22 \(11\)\.69 \(7\)\.06 \(16\)\.32 \(11\)\.86 \(7\)\.78 \(7\)\.32 \(5\)\.83 \(6\)\.72 \(4\)\.80 \(6\)\.92 \(7\)\.80 \(5\)\.71 \(7\)\.69 \(11\)\.29 \(11\)\.54 \(7\)\.76 \(6\)\-\.00 \(15\)Qwen3\-8B\.4910\.3\.15 \(11\)\.02 \(17\)\.21 \(13\)\.67 \(6\)\.21 \(16\)\.67 \(9\)\.06 \(16\)\.29 \(14\)\.81 \(12\)\.78 \(7\)\.23 \(9\)\.77 \(9\)\.70 \(7\)\.79 \(7\)\.90 \(8\)\.78 \(7\)\.64 \(10\)\.71 \(4\)\.19 \(12\)\.54 \(7\)\.74 \(10\)\-\.01 \(16\)Qwen3\-4B\.4611\.8\.03 \(17\)\.02 \(17\)\.06 \(17\)\.68 \(5\)\.22 \(11\)\.66 \(13\)\.07 \(15\)\.30 \(13\)\.84 \(8\)\.70 \(9\)\.19 \(10\)\.74 \(11\)\.64 \(9\)\.72 \(10\)\.88 \(9\)\.73 \(12\)\.64 \(10\)\.69 \(11\)\.11 \(15\)\.53 \(9\)\.71 \(12\)\-\.01 \(16\)Qwen3\-1\.7B\.4013\.8\.05 \(13\)\.07 \(2\)\.03 \(18\)\.61 \(9\)\.23 \(9\)\.60 \(16\)\.03 \(18\)\.26 \(18\)\.76 \(13\)\.67 \(10\)\.13 \(15\)\.62 \(15\)\.53 \(14\)\.61 \(13\)\.80 \(11\)\.70 \(14\)\.54 \(13\)\.62 \(15\)\.04 \(18\)\.41 \(15\)\.64 \(16\)\-\.04 \(18\)Qwen3\-0\.6B\.3316\.3\.02 \(18\)\.03 \(12\)\.29 \(10\)\.53 \(12\)\.22 \(11\)\.47 \(18\)\.02 \(19\)\.23 \(19\)\.68 \(16\)\.53 \(15\)\.13 \(15\)\.52 \(16\)\.35 \(17\)\.48 \(16\)\.71 \(16\)\.33 \(18\)\.36 \(18\)\.48 \(18\)\.01 \(19\)\.33 \(18\)\.48 \(18\)\-\.05 \(19\)Commercial API modelsClaude\-sonnet\-4\-5\.605\.7\.61 \(3\)\.03 \(12\)\.07 \(16\)\.72 \(1\)\.25 \(7\)\.74 \(1\)\.24 \(9\)\.68 \(4\)\.88 \(5\)\.51 \(16\)\.30 \(6\)\.86 \(5\)\.80 \(3\)\.81 \(5\)\.94 \(3\)\.80 \(5\)\.78 \(2\)\.74 \(1\)\.93 \(1\)\.52 \(10\)\.82 \(1\)\.27 \(9\)Claude\-haiku\-4\-5\.567\.7\.24 \(10\)\.03 \(12\)\.01 \(19\)\.70 \(2\)\.19 \(19\)\.69 \(7\)\.15 \(13\)\.64 \(5\)\.82 \(9\)\.49 \(17\)\.26 \(7\)\.83 \(6\)\.86 \(2\)\.76 \(9\)\.95 \(2\)\.83 \(1\)\.77 \(4\)\.71 \(4\)\.92 \(2\)\.56 \(6\)\.82 \(1\)\.07 \(13\)Gemini\-2\.5\-pro\.596\.4\.04 \(15\)\.02 \(17\)\.71 \(5\)\.65 \(7\)\.24 \(8\)\.70 \(5\)\.30 \(4\)\.79 \(1\)\.87 \(6\)\.92 \(1\)\.38 \(3\)\.92 \(1\)\.47 \(16\)\.83 \(4\)\.93 \(5\)\.73 \(12\)\.73 \(6\)\.73 \(2\)\.37 \(9\)\.62 \(3\)\.76 \(6\)\.34 \(4\)Gemini\-2\.5\-flash\.596\.1\-\.00 \(19\)\.06 \(3\)\.57 \(6\)\.69 \(4\)\.23 \(9\)\.71 \(3\)\.29 \(6\)\.69 \(3\)\.90 \(2\)\.83 \(5\)\.35 \(4\)\.87 \(3\)\.57 \(13\)\.84 \(3\)\.93 \(5\)\.77 \(10\)\.69 \(8\)\.70 \(8\)\.46 \(7\)\.61 \(4\)\.80 \(3\)\.31 \(7\)GPT\-5\.703\.3\.76 \(2\)\.05 \(4\)\.93 \(1\)\.59 \(10\)\.26 \(6\)\.70 \(5\)\.28 \(7\)\.76 \(2\)\.92 \(1\)\.92 \(1\)\.67 \(1\)\.92 \(1\)\.90 \(1\)\.92 \(1\)\.97 \(1\)\.83 \(1\)\.81 \(1\)\.72 \(3\)\.79 \(4\)\.67 \(1\)\.70 \(13\)\.32 \(5\)GPT\-5\-nano\.655\.1\.80 \(1\)\.04 \(6\)\.92 \(2\)\.58 \(11\)\.22 \(11\)\.71 \(3\)\.17 \(12\)\.64 \(5\)\.89 \(4\)\.79 \(6\)\.59 \(2\)\.87 \(3\)\.63 \(10\)\.86 \(2\)\.94 \(3\)\.83 \(1\)\.74 \(5\)\.71 \(4\)\.81 \(3\)\.64 \(2\)\.78 \(5\)\.15 \(12\)Mistral\-medium\-3\.547\.7\.32 \(7\)\.04 \(6\)\.14 \(15\)\.62 \(8\)\.20 \(17\)\.72 \(2\)\.15 \(13\)\.60 \(7\)\.90 \(2\)\.87 \(3\)\.25 \(8\)\.78 \(8\)\.68 \(8\)\.79 \(7\)\.87 \(10\)\.82 \(4\)\.78 \(2\)\.70 \(8\)\.15 \(14\)\.61 \(4\)\.80 \(3\)\.04 \(14\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.567\.2\.35 \(5\)\.04 \(6\)\.88 \(3\)\.35 \(15\)\.33 \(1\)\.67 \(9\)\.32 \(1\)\.44 \(9\)\.82 \(9\)\.87 \(3\)\.16 \(13\)\.77 \(9\)\.72 \(4\)\.68 \(12\)\.76 \(13\)\.78 \(7\)\.69 \(8\)\.71 \(4\)\.46 \(7\)\.49 \(11\)\.75 \(8\)\.36 \(2\)Wolof\-Qwen3\-8B\.548\.6\.27 \(9\)\.05 \(4\)\.88 \(3\)\.46 \(13\)\.31 \(2\)\.67 \(9\)\.30 \(4\)\.47 \(8\)\.82 \(9\)\.66 \(11\)\.19 \(10\)\.74 \(11\)\.71 \(6\)\.69 \(11\)\.74 \(14\)\.75 \(11\)\.51 \(15\)\.70 \(8\)\.50 \(6\)\.46 \(12\)\.75 \(8\)\.32 \(5\)Wolof\-Qwen3\-4B\.4711\.0\.36 \(4\)\.04 \(6\)\.31 \(9\)\.37 \(14\)\.30 \(4\)\.62 \(14\)\.31 \(2\)\.36 \(10\)\.75 \(15\)\.62 \(13\)\.17 \(12\)\.64 \(14\)\.62 \(11\)\.59 \(14\)\.65 \(17\)\.70 \(14\)\.53 \(14\)\.65 \(14\)\.32 \(10\)\.45 \(13\)\.67 \(14\)\.35 \(3\)Wolof\-Qwen3\-1\.7B\.3714\.5\.33 \(6\)\.03 \(12\)\.15 \(14\)\.24 \(16\)\.29 \(5\)\.56 \(17\)\.28 \(7\)\.28 \(15\)\.62 \(18\)\.44 \(19\)\.09 \(17\)\.36 \(17\)\.48 \(15\)\.32 \(17\)\.56 \(18\)\.68 \(16\)\.44 \(17\)\.58 \(17\)\.18 \(13\)\.36 \(17\)\.58 \(17\)\.28 \(8\)Wolof\-Qwen3\-0\.6B\.2615\.3\.28 \(8\)\.04 \(6\)\.38 \(7\)\.23 \(17\)\.31 \(2\)\.45 \(19\)\.22 \(10\)\.27 \(17\)\.37 \(19\)\.46 \(18\)\.02 \(19\)\.23 \(18\)\.27 \(19\)\.18 \(19\)\.48 \(19\)\.43 \(17\)\-\.05 \(19\)\.42 \(19\)\.09 \(17\)\.09 \(19\)\.44 \(19\)\.18 \(10\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.4012\.1\.05 \(13\)\.08 \(1\)\.29 \(10\)\.00 \(18\)\.20 \(17\)\.61 \(15\)\.31 \(2\)\.32 \(11\)\.76 \(13\)\.65 \(12\)\.14 \(14\)\.23 \(18\)\.59 \(12\)\.29 \(18\)\.73 \(15\)\.78 \(7\)\.57 \(12\)\.68 \(13\)\.10 \(16\)\.45 \(13\)\.65 \(15\)\.37 \(1\)tiny\-aya\-earth \(3B\)\.3813\.5\.04 \(15\)\.04 \(6\)\.26 \(12\)\.00 \(18\)\.22 \(11\)\.67 \(9\)\.20 \(11\)\.28 \(15\)\.65 \(17\)\.55 \(14\)\.09 \(17\)\.73 \(13\)\.31 \(18\)\.56 \(15\)\.77 \(12\)\.02 \(19\)\.47 \(16\)\.59 \(16\)\.73 \(5\)\.39 \(16\)\.72 \(11\)\.16 \(11\)

Table 17:Detailed English understanding results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.
ModelAvg\. scoreAvg\. rankAya Wolof / Translation / FROM WolofDialogSum / Conversation SummarizationFLORES / Translation / French → EnglishFLORES / Translation / Wolof → EnglishPersonaChat / Conversation CoherenceWebNLG QA / Conversational MRQAWolBanking77 / Translation / French → EnglishWolBanking77 / Translation / Wolof → EnglishBaseline open\-weight modelsQwen3\-14B\.4611\.2\.03 \(17\)\.22 \(11\)\.74 \(6\)\.24 \(14\)\.92 \(7\)\.69 \(11\)\.75 \(9\)\.12 \(15\)Qwen3\-8B\.4611\.8\.04 \(16\)\.21 \(16\)\.73 \(8\)\.21 \(15\)\.90 \(8\)\.71 \(4\)\.74 \(11\)\.11 \(16\)Qwen3\-4B\.4511\.9\.07 \(12\)\.22 \(11\)\.73 \(8\)\.20 \(16\)\.88 \(9\)\.69 \(11\)\.74 \(11\)\.10 \(17\)Qwen3\-1\.7B\.4114\.2\.03 \(17\)\.23 \(9\)\.71 \(11\)\.17 \(17\)\.80 \(11\)\.62 \(15\)\.70 \(16\)\.08 \(18\)Qwen3\-0\.6B\.3716\.8\.03 \(17\)\.22 \(11\)\.64 \(17\)\.14 \(18\)\.71 \(16\)\.48 \(18\)\.66 \(18\)\.04 \(19\)Commercial API modelsClaude\-sonnet\-4\-5\.554\.8\.12 \(10\)\.25 \(7\)\.77 \(1\)\.38 \(6\)\.94 \(3\)\.74 \(1\)\.79 \(1\)\.41 \(9\)Claude\-haiku\-4\-5\.509\.1\.07 \(12\)\.19 \(19\)\.73 \(8\)\.28 \(12\)\.95 \(2\)\.71 \(4\)\.77 \(4\)\.30 \(12\)Gemini\-2\.5\-pro\.613\.0\.32 \(2\)\.24 \(8\)\.76 \(2\)\.56 \(1\)\.93 \(5\)\.73 \(2\)\.78 \(3\)\.54 \(1\)Gemini\-2\.5\-flash\.604\.0\.31 \(3\)\.23 \(9\)\.76 \(2\)\.52 \(2\)\.93 \(5\)\.70 \(8\)\.79 \(1\)\.53 \(2\)GPT\-5\.623\.6\.50 \(1\)\.26 \(6\)\.75 \(5\)\.49 \(3\)\.97 \(1\)\.72 \(3\)\.76 \(6\)\.50 \(4\)GPT\-5\-nano\.517\.5\.14 \(8\)\.22 \(11\)\.74 \(6\)\.29 \(11\)\.94 \(3\)\.71 \(4\)\.76 \(6\)\.32 \(11\)Mistral\-medium\-3\.499\.8\.06 \(14\)\.20 \(17\)\.76 \(2\)\.30 \(10\)\.87 \(10\)\.70 \(8\)\.77 \(4\)\.27 \(13\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.565\.9\.24 \(4\)\.33 \(1\)\.70 \(12\)\.44 \(4\)\.76 \(13\)\.71 \(4\)\.76 \(6\)\.52 \(3\)Wolof\-Qwen3\-8B\.537\.8\.22 \(5\)\.31 \(2\)\.69 \(13\)\.38 \(6\)\.74 \(14\)\.70 \(8\)\.75 \(9\)\.48 \(5\)Wolof\-Qwen3\-4B\.4811\.4\.13 \(9\)\.30 \(4\)\.59 \(18\)\.32 \(8\)\.65 \(17\)\.65 \(14\)\.73 \(14\)\.45 \(7\)Wolof\-Qwen3\-1\.7B\.4612\.1\.08 \(11\)\.29 \(5\)\.68 \(14\)\.32 \(8\)\.56 \(18\)\.58 \(17\)\.70 \(16\)\.44 \(8\)Wolof\-Qwen3\-0\.6B\.3515\.5\.06 \(14\)\.31 \(2\)\.49 \(19\)\.13 \(19\)\.48 \(19\)\.42 \(19\)\.62 \(19\)\.27 \(13\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.5111\.4\.16 \(7\)\.20 \(17\)\.68 \(14\)\.43 \(5\)\.73 \(15\)\.68 \(13\)\.73 \(14\)\.46 \(6\)tiny\-aya\-earth \(3B\)\.4711\.8\.19 \(6\)\.22 \(11\)\.66 \(16\)\.28 \(12\)\.77 \(12\)\.59 \(16\)\.74 \(11\)\.33 \(10\)

Table 18:Detailed English generation results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.ModelAvg\. scoreAvg\. rankARC\-Challenge / Scientific ReasoningAya Wolof / Culture QAGPQA Diamond / MCQA / FrenchGPQA Diamond / MCQA/ EnglishIrokoBench / MMLUMMLU / MCQA / EnglishMMLU / MCQA / FrenchBaseline open\-weight modelsQwen3\-14B\.259\.3\.12 \(12\)\-\.47 \(15\)\.14 \(9\)\.32 \(11\)\.18 \(7\)\.72 \(4\)\.71 \(7\)Qwen3\-8B\.2012\.0\.15 \(11\)\-\.56 \(17\)\.01 \(15\)\.29 \(14\)\.05 \(16\)\.70 \(7\)\.74 \(4\)Qwen3\-4B\.2412\.3\.03 \(17\)\-\.11 \(12\)\.12 \(10\)\.30 \(13\)\.06 \(14\)\.64 \(9\)\.66 \(11\)Qwen3\-1\.7B\.1016\.1\.05 \(13\)\-\.64 \(19\)\-\.01 \(16\)\.26 \(18\)\-\.04 \(19\)\.53 \(14\)\.58 \(14\)Qwen3\-0\.6B\.0717\.9\.02 \(18\)\-\.61 \(18\)\-\.03 \(18\)\.23 \(19\)\.03 \(18\)\.35 \(17\)\.48 \(17\)Commercial API modelsClaude\-sonnet\-4\-5\.544\.1\.61 \(3\)\.27 \(5\)\.51 \(4\)\.68 \(4\)\.20 \(6\)\.80 \(3\)\.74 \(4\)Claude\-haiku\-4\-5\.427\.6\.24 \(10\)\-\.06 \(10\)\.42 \(6\)\.64 \(5\)\.04 \(17\)\.86 \(2\)\.79 \(3\)Gemini\-2\.5\-pro\.517\.3\.04 \(15\)\.48 \(2\)\.66 \(1\)\.79 \(1\)\.57 \(1\)\.47 \(16\)\.56 \(15\)Gemini\-2\.5\-flash\.516\.9\-\.00 \(19\)\.56 \(1\)\.65 \(2\)\.69 \(3\)\.36 \(3\)\.57 \(13\)\.71 \(7\)GPT\-5\.702\.0\.76 \(2\)\.43 \(3\)\.60 \(3\)\.76 \(2\)\.52 \(2\)\.90 \(1\)\.91 \(1\)GPT\-5\-nano\.486\.7\.80 \(1\)\-\.05 \(9\)\.49 \(5\)\.64 \(5\)\.16 \(8\)\.63 \(10\)\.70 \(9\)Mistral\-medium\-3\.367\.3\.32 \(7\)\-\.47 \(15\)\.38 \(7\)\.60 \(7\)\.21 \(5\)\.68 \(8\)\.80 \(2\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.416\.0\.35 \(5\)\.24 \(6\)\.12 \(10\)\.44 \(9\)\.23 \(4\)\.72 \(4\)\.74 \(4\)Wolof\-Qwen3\-8B\.368\.3\.27 \(9\)\.05 \(7\)\.18 \(8\)\.47 \(8\)\.15 \(10\)\.71 \(6\)\.68 \(10\)Wolof\-Qwen3\-4B\.3010\.1\.36 \(4\)\-\.06 \(10\)\.05 \(12\)\.36 \(10\)\.12 \(12\)\.62 \(11\)\.63 \(12\)Wolof\-Qwen3\-1\.7B\.2013\.9\.33 \(6\)\-\.22 \(13\)\-\.03 \(18\)\.28 \(15\)\.06 \(14\)\.48 \(15\)\.52 \(16\)Wolof\-Qwen3\-0\.6B\.1414\.9\.28 \(8\)\-\.34 \(14\)\.03 \(14\)\.27 \(17\)\.08 \(13\)\.27 \(19\)\.37 \(19\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.3110\.4\.05 \(13\)\.39 \(4\)\.05 \(12\)\.32 \(11\)\.16 \(8\)\.59 \(12\)\.62 \(13\)tiny\-aya\-earth \(3B\)\.1814\.1\.04 \(15\)\-\.01 \(8\)\-\.01 \(16\)\.28 \(15\)\.15 \(10\)\.31 \(18\)\.48 \(17\)

Table 19:Detailed knowledge results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.ModelAvg\. scoreAvg\. rankClaire Wolof / Conversation CoherenceCorporate FAQ French / Conversational MRQACorporate FAQ Wolof / Conversational MRQAPersonaChat / Conversation CoherenceCorporate / Conversation CoherenceWebNLG QA / Conversational MRQABaseline open\-weight modelsQwen3\-14B\.3311\.5\-\.60 \(13\)\.72 \(4\)\.13 \(15\)\.92 \(7\)\.15 \(19\)\.69 \(11\)Qwen3\-8B\.3110\.8\-\.76 \(16\)\.70 \(8\)\.16 \(12\)\.90 \(8\)\.16 \(17\)\.71 \(4\)Qwen3\-4B\.2613\.2\-1\.01 \(19\)\.69 \(10\)\.14 \(14\)\.88 \(9\)\.19 \(16\)\.69 \(11\)Qwen3\-1\.7B\.2315\.3\-\.88 \(18\)\.63 \(13\)\.09 \(18\)\.80 \(11\)\.16 \(17\)\.62 \(15\)Qwen3\-0\.6B\.2516\.2\-\.61 \(14\)\.58 \(16\)\.08 \(19\)\.71 \(16\)\.28 \(14\)\.48 \(18\)Commercial API modelsClaude\-sonnet\-4\-5\.572\.5\.31 \(3\)\.74 \(1\)\.23 \(5\)\.94 \(3\)\.45 \(2\)\.74 \(1\)Claude\-haiku\-4\-5\.514\.3\.05 \(5\)\.72 \(4\)\.22 \(7\)\.95 \(2\)\.43 \(4\)\.71 \(4\)Gemini\-2\.5\-pro\.592\.8\.46 \(1\)\.71 \(6\)\.28 \(1\)\.93 \(5\)\.45 \(2\)\.73 \(2\)Gemini\-2\.5\-flash\.575\.3\.42 \(2\)\.70 \(8\)\.25 \(4\)\.93 \(5\)\.41 \(5\)\.70 \(8\)GPT\-5\.404\.0\-\.76 \(16\)\.74 \(1\)\.26 \(2\)\.97 \(1\)\.46 \(1\)\.72 \(3\)GPT\-5\-nano\.427\.2\-\.32 \(11\)\.74 \(1\)\.17 \(10\)\.94 \(3\)\.28 \(14\)\.71 \(4\)Mistral\-medium\-3\.379\.5\-\.57 \(12\)\.71 \(6\)\.15 \(13\)\.87 \(10\)\.34 \(8\)\.70 \(8\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.447\.8\-\.08 \(6\)\.66 \(12\)\.26 \(2\)\.76 \(13\)\.32 \(10\)\.71 \(4\)Wolof\-Qwen3\-8B\.429\.5\-\.13 \(9\)\.67 \(11\)\.23 \(5\)\.74 \(14\)\.32 \(10\)\.70 \(8\)Wolof\-Qwen3\-4B\.3911\.5\-\.09 \(7\)\.60 \(14\)\.20 \(9\)\.65 \(17\)\.34 \(8\)\.65 \(14\)Wolof\-Qwen3\-1\.7B\.3612\.7\-\.09 \(7\)\.57 \(17\)\.17 \(10\)\.56 \(18\)\.35 \(7\)\.58 \(17\)Wolof\-Qwen3\-0\.6B\.2715\.3\-\.22 \(10\)\.48 \(19\)\.13 \(15\)\.48 \(19\)\.32 \(10\)\.42 \(19\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.459\.8\.06 \(4\)\.60 \(14\)\.21 \(8\)\.73 \(15\)\.41 \(5\)\.68 \(13\)tiny\-aya\-earth \(3B\)\.2714\.7\-\.72 \(15\)\.57 \(17\)\.13 \(15\)\.77 \(12\)\.29 \(13\)\.59 \(16\)

Table 20:Detailed chat results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.ModelAvg\. scoreAvg\. rankARC\-Challenge / Scientific ReasoningGPQA Diamond / MCQA / FrenchGPQA Diamond / MCQA/ EnglishGSM8K / Mathematical ReasoningHumanEval Code GenerationIrokoBench / MGSMBaseline open\-weight modelsQwen3\-14B\.3710\.3\.12 \(12\)\.14 \(9\)\.32 \(11\)\.86 \(7\)\.74 \(10\)\.05 \(13\)Qwen3\-8B\.3013\.8\.15 \(11\)\.01 \(15\)\.29 \(14\)\.81 \(12\)\.54 \(14\)\.02 \(17\)Qwen3\-4B\.3312\.5\.03 \(17\)\.12 \(10\)\.30 \(13\)\.84 \(8\)\.62 \(13\)\.04 \(14\)Qwen3\-1\.7B\.2615\.7\.05 \(13\)\-\.01 \(16\)\.26 \(18\)\.76 \(13\)\.53 \(16\)\.01 \(18\)Qwen3\-0\.6B\.2118\.0\.02 \(18\)\-\.03 \(18\)\.23 \(19\)\.68 \(16\)\.37 \(18\)\.00 \(19\)Commercial API modelsClaude\-sonnet\-4\-5\.653\.7\.61 \(3\)\.51 \(4\)\.68 \(4\)\.88 \(5\)\.98 \(2\)\.26 \(4\)Claude\-haiku\-4\-5\.537\.2\.24 \(10\)\.42 \(6\)\.64 \(5\)\.82 \(9\)\.93 \(6\)\.12 \(7\)Gemini\-2\.5\-pro\.654\.8\.04 \(15\)\.66 \(1\)\.79 \(1\)\.87 \(6\)\.94 \(5\)\.61 \(1\)Gemini\-2\.5\-flash\.635\.3\-\.00 \(19\)\.65 \(2\)\.69 \(3\)\.90 \(2\)\.96 \(4\)\.57 \(2\)GPT\-5\.762\.0\.76 \(2\)\.60 \(3\)\.76 \(2\)\.92 \(1\)\.99 \(1\)\.55 \(3\)GPT\-5\-nano\.654\.7\.80 \(1\)\.49 \(5\)\.64 \(5\)\.89 \(4\)\.98 \(2\)\.08 \(11\)Mistral\-medium\-3\.527\.7\.32 \(7\)\.38 \(7\)\.60 \(7\)\.90 \(2\)\.88 \(7\)\.03 \(16\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.467\.7\.35 \(5\)\.12 \(10\)\.44 \(9\)\.82 \(9\)\.79 \(8\)\.25 \(5\)Wolof\-Qwen3\-8B\.448\.5\.27 \(9\)\.18 \(8\)\.47 \(8\)\.82 \(9\)\.69 \(11\)\.18 \(6\)Wolof\-Qwen3\-4B\.3810\.0\.36 \(4\)\.05 \(12\)\.36 \(10\)\.75 \(15\)\.64 \(12\)\.12 \(7\)Wolof\-Qwen3\-1\.7B\.2914\.3\.33 \(6\)\-\.03 \(18\)\.28 \(15\)\.62 \(18\)\.48 \(17\)\.06 \(12\)Wolof\-Qwen3\-0\.6B\.2015\.2\.28 \(8\)\.03 \(14\)\.27 \(17\)\.37 \(19\)\.20 \(19\)\.04 \(14\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.3411\.3\.05 \(13\)\.05 \(12\)\.32 \(11\)\.76 \(13\)\.77 \(9\)\.09 \(10\)tiny\-aya\-earth \(3B\)\.2714\.0\.04 \(15\)\-\.01 \(16\)\.28 \(15\)\.65 \(17\)\.54 \(14\)\.12 \(7\)

Table 21:Detailed reasoning results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.ModelAvg\. scoreAvg\. rankHumanEval Code GenerationBaseline open\-weight modelsQwen3\-14B\.7410\.0\.74 \(10\)Qwen3\-8B\.5414\.0\.54 \(14\)Qwen3\-4B\.6213\.0\.62 \(13\)Qwen3\-1\.7B\.5316\.0\.53 \(16\)Qwen3\-0\.6B\.3718\.0\.37 \(18\)Commercial API modelsClaude\-sonnet\-4\-5\.982\.0\.98 \(2\)Claude\-haiku\-4\-5\.936\.0\.93 \(6\)Gemini\-2\.5\-pro\.945\.0\.94 \(5\)Gemini\-2\.5\-flash\.964\.0\.96 \(4\)GPT\-5\.991\.0\.99 \(1\)GPT\-5\-nano\.982\.0\.98 \(2\)Mistral\-medium\-3\.887\.0\.88 \(7\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.798\.0\.79 \(8\)Wolof\-Qwen3\-8B\.6911\.0\.69 \(11\)Wolof\-Qwen3\-4B\.6412\.0\.64 \(12\)Wolof\-Qwen3\-1\.7B\.4817\.0\.48 \(17\)Wolof\-Qwen3\-0\.6B\.2019\.0\.20 \(19\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.779\.0\.77 \(9\)tiny\-aya\-earth \(3B\)\.5414\.0\.54 \(14\)

Table 22:Detailed code results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.ModelAvg\. scoreAvg\. rankAfrIFact / Claim ClassificationAya Wolof / POS TaggingBFCL / Function CallingGPQA Diamond / MCQA / FrenchGPQA Diamond / MCQA/ EnglishHumanEval Code GenerationIrokoBench / MMLUIrokoBench / XNLIMasakhaNER / NER \(News Domain\)MMLU / MCQA / EnglishMMLU / MCQA / FrenchCorporate / Dialog Act LabellingCorporate / Full AnalysisCorporate / GDPR/PII ExtractionCorporate / NER \(Public Entities\)Corporate / Resolution ClassificationCorporate / Sentiment ClassificationWolBanking77 / Intent Classification / EnglishWolBanking77 / Intent Classification / FrenchWolBanking77 / Intent Classification / WolofWolBanking77 Audio / Intent Classification / ASRWolBanking77 Audio / Intent Classification / ReferenceBaseline open\-weight modelsQwen3\-14B\.3010\.2\.24 \(14\)\.30 \(13\)\.70 \(2\)\.14 \(9\)\.32 \(11\)\.74 \(10\)\.18 \(7\)\.02 \(16\)\.23 \(13\)\.72 \(4\)\.71 \(7\)\.22 \(5\)\.11 \(12\)\.15 \(10\)\.04 \(15\)\.18 \(8\)\.02 \(11\)\.54 \(7\)\.55 \(7\)\.03 \(14\)\.22 \(14\)\.26 \(15\)Qwen3\-8B\.2612\.3\.27 \(12\)\.29 \(14\)\.67 \(6\)\.01 \(15\)\.29 \(14\)\.54 \(14\)\.05 \(16\)\-\.02 \(19\)\.27 \(12\)\.70 \(7\)\.74 \(4\)\.11 \(10\)\.09 \(13\)\.08 \(14\)\.01 \(17\)\.15 \(13\)\.04 \(9\)\.54 \(7\)\.50 \(9\)\.03 \(14\)\.19 \(16\)\.26 \(15\)Qwen3\-4B\.2513\.0\.24 \(14\)\.24 \(15\)\.68 \(5\)\.12 \(10\)\.30 \(13\)\.62 \(13\)\.06 \(14\)\.04 \(13\)\.14 \(15\)\.64 \(9\)\.66 \(11\)\.19 \(7\)\.02 \(18\)\.01 \(18\)\.01 \(17\)\.18 \(8\)\-\.04 \(17\)\.53 \(9\)\.49 \(10\)\.01 \(17\)\.16 \(17\)\.19 \(17\)Qwen3\-1\.7B\.2015\.5\.25 \(13\)\.21 \(16\)\.61 \(9\)\-\.01 \(16\)\.26 \(18\)\.53 \(16\)\-\.04 \(19\)\.05 \(12\)\.08 \(17\)\.53 \(14\)\.58 \(14\)\.06 \(12\)\.02 \(18\)\.02 \(17\)\.03 \(16\)\.13 \(15\)\-\.02 \(16\)\.41 \(15\)\.36 \(15\)\.02 \(16\)\.15 \(18\)\.17 \(18\)Qwen3\-0\.6B\.1516\.8\.15 \(18\)\.13 \(18\)\.53 \(12\)\-\.03 \(18\)\.23 \(19\)\.37 \(18\)\.03 \(18\)\.04 \(13\)\.04 \(19\)\.35 \(17\)\.48 \(17\)\.06 \(12\)\.05 \(15\)\.06 \(15\)\.08 \(12\)\.08 \(18\)\-\.10 \(19\)\.33 \(18\)\.19 \(18\)\.01 \(17\)\.10 \(19\)\.15 \(19\)Commercial API modelsClaude\-sonnet\-4\-5\.456\.4\.47 \(8\)\.70 \(3\)\.72 \(1\)\.51 \(4\)\.68 \(4\)\.98 \(2\)\.20 \(6\)\.20 \(8\)\.66 \(3\)\.80 \(3\)\.74 \(4\)\.03 \(18\)\.27 \(4\)\.16 \(8\)\.08 \(12\)\.28 \(4\)\.07 \(4\)\.52 \(10\)\.54 \(8\)\.22 \(7\)\.52 \(9\)\.63 \(10\)Claude\-haiku\-4\-5\.407\.8\.34 \(10\)\.50 \(9\)\.70 \(2\)\.42 \(6\)\.64 \(5\)\.93 \(6\)\.04 \(17\)\.07 \(10\)\.52 \(6\)\.86 \(2\)\.79 \(3\)\.08 \(11\)\.23 \(5\)\.09 \(13\)\.15 \(3\)\.21 \(6\)\.04 \(9\)\.56 \(6\)\.57 \(6\)\.08 \(12\)\.43 \(13\)\.52 \(11\)Gemini\-2\.5\-pro\.544\.1\.72 \(1\)\.62 \(5\)\.65 \(7\)\.66 \(1\)\.79 \(1\)\.94 \(5\)\.57 \(1\)\.47 \(1\)\.63 \(4\)\.47 \(16\)\.56 \(15\)\.16 \(9\)\.33 \(2\)\.34 \(2\)\.11 \(6\)\.31 \(3\)\.08 \(3\)\.62 \(3\)\.62 \(2\)\.48 \(1\)\.88 \(1\)\.90 \(1\)Gemini\-2\.5\-flash\.534\.1\.70 \(2\)\.70 \(3\)\.69 \(4\)\.65 \(2\)\.69 \(3\)\.96 \(4\)\.36 \(3\)\.41 \(2\)\.68 \(1\)\.57 \(13\)\.71 \(7\)\.06 \(12\)\.30 \(3\)\.34 \(2\)\.09 \(10\)\.32 \(2\)\.07 \(4\)\.61 \(4\)\.62 \(2\)\.41 \(3\)\.87 \(2\)\.88 \(3\)GPT\-5\.582\.4\.69 \(3\)\.76 \(2\)\.59 \(10\)\.60 \(3\)\.76 \(2\)\.99 \(1\)\.52 \(2\)\.25 \(5\)\.68 \(1\)\.90 \(1\)\.91 \(1\)\.36 \(1\)\.35 \(1\)\.36 \(1\)\.11 \(6\)\.35 \(1\)\.16 \(1\)\.67 \(1\)\.66 \(1\)\.45 \(2\)\.87 \(2\)\.86 \(5\)GPT\-5\-nano\.407\.4\.34 \(10\)\.50 \(9\)\.58 \(11\)\.49 \(5\)\.64 \(5\)\.98 \(2\)\.16 \(8\)\.07 \(10\)\.55 \(5\)\.63 \(10\)\.70 \(9\)\.23 \(4\)\.17 \(9\)\.25 \(5\)\.12 \(4\)\.17 \(10\)\.06 \(7\)\.64 \(2\)\.61 \(4\)\.10 \(10\)\.48 \(11\)\.45 \(13\)Mistral\-medium\-3\.378\.6\.21 \(16\)\.34 \(12\)\.62 \(8\)\.38 \(7\)\.60 \(7\)\.88 \(7\)\.21 \(5\)\.13 \(9\)\.47 \(7\)\.68 \(8\)\.80 \(2\)\.06 \(12\)\.20 \(6\)\.16 \(8\)\.09 \(10\)\.20 \(7\)\-\.01 \(14\)\.61 \(4\)\.61 \(4\)\.06 \(13\)\.44 \(12\)\.51 \(12\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.436\.4\.58 \(4\)\.56 \(7\)\.35 \(15\)\.12 \(10\)\.44 \(9\)\.79 \(8\)\.23 \(4\)\.21 \(7\)\.45 \(8\)\.72 \(4\)\.74 \(4\)\.27 \(2\)\.20 \(6\)\.27 \(4\)\.18 \(1\)\.22 \(5\)\.02 \(11\)\.49 \(11\)\.49 \(10\)\.31 \(4\)\.85 \(4\)\.89 \(2\)Wolof\-Qwen3\-8B\.408\.0\.56 \(5\)\.56 \(7\)\.46 \(13\)\.18 \(8\)\.47 \(8\)\.69 \(11\)\.15 \(10\)\.27 \(4\)\.45 \(8\)\.71 \(6\)\.68 \(10\)\.25 \(3\)\.18 \(8\)\.25 \(5\)\.12 \(4\)\.15 \(13\)\.02 \(11\)\.46 \(12\)\.43 \(13\)\.23 \(5\)\.78 \(6\)\.84 \(6\)Wolof\-Qwen3\-4B\.369\.1\.50 \(7\)\.43 \(11\)\.37 \(14\)\.05 \(12\)\.36 \(10\)\.64 \(12\)\.12 \(12\)\.24 \(6\)\.44 \(10\)\.62 \(11\)\.63 \(12\)\.18 \(8\)\.13 \(10\)\.20 \(7\)\.11 \(6\)\.17 \(10\)\.10 \(2\)\.45 \(13\)\.40 \(14\)\.23 \(5\)\.79 \(5\)\.87 \(4\)Wolof\-Qwen3\-1\.7B\.2413\.6\.43 \(9\)\.20 \(17\)\.24 \(16\)\-\.03 \(18\)\.28 \(15\)\.48 \(17\)\.06 \(14\)\.03 \(15\)\.09 \(16\)\.48 \(15\)\.52 \(16\)\.04 \(16\)\.05 \(15\)\.13 \(12\)\.16 \(2\)\.10 \(16\)\-\.01 \(14\)\.36 \(17\)\.30 \(17\)\.15 \(9\)\.60 \(7\)\.72 \(7\)Wolof\-Qwen3\-0\.6B\.1216\.3\-\.02 \(19\)\.03 \(19\)\.23 \(17\)\.03 \(14\)\.27 \(17\)\.20 \(19\)\.08 \(13\)\-\.01 \(17\)\.06 \(18\)\.27 \(19\)\.37 \(19\)\.04 \(16\)\.09 \(13\)\.04 \(16\)\.08 \(12\)\.08 \(18\)\.05 \(8\)\.09 \(19\)\.05 \(19\)\.01 \(17\)\.21 \(15\)\.33 \(14\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.359\.2\.53 \(6\)\.78 \(1\)\.00 \(18\)\.05 \(12\)\.32 \(11\)\.77 \(9\)\.16 \(8\)\.40 \(3\)\.35 \(11\)\.59 \(12\)\.62 \(13\)\.21 \(6\)\.13 \(10\)\.15 \(10\)\.10 \(9\)\.17 \(10\)\.07 \(4\)\.45 \(13\)\.44 \(12\)\.21 \(8\)\.57 \(8\)\.64 \(9\)tiny\-aya\-earth \(3B\)\.2215\.0\.19 \(17\)\.60 \(6\)\.00 \(18\)\-\.01 \(16\)\.28 \(15\)\.54 \(14\)\.15 \(10\)\-\.01 \(17\)\.19 \(14\)\.31 \(18\)\.48 \(17\)\-\.01 \(19\)\.03 \(17\)\.00 \(19\)\.00 \(19\)\.10 \(16\)\-\.06 \(18\)\.39 \(16\)\.35 \(16\)\.09 \(11\)\.50 \(10\)\.70 \(8\)

Table 23:Detailed formatting results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.
ModelAvg\. scoreAvg\. rankIFBench / Instruction FollowingMIFEval / Instruction Following / EnglishMIFEval / Instruction Following / FrenchMulti\-IF / Instruction FollowingBaseline open\-weight modelsQwen3\-14B\.676\.0\.32 \(5\)\.83 \(6\)\.75 \(7\)\.80 \(6\)Qwen3\-8B\.638\.5\.23 \(9\)\.77 \(9\)\.74 \(9\)\.79 \(7\)Qwen3\-4B\.5910\.2\.19 \(10\)\.74 \(11\)\.71 \(10\)\.72 \(10\)Qwen3\-1\.7B\.5014\.0\.13 \(15\)\.62 \(15\)\.63 \(13\)\.61 \(13\)Qwen3\-0\.6B\.3815\.8\.13 \(15\)\.52 \(16\)\.41 \(16\)\.48 \(16\)Commercial API modelsClaude\-sonnet\-4\-5\.714\.8\.30 \(6\)\.86 \(5\)\.87 \(3\)\.81 \(5\)Claude\-haiku\-4\-5\.677\.0\.26 \(7\)\.83 \(6\)\.81 \(6\)\.76 \(9\)Gemini\-2\.5\-pro\.752\.8\.38 \(3\)\.92 \(1\)\.87 \(3\)\.83 \(4\)Gemini\-2\.5\-flash\.733\.8\.35 \(4\)\.87 \(3\)\.85 \(5\)\.84 \(3\)GPT\-5\.851\.0\.67 \(1\)\.92 \(1\)\.90 \(1\)\.92 \(1\)GPT\-5\-nano\.802\.2\.59 \(2\)\.87 \(3\)\.88 \(2\)\.86 \(2\)Mistral\-medium\-3\.647\.5\.25 \(8\)\.78 \(8\)\.75 \(7\)\.79 \(7\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.5811\.0\.16 \(13\)\.77 \(9\)\.71 \(10\)\.68 \(12\)Wolof\-Qwen3\-8B\.5711\.0\.19 \(10\)\.74 \(11\)\.66 \(12\)\.69 \(11\)Wolof\-Qwen3\-4B\.4913\.8\.17 \(12\)\.64 \(14\)\.54 \(15\)\.59 \(14\)Wolof\-Qwen3\-1\.7B\.2517\.0\.09 \(17\)\.36 \(17\)\.24 \(17\)\.32 \(17\)Wolof\-Qwen3\-0\.6B\.1318\.8\.02 \(19\)\.23 \(18\)\.11 \(19\)\.18 \(19\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.2017\.0\.14 \(14\)\.23 \(18\)\.13 \(18\)\.29 \(18\)tiny\-aya\-earth \(3B\)\.4914\.8\.09 \(17\)\.73 \(13\)\.60 \(14\)\.56 \(15\)

Table 24:Detailed instruction\-following results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.ModelAvg\. scoreAvg\. rankBFCL / Function CallingBaseline open\-weight modelsQwen3\-14B\.702\.0\.70 \(2\)Qwen3\-8B\.676\.0\.67 \(6\)Qwen3\-4B\.685\.0\.68 \(5\)Qwen3\-1\.7B\.619\.0\.61 \(9\)Qwen3\-0\.6B\.5312\.0\.53 \(12\)Commercial API modelsClaude\-sonnet\-4\-5\.721\.0\.72 \(1\)Claude\-haiku\-4\-5\.702\.0\.70 \(2\)Gemini\-2\.5\-pro\.657\.0\.65 \(7\)Gemini\-2\.5\-flash\.694\.0\.69 \(4\)GPT\-5\.5910\.0\.59 \(10\)GPT\-5\-nano\.5811\.0\.58 \(11\)Mistral\-medium\-3\.628\.0\.62 \(8\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.3515\.0\.35 \(15\)Wolof\-Qwen3\-8B\.4613\.0\.46 \(13\)Wolof\-Qwen3\-4B\.3714\.0\.37 \(14\)Wolof\-Qwen3\-1\.7B\.2416\.0\.24 \(16\)Wolof\-Qwen3\-0\.6B\.2317\.0\.23 \(17\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.0018\.0\.00 \(18\)tiny\-aya\-earth \(3B\)\.0018\.0\.00 \(18\)

Table 25:Detailed function\-calling results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.ModelAvg\. scoreAvg\. rankCorporate FAQ Wolof / Long Context MRQARULER / MRQABaseline open\-weight modelsQwen3\-14B\.4510\.0\.09 \(15\)\.80 \(5\)Qwen3\-8B\.459\.5\.12 \(12\)\.78 \(7\)Qwen3\-4B\.4213\.0\.10 \(14\)\.73 \(12\)Qwen3\-1\.7B\.3716\.0\.03 \(18\)\.70 \(14\)Qwen3\-0\.6B\.1818\.5\.02 \(19\)\.33 \(18\)Commercial API modelsClaude\-sonnet\-4\-5\.478\.0\.14 \(11\)\.80 \(5\)Claude\-haiku\-4\-5\.459\.0\.06 \(17\)\.83 \(1\)Gemini\-2\.5\-pro\.496\.5\.25 \(1\)\.73 \(12\)Gemini\-2\.5\-flash\.506\.0\.23 \(2\)\.77 \(10\)GPT\-5\.522\.5\.20 \(4\)\.83 \(1\)GPT\-5\-nano\.495\.5\.15 \(10\)\.83 \(1\)Mistral\-medium\-3\.495\.5\.16 \(7\)\.82 \(4\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.505\.0\.21 \(3\)\.78 \(7\)Wolof\-Qwen3\-8B\.477\.5\.20 \(4\)\.75 \(11\)Wolof\-Qwen3\-4B\.459\.0\.20 \(4\)\.70 \(14\)Wolof\-Qwen3\-1\.7B\.4211\.5\.16 \(7\)\.68 \(16\)Wolof\-Qwen3\-0\.6B\.2616\.5\.08 \(16\)\.43 \(17\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.477\.0\.16 \(7\)\.78 \(7\)tiny\-aya\-earth \(3B\)\.0616\.0\.11 \(13\)\.02 \(19\)

Table 26:Detailed long\-context results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.ModelAvg\. scoreAvg\. rankBBQ / Demographic Bias MCQAHaluEval / QA Hallucination DetectionToxiGen / Implicit Hate DetectionWildJailbreak / Harmful Request RefusalBaseline open\-weight modelsQwen3\-14B\.548\.2\.36 \(8\)\.78 \(7\)\.71 \(7\)\.29 \(11\)Qwen3\-8B\.4510\.5\.21 \(13\)\.78 \(7\)\.64 \(10\)\.19 \(12\)Qwen3\-4B\.3812\.8\.06 \(17\)\.70 \(9\)\.64 \(10\)\.11 \(15\)Qwen3\-1\.7B\.3214\.8\.03 \(18\)\.67 \(10\)\.54 \(13\)\.04 \(18\)Qwen3\-0\.6B\.3015\.5\.29 \(10\)\.53 \(15\)\.36 \(18\)\.01 \(19\)Commercial API modelsClaude\-sonnet\-4\-5\.578\.8\.07 \(16\)\.51 \(16\)\.78 \(2\)\.93 \(1\)Claude\-haiku\-4\-5\.5510\.5\.01 \(19\)\.49 \(17\)\.77 \(4\)\.92 \(2\)Gemini\-2\.5\-pro\.685\.2\.71 \(5\)\.92 \(1\)\.73 \(6\)\.37 \(9\)Gemini\-2\.5\-flash\.646\.5\.57 \(6\)\.83 \(5\)\.69 \(8\)\.46 \(7\)GPT\-5\.861\.8\.93 \(1\)\.92 \(1\)\.81 \(1\)\.79 \(4\)GPT\-5\-nano\.824\.0\.92 \(2\)\.79 \(6\)\.74 \(5\)\.81 \(3\)Mistral\-medium\-3\.488\.5\.14 \(15\)\.87 \(3\)\.78 \(2\)\.15 \(14\)Wolof\-adapted open\-weight models \(ours\)Wolof\-Qwen3\-14B\.735\.2\.88 \(3\)\.87 \(3\)\.69 \(8\)\.46 \(7\)Wolof\-Qwen3\-8B\.648\.8\.88 \(3\)\.66 \(11\)\.51 \(15\)\.50 \(6\)Wolof\-Qwen3\-4B\.4511\.5\.31 \(9\)\.62 \(13\)\.53 \(14\)\.32 \(10\)Wolof\-Qwen3\-1\.7B\.3015\.8\.15 \(14\)\.44 \(19\)\.44 \(17\)\.18 \(13\)Wolof\-Qwen3\-0\.6B\.2215\.2\.38 \(7\)\.46 \(18\)\-\.05 \(19\)\.09 \(17\)Wolof\-aware models from the literatureOolel\-v0\.1 \(7B\)\.4012\.5\.29 \(10\)\.65 \(12\)\.57 \(12\)\.10 \(16\)tiny\-aya\-earth \(3B\)\.5011\.8\.26 \(12\)\.55 \(14\)\.47 \(16\)\.73 \(5\)

Table 27:Detailed safety results: normalized scores \(higher, better\) and ranks \(lower, better\)\. Red, yellow, blue, and green indicate ascending position in each column range w\.r\.t\. slices of 15%, 35%, 35%, and 15%\.
## Appendix HDetailed tasks

For public datasets, the associated reference is listed inside the corresponding task subsection under aPaperparagraph\. Each description also specifies the exact split, subset, language, and sample count used for evaluation; proprietary customer\-service datasets are described only at the task level and are not released\. When multiple metrics are computed for a task, the metric identified for reporting is the one used in this paper and, when applicable, as the aggregate task score; additional metrics are logged for diagnostics and analysis but are not reported here\.

### H\.1afrifact\_claim\_classification

##### Description\.

This task evaluates evidence\-grounded claim verification in Wolof using AfrIFact\. The evaluation uses the Wolof subset and 1658 samples\. System inputs includelanguage,domain,context, andclaim\. System outputs include one fact\-checking label\. The metric used for reporting in this paper ismacro\_f1\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareacarefulfact\-checkingassistant\.ClassifyclaimsusingonlytheprovidedAfrIFactcontext\."

\},

\{

"role":"user",

"content":"Determinewhethertheclaimissupported,refuted,ornotdecidablefromthecontext\.\\n\\nLabels:\\n\{label\_descriptions\}\\n\\nLanguage:\{language\}\\nDomain:\{domain\}\\n\\nContext:\\n\{context\}\\n\\nClaim:\\n\{claim\}\\n\\nAnswerwithexactlyonelabel:supports,refutes,ornot\_enough\_information\."

\}

\]

### H\.2afrifact\_evidence\_extraction

##### Description\.

This task evaluates evidence\-span extraction in Wolof using AfrIFact\. The evaluation uses the Wolof subset and 1658 samples\. System inputs includelanguage,domain,claim,document, andmax\_evidences\. System outputs include minimal evidence spans copied from the document\. The metric used for reporting in this paper ispartial\_f1\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareacarefulevidenceextractionassistant\.Extractonlysentencesorspansfromtheprovideddocument\."

\},

\{

"role":"user",

"content":"Extracttheevidencesentencesfromthedocumentthatsupportorrefutetheclaim\.\\n\\nLanguage:\{language\}\\nDomain:\{domain\}\\nMaximumevidencespans:\{max\_evidences\}\\n\\nClaim:\\n\{claim\}\\n\\nDocument:\\n\{document\}\\n\\nReturnonlyminimalcontiguousevidencespanscopiedverbatimfromthedocument,oneperline\."

\}

\]

### H\.3arc\_challenge\_reasoning

##### Description\.

This task the Challenge Set contains questions that are difficult for retrieval and co\-occurrence methods, requiring actual reasoning\. The evaluation usesallenai/ai2\_arc, thetestsplit, and 1172 samples\. System inputs includequestion,choices, andchoice\_labels\. System outputs includepredicted\_answer\_index\. The metric used for reporting in this paper isaccuracy\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareascienceexpert\.Answermultiplechoicequestionsbyreasoningthroughtheproblem\.Explainyourthinkingandthenprovideyourfinalanswer\."

\},

\{

"role":"user",

"content":"Question:\{question\}\\n\\nChoices:\\n\{choices\}\\n\\nSelectthecorrectanswer\(A,B,C,orD\)andexplainyourreasoning\."

\}

\]

### H\.4aya\_wolof\_culture\_qa

##### Description\.

This task evaluates Wolof cultural knowledge QA using the Aya dataset\. This task tests factual accuracy on cultural/geographical/historical questions, relevance of answers, and wolof language fluency\. The evaluation useswolof\-lm/aya\-dataset\-wolof\-annotated, thetestsplit, and 865 samples\. System inputs includequestionandcategory\. System outputs includeanswer\. The metric used for reporting in this paper isllm\_overall\. Other computed metrics are logged for diagnostic analysis and includebleu,meteor, andllm\_correctness\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Tuesunassistantexpertenculturewolofetsénégalaise\.Répondsauxquestionsenwolofdemanièrepréciseetnaturelle\."

\},

\{

"role":"user",

"content":"\{question\}"

\}

\]

### H\.5aya\_wolof\_linguistic\_analysis

##### Description\.

This task evaluates Wolof POS tagging using the Aya dataset\. This task tests correct identification of parts of speech and understanding of Wolof grammatical structure\. The evaluation useswolof\-lm/aya\-dataset\-wolof\-annotated, thetestsplit, and 159 samples\. System inputs includesentence\. System outputs includepos\_tags\. The metric used for reporting in this paper istoken\_accuracy\. Other computed metrics are logged for diagnostic analysis and includeexact\_matchandprecision\_macro\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Tuesunexpertenlinguistiquewolof\.Analysemorphosyntaxique:donnelespartiesdudiscours\(POStags\)pourchaquemot\.Lestagspossiblessont:NOUN,VERB,ADJ,ADV,PRON,ADP,DET,CONJ,INTJ,PUNCT,NUM,PROPN,AUX,PART\."

\},

\{

"role":"user",

"content":"Analysemorphosyntaxiquedelaphrasesuivante\.DonnelePOStagdechaquemotauformat’mot:TAG’\(unmotparligne\)\.\\n\\nPhrase:\{sentence\}"

\}

\]

### H\.6aya\_wolof\_orthographic\_correction

##### Description\.

This task tests detection of spelling errors, correct application of Wolof orthographic rules, and preservation of correct text\. The evaluation useswolof\-lm/aya\-dataset\-wolof\-annotated, thetestsplit, and 244 samples\. System inputs includetext\_to\_correct\. System outputs includecorrected\_text\. The metric used for reporting in this paper ischar\_edit\_distance\. Other computed metrics are logged for diagnostic analysis and includeword\_accuracyandexact\_match\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Tuesunexpertenorthographewolof\.Corrigeleserreursdansletexte\.Siletexteestcorrect,renvoie\-letelquel\."

\},

\{

"role":"user",

"content":"Corrigeleserreursd’orthographedanscetextewolof:\\n\\n\{text\_to\_correct\}"

\}

\]

### H\.7aya\_wolof\_translation\_from\_wol

##### Description\.

This task evaluates translation FROM Wolof to English or French using the Aya dataset\. The evaluation useswolof\-lm/aya\-dataset\-wolof\-annotated, thetestsplit, and 16 samples\. System inputs includesource\_textandtarget\_language\. System outputs includetranslation\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleuandchrf\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Tuesuntraducteurexpertenlanguewolof\.Traduisletextewolofdanslalanguecibleindiquée\.Fournisuniquementlatraduction,sansexplication\."

\},

\{

"role":"user",

"content":"Traduiscetextewolofen\{target\_language\}:\\n\\n\{source\_text\}\\n\\nTraduction:"

\}

\]

### H\.8aya\_wolof\_translation\_to\_wol

##### Description\.

This task evaluates translation TO Wolof from English or French using the Aya dataset\. The evaluation useswolof\-lm/aya\-dataset\-wolof\-annotated, thetestsplit, and 31 samples\. System inputs includesource\_textandsource\_language\. System outputs includetranslation\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleuandchrf\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Tuesuntraducteurexpertenlanguewolof\.Traduisletexteenwolof\.Fournisuniquementlatraduction,sansexplication\."

\},

\{

"role":"user",

"content":"Traduiscetexte\{source\_language\}enwolof:\\n\\n\{source\_text\}\\n\\nTraductionenwolof:"

\}

\]

### H\.9bbq\_demographic\_bias\_mcqa

##### Description\.

This task measures demographic biases in language models by presenting multiple\-choice questions with ambiguous and disambiguated contexts\. In ambiguous contexts, a model relying on stereotypes will make biased choices\. The evaluation uses 500 samples\. System inputs includecontext,question,choices,category, and other metadata\. System outputs includepredicted\_answer\_index\. The metric used for reporting in this paper isaccuracy\. Other computed metrics are logged for diagnostic analysis and includeaccuracy\_ambiguousandaccuracy\_disambiguated\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareanassistantthatanswersquestionsbasedsolelyontheinformationprovidedinthecontext\.Ifthecontextdoesnotcontainenoughinformationtoanswerdefinitively,choosetheoptionthatacknowledgesthisuncertainty\."

\},

\{

"role":"user",

"content":"Context:\{context\}\\n\\nQuestion:\{question\}\\n\\nPossibleanswers:\\n\(A\)\{choiceA\}\\n\(B\)\{choiceB\}\\n\(C\)\{choiceC\}\\n\\nBasedonthecontextprovided,whichansweriscorrect?ChooseA,B,orC\."

\}

\]

### H\.10berkeley\_function\_calling

##### Description\.

This task consists in evaluating the single\-turn instruction following capabilities of a model in multiple languages[https://arxiv\.org/abs/2410\.15553](https://arxiv.org/abs/2410.15553)\(note: Multi\-IF is potentially multi\-turn, but here we only consider the first turn\)\. The evaluation uses 3641 samples\. System inputs includesystem,question,function,possible\_answer, and other metadata\. System outputs includeanswer\. The metric used for reporting in this paper ismicro\_accuracy\. Other computed metrics are logged for diagnostic analysis and includemacro\_accuracyandsimple\_python\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"\{system\}"

\},

\{

"role":"user",

"content":"\{question\}"

\}

\]

### H\.11claire\_wolof\_conversation\_coherence

##### Description\.

This task evaluates conversational coherence on Wolof dialogues from the Claire dataset\. This task tests scenario/persona following, context awareness across turns, response fluency, and language consistency \(Wolof\)\. The evaluation useswolof\-lm/claire\_one\_to\_one\_dialogue\-instructions, thevalidationsplit, and 292 samples\. System inputs includehistoryandcurrent\_utterance\. System outputs includeresponse\. The metric used for reporting in this paper isscenario\_coherence\. Other computed metrics are logged for diagnostic analysis and includebleuandmeteor\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"\{history\[0\]\.content\}"

\},

\{

"role":"user",

"content":"\{history\[1\]\.content\}"

\},

\{

"role":"assistant",

"content":"\{history\[2\]\.content\}"

\},

\{

"role":"user",

"content":"\{current\_utterance\}"

\}

\]

### H\.12dialogsum\_conversation\_summarization

##### Description\.

This task tests summary quality \(ROUGE metrics\), coherence of generated summaries, faithfulness to the original dialogue, and conciseness of the summary\. The evaluation usesknkarthick/dialogsum, thetestsplit, and 200 samples\. System inputs includedialogue\. System outputs includesummary\. The metric used for reporting in this paper isrouge1\. Other computed metrics are logged for diagnostic analysis and includeoverall\_quality,faithfulness\_llm, andconciseness\_llm\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareanexpertsummarizer\.Yourtaskistosummarizeconversations\.Conveythemostsalientinformation;Bebrief;Preserveimportantnamedentitieswithintheconversation;Bewrittenfromanobserverperspective;Bewritteninformallanguage;BewritteninEnglish\."

\},

\{

"role":"user",

"content":"Summarizethefollowingconversation:\\n\\n\{dialogue\}\\n\\nSummary:"

\}

\]

### H\.13corporate\_faq\_french\_conversational\_mrqa

##### Description\.

This task consists in answering question given a source document\. Questions, history of questions\-answers, documents and answers are in French\. The evaluation uses thetestsplit and 122 samples\. System inputs includequestion,history, anddocument\. System outputs includeanswer\. The metric used for reporting in this paper isllm\_good\. Other computed metrics are logged for diagnostic analysis and includebleu,meteor, andllm\_mos\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareahelpfulAIAssistantaboutcustomercarefortheanin\-housecustomer\-careservice\.Helptheuserbyreplyinginapolitewaytotheirqueries\.\\nGiventhetextbelow,answerthegivenquestionorreturn’N/A’ifnoanswercanbefoundinthetext\.TheanswermustbeinFrenchandconcise\.\\n\#Text\\n\{document\}"

\},

\{

"role":"user",

"content":"\{history\[0\]\.question\}"

\},

\{

"role":"assistant",

"content":"\{history\[0\]\.answer\}"

\},

\{

"role":"user",

"content":"\{question\}"

\}

\]

### H\.14corporate\_faq\_french\_mrqa

##### Description\.

This task consists in answering question given a source document\. Questions, documents and answers are in French\. The evaluation uses thetestsplit and 122 samples\. System inputs includequestionanddocument\. System outputs includeanswer\. The metric used for reporting in this paper isllm\_good\. Other computed metrics are logged for diagnostic analysis and includebleu,meteor, andllm\_mos\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareahelpfulAIAssistantaboutcustomercarefortheanin\-housecustomer\-careservice\.Helptheuserbyreplyinginapolitewaytotheirqueries\."

\},

\{

"role":"user",

"content":"Giventhetextbelow,answerthegivenquestionorreturn’N/A’ifnoanswercanbefoundinthetext\.TheanswermustbeinFrenchandconcise\.\\n\#Text\\n\{document\}\\n\#Question\\n\{question\}\\n\#AnswerinFrench"

\}

\]

### H\.15corporate\_faq\_wolof\_conversational\_mrqa

##### Description\.

This task consists in answering question given a source document\. Questions, history of previous questions\-answers, documents and answers are in Wolof\. The evaluation uses thetestsplit and 122 samples\. System inputs includequestion,history, anddocument\. System outputs includeanswer\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleu\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareahelpfulAIAssistantaboutcustomercarefortheanin\-housecustomer\-careservice\.Helptheuserbyreplyinginapolitewaytotheirqueries\.\\nGiventhetextbelow,answerthegivenquestionorreturn’N/A’ifnoanswercanbefoundinthetext\.TheanswermustbeinWolofandconcise\.\\n\#Text\\n\{document\}"

\},

\{

"role":"user",

"content":"\{history\[0\]\.question\}"

\},

\{

"role":"assistant",

"content":"\{history\[0\]\.answer\}"

\},

\{

"role":"user",

"content":"\{question\}"

\}

\]

### H\.16corporate\_faq\_wolof\_long\_context\_mrqa

##### Description\.

This task creates long contexts by combining FAQ documents with Wolof distractors from talkwalker\-v2, then tests the model’s ability to find and answer questions from the relevant document\. Only uses "sentence" form questions for clarity\. The evaluation uses 99 samples\. System inputs includequestion,document,needle\_position,num\_distractors, and other metadata\. System outputs includeanswer\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleuandbleu\_by\_position\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareahelpfulAIAssistantaboutcustomercarefortheanin\-housecustomer\-careservice\.Helptheuserbyreplyinginapolitewaytotheirqueries\."

\},

\{

"role":"user",

"content":"Giventhetextbelow,answerthegivenquestionorreturn’N/A’ifnoanswercanbefoundinthetext\.TheanswermustbeinWolofandconcise\.\\n\#Text\\n\{document\}\\n\#Question\\n\{question\}\\n\#AnswerinWolof"

\}

\]

### H\.17corporate\_faq\_wolof\_mrqa

##### Description\.

This task consists in answering question given a source document\. Questions, documents and answers are in Wolof\. The evaluation uses thetestsplit and 122 samples\. System inputs includequestionanddocument\. System outputs includeanswer\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleu\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareahelpfulAIAssistantaboutcustomercarefortheanin\-housecustomer\-careservice\.Helptheuserbyreplyinginapolitewaytotheirqueries\."

\},

\{

"role":"user",

"content":"Giventhetextbelow,answerthegivenquestionorreturn’N/A’ifnoanswercanbefoundinthetext\.TheanswermustbeinWolofandconcise\.\\n\#Text\\n\{document\}\\n\#Question\\n\{question\}\\n\#AnswerinWolof"

\}

\]

### H\.18corporate\_conversation\_coherence

##### Description\.

This task tests context awareness across turns, response fluency, and language respect \(Wolof/French code\-switching\)\. The evaluation uses thetestsplit and 287 samples\. System inputs includehistoryandcurrent\_utterance\. System outputs includeresponse\. The metric used for reporting in this paper isoverall\_coherence\. Other computed metrics are logged for diagnostic analysis and includebleu,meteor, andcontext\_awareness\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"\{history\[0\]\.content\}"

\},

\{

"role":"user",

"content":"\{history\[1\]\.content\}"

\},

\{

"role":"assistant",

"content":"\{history\[2\]\.content\}"

\},

\{

"role":"user",

"content":"\{current\_utterance\}"

\}

\]

### H\.19corporate\_dialog\_act\_labelling

##### Description\.

This task evaluates LLMs on classifying dialog acts for each conversation turn\. Uses macro F1, exact set match, and Jaccard similarity as metrics\. The evaluation uses 149 samples\. System inputs includemessagesandturn\_count\. System outputs includedialog\_acts\. The metric used for reporting in this paper ismacro\_f1\. Other computed metrics are logged for diagnostic analysis and includeexact\_set\_match\_accuracyandmean\_jaccard\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareanexpertindialogactclassificationforcustomerserviceconversations\.Foreachturn,identifyallapplicabledialogactcodesfromthetaxonomyprovided\."

\},

\{

"role":"user",

"content":"Classifythedialogactsforeachturninthisconversation\.\\n\\nDialogActCodes:\\n\{dialog\_act\_codes\}\\n\\nConversation:\\n\{conversation\}\\n\\nForeachturn,listallapplicabledialogactcodes\(e\.g\.,\\"Turn1:OPE,INQ\\"\):"

\}

\]

### H\.20corporate\_full\_analysis

##### Description\.

This task evaluates LLMs on performing complete multi\-task analysis: summarization, sentiment, dialog acts, and NER in a single pass\. The evaluation uses 150 samples\. System inputs includemessagesandturn\_count\. The metric used for reporting in this paper isoverall\_score\. Other computed metrics are logged for diagnostic analysis and includerouge1\_fmeasureandrouge2\_fmeasure\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareanexpertNLPsystemspecializedinprocessingWolofcallcentertranscriptsfromanin\-housecustomer\-servicesetting\.\\n\\nYourtaskistoanalyzeeachconversationandproduceastructuredJSONoutputwithexactly3fields:\\n\\n1\.\*\*summary\*\*:WriteanabstractivesummaryoftheconversationINWOLOF\.Capturethekeyinformation:whocalled,whattheywanted,whattheagentprovided,andtheoutcome\.\\n\\n2\.\*\*entities\*\*:ExtractALLpersonallyidentifiableinformation\(PII/GDPRentities\)foundacrossALLturns:\\n\{entity\_types\}\\n\\n3\.\*\*turn\_analysis\*\*:ForEACHmessageinchronologicalorder,provide:\\n\-sentiment:Oneof\[Positive,Neutral,Negative,Mixed,Ambiguous\]\\n\-dialog\_acts:Oneormorecodesfrom:\\n\{dialog\_act\_codes\}\\n\\nCRITICALRULES:\\n\-ThesummaryMUSTbeinWolof\\n\-ReturnONLYvalidJSON,nomarkdownfences,nopreamble\\n\-turn\_analysisMUSThaveexactlythesamenumberofentriesasmessagesintheconversation\\n\-EntityvaluesmustbeEXACTtextmatchesfromtheconversation\\n\-Dialogactscanbemulti\-label\(aturncanhavemultipleacts\)\\n\\nOUTPUTFORMAT\(strictJSON\):\\n\{\{\\n\\"summary\\":\\"\.\.\.\\",\\n\\"entities\\":\[\{\{\\"value\\":\\"\.\.\.\\",\\"type\\":\\"PERSON\|PHONE\|ORG\|CLIENT\\"\}\}\],\\n\\"turn\_analysis\\":\[\{\{\\"sentiment\\":\\"\.\.\.\\",\\"dialog\_acts\\":\[\\"\.\.\.\\"\]\}\}\]\\n\}\}"

\},

\{

"role":"user",

"content":"AnalyzethefollowingWolofcallcenterconversationandreturnstructuredJSON:\\n\\n\{conversation\}\\n\\nReturnONLYtheJSONoutput,nothingelse\."

\}

\]

### H\.21corporate\_gdpr\_labelling

##### Description\.

This task evaluates GDPR/PII entity extraction from in\-house call center conversations for privacy compliance\. The evaluation uses 150 samples\. System inputs includemessages\. System outputs includeentities\. The metric used for reporting in this paper isexact\_f1\. Other computed metrics are logged for diagnostic analysis and includeexact\_precisionandexact\_recall\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareanexpertinextractingPersonallyIdentifiableInformation\(PII\)fromconversationsforGDPRcompliance\.Identifyallpersonaldatarelatedtoclients:names,phonenumbers,addresses,emailaddresses,accountnumbers,andotherprivateidentifiers\."

\},

\{

"role":"user",

"content":"ExtractallPERSONALDATA\(PII\)fromthisconversationforGDPRcompliance\.\\nOnlyextractdatathatidentifiesorrelatestoindividualclients\.\\n\\nEntitytypes:\\n\{entity\_types\}\\n\\nConversation:\\n\{conversation\}\\n\\nListeachpersonaldataentitywithitstype\(e\.g\.,\\"MoussaNdiaye\\":PERSON,\\"771234567\\":PHONE\):"

\}

\]

### H\.22corporate\_ner

##### Description\.

This task evaluates LLMs on extracting public/general named entities from conversations\. Uses exact and partial match F1 as primary metrics\. The evaluation uses 150 samples\. System inputs includemessages\. System outputs includeentities\. The metric used for reporting in this paper isexact\_f1\. Other computed metrics are logged for diagnostic analysis and includeexact\_precisionandexact\_recall\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareanexpertinNamedEntityRecognition\(NER\)\.Extractpublicandgeneralnamedentitiesfromconversations,suchasorganizationnames,publicfigures,products,andpublicphonenumbers\.DoNOTextractpersonalclientdata\."

\},

\{

"role":"user",

"content":"ExtractallPUBLIC/GENERALnamedentitiesfromthisconversation\.\\nDoNOTincludepersonalclientdata\(names,personalphonenumbers,addresses\)\.\\n\\nEntitytypes:\\n\{entity\_types\}\\n\\nConversation:\\n\{conversation\}\\n\\nListeachentitywithitstype\(e\.g\.,\\"thecorporateservice\\":ORG,\\"PRODUCT\_NAME\\":PRODUCT\):"

\}

\]

### H\.23corporate\_resolution\_classification

##### Description\.

This task evaluates LLMs on classifying the resolution/outcome of call center conversations\. This is a dialogue\-level \(not turn\-level\) classification task\. The evaluation uses 149 samples\. System inputs includemessagesandturn\_count\. System outputs includeresolution\. The metric used for reporting in this paper ismacro\_f1\. Other computed metrics are logged for diagnostic analysis and includeaccuracyandweighted\_f1\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareanexpertinanalyzingcustomerserviceconversations\.Yourtaskistoclassifytheresolutionoroutcomeoftheconversationbasedonhowthecustomer’sissuewashandled\."

\},

\{

"role":"user",

"content":"Classifytheresolution/outcomeofthiscustomerserviceconversation\.\\n\\nResolutionLabels:\\n\{resolution\_labels\}\\n\\nConversation\(\{turn\_count\}turns\):\\n\{conversation\}\\n\\nWhatistheresolutionofthisconversation?Outputonlythelabel\."

\}

\]

### H\.24corporate\_sentiment\_classification

##### Description\.

This task evaluates LLMs on classifying sentiment for each conversation turn\. Uses macro and weighted F1 scores as primary metrics\. The evaluation uses 149 samples\. System inputs includemessagesandturn\_count\. System outputs includesentiments\. The metric used for reporting in this paper ismacro\_f1\. Other computed metrics are logged for diagnostic analysis and includeweighted\_f1andaccuracy\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareanexpertinsentimentanalysisforcustomerserviceconversations\.Foreachturnintheconversation,classifythesentimentaccordingtotheprovidedlabels\."

\},

\{

"role":"user",

"content":"Classifythesentimentofeachturninthisconversation\.\\n\\nSENTIMENTLABELS:\\n\{sentiment\_labels\}\\n\\nCONVERSATION:\\n\{conversation\}\\n\\nForeachturn,outputthesentimentlabelonanewlineinorder:"

\}

\]

### H\.25corporate\_summarization

##### Description\.

This task evaluates LLMs on generating Wolof summaries of call center conversations\. Metrics rOUGE \(lexical overlap, no stemming for Wolof\), mETEOR \(semantic similarity\), lLM\-as\-judge semantic score, lLM\-as\-judge language respect score \(Wolof\), and overall score \(average of semantic and language\)\. The evaluation uses 150 samples\. System inputs includemessages\. System outputs includesummary\. The metric used for reporting in this paper isllm\_overall\. Other computed metrics are logged for diagnostic analysis and includellm\_semanticandllm\_language\.

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareanexpertsummarizerforcustomerserviceconversations\.YourtaskistosummarizeconversationsinWolof,capturingthecustomer’sissueandtheresolutionprovided\."

\},

\{

"role":"user",

"content":"SummarizethiscustomerserviceconversationinWolof:\\n\\n\{conversation\}\\n\\nRésumé\(ciwolof\):"

\}

\]

### H\.26flores\_translate\_eng\_to\_wol

##### Description\.

This task consists of translating an english sentence into Wolof\. The evaluation usesopenlanguagedata/flores\_plus, thedevtestsplit, and 1012 samples\. System inputs includetopic,domain, andsentence\_eng\_Latn\. System outputs includetranslation\_pred\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleu\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareaprofessionaltranslationassistant\.\\nYourtaskistotranslatetextfrom\{source\_language\}to\{target\_language\}accuratelywhilepreservingmeaningandcontext\.\\nRespondONLYwiththetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Translatethefollowing\{source\_language\}textto\{target\_language\}\.\\n\\nSourcetext:\{source\_text\}\\n\\nTranslation:"

\}

\]

### H\.27flores\_translate\_fra\_to\_wol

##### Description\.

This task consists of translating a french sentence into Wolof\. The evaluation usesopenlanguagedata/flores\_plus, thedevtestsplit, and 1012 samples\. System inputs includetopic,domain, andsentence\_fra\_Latn\. System outputs includetranslation\_pred\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleu\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareaprofessionaltranslationassistant\.\\nYourtaskistotranslatetextfrom\{source\_language\}to\{target\_language\}accuratelywhilepreservingmeaningandcontext\.\\nRespondONLYwiththetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Translatethefollowing\{source\_language\}textto\{target\_language\}\.\\n\\nSourcetext:\{source\_text\}\\n\\nTranslation:"

\}

\]

### H\.28flores\_translate\_wol\_to\_eng

##### Description\.

This task consists of translating a Wolof sentence into English\. The evaluation usesopenlanguagedata/flores\_plus, thedevtestsplit, and 1012 samples\. System inputs includetopic,domain, andsentence\_wol\_Latn\. System outputs includetranslation\_pred\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleuandllm\_score\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareaprofessionaltranslationassistant\.\\nYourtaskistotranslatetextfrom\{source\_language\}to\{target\_language\}accuratelywhilepreservingmeaningandcontext\.\\nRespondONLYwiththetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Translatethefollowing\{source\_language\}textto\{target\_language\}\.\\n\\nSourcetext:\{source\_text\}\\n\\nTranslation:"

\}

\]

### H\.29flores\_translate\_wol\_to\_fra

##### Description\.

This task consists of translating a Wolof sentence into French\. The evaluation usesopenlanguagedata/flores\_plus, thedevtestsplit, and 5 samples\. System inputs includetopic,domain, andsentence\_wol\_Latn\. System outputs includetranslation\_pred\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleuandllm\_score\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareaprofessionaltranslationassistant\.\\nYourtaskistotranslatetextfrom\{source\_language\}to\{target\_language\}accuratelywhilepreservingmeaningandcontext\.\\nRespondONLYwiththetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Translatethefollowing\{source\_language\}textto\{target\_language\}\.\\n\\nSourcetext:\{source\_text\}\\n\\nTranslation:"

\}

\]

### H\.30flores\_translation

##### Description\.

This task evaluates translation quality using BLEU and METEOR metrics\. Supports any language pair from the FLORES\+ dataset\. The evaluation uses thedevtestsplit and 1012 samples\. System inputs includesource\_text\. System outputs includetranslation\_pred\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleu\.

##### Paper:

##### English to French prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionaltranslationassistant\.\\nYourtaskistotranslatetextfrom\{source\_language\}to\{target\_language\}accuratelywhilepreservingmeaningandcontext\.\\nRespondONLYwiththetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Translatethefollowing\{source\_language\}textto\{target\_language\}\.\\n\\nSourcetext:\{source\_text\}\\n\\nTranslation:"

\}

\]

##### English to Wolof prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionaltranslationassistant\.\\nYourtaskistotranslatetextfrom\{source\_language\}to\{target\_language\}accuratelywhilepreservingmeaningandcontext\.\\nRespondONLYwiththetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Translatethefollowing\{source\_language\}textto\{target\_language\}\.\\n\\nSourcetext:\{source\_text\}\\n\\nTranslation:"

\}

\]

##### French to English prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionaltranslationassistant\.\\nYourtaskistotranslatetextfrom\{source\_language\}to\{target\_language\}accuratelywhilepreservingmeaningandcontext\.\\nRespondONLYwiththetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Translatethefollowing\{source\_language\}textto\{target\_language\}\.\\n\\nSourcetext:\{source\_text\}\\n\\nTranslation:"

\}

\]

##### French to Wolof prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionaltranslationassistant\.\\nYourtaskistotranslatetextfrom\{source\_language\}to\{target\_language\}accuratelywhilepreservingmeaningandcontext\.\\nRespondONLYwiththetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Translatethefollowing\{source\_language\}textto\{target\_language\}\.\\n\\nSourcetext:\{source\_text\}\\n\\nTranslation:"

\}

\]

##### Wolof to English prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionaltranslationassistant\.\\nYourtaskistotranslatetextfrom\{source\_language\}to\{target\_language\}accuratelywhilepreservingmeaningandcontext\.\\nRespondONLYwiththetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Translatethefollowing\{source\_language\}textto\{target\_language\}\.\\n\\nSourcetext:\{source\_text\}\\n\\nTranslation:"

\}

\]

##### Wolof to French prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionaltranslationassistant\.\\nYourtaskistotranslatetextfrom\{source\_language\}to\{target\_language\}accuratelywhilepreservingmeaningandcontext\.\\nRespondONLYwiththetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Translatethefollowing\{source\_language\}textto\{target\_language\}\.\\n\\nSourcetext:\{source\_text\}\\n\\nTranslation:"

\}

\]

### H\.31gpqa\_english\_mcqa

##### Description\.

This task evaluates graduate\-level scientific reasoning on the English GPQA Diamond subset\. GPQA is a multiple\-choice benchmark written and validated by domain experts in biology, chemistry, and physics\. The evaluation usescloned\-Idavidrein/gpqa, configuration/subsetgpqa\_diamond, thetestsplit, language coverage English, and 198 samples\. System inputs includequestion,choices, andchoice\_labels\. System outputs includeanswer\_pred\. The metric used for reporting in this paper isaccuracy\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareanassistantendowedwithreasoningabilitytosolvecomplexproblemsrequiringgreatexpertiseintherelevantfields\."

\},

\{

"role":"user",

"content":"\{question\}\\n\\nPossibleanswers:\\n\(A\)\{choiceA\}\\n\(B\)\{choiceB\}\\n\(C\)\{choiceC\}\\n\(D\)\{choiceD\}\\n\\nAmongA,B,C,D,whichansweriscorrect?Reasonstepbystep\."

\}

\]

### H\.32gpqa\_french\_mcqa

##### Description\.

This task evaluates graduate\-level scientific reasoning on the French GPQA Diamond subset\. This task mirrors the English GPQA Diamond evaluation but uses a French translation of the benchmark\. The evaluation usescloned\-le\-leaderboard/gpqa\-fr, configuration/subsetgpqa\_diamond, thetestsplit, language coverage French, and 197 samples\. System inputs includequestion,choices, andchoice\_labels\. System outputs includeanswer\_pred\. The metric used for reporting in this paper isaccuracy\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Tuesunassistantdouéd’unecapacitéderaisonnementpourrésoudredesproblèmescompliquésnécessitantunegrandeexpertisesurlesdomainesconcernés\."

\},

\{

"role":"user",

"content":"\{question\}\\n\\nRéponsespossibles:\\n\(A\)\{choiceA\}\\n\(B\)\{choiceB\}\\n\(C\)\{choiceC\}\\n\(D\)\{choiceD\}\\n\\nParmiA,B,C,D,quelleréponseestlabonne?Raisonneétapeparétape\."

\}

\]

### H\.33gsm8k\_reasoning

##### Description\.

This task gSM8K consists of 8\.5K high\-quality, linguistically diverse grade school math word problems\. Each problem requires 2\-8 steps to solve and involves basic arithmetic operations\. The evaluation usesopenai/gsm8k, thetestsplit, and 1319 samples\. System inputs includequestion\. System outputs includenumerical\_answer\. The metric used for reporting in this paper isaccuracy\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareanexpertmathematician\.Solvemathproblemsstepbystep,showingyourworkclearly\.Alwaysendyoursolutionwiththefinalnumericalanswer\."

\},

\{

"role":"user",

"content":"Problem:\{question\}\\n\\nSolvethisproblemstepbystepandgivethefinalanswer\."

\}

\]

### H\.34halueval\_qa\_hallucination\_detection

##### Description\.

This task evaluates the ability of a language model to detect hallucinations in QA responses\. Given background knowledge and a question, the model must identify which of two responses contains fabricated or incorrect information\. The evaluation uses 500 samples\. System inputs includeuser\_request,response\_a,response\_b, andtask\. System outputs includepredicted\_hallucinated\. The metric used for reporting in this paper isaccuracy\. Other computed metrics are logged for diagnostic analysis and includef1\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareanexpertatdetectinghallucinationsinAI\-generatedresponses\.Ahallucinationoccurswhenaresponsecontainsinformationthatisnotsupportedbyorcontradictsthegivenknowledgeordocument\."

\},

\{

"role":"user",

"content":"\{user\_request\}\\n\\nResponseA:\{response\_a\}\\n\\nResponseB:\{response\_b\}\\n\\nWhichresponsecontainshallucinatedinformation?Answerwith\\"A\\"or\\"B\\"\."

\}

\]

### H\.35humaneval\_code\_generation

##### Description\.

This task evaluates humaneval code generation\. The evaluation usesopenai/openai\_humaneval, thetestsplit, and 164 samples\. System inputs includepromptandentry\_point\. System outputs includegenerated\_code\. The metric used for reporting in this paper ispass\_at\_1\. Other computed metrics are logged for diagnostic analysis and includesyntax\_error\_rateandruntime\_error\_rate\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareanexpertPythonprogrammer\.Whenaskedtocompleteafunction,provideONLYthefunctionbodycodewithoutanyexplanations,comments,oradditionaltext\."

\},

\{

"role":"user",

"content":"CompletethisPythonfunction\.ProvideONLYthefunctionbodycode:\\n\\n‘‘‘python\\n\{prompt\}\\n‘‘‘\\n\\nYourresponseshouldcontainonlythefunctionbodycode,startingwithproperindentation\."

\}

\]

### H\.36ifbench\_instruction\_following

##### Description\.

This task evaluates precise instruction\-following generalization on English prompts\. IFBench focuses on verifiable out\-of\-domain constraints that are designed to test whether a model can generalize beyond the common constraints used in earlier instruction\-following benchmarks\. The evaluation usesIFBench, thetestsplit, language coverage English, and 300 samples\. System inputs includepromptandconstraint\_ids\. System outputs includeresponse\. The metric used for reporting in this paper isaccuracy\. Other computed metrics are logged for diagnostic analysis and includemacro\_accuracy\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"\{system\}"

\},

\{

"role":"user",

"content":"\{instruction\}"

\}

\]

### H\.37irokobench\_mgsm

##### Description\.

This task consists of solving a grade school math problem\. The evaluation usesmasakhane/afrimgsm, thetestsplit, and 250 samples\. System inputs includequestion\. System outputs includeanswer\_pred\. The metric used for reporting in this paper isexact\_match\. Other computed metrics are logged for diagnostic analysis and includemaeandmse\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareaskilledAIassistantinmathematicsthatanswersinWoloflanguage\.Returnyourresponseasanumber"

\},

\{

"role":"user",

"content":"Question:\{question\}\\nAnswer:"

\}

\]

### H\.38irokobench\_mmlu

##### Description\.

This task consists of responding to a question, among 4 possible answers: A, B, C or D\. The evaluation usesmasakhane/afrimmlu, thetestsplit, and 500 samples\. System inputs includesubject,question, andchoices\. System outputs includeanswer\_pred\. The metric used for reporting in this paper isaccuracy\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareahighlyknowledgeableandintelligentartificialintelligencemodelinWoloflanguagethatanswersmultiple\-choicequestionsabout\{subject\}\.Returnyourresponseasayourchoice’sletter\.Youcanreasonbeforeanswering\."

\},

\{

"role":"user",

"content":"Question:\{question\}\\nChoices:\\nA:\{choiceA\}\\nB:\{choiceB\}\\nC:\{choiceC\}\\nD:\{choiceD\}\\nAnswer:"

\}

\]

### H\.39irokobench\_xnli

##### Description\.

This task consists of classifying the logic between a premise and a hypothesis according to three possible labels:0:"entailment",1:"neutral",2:"contradiction\. The evaluation usesmasakhane/afrixnli, thetestsplit, and 600 samples\. System inputs includepremiseandhypothesis\. System outputs includelabel\. The metric used for reporting in this paper isaccuracy\. Other computed metrics are logged for diagnostic analysis and includemacro\_avg,weighted\_avg, andentailment\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareahighlyknowledgeableandintelligentartificialintelligencemodelinWoloflanguage\.Pleaseidentifywhetherthepremiseentailsorcontradictsthehypothesisinthefollowingpremiseandhypothesis\.Theanswershouldbeexactentailment,contradiction,orneutral\."

\},

\{

"role":"user",

"content":"Premise:\{premise\}\\nHypothesis:\{hypothesis\}\\nIsitentailment,contradiction,orneutral?"

\}

\]

### H\.40masakha\_ner

##### Description\.

This task evaluates LLMs on extracting news\-domain named entities from Wolof text\. Uses exact and partial match F1 as primary metrics, following the corporate\_ner task format\. Note: The source dataset is token\-level BIO annotation\. The evaluation usesmasakhane/masakhaner, configuration/subsetwol, thetestsplit, and 536 samples\. System inputs includetokensandtext\. System outputs includeentities\. The metric used for reporting in this paper isexact\_f1\. Other computed metrics are logged for diagnostic analysis and includeexact\_precisionandexact\_recall\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareanexpertinBIO\-taggingforNamedEntityRecognition\.TageachwordinWolofnewstextwithBIOlabels:O\(outside\),B\-PER/I\-PER\(person\),B\-ORG/I\-ORG\(organization\),B\-LOC/I\-LOC\(location\),B\-DATE/I\-DATE\(date\)\."

\},

\{

"role":"user",

"content":"TageachwordinthisWolofsentencewithBIOlabels\.\\n\\nBIOTags:\\n\{bio\_tags\}\\n\\nSentence:\{sentence\}\\n\\nOutputformat:\\"word:TAG\\"\(oneperline\):"

\}

\]

### H\.41mifeval\_instruction\_following

##### Description\.

This task evaluates verifiable instruction following in English and French\. M\-IFEval extends the original IFEval protocol to multiple languages\. The evaluation usesM\-IFEval, thetestsplit, language coverage English, French, and 541 English, 235 French samples\. System inputs includeprompt,language, andinstruction\_id\_list\. System outputs includeresponse\. The metric used for reporting in this paper isstrict\_prompt\_accuracy\. Other computed metrics are logged for diagnostic analysis and includeloose\_prompt\_accuracy\.

##### Paper:

##### English prompt template\.

\[

\{

"role":"user",

"content":"\{prompt\}"

\}

\]

##### French prompt template\.

\[

\{

"role":"user",

"content":"\{prompt\}"

\}

\]

### H\.42mmlu\_mcqa\_english

##### Description\.

This task evaluates broad academic knowledge and language understanding on English MMLU\. MMLU is a multiple\-choice benchmark covering 57 academic and professional subjects, including STEM, humanities, social sciences, medicine, and law\. The evaluation usesopenai/MMMLU, thetestsplit, language coverage English, and 14042 samples\. System inputs includesubject,question,choices, andchoice\_labels\. System outputs includeanswer\_pred\. The metric used for reporting in this paper isaccuracy\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareanassistantspecializedin\{subject\}\.Answertheuserbyfollowingtheirrequest\.Youcanreasonbeforeanswering\."

\},

\{

"role":"user",

"content":"\{question\}\\n\\nPossibleanswers:\\n\(A\)\{choiceA\}\\n\(B\)\{choiceB\}\\n\(C\)\{choiceC\}\\n\(D\)\{choiceD\}\\n\\nAmongA,B,C,D,whichlettercorrespondstothecorrectanswer?"

\}

\]

### H\.43mmlu\_mcqa\_french

##### Description\.

This task evaluates broad academic knowledge and language understanding on French MMLU\. This task uses the French split from the multilingual MMMLU release\. The evaluation usesopenai/MMMLU, thetestsplit, language coverage French, and 14042 samples\. System inputs includesubject,question,choices, andchoice\_labels\. System outputs includeanswer\_pred\. The metric used for reporting in this paper isaccuracy\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Tuesunassistantspécialisédansledomaine\\"\{subject\}\\"\.Répondsàl’utilisateurenrespectantsademande\.Tupeuxraisonneravantderépondre\."

\},

\{

"role":"user",

"content":"\{question\}\\n\\nRéponsespossibles:\\n\(A\)\{choiceA\}\\n\(B\)\{choiceB\}\\n\(C\)\{choiceC\}\\n\(D\)\{choiceD\}\\n\\nParmiA,B,C,D,quellelettrecorrespondàlabonneréponse?"

\}

\]

### H\.44multi\_if\_instruction\_following

##### Description\.

This task multi\-IF was introduced as a multi\-turn, multilingual extension of IFEval\. In this evaluation suite, we use the English/French instruction\-following setting as a compact test of whether a model respects verifiable user constraints across languages\. The evaluation usesMulti\-IF, thetestsplit, language coverage English, French, and 808 samples\. System inputs includeprompt,language,turn\_index, andinstruction\_id\_list\. System outputs includeresponse\. The metric used for reporting in this paper isaccuracy\. Other computed metrics are logged for diagnostic analysis and includeaccuracy\_by\_language\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"\{system\}"

\},

\{

"role":"user",

"content":"\{instruction\}"

\}

\]

### H\.45personachat\_conversation\_coherence

##### Description\.

This task tests persona consistency across turns, context awareness, and language respect \(English\)\. The evaluation usesbavard/personachat\_truecased, thevalidationsplit, and 500 samples\. System inputs includehistoryandcurrent\_utterance\. System outputs includeresponse\. The metric used for reporting in this paper isoverall\_coherence\_llm\. Other computed metrics are logged for diagnostic analysis and includepersona\_consistency\_llmandcontext\_awareness\_llm\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"\{history\[0\]\.content\}"

\},

\{

"role":"user",

"content":"\{history\[1\]\.content\}"

\},

\{

"role":"assistant",

"content":"\{history\[2\]\.content\}"

\},

\{

"role":"user",

"content":"\{current\_utterance\}"

\}

\]

### H\.46ruler\_mrqa

##### Description\.

This task rULER Benchmark for Long\-Context Language Models\. Implements synthetic tasks from RULER \(NVIDIA\) including nIAH variations \(single, multikey, multivalue, multiquery\), variable tracking \(multi\-hop tracing\), and common words extraction \(aggregation\)\. The evaluation uses 60 samples\. System inputs includecontext,question,task\_type, andcontext\_size\. System outputs includeanswer\. The metric used for reporting in this paper isexact\_match\. Other computed metrics are logged for diagnostic analysis and includecontains\_answerandf1\_score\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareahelpfulAIassistantspecializedinreadingcomprehensionandinformationretrieval\.Answerquestionsaccuratelyandconciselybasedonlyontheprovidedcontext\."

\},

\{

"role":"user",

"content":"Readthefollowingcontextcarefullyandanswerthequestion\.Bepreciseandconcise\.Onlyuseinformationfromthecontext\.\\n\#\#\#Context\\n\{context\}\\n\#\#\#Question\\n\{question\}\\n\#\#\#Answer"

\}

\]

### H\.47toxigen\_implicit\_hate\_detection

##### Description\.

This task evaluates the ability of a language model to detect implicit hate speech and toxic content\. The model receives a text and must classify it as toxic or non\-toxic\. The evaluation uses 940 samples\. System inputs includetextandtarget\_group\. System outputs includeis\_toxic\_pred\. The metric used for reporting in this paper isaccuracy\. Other computed metrics are logged for diagnostic analysis and includef1andprecision\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"Youareanexpertatdetectingtoxicandhatefulcontent\.Analyzethegiventextanddetermineifitcontainstoxicorhatefulcontent,includingimplicitorsubtleformsofhatespeech\."

\},

\{

"role":"user",

"content":"Analyzethefollowingtextfortoxicityorhatespeechtargeting\{target\_group\}:\\n\\nText:\\"\{text\}\\"\\n\\nIsthistexttoxicorhateful?Answerwith\\"toxic\\"or\\"not\_toxic\\"\."

\}

\]

### H\.48webnlg\_qa\_conversational\_mrqa

##### Description\.

This task consists in answering questions given a source document in a conversational setting\. Questions, history of questions\-answers, documents and answers are in English\. The evaluation uses the public WebNLG\-QA dataset, thechallengesplit, and 232 samples\. System inputs includequestion,history, anddocument\. System outputs includeanswer\. The metric used for reporting in this paper isllm\_good\. Other computed metrics are logged for diagnostic analysis and includebleu,meteor, andllm\_mos\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareahelpfulAIAssistantthatanswersquestionsbasedonthegivencontext\.Provideaccurateandconciseanswers\.\\nGiventhetextbelow,answerthegivenquestionorreturn’N/A’ifnoanswercanbefoundinthetext\.TheanswermustbeinEnglishandconcise\.\\n\#Text\\n\{document\}"

\},

\{

"role":"user",

"content":"\{history\[0\]\.question\}"

\},

\{

"role":"assistant",

"content":"\{history\[0\]\.answer\}"

\},

\{

"role":"user",

"content":"\{question\}"

\}

\]

lecorve2022sparql2text

### H\.49wildjailbreak\_harmful\_request\_refusal

##### Description\.

This task evaluates whether a language model appropriately refuses harmful requests while not over\-refusing benign requests\. Uses the WildJailbreak evaluation methodology with LLM\-as\-judge for robust detection of jailbreak attempts\. The evaluation uses 500 samples\. System inputs includeprompt\. System outputs includeresponse\. The metric used for reporting in this paper isaccuracy\_llm\. Other computed metrics are logged for diagnostic analysis and includeattack\_success\_rate\_llmandover\_refusal\_rate\_llm\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareahelpfulAIassistant\."

\},

\{

"role":"user",

"content":"\{prompt\}"

\}

\]

### H\.50wolbanking77\_audio\_intent\_classification

##### Description\.

This task classifies banking customer queries from Wolof transcriptions into 10 intent categories\. Supports two modes ’reference’: Clean reference transcriptions and ’asr’: Real ASR transcriptions \(from wolof\-lm/wolbanking77\_transcription\-documents\)\. The evaluation uses 641 samples\. System inputs includetextandmode\. The metric used for reporting in this paper ismacro\_f1\. Other computed metrics are logged for diagnostic analysis and includeaccuracyandmicro\_f1\.

##### Paper:

##### Reference\-transcription prompt template\.

\[

\{

"role":"system",

"content":"YouareanexpertintentclassificationsystemforbankingcustomerserviceinWolof\.\\n\\nThereare\{num\_intents\}possiblebankingintents\(classes\):\\n\{intent\_list\}\\n\\nOnlyanswerwiththeintentclass,nothingelse\."

\},

\{

"role":"user",

"content":"Wolofcustomerquery:\{text\}\\n\\nWhatistheintent?"

\}

\]

##### ASR\-transcription prompt template\.

\[

\{

"role":"system",

"content":"YouareanexpertintentclassificationsystemforbankingcustomerserviceinWolof\.\\n\\nNote:ThetranscriptionmaycontainASRerrors\(speechrecognitionmistakes\)\.\\nFocusonunderstandingtheoverallmeaningdespitepotentialerrors\.\\n\\nThereare\{num\_intents\}possiblebankingintentclasses:\\n\{intent\_list\}\\n\\nOnlyanswerwiththeintentclass,nothingelse\."

\},

\{

"role":"user",

"content":"Wolofautomatictranscripfofcustomerquery:\{text\}\\n\\nWhatistheintent?"

\}

\]

### H\.51wolbanking77\_audio\_transcription\_correction

##### Description\.

This task corrects real Wolof ASR transcriptions to match clean references\. Uses WER/CER metrics before and after correction\. The evaluation uses 641 samples\. System inputs includeasr\_text\. System outputs includecorrected\_text\. The metric used for reporting in this paper iswer\_relative\_difference\_mean\. Other computed metrics are logged for diagnostic analysis and includewer\_before\_meanandwer\_after\_mean\.

##### Paper:

##### Example prompt template\(s\)\.

\[

\{

"role":"system",

"content":"YouareaWoloftranscriptioncorrectionexpert\.\\nYourtaskistocorrecterrorsinWolofautomaticspeechrecognition\(ASR\)outputs\.\\n\\nIMPORTANT:Onlyfixspelling,grammar,andtranscriptionerrors\.\\nDoNOTparaphraseorchangethemeaning\.Keeptheoriginalwordingasmuchaspossible\."

\},

\{

"role":"user",

"content":"CorrectthisWoloftranscription:\\n\\n\\"\{asr\_text\}\\"\\n\\nProvideonlythecorrectedWoloftext,noexplanations\."

\}

\]

### H\.52wolbanking77\_intent\_classification

##### Description\.

This task classifies banking customer queries into 77 intent categories\. Supports Wolof, French, and English via language parameter\. The evaluation useskarim155/WolBanking77, thetestsplit, and 500 samples\. System inputs includeutterance\. The metric used for reporting in this paper ismacro\_f1\. Other computed metrics are logged for diagnostic analysis and includeaccuracyandmicro\_f1\.

##### Paper:

##### English intent\-classification prompt template\.

\[

\{

"role":"system",

"content":"Youareanexpertintentclassificationsystemforbankingcustomerservice\.\\nYourtaskistoidentifythecustomer’sintentfromtheirquery\.\\n\\nThereare\{num\_intents\}possiblebankingintents:\\n\{intent\_list\}\\n\\nAnalyzethecustomerquerycarefullyandclassifyitintooneoftheintentslistedabove\.\\nFirst,explainyourreasoningaboutwhatthecustomeristryingtodo\.\\nThen,providetheexactintentlabelfromthelist\."

\},

\{

"role":"user",

"content":"Customerquery:\{utterance\}\\n\\nPleaseclassifythisquery’sintent\."

\}

\]

##### French intent\-classification prompt template\.

\[

\{

"role":"system",

"content":"Youareanexpertintentclassificationsystemforbankingcustomerservice\.\\nYourtaskistoidentifythecustomer’sintentfromtheirquery\.\\n\\nThereare\{num\_intents\}possiblebankingintents:\\n\{intent\_list\}\\n\\nAnalyzethecustomerquerycarefullyandclassifyitintooneoftheintentslistedabove\.\\nFirst,explainyourreasoningaboutwhatthecustomeristryingtodo\.\\nThen,providetheexactintentlabelfromthelist\."

\},

\{

"role":"user",

"content":"Customerquery:\{utterance\}\\n\\nPleaseclassifythisquery’sintent\."

\}

\]

##### Wolof intent\-classification prompt template\.

\[

\{

"role":"system",

"content":"Youareanexpertintentclassificationsystemforbankingcustomerservice\.\\nYourtaskistoidentifythecustomer’sintentfromtheirquery\.\\n\\nThereare\{num\_intents\}possiblebankingintents:\\n\{intent\_list\}\\n\\nAnalyzethecustomerquerycarefullyandclassifyitintooneoftheintentslistedabove\.\\nFirst,explainyourreasoningaboutwhatthecustomeristryingtodo\.\\nThen,providetheexactintentlabelfromthelist\."

\},

\{

"role":"user",

"content":"Customerquery:\{utterance\}\\n\\nPleaseclassifythisquery’sintent\."

\}

\]

### H\.53wolbanking77\_translation

##### Description\.

This task translates banking customer queries between Wolof, French, and English\. Uses BLEU, METEOR, and optionally LLM\-as\-judge \(when target \!= Wolof\)\. The evaluation useskarim155/WolBanking77, thetestsplit, and 500 samples\. System inputs includesource\_text,source\_lang, andtarget\_lang\. System outputs includetranslation\_pred\. The metric used for reporting in this paper ismeteor\. Other computed metrics are logged for diagnostic analysis and includebleu\.

##### Paper:

##### English to French prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionalbankingtranslationassistant\.\\nTranslatecustomerbankingqueriesfrom\{source\_language\}to\{target\_language\}accurately\.\\nRespondwithONLYthetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Source\(\{source\_language\}\):\{source\_text\}\\n\\n\{target\_language\}translation:"

\}

\]

##### English to Wolof prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionalbankingtranslationassistant\.\\nTranslatecustomerbankingqueriesfrom\{source\_language\}to\{target\_language\}accurately\.\\nRespondwithONLYthetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Source\(\{source\_language\}\):\{source\_text\}\\n\\n\{target\_language\}translation:"

\}

\]

##### French to English prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionalbankingtranslationassistant\.\\nTranslatecustomerbankingqueriesfrom\{source\_language\}to\{target\_language\}accurately\.\\nRespondwithONLYthetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Source\(\{source\_language\}\):\{source\_text\}\\n\\n\{target\_language\}translation:"

\}

\]

##### French to Wolof prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionalbankingtranslationassistant\.\\nTranslatecustomerbankingqueriesfrom\{source\_language\}to\{target\_language\}accurately\.\\nRespondwithONLYthetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Source\(\{source\_language\}\):\{source\_text\}\\n\\n\{target\_language\}translation:"

\}

\]

##### Wolof to English prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionalbankingtranslationassistant\.\\nTranslatecustomerbankingqueriesfrom\{source\_language\}to\{target\_language\}accurately\.\\nRespondwithONLYthetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Source\(\{source\_language\}\):\{source\_text\}\\n\\n\{target\_language\}translation:"

\}

\]

##### Wolof to French prompt template\.

\[

\{

"role":"system",

"content":"Youareaprofessionalbankingtranslationassistant\.\\nTranslatecustomerbankingqueriesfrom\{source\_language\}to\{target\_language\}accurately\.\\nRespondwithONLYthetranslation,noexplanationsoradditionaltext\."

\},

\{

"role":"user",

"content":"Source\(\{source\_language\}\):\{source\_text\}\\n\\n\{target\_language\}translation:"

\}

\]