非洲语言AI安全就绪度报告

# African Languages AI Safety Readiness Report
Source: [https://arxiv.org/html/2610.27269](https://arxiv.org/html/2610.27269)
Israel Abebe Azime††thanks:Equal contribution\.Jesujoba O\. AlabiAffiliation:MasakhaneTajuddeen GwadabeAffiliation:MasakhaneDavid Ifeoluwa AdelaniAffiliation:MasakhaneAffiliation:Mila \- Quebec AI Institute, McGill University, and Canada CIFAR AI Chair

###### Abstract

With the widespread adoption of large language models \(LLMs\) across languages and domains, AI safety has emerged as a critical global concern\. However, due to resource constraints, research and development efforts tend to prioritize a limited number of high\-resource languages, leaving many others, especially those spoken in Africa, underrepresented\. While existing research on language “resource\-ness” has largely examined whether sufficient resources are available to develop language technologies, the question of whether languages are adequately represented in AI safety research has received attention only more recently\. This report examines the readiness of African languages for inclusion in AI safety discussions, drawing on existing literature and data\. It considers four factors: the availability of AI\-relevant resources, including text data; national AI policy support; performance on established text benchmarks; and speaker population\. Based on these factors, languages are grouped into four readiness categories:fully ready,mostly ready,emerging, andfoundational support needed\. This classification is intended to inform the inclusion of African languages in AI safety benchmarks, highlight varying levels of preparedness, and guide strategic investment in data development, policy, and community engagement for underrepresented languages\.

*K*eywordsAI Safety⋅\\cdotLLM⋅\\cdotMultilingual Safety

## 1Introduction

The emergence of large language models \(LLMs\) has advanced natural language processing \(NLP\), enabling models to understand and generate human\-like text across a range of tasks and domains\[[1](https://arxiv.org/html/2610.27269#bib.bib1),[2](https://arxiv.org/html/2610.27269#bib.bib2)\]\.

Although there are over 7,000 languages spoken worldwide\[[3](https://arxiv.org/html/2610.27269#bib.bib3)\], mainstream NLP research has historically focused primarily on English\. In recent years, more studies have begun to address low\-resource and underrepresented languages\[[4](https://arxiv.org/html/2610.27269#bib.bib4)\]\. This work includes developing language\-specific models\[[5](https://arxiv.org/html/2610.27269#bib.bib5),[6](https://arxiv.org/html/2610.27269#bib.bib6)\], evaluating their performance using appropriate benchmarks\[[7](https://arxiv.org/html/2610.27269#bib.bib7),[8](https://arxiv.org/html/2610.27269#bib.bib8)\], and evaluating fairness and bias in NLP systems\[[9](https://arxiv.org/html/2610.27269#bib.bib9)\]\. Recent surveys of multilingual LLMs also highlight ongoing challenges in evaluating models across languages and addressing differences in their performance\[[10](https://arxiv.org/html/2610.27269#bib.bib10)\]\.

These differences also raise concerns about AI safety\. LLMs can generate biased, misleading, or harmful content, particularly when safety safeguards are inadequate or training data do not adequately represent the languages and communities they serve\[[11](https://arxiv.org/html/2610.27269#bib.bib11),[12](https://arxiv.org/html/2610.27269#bib.bib12)\]\. These risks raise particular concerns for low\-resource languages, where limited digital corpora and annotated datasets can constrain model development and adaptation\[[13](https://arxiv.org/html/2610.27269#bib.bib13)\]\. This scarcity not only hampers the performance of AI systems in these languages but also exacerbates safety concerns\[[14](https://arxiv.org/html/2610.27269#bib.bib14)\], as most AI safety research has primarily focused on English\[[15](https://arxiv.org/html/2610.27269#bib.bib15)\]\.

In this report, we focus on answering a central question:which languages native to Africa are prime candidates for inclusion in future AI safety benchmarks?To address this question, we conduct a comprehensive investigation comprising four key components\. We did the following:\(1\)analyzed the language coverage in widely\-used datasets to understand existing representation disparities;\(2\)evaluated popular multilingual benchmarks, examining both language inclusion and performance scores across languages;\(3\)benchmarked state\-of\-the\-art large language models \(LLMs\) using a carefully curated test suite designed to assess their capabilities in understanding and generating text in these languages; and\(4\)conducted interviews with native speakers to gain qualitative insights into the current state of AI readiness, as well as to identify challenges, opportunities, and directions for future development and support\.

Figure 1:A layered framework for evaluating the presence and readiness of African languages in machine learning ecosystems\. The analysis spans dataset coverage in pretraining corpora, representation in major AI systems, and performance across both popular multilingual benchmarks and African\-specific LLM benchmarks\. It also incorporates qualitative dimensions such as national AI strategies, policy landscapes, and community\-centered insights, including real\-world usage and feedback from related research\. This comprehensive view is used to assess not only technological readiness but also to guide the inclusion of languages in safety\-critical benchmarks and broader AI resource prioritization\. The result is highly dependent on Language ID used\.[Figure1](https://arxiv.org/html/2610.27269#S1.F1)shows a summary of our study, showing the layered framework we used to arrive at the categorization of African languages into four levels of AI safety readiness\.

## 2Background

### 2\.1Languages

Africa has one of the richest linguistic landscapes in the world, with over 2,000 languages spoken across the continent\[[16](https://arxiv.org/html/2610.27269#bib.bib16)\]\. These languages belong to several language families, such as Afroasiatic, Nilo\-Saharan, and Niger\-Congo, with the Niger\-Congo family being the largest, both in terms of the number of languages and speakers\[[17](https://arxiv.org/html/2610.27269#bib.bib17),[16](https://arxiv.org/html/2610.27269#bib.bib16)\]\. Despite this vast diversity, African languages are not well represented in language technologies\. However, there are efforts from various quarters, including research institutions, and local language communities, in the field of NLP for African languages, including the development of datasets, embeddings, language models, and other essential resources\[[18](https://arxiv.org/html/2610.27269#bib.bib18)\]\. Nevertheless, the readiness of many African languages for integration into AI technologies remains an open research question, due to challenges such as limited data and resource availability\. Therefore, in this paper we study and analyse the readiness of readiness of many African languages for safety use in AI\.

### 2\.2Datasets

Several datasets exist for African languages to support various NLP downstream tasks such as machine translation\[[19](https://arxiv.org/html/2610.27269#bib.bib19),[20](https://arxiv.org/html/2610.27269#bib.bib20)\], sentiment analysis\[[21](https://arxiv.org/html/2610.27269#bib.bib21),[22](https://arxiv.org/html/2610.27269#bib.bib22)\], named entity recognition\[[23](https://arxiv.org/html/2610.27269#bib.bib23),[24](https://arxiv.org/html/2610.27269#bib.bib24)\], and question answering\[[25](https://arxiv.org/html/2610.27269#bib.bib25)\]\. While some datasets are developed for individual languages\[[26](https://arxiv.org/html/2610.27269#bib.bib26),[27](https://arxiv.org/html/2610.27269#bib.bib27),[28](https://arxiv.org/html/2610.27269#bib.bib28)\], others are part of multilingual efforts\[[19](https://arxiv.org/html/2610.27269#bib.bib19),[21](https://arxiv.org/html/2610.27269#bib.bib21),[24](https://arxiv.org/html/2610.27269#bib.bib24)\]\. These multilingual datasets may be created through translation of existing resources in high\-resource languages\[[29](https://arxiv.org/html/2610.27269#bib.bib29)\], annotation projection\[[30](https://arxiv.org/html/2610.27269#bib.bib30)\], or extracting massive unannotated data from the web\[[31](https://arxiv.org/html/2610.27269#bib.bib31),[13](https://arxiv.org/html/2610.27269#bib.bib13),[32](https://arxiv.org/html/2610.27269#bib.bib32)\], which are often used for language modeling\.\[[18](https://arxiv.org/html/2610.27269#bib.bib18)\], in their work, showed that efforts on NLP tasks for African languages are skewed toward popular tasks, which can be attributed to the availability of datasets\.

### 2\.3Multilingual Language Models

With the curation of massive multilingual resources from the web, recent language models now cover hundreds of languages, including many African languages\[[33](https://arxiv.org/html/2610.27269#bib.bib33),[34](https://arxiv.org/html/2610.27269#bib.bib34),[32](https://arxiv.org/html/2610.27269#bib.bib32)\]\. These models follow one of three transformer architectures\[[35](https://arxiv.org/html/2610.27269#bib.bib35)\], encoder\-only\[[33](https://arxiv.org/html/2610.27269#bib.bib33)\], decoder\-only\[[34](https://arxiv.org/html/2610.27269#bib.bib34),[32](https://arxiv.org/html/2610.27269#bib.bib32)\], and encoder\-decoder\[[31](https://arxiv.org/html/2610.27269#bib.bib31)\]\.[Figure2](https://arxiv.org/html/2610.27269#S2.F2)shows the categorization of African\-centric LLMs based on these architecture\. While some of these models are trained from scratch, others are developed through continued pretraining\. Recently, decoder\-only models have become more common, contributing to a shift in how NLP tasks are formulated\[[36](https://arxiv.org/html/2610.27269#bib.bib36),[37](https://arxiv.org/html/2610.27269#bib.bib37),[38](https://arxiv.org/html/2610.27269#bib.bib38)\]\. Tasks that were once treated as classification or structured prediction problems are now increasingly rendered as generative tasks\. As these models continue to scale in size, they have demonstrated impressive reasoning capabilities and emergent abilities\[[37](https://arxiv.org/html/2610.27269#bib.bib37),[39](https://arxiv.org/html/2610.27269#bib.bib39)\], skills that were not explicitly programmed but arise from model complexity and size\. However, this rapid advancement also raises significant concerns about their safety and reliability\.\.

Furthermore, many of the recent and popular LLMs are trained predominantly on English data\[[40](https://arxiv.org/html/2610.27269#bib.bib40),[41](https://arxiv.org/html/2610.27269#bib.bib41),[42](https://arxiv.org/html/2610.27269#bib.bib42)\], with other languages, particularly low\-resource ones, being underrepresented\. Nevertheless, some have shown an ability to generalize across languages\. Also, there are growing efforts to develop models specifically for African languages, either by training from scratch using curated local datasets or by adapting existing multilingual models through continued pretraining\.

\{forest\}

Figure 2:Classification of African\-centric Large Language Models \(LLMs\) based on their base transformer architecture\.

## 3Methodology

To assess the AI readiness of African languages, we developed a multi\-faceted methodology that draws on existing data, quantitative analysis, and community insights\. In this work, we consider languages to be ’ready’ when they have accessible corpora, pretrained models, established benchmarks, supportive national AI policies, and an active community of speakers, used in this report as a proxy for societal importance\. Each subsection below describes one of the core components of our approach\.

### 3\.1Dataset Coverage in Pretraining Corpora

The development of language technologies based on deep learning and transformer models requires large volumes of data\. In particular, modern language models, which underpin a wide range of NLP applications, depend heavily on large text corpora, typically sourced from the web\. To assess the AI safety readiness of African languages, we began by examining their representation in large\-scale pretraining datasets, focusing on recent massively multilingual corpora such as MADLAD\-400\[[32](https://arxiv.org/html/2610.27269#bib.bib32)\]and FineWeb\-2\[[13](https://arxiv.org/html/2610.27269#bib.bib13)\], and Wikipedia\. These corpora are among the most widely used in the training of contemporary multilingual language models\. For each language, we extracted the number of tokens, sentences, documents, or articles, depending on what the dataset provided\. We then compared these amounts to English and other widely used languages\. This allowed us to gauge the extent of exposure each language receives during pretraining, a key factor influencing downstream model performance\.

### 3\.2Language Presence in Major AI Systems

To assess global inclusion efforts, we analyzed the presence of African languages in widely\-used AI systems such as Google Translate111https://translate\.google\.com/and Meta’s NLLB\[[64](https://arxiv.org/html/2610.27269#bib.bib64)\]\. These systems serve as proxies for practical deployment readiness and the extent to which a language is accessible via commercial or research\-grade models\. We documented whether the language is supported, and the performance it has\.

#### 3\.2\.1Model Support and Benchmarking

To understand how African languages perform in standard multilingual evaluation settings, we analyzed results reported in\[[65](https://arxiv.org/html/2610.27269#bib.bib65)\], which evaluated LLMs on benchmarks such as FLORES\-200\[[66](https://arxiv.org/html/2610.27269#bib.bib66)\]for machine translation, SIB\-200\[[30](https://arxiv.org/html/2610.27269#bib.bib30)\]for topic classification, and BELEBELE\[[67](https://arxiv.org/html/2610.27269#bib.bib67)\]for reading comprehension based question answering, covering over 100 languages\. We examined metrics such as chrF\+\+\[[68](https://arxiv.org/html/2610.27269#bib.bib68)\], BLEU\[[69](https://arxiv.org/html/2610.27269#bib.bib69)\], and accuracy\. We also compared the performance of African languages to languages from other world regions to contextualize their relative standing\. In addition, we used results from\[[70](https://arxiv.org/html/2610.27269#bib.bib70)\], which introduced AfroBench, a collection of benchmark tasks specifically focused on African languages to further analyze model performance across tasks such as sentiment analysis, named entity recognition, and question answering\. These comparisons allowed us to identify languages that are underperforming despite moderate representation and those that show strong results even with limited resources\.

Languages that achieve stronger performance on these benchmarks can be considered more ready for AI applications, as they are better supported by current models\. Benchmark performance serves as a proxy for model usability in real\-world tasks and reflects how well the language has been represented and learned during pretraining\.

### 3\.3Policy Landscape and Community\-Centered Analysis

#### 3\.3\.1Existence of National AI Strategies

In this report, we view the presence of a national AI policy as an indicator of potential for regulatory oversight, investment in local capacity, and the development of infrastructure needed to monitor and guide the deployment of AI technologies\. Consequently, languages spoken in countries with such policies may be better positioned for safe and responsible AI integration\. Hence, using online sources, we identified African languages spoken in at least one country with a national AI policy\.

#### 3\.3\.2Community Input on Real\-World Usage

We complemented our technical analysis with qualitative insights gathered through forms filled by native speakers and local technologists\. These community responses helped contextualize the technical findings, offering views on language use in digital spaces, AI\-related challenges, and perceived progress in AI inclusion\. Responses were thematically coded to identify recurring patterns and challenges\.

#### 3\.3\.3Insights from Related Research

A considerable body of research has been dedicated to assessing language resources and capabilities, with the aim of categorizing languages based on their level of support and identifying the efforts required to ensure equitable technological inclusion\. Notable examples of such work include\[[71](https://arxiv.org/html/2610.27269#bib.bib71),[72](https://arxiv.org/html/2610.27269#bib.bib72)\]\. This work defines what it means for a language to be low\-resource by carefully analyzing sociopolitical, resource\-related, and artifact\-based factors\. Even though we share good subset of factors to look at languages, the aim of this paper is compiling several factors to decide on which subset of languages to build our safety benchmark\.

### 3\.4Criteria for Language Inclusion

As an outcome of this work, we used the inputs from the sections above to classify languages into four readiness levels:fully ready,mostly ready,emerging, andfoundational support neededfor integration into AI safety research and benchmarks\. Based on our analysis, languages not included in FLORES\-200 are categorized asfoundational support needed, with the exception of Nigerian Pidgin, which, according to\[[16](https://arxiv.org/html/2610.27269#bib.bib16)\], is the most widely spoken language in Africa and a creole known for high cross\-lingual transfer\. As a result, the 57 African languages included in FLORES\-200, along with Nigerian Pidgin, were ranked into the remaining three readiness categories\.

\(a\)Wikipedia\(b\)FineWeb
Figure 3:Top 20 African languages in publicly available multilingual unlabeled corpora: Wikipedia and FineWeb\.

## 4Result and Analysis

Based on the described methodology, we proceed to answer a series of sub\-questions that collectively address the overall research question: which African languages are ready for AI safety?

### 4\.1What languages are represented in existing Corpora?

Our analysis of Wikipedia statistics222retrived on 06\.06\.2025 from[List of Wikipedias](https://en.wikipedia.org/wiki/List_of_Wikipedias)\.shows that of 341 active languages on Wikipedia, only 53 are African languages\. Furthermore, of the more than 65 million articles, just over 10% are in English, German accounts for 4\.6%, while all 53 African languages combined account for less than 5%\.

Aside from Wikipedia, there have been efforts to curate massive unannotated corpora for several languages, including African languages, from the entire web\. Recent datasets include mC4\[[31](https://arxiv.org/html/2610.27269#bib.bib31)\], Fineweb2\[[13](https://arxiv.org/html/2610.27269#bib.bib13)\], and MADLAD\[[32](https://arxiv.org/html/2610.27269#bib.bib32)\]\. Our analysis of the Fineweb2 dataset statistics provided by its authors, excluding English, shows that out of over 4 billion documents, 605 million \(13\.3%\) are in Russian, while 332 million \(7%\) are in French\. In contrast, African languages make up less than 0\.5% of the dataset\.

Table 1:Average text\-based evaluation results across languages by region, using Gemma \(3 27B\), Gemini\-2\-Flash, and GPT\-4o \(08/24\)\. Results obtained from\[[65](https://arxiv.org/html/2610.27269#bib.bib65)\][Figure3](https://arxiv.org/html/2610.27269#S3.F3)shows the top 20 African languages according to the number of Wikipedia articles and Fineweb2 documents\. Even among African languages, there is a skewed distribution of available web resources\. Arabic dialects \(Egyptian and Moroccan\) have significantly more content on Wikipedia and Fineweb2, respectively, than other languages\. Afrikaans consistently ranks second across both data sources, while Swahili, the most widely spoken indigenous African language, holds the third and fourth positions, respectively\.

### 4\.2How good are popular LLMs on existing benchmark?

In[Section4\.1](https://arxiv.org/html/2610.27269#S4.SS1), we showed that web\-based resources such as Wikipedia and FineWeb\-2 are skewed, with African languages significantly underrepresented\. In this section, we analyze and evaluate several existing LLMs with a focus on African languages\. We begin with a global perspective before narrowing our evaluation to concentrate specifically on African languages\.

#### 4\.2\.1On a global perspective

We analyze the multilingual evaluation results across two broad categories of tasks: natural language understanding \(NLU\) and natural language generation \(NLG\), using the results from\[[65](https://arxiv.org/html/2610.27269#bib.bib65)\]\. Specifically, we focus on language identification, topic classification, and reading comprehension\-based question answering for NLU, and machine translation for NLG\. All tasks are based on the massively multilingual FLORES\-200 dataset\[[66](https://arxiv.org/html/2610.27269#bib.bib66)\], which covers 200 languages\. FLORES\-200 was used to create SIB\-200\[[30](https://arxiv.org/html/2610.27269#bib.bib30)\]for topic classification, BELEBELE\[[67](https://arxiv.org/html/2610.27269#bib.bib67)\]for reading comprehension question answering \(available for 122 languages\), and also served as the benchmark for machine translation evaluation\.

The results, broken down by region and presented in[Table1](https://arxiv.org/html/2610.27269#S4.T1), compare three popular LLMs across these tasks\. On average, the findings show that African languages consistently achieve the lowest performance across all models\. A closer look at the individual tasks confirms this same trend\. We hypothesize that this underperformance is partly due to the underrepresentation of African languages during the pretraining and adaptation phases of these LLMs\.

Additionally, proprietary LLMs tend to outperform open\-source models\. While the reasons for this disparity are not entirely clear, it is possible that the companies behind proprietary models have access to higher\-quality or more extensive training data\.

Table 2:LLM performance on AfroBench\-Lite, covering 14 African languages across 7 tasks: intent classification \(Intent\), topic classification \(TC\), natural language inference \(NLI\), reading comprehension \(RC\), knowledge base question answering \(MMLU\), mathematical reasoning \(Math\), and machine translation \(MT\)\. Results are obtained from\[[70](https://arxiv.org/html/2610.27269#bib.bib70)\]\.\(a\)SIB\-200\(b\)MT
Figure 4:Correlation Between Data Size and Performance for African Languages\. \(Left\) Accuracy ratio on SIB\-200; \(Right\) chrF\+\+ ratio on FLORES\-200\. Both plots compare performance against the amount of pretraining data in Fineweb2\.
#### 4\.2\.2A focus on only African languages

In addition to the analysis in[Section4\.2\.1](https://arxiv.org/html/2610.27269#S4.SS2.SSS1), we evaluate the performance of several LLMs on AfroBench\-Lite\[[70](https://arxiv.org/html/2610.27269#bib.bib70)\], a benchmark that spans 7 tasks across 14 diverse African languages333English, Kiswahili, Kinyarwanda, Hausa, Amharic, isiXhosa, chiShona, isiZulu, Igbo, Yorùbá, Sesotho, Lingala, Oromo, Luganda, and Wolof\.\. The tasks include intent classification \(Intent\), topic classification \(TC\), natural language inference \(NLI\), reading comprehension \(RC\), knowledge base question answering \(MMLU\), mathematical reasoning \(Math\), and machine translation \(MT\)\. While there is some overlap between AfroBench\-Lite and the mSTEB benchmark, this analysis focuses specifically on African languages and a broader set of tasks\. Accordingly, in this section, we compare the performance of 9 LLMs using results reported for 7 models by\[[70](https://arxiv.org/html/2610.27269#bib.bib70)\]444[https://mcgill\-nlp\.github\.io/AfroBench/leaderboard\.html](https://mcgill-nlp.github.io/AfroBench/leaderboard.html)along with our own experiments on 2 additional models\. The results are presented in[Table2](https://arxiv.org/html/2610.27269#S4.T2)\.

The results show that Gemini 2\.0 and GPT\-4\.1 perform relatively well on both English and, on average, across African languages\. Across all evaluated models, performance on English is consistently higher than the average performance on African languages\. The translation task, particularly translating from English or French into African languages, emerges as notably more challenging\. This is expected, as machine translation is inherently more challenging, especially for low\-resource African languages, because it requires both the understanding and generation capabilities of language models, unlike the other six tasks\. Therefore, in our final language selection, we prioritize translation performance as a key indicator of readiness\.

A closer examination of individual language performance in Gemini 2\.0 reveals that Swahili, Zulu, Kinyarwanda, and Amharic are among the top\-performing African languages in the topic classification task, achieving results comparable to English\. In contrast, Wolof, Luganda, and Oromo exhibited the lowest performance\. As previously hypothesized, this disparity is likely due to representation bias: these models are primarily English\-centric, and languages with greater resource availability tend to perform better\.

For the machine translation task, Swahili again outperformed other languages by a notable margin\. However, Amharic showed a significant drop in performance, which we hypothesize may be due to tokenization challenges associated with its unique script\. Yoruba also demonstrated relatively low performance, similar to Wolof\. We attribute this to known issues with the FLORES dataset\[[73](https://arxiv.org/html/2610.27269#bib.bib73)\], which has been shown to extend to other datasets derived from it\[[74](https://arxiv.org/html/2610.27269#bib.bib74)\]\.

To further explore the role of representation bias, we evaluated the correlation between model performance and language resource availability\. Specifically, we compared the proportion of FineWeb resources relative to English with performance on topic classification and machine translation tasks using GPT\-4o, across the 14 languages in AfroBench\-Lite\. The results, shown in[Figure4](https://arxiv.org/html/2610.27269#S4.F4), provide partial support for the hypothesis\. Swahili which is the most resourced language among the 14, consistently achieves the highest performance\. Mid\-resource languages such as Zulu and Hausa follow, while low\-resource languages like Wolof show the weakest results\. Given the limited sample size, we computed Spearman’s rank correlation between relative performance \(compared to English\) and FineWeb2 language size\. Although the results suggest a positive trend \(0\.85 for SIB and 0\.64 for MT\), we interpret them with caution due to the small number of data points\.

Figure[5](https://arxiv.org/html/2610.27269#S4.F5), visualizes the language understanding capability of an LLM by measuring its refusal rate on the FLORES task\. To assess this, we prompted the LLM in various languages and used automatic language identification to verify if the response was in the same language as the input\. A high accuracy indicates that the model correctly understood and responded in the intended language, while a low score suggests that the model either refused, responded in another language \(typically English\), or misunderstood the prompt\. This method serves as a practical proxy for identifying whether the LLM can meaningfully engage with a given language\. Our findings reveal that while the LLM accurately responds in high\-resource languages like French, Portuguese, and Swahili, it frequently fails or defaults to English for many low\-resource African languages such as Fon, Sotho, Sswati, Tigrinya, Oromo, and Bambara, indicating substantial gaps in the model’s multilingual capabilities, especially for languages with limited pretraining exposure or benchmark representation\.

Figure 5:Variation in language understanding accuracy of GPT\-4o when prompted in different African languages from the FLORES\-200 benchmark\. Higher scores indicate successful comprehension and response in the correct language, while lower scores highlight failure modes such as refusal or defaulting to English\. The results underscore performance disparities linked to resource availability and benchmark representation\.

### 4\.3Performance and Language Presence in Major AI Systems

As shown in Figure[6](https://arxiv.org/html/2610.27269#S4.F6), we present a visual comparison of the translation performance of NLLB and Google Translate across several target languages, plotted against the amount of data available from the Fineweb2 dataset\. This comparison allows us to assess whether major AI systems accurately and reliably represent these languages in machine translation tasks\. Initial observations from the plot suggest a general correlation: languages with larger character\-level datasets tend to yield higher translation quality scores\.

The under\-representation of certain languages in major AI systems has significant downstream implications\. In particular, it hinders equitable access to information and knowledge for speakers of these languages\. With regards to safety the first approach of making systems safe is by leveraging machine translated dataset for development or testing\. so the fact that the MT scores are low also makes this languages vulnerable to safety and performance issues that are not discovered while using highly resource languages\.

### 4\.4Which languages are spoken in countries with a national AI policy?

According to\[[75](https://arxiv.org/html/2610.27269#bib.bib75)\], only 12 out of the 54 African countries, just 20% have a national AI policy\. These countries include Tunisia, Algeria, and Egypt inNorth Africa; Benin, Ghana, Nigeria, and Senegal inWest Africa; Ethiopia, Mauritius, and Rwanda inEast Africa; and South Africa and Zambia inSouthern Africa\. Meanwhile, Kenya and Morocco are currently formulating their own policies, with Kenya’s draft already available for public review\.

[Figure6\(a\)](https://arxiv.org/html/2610.27269#S4.F6.sf1)shows a colored map of Africa highlighting countries with national AI policies and those with policies in progress\. This reflects a skewed distribution of efforts, indicating that much more remains to be done\. However, of the 57 African languages in FLORES, 41 are spoken in at least one country with a national AI policy, a few in countries with policies in progress, and some in countries with no policy at all, such as Kabuverdianu, Ganda, Kikongo, and Tamasheq\.

\(a\)Countries with national AI policy\(b\)Presence in major AI systems \(Chrf2\+\+ vs Characters\)
Figure 6:Overview of African AI Policy and Language Translation Service Presence\.\(a\) Map showing the status of national AI policies across African countries, categorized asYes,Ongoing, andNo\. \(b\) Scatter plot illustrating the performance of NLLB and Google Translate scores against language character counts \(log scale\), color\-coded by language \(BCP\-47\)\.
### 4\.5Community Input observation

To better understand the real\-world usage and community perspectives on AI readiness across African countries, we designed and distributed a structured Google Forms questionnaire\. The form included both multiple\-choice and open\-ended questions targeting language accessibility, tool availability, and national AI infrastructure\. Responses collected from diverse participants across several countries provided valuable insight into the current state of digital inclusion and AI engagement in the region\.

From the submissions, we observed that access to digital content in native African languages remains limited\. Most respondents found it difficult to locate online resources in their local languages, with only a few describing the experience as “somewhat easy\.” Translation and language processing tools were noted to be scarce or underperforming\. Google Translate was the most commonly referenced tool, but its effectiveness was often questioned\. Community\-led efforts such as Nkowaokwu for Igbo, the Academy of Ewe Language and Masakhane Community show promise but are still in early stages and not widely integrated into AI applications\.

Additionally, the survey revealed a general lack of awareness or infrastructure surrounding AI development at the national level\. Most participants were unsure or unaware of their country’s AI strategies, government investments, or regulatory frameworks\. Government support was generally viewed as minimal or unclear\. While a few respondents mentioned the existence of university departments or research centers, very few could cite active collaborations between academia and industry\. These findings highlight an important gap between grassroots interest and institutional readiness, underscoring the need for more coordinated, inclusive AI development efforts across the continent\.

## 5Recommendations and Conclusion

The objective of this report is to identify African languages that are strong candidates for inclusion in AI safety benchmarks, highlight the types of resources currently missing to support their readiness, and outline the gaps and efforts needed to prepare these languages for meaningful participation in AI safety discussions\. In this report, we have assessed the status of these languages based on resource availability, the performance of LLMs, using English as a baseline for comparison, and the presence of national AI policies in the countries where these languages are spoken\.

Based on our analyses and findings, we classified languages into four readiness categories:fully ready,mostly ready,emerging, andfoundational support needed\. Languages in thefoundational support neededcategory include all those not represented in FLORES\-200, except the Nigerian Pidgin\. Although some of these languages may appear in large multilingual corpora, the lack of standardized evaluation benchmarks to assess their performance relative to other languages supports their placement in this category\.

Accordingly, the 58 African languages including the 57 in FLORES\-200 were divided into the three remaining categories: we recommend 15 languages asfully ready, 10 languages asmostly ready, while the remaining 33 are categorized asemerging\. We compared their presence in Fineweb2, Gemini’s performance on translation tasks from English or French to these languages \(using FLORES\-200\), the speaking population, and the existence of a national AI policy in countries where the languages are spoken\.[Table3](https://arxiv.org/html/2610.27269#S5.T3)presents the final recommendations\.

Table 3:Categorization of African Languages for AI Safety\. the languages are arranged in no particular order\.Aside from the above language categorization and recommendation, we highlight below some activities and effort required for AI safety readiness in African languages\.

1. 1\.Strategic Policy Action for Language\-Inclusive AIGovernments and policymakers should develop comprehensive national AI policies that explicitly prioritize language inclusion\. They should allocate targeted resources, provide grants, and create incentives to support local talent working on language technologies\. Their leadership is crucial to ensure that local and underrepresented languages are not left behind in AI advancements and that linguistic diversity is embedded in the nation’s digital future\.
2. 2\.Community\-Driven Language InclusionLocal communities and language speakers should take stewardship of collecting clean, verified monolingual data that embodies their cultural values, knowledge systems, and social norms\. Their role extends beyond data collection, they are also key advocates for raising awareness about the importance of language representation in AI\. This is vital not only for preserving linguistic identity, but also for empowering their communities to participate meaningfully in the AI\-driven economy\.
3. 3\.Ensuring AI Safety Through Inclusive ResearchResearch and Academic Institutions should play a leading role in ensuring AI safety by creating quality evaluation datasets for African languages\. These datasets are essential for assessing how large language models \(LLMs\) perform across different linguistic and cultural contexts\. By identifying biases, inaccuracies, and performance gaps, researchers can help prevent harmful outputs and ensure that AI systems behave reliably and ethically for all language communities—not just those with abundant data resources\.

## Acknowledgments

This project was supported in MLCommons through a grant with Masakhane Research Foundation\.

## References

- \[1\]Libo Qin, Qiguang Chen, Xiachong Feng, Yang Wu, Yongheng Zhang, Yinghui Li, Min Li, Wanxiang Che, and Philip S Yu\.Large language models meet nlp: A survey\.Frontiers of Computer Science, 20\(11\):2095–2236, 2026\.
- \[2\]Humza Naveed, Asad Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Saeed Mian\.A comprehensive overview of large language models\.ACM Transactions on Intelligent Systems and Technology, 16:1–72, 2025\.
- \[3\]Lyle Campbell and Verónica Grondona\.Ethnologue: Languages of the world\.Language, 84\(3\):636–641, 2008\.
- \[4\]Michael A\. Hedderich, Lukas Lange, Heike Adel, Jannik Strötgen, and Dietrich Klakow\.A survey on recent approaches for natural language processing in low\-resource scenarios\.In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani\-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou, editors,Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2545–2568, Online, June 2021\. Association for Computational Linguistics\.
- \[5\]Shijie Wu and Mark Dredze\.Are all languages created equal in multilingual BERT?In Spandana Gella, Johannes Welbl, Marek Rei, Fabio Petroni, Patrick Lewis, Emma Strubell, Minjoon Seo, and Hannaneh Hajishirzi, editors,Proceedings of the 5th Workshop on Representation Learning for NLP, pages 120–130, Online, July 2020\. Association for Computational Linguistics\.
- \[6\]Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov\.Unsupervised cross\-lingual representation learning at scale\.In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors,Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440–8451, Online, July 2020\. Association for Computational Linguistics\.
- \[7\]Amirkeivan Mohtashami, Mauro Verzetti, and Paul K Rubenstein\.Learning translation quality evaluation on low resource languages from large language models\.arXiv preprint arXiv:2302\.03491, 2023\.
- \[8\]Júlia Falcão, Claudia Borg, Nora Aranberri, and Kurt Abela\.COMET for low\-resource machine translation evaluation: A case study of English\-Maltese and Spanish\-Basque\.In Nicoletta Calzolari, Min\-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue, editors,Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\), pages 3553–3565, Torino, Italia, May 2024\. ELRA and ICCL\.
- \[9\]Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed\.Bias and fairness in large language models: A survey\.Computational Linguistics, 50\(3\):1097–1179, 2024\.
- \[10\]Li\-Bo Qin, Qiguang Chen, Yu\-Hang Zhou, Zhi Chen, Ying\-Hui Li, Lizi Liao, Min Li, Wan\-Xiang Che, and Philip S\. Yu\.A survey of multilingual large language models\.Patterns, 6, 2025\.
- \[11\]Emily M\. Bender, Timnit Gebru, Angelina McMillan\-Major, and Shmargaret Shmitchell\.On the dangers of stochastic parrots: Can language models be too big?Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021\.
- \[12\]Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po\-Sen Huang, John F\. J\. Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sande Minnich Brown, Zachary Kenton, William T\. Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William S\. Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and Iason Gabriel\.Taxonomy of risks posed by language models\.Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022\.
- \[13\]Guilherme Penedo, Hynek Kydlíček, Vinko Sabolčec, Bettina Messmer, Negar Foroutan, Amir Hossein Kargaran, Colin Raffel, Martin Jaggi, Leandro Von Werra, and Thomas Wolf\.Fineweb2: One pipeline to scale them all — adapting pre\-training data processing to every language\.InSecond Conference on Language Modeling, 2025\.
- \[14\]Wenxuan Wang, Zhaopeng Tu, Chang Chen, Youliang Yuan, Jen\-tse Huang, Wenxiang Jiao, and Michael Lyu\.All languages matter: On the multilingual safety of LLMs\.In Lun\-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguistics: ACL 2024, pages 5865–5877, Bangkok, Thailand, August 2024\. Association for Computational Linguistics\.
- \[15\]Zheng Xin Yong, Beyza Ermis, Marzieh Fadaee, Stephen Bach, and Julia Kreutzer\.The state of multilingual LLM safety research: From measuring the language gap to mitigating it\.In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 15845–15860, Suzhou, China, November 2025\. Association for Computational Linguistics\.
- \[16\]David M\. Eberhard, Gary F\. Simons, and Charles D\. Fennig, editors\.Ethnologue: Languages of the World\.SIL International, Dallas, Texas, 27 edition, 2025\.Accessed June 22, 2025\.
- \[17\]H Hammarström, R Forkel, and M Haspelmath\.Bank, s\. glottolog/glottolog: Glottolog database 5\.1\.Zenodo https://doi\.org/10\.5281/zenodo, 10804357, 2024\.
- \[18\]Jesujoba Oluwadara Alabi, Michael A\. Hedderich, David Ifeoluwa Adelani, and Dietrich Klakow\.Charting the landscape of African NLP: Mapping progress and shaping the road ahead\.In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27807–27841, Suzhou, China, November 2025\. Association for Computational Linguistics\.
- \[19\]David Ifeoluwa Adelani, Jesujoba Oluwadara Alabi, Angela Fan, Julia Kreutzer, Xiaoyu Shen, Machel Reid, Dana Ruiter, Dietrich Klakow, Peter Nabende, Ernie Chang, Tajuddeen Gwadabe, Freshia Sackey, Bonaventure F\. P\. Dossou, Chris Emezue, Colin Leong, Michael Beukman, Shamsuddeen H\. Muhammad, Guyo D\. Jarso, Oreen Yousuf, Andre N\. Niyongabo Rubungo, Gilles Hacheme, Eric Peter Wairagala, Muhammad Umair Nasir, Benjamin A\. Ajibade, Tunde Oluwaseyi Ajayi, Yvonne Wambui Gitau, Jade Abbott, Mohamed Ahmed, Millicent Ochieng, Anuoluwapo Aremu, Perez Ogayo, Jonathan Mukiibi, Fatoumata Ouoba Kabore, Godson Koffi Kalipe, Derguene Mbaye, Allahsera Auguste Tapo, Victoire M\. Memdjokam Koagne, Edwin Munkoh\-Buabeng, Valencia Wagner, Idris Abdulmumin, Ayodele Awokoya, Happy Buzaaba, Blessing Sibanda, Andiswa Bukula, and Sam Manthalu\.A few thousand translations go a long way\! leveraging pre\-trained models for African news translation\.In Marine Carpuat, Marie\-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz, editors,Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3053–3070, Seattle, United States, July 2022\. Association for Computational Linguistics\.
- \[20\]Jesujoba Oluwadara Alabi, Israel Abebe Azime, Miaoran Zhang, Cristina España\-Bonet, Rachel Bawden, Dawei Zhu, David Ifeoluwa Adelani, Clement Oyeleke Odoje, Idris Akinade, Iffat Maab, Davis David, Shamsuddeen Hassan Muhammad, Neo Putini, David O\. Ademuyiwa, Andrew Caines, and Dietrich Klakow\.AFRIDOC\-MT: Document\-level MT corpus for African languages\.In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27770–27806, Suzhou, China, November 2025\. Association for Computational Linguistics\.
- \[21\]Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, Nedjma Ousidhoum, David Ifeoluwa Adelani, Seid Muhie Yimam, Ibrahim Sa’id Ahmad, Meriem Beloucif, Saif M\. Mohammad, Sebastian Ruder, Oumaima Hourrane, Pavel Brazdil, Alipio Jorge, Felermino Dário Mário António Ali, Davis David, Salomey Osei, Bello Shehu Bello, Falalu Ibrahim, Tajuddeen Gwadabe, Samuel Rutunda, Tadesse Belay, Wendimu Baye Messelle, Hailu Beshada Balcha, Sisay Adugna Chala, Hagos Tesfahun Gebremichael, Bernard Opoku, and Stephen Arthur\.AfriSenti: A Twitter sentiment analysis benchmark for African languages\.In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13968–13981, Singapore, December 2023\. Association for Computational Linguistics\.
- \[22\]Shamsuddeen Hassan Muhammad, David Ifeoluwa Adelani, Sebastian Ruder, Ibrahim Sa’id Ahmad, Idris Abdulmumin, Bello Shehu Bello, Monojit Choudhury, Chris Chinenye Emezue, Saheed Salahudeen Abdullahi, Anuoluwapo Aremu, Alípio Jorge, and Pavel Brazdil\.NaijaSenti: A Nigerian Twitter sentiment corpus for multilingual sentiment analysis\.In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, and Stelios Piperidis, editors,Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 590–602, Marseille, France, June 2022\. European Language Resources Association\.
- \[23\]David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D’souza, Julia Kreutzer, Constantine Lignos, Chester Palen\-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsuddeen H\. Muhammad, Chris Chinenye Emezue, Joyce Nakatumba\-Nabende, Perez Ogayo, Aremu Anuoluwapo, Catherine Gitau, Derguene Mbaye, Jesujoba Alabi, Seid Muhie Yimam, Tajuddeen Rabiu Gwadabe, Ignatius Ezeani, Rubungo Andre Niyongabo, Jonathan Mukiibi, Verrah Otiende, Iroro Orife, Davis David, Samba Ngom, Tosin Adewumi, Paul Rayson, Mofetoluwa Adeyemi, Gerald Muriuki, Emmanuel Anebi, Chiamaka Chukwuneke, Nkiruka Odu, Eric Peter Wairagala, Samuel Oyerinde, Clemencia Siro, Tobius Saul Bateesa, Temilola Oloyede, Yvonne Wambui, Victor Akinode, Deborah Nabagereka, Maurice Katusiime, Ayodele Awokoya, Mouhamadane MBOUP, Dibora Gebreyohannes, Henok Tilaye, Kelechi Nwaike, Degaga Wolde, Abdoulaye Faye, Blessing Sibanda, Orevaoghene Ahia, Bonaventure F\. P\. Dossou, Kelechi Ogueji, Thierno Ibrahima DIOP, Abdoulaye Diallo, Adewale Akinfaderin, Tendai Marengereke, and Salomey Osei\.MasakhaNER: Named entity recognition for African languages\.Transactions of the Association for Computational Linguistics, 9:1116–1131, 2021\.
- \[24\]David Ifeoluwa Adelani, Graham Neubig, Sebastian Ruder, Shruti Rijhwani, Michael Beukman, Chester Palen\-Michel, Constantine Lignos, Jesujoba O\. Alabi, Shamsuddeen H\. Muhammad, Peter Nabende, Cheikh M\. Bamba Dione, Andiswa Bukula, Rooweither Mabuya, Bonaventure F\. P\. Dossou, Blessing Sibanda, Happy Buzaaba, Jonathan Mukiibi, Godson Kalipe, Derguene Mbaye, Amelia Taylor, Fatoumata Kabore, Chris Chinenye Emezue, Anuoluwapo Aremu, Perez Ogayo, Catherine Gitau, Edwin Munkoh\-Buabeng, Victoire Memdjokam Koagne, Allahsera Auguste Tapo, Tebogo Macucwa, Vukosi Marivate, Elvis Mboning, Tajuddeen Gwadabe, Tosin Adewumi, Orevaoghene Ahia, Joyce Nakatumba\-Nabende, Neo L\. Mokono, Ignatius Ezeani, Chiamaka Chukwuneke, Mofetoluwa Adeyemi, Gilles Q\. Hacheme, Idris Abdulmumin, Odunayo Ogundepo, Oreen Yousuf, Tatiana Moteu Ngoli, and Dietrich Klakow\.MasakhaNER 2\.0: Africa\-centric transfer learning for named entity recognition\.In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors,Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4488–4508, Abu Dhabi, United Arab Emirates, December 2022\. Association for Computational Linguistics\.
- \[25\]Odunayo Ogundepo, Tajuddeen R\. Gwadabe, Clara E\. Rivera, Jonathan H\. Clark, Sebastian Ruder, David Ifeoluwa Adelani, Bonaventure F\. P\. Dossou, Abdou Aziz Diop, Claytone Sikasote, Gilles Hacheme, Happy Buzaaba, Ignatius Ezeani, Rooweither Mabuya, Salomey Osei, Chris Emezue, Albert Njoroge Kahira, Shamsuddeen Hassan Muhammad, Akintunde Oladipo, Abraham Toluwase Owodunni, Atnafu Lambebo Tonja, Iyanuoluwa Shode, Akari Asai, Tunde Oluwaseyi Ajayi, Clemencia Siro, Steven Arthur, Mofetoluwa Adeyemi, Orevaoghene Ahia, Anuoluwapo Aremu, Oyinkansola Awosan, Chiamaka Chukwuneke, Bernard Opoku, Awokoya Ayodele, Verrah Otiende, Christine Mwase, Boyd Sinkala, Andre Niyongabo Rubungo, Daniel A\. Ajisafe, Emeka Felix Onwuegbuzia, Habib Mbow, Emile Niyomutabazi, Eunice Mukonde, Falalu Ibrahim Lawan, Ibrahim Said Ahmad, Jesujoba O\. Alabi, Martin Namukombo, Mbonu Chinedu, Mofya Phiri, Neo Putini, Ndumiso Mngoma, Priscilla A\. Amouk, Ruqayya Nasir Iro, and Sonia Adhiambo\.AfriQA: Cross\-lingual open\-retrieval question answering for African languages\.In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14957–14972, Singapore, December 2023\. Association for Computational Linguistics\.
- \[26\]David Ifeoluwa Adelani, Dana Ruiter, Jesujoba O\. Alabi, Damilola Adebonojo, Adesina Ayeni, Mofe Adeyemi, Ayodele Esther Awokoya, and Cristina España\-Bonet\.The effect of domain and diacritics in Yoruba–English neural machine translation\.In Kevin Duh and Francisco Guzmán, editors,Proceedings of Machine Translation Summit XVIII: Research Track, pages 61–75, Virtual, August 2021\. Association for Machine Translation in the Americas\.
- \[27\]Barack Wamkaya Wanjawa, Lilian D\. A\. Wanzare, Florence Indede, Owen McOnyango, Lawrence Muchemi, and Edward Ombui\.Kenswquad—a question answering dataset for swahili low\-resource language\.ACM Transactions on Asian and Low\-Resource Language Information Processing, 22:1 – 20, 2022\.
- \[28\]Tilahun Abedissa Taffa, Ricardo Usbeck, and Yaregal Assabie\.Low resource question answering: An Amharic benchmarking dataset\.In Rooweither Mabuya, Muzi Matfunjwa, Mmasibidi Setaka, and Menno van Zaanen, editors,Proceedings of the Fifth Workshop on Resources for African Indigenous Languages @ LREC\-COLING 2024, pages 124–132, Torino, Italia, May 2024\. ELRA and ICCL\.
- \[29\]David Ifeoluwa Adelani, Jessica Ojo, Israel Abebe Azime, Jian Yun Zhuang, Jesujoba Oluwadara Alabi, Xuanli He, Millicent Ochieng, Sara Hooker, Andiswa Bukula, En\-Shiun Annie Lee, Chiamaka Ijeoma Chukwuneke, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson Koffi Kalipe, Jonathan Mukiibi, Salomon Kabongo Kabenamualu, Foutse Yuehgoh, Mmasibidi Setaka, Lolwethu Ndolela, Nkiruka Odu, Rooweither Mabuya, Salomey Osei, Shamsuddeen Hassan Muhammad, Sokhar Samb, Tadesse Kebede Guge, Tombekai Vangoni Sherman, and Pontus Stenetorp\.IrokoBench: A new benchmark for African languages in the age of large language models\.In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\), pages 2732–2757, Albuquerque, New Mexico, April 2025\. Association for Computational Linguistics\.
- \[30\]David Ifeoluwa Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba O\. Alabi, Yanke Mao, Haonan Gao, and En\-Shiun Annie Lee\.SIB\-200: A simple, inclusive, and big evaluation dataset for topic classification in 200\+ languages and dialects\.In Yvette Graham and Matthew Purver, editors,Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\), pages 226–245, St\. Julian’s, Malta, March 2024\. Association for Computational Linguistics\.
- \[31\]Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al\-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel\.mT5: A massively multilingual pre\-trained text\-to\-text transformer\.In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani\-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou, editors,Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483–498, Online, June 2021\. Association for Computational Linguistics\.
- \[32\]Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Christopher A\. Choquette\-Choo, Katherine Lee, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat\.Madlad\-400: A multilingual and document\-level large audited dataset, 2023\.
- \[33\]Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov\.Unsupervised cross\-lingual representation learning at scale\.In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors,Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440–8451, Online, July 2020\. Association for Computational Linguistics\.
- \[34\]Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, and Xian Li\.Few\-shot learning with multilingual generative language models\.In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors,Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9019–9052, Abu Dhabi, United Arab Emirates, December 2022\. Association for Computational Linguistics\.
- \[35\]Ashish Vaswani, Noam M\. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N\. Gomez, Lukasz Kaiser, and Illia Polosukhin\.Attention is all you need\.InNeural Information Processing Systems, 2017\.
- \[36\]Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al\-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony S\. Hartshorn, and et al\.The llama 3 herd of models\.ArXiv, abs/2407\.21783, 2024\.
- \[37\]Tom B\. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert\-Voss, Gretchen Krueger, T\. J\. Henighan, Rewon Child, Aditya Ramesh, Daniel M\. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Ma teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei\.Language models are few\-shot learners\.ArXiv, abs/2005\.14165, 2020\.
- \[38\]Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu\.Exploring the limits of transfer learning with a unified text\-to\-text transformer\.Journal of machine learning research, 21\(140\):1–67, 2020\.
- \[39\]Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H\. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus\.Emergent abilities of large language models\.Transactions on Machine Learning Research, 2022\.Survey Certification\.
- \[40\]OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, and et al\.Gpt\-4 technical report\.2023\.
- \[41\]Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al\.Gemma 3 technical report\.arXiv preprint arXiv:2503\.19786, 2025\.
- \[42\]Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al\.Gemini 1\.5: Unlocking multimodal understanding across millions of tokens of context\.arXiv preprint arXiv:2403\.05530, 2024\.
- \[43\]Kelechi Ogueji, Yuxin Zhu, and Jimmy Lin\.Small data? no problem\! exploring the viability of pretrained multilingual language models for low\-resourced languages\.In Duygu Ataman, Alexandra Birch, Alexis Conneau, Orhan Firat, Sebastian Ruder, and Gozde Gul Sahin, editors,Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 116–126, Punta Cana, Dominican Republic, November 2021\. Association for Computational Linguistics\.
- \[44\]Akintunde Oladipo\.Scaling pre\-training data and language models for african languages\.2024\.
- \[45\]Odunayo Jude Ogundepo, Akintunde Oladipo, Mofetoluwa Adeyemi, Kelechi Ogueji, and Jimmy Lin\.AfriTeVA: Extending ?small data? pretraining approaches to sequence\-to\-sequence models\.In Colin Cherry, Angela Fan, George Foster, Gholamreza \(Reza\) Haffari, Shahram Khadivi, Nanyun \(Violet\) Peng, Xiang Ren, Ehsan Shareghi, and Swabha Swayamdipta, editors,Proceedings of the Third Workshop on Deep Learning for Low\-Resource Natural Language Processing, pages 126–135, Hybrid, July 2022\. Association for Computational Linguistics\.
- \[46\]Akintunde Oladipo, Mofetoluwa Adeyemi, Orevaoghene Ahia, Abraham Toluwalase Owodunni, Odunayo Ogundepo, David Ifeoluwa Adelani, and Jimmy Lin\.Better quality pre\-training data and t5 models for African languages\.In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 158–168, Singapore, December 2023\. Association for Computational Linguistics\.
- \[47\]Atnafu Lambebo Tonja, Israel Abebe Azime, Tadesse Destaw Belay, Mesay Gemeda Yigezu, Moges Ahmed Ah Mehamed, Abinew Ali Ayele, Ebrahim Chekol Jibril, Michael Melese Woldeyohannis, Olga Kolesnikova, Philipp Slusallek, Dietrich Klakow, and Seid Muhie Yimam\.EthioLLM: Multilingual large language models for Ethiopian languages with task evaluation\.In Nicoletta Calzolari, Min\-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue, editors,Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\), pages 6341–6352, Torino, Italia, May 2024\. ELRA and ICCL\.
- \[48\]Ife Adebara, AbdelRahim Elmadany, and Muhammad Abdul\-Mageed\.Cheetah: Natural language generation for 517 African languages\.In Lun\-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), pages 12798–12823, Bangkok, Thailand, August 2024\. Association for Computational Linguistics\.
- \[49\]Akindele Michael Olawole, Jesujoba Oluwadara Alabi, Aderonke Busayo Sakpere, and David Ifeoluwa Adelani\.YAD: Leveraging t5 for improved automatic diacritization of yorùbá text\.In5th Workshop on African Natural Language Processing, 2024\.
- \[50\]Atnafu Lambebo Tonja, Bonaventure F\. P\. Dossou, Jessica Ojo, Jenalea Rajab, Fadel Thior, Eric Peter Wairagala, Anuoluwapo Aremu, Pelonomi Moiloa, Jade Abbott, Vukosi Marivate, and Benjamin Rosman\.Inkubalm: A small language model for low\-resource african languages, 2024\.
- \[51\]Happy Buzaaba, Alexander Wettig, David Ifeoluwa Adelani, and Christiane Fellbaum\.Lugha\-llama: Adapting large language models for african languages\.ArXiv, abs/2504\.06536, 2025\.
- \[52\]Kosei Uemura, Mahe Chen, Alex Pejovic, Chika Maduabuchi, Yifei Sun, and En\-Shiun Annie Lee\.AfriInstruct: Instruction tuning of African languages for diverse tasks\.In Yaser Al\-Onaizan, Mohit Bansal, and Yun\-Nung Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13571–13585, Miami, Florida, USA, November 2024\. Association for Computational Linguistics\.
- \[53\]Michael Andersland\.Amharic llama and llava: Multimodal llms for low resource languages, 2024\.
- \[54\]Israel Abebe Azime, Atnafu Lambebo Tonja, Tadesse Destaw Belay, Mitiku Yohannes Fuge, Aman Kassahun Wassie, Eyasu Shiferaw Jada, Yonas Chanie, Walelign Tewabe Sewunetie, and Seid Muhie Yimam\.Walia\-LLM: Enhancing Amharic\-LLaMA by integrating task\-specific and generative datasets\.In Yaser Al\-Onaizan, Mohit Bansal, and Yun\-Nung Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024, pages 432–444, Miami, Florida, USA, November 2024\. Association for Computational Linguistics\.
- \[55\]Jesujoba O\. Alabi, David Ifeoluwa Adelani, Marius Mosbach, and Dietrich Klakow\.Adapting pre\-trained language models to African languages via multilingual adaptive fine\-tuning\.InProceedings of the 29th International Conference on Computational Linguistics, pages 4336–4349, Gyeongju, Republic of Korea, October 2022\. International Committee on Computational Linguistics\.
- \[56\]Tadesse Destaw Belay, Israel Abebe Azime, Ibrahim Said Ahmad, David Ifeoluwa Adelani, Idris Abdulmumin, Abinew Ali Ayele, Shamsuddeen Hassan Muhammad, and Seid Muhie Yimam\.Afroxlmr\-social: Adapting pre\-trained language models for african languages social media text, 2025\.
- \[57\]Francois Meyer, Haiyue Song, Abhisek Chakrabarty, Jan Buys, Raj Dabre, and Hideki Tanaka\.NGLUEni: Benchmarking and adapting pretrained language models for nguni languages\.In Nicoletta Calzolari, Min\-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue, editors,Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\), pages 12247–12258, Torino, Italia, May 2024\. ELRA and ICCL\.
- \[58\]Antoine Nzeyimana and Andre Niyongabo Rubungo\.KinyaBERT: a morphology\-aware Kinyarwanda language model\.In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors,Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), pages 5347–5363, Dublin, Ireland, May 2022\. Association for Computational Linguistics\.
- \[59\]Vukosi Marivate, Moseli Mots’Oehli, Valencia Wagner, Richard Lastrucci, and Isheanesu Dzingirai\.Puoberta: Training and evaluation of a curated language model for setswana\.InArtificial Intelligence Research\. SACAIR 2023\. Communications in Computer and Information Science, 2023\.
- \[60\]David Adelani\.oyo\-bert\-base \(revision f9d07fb\), 2023\.
- \[61\]Gati Martin, Medard Edmund Mswahili, Young\-Seob Jeong, and Jiyoung Woo\.SwahBERT: Language model of Swahili\.In Marine Carpuat, Marie\-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz, editors,Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 303–313, Seattle, United States, July 2022\. Association for Computational Linguistics\.
- \[62\]Faisal Qarah\.Egybert: A large language model pretrained on egyptian dialect corpora\.arXiv preprint arXiv:2408\.03524, 2024\.
- \[63\]Chiamaka Chukwuneke, Ignatius Ezeani, Paul Rayson, and Mahmoud El\-Haj\.IgboBERT models: Building and training transformer models for the Igbo language\.In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, and Stelios Piperidis, editors,Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5114–5122, Marseille, France, June 2022\. European Language Resources Association\.
- \[64\]Marta R Costa\-Jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al\.No language left behind: Scaling human\-centered machine translation\.arXiv preprint arXiv:2207\.04672, 2022\.
- \[65\]Luel Hagos Beyene, Vivek Singh Verma, Min Ma, Jesujoba Oluwadara Alabi, Fabian David Schmidt, Joyce Nakatumba\-Nabende, and David Ifeoluwa Adelani\.msteb: Massively multilingual evaluation of llms on speech and text tasks\.2025 IEEE Automatic Speech Recognition and Understanding Workshop \(ASRU\), pages 1–8, 2025\.
- \[66\]NLLB Team, Marta R\. Costa\-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang\.No language left behind: Scaling human\-centered machine translation\.2022\.
- \[67\]Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa\.The belebele benchmark: a parallel reading comprehension dataset in 122 language variants\.In Lun\-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), pages 749–775, Bangkok, Thailand, August 2024\. Association for Computational Linguistics\.
- \[68\]Maja Popović\.chrF: character n\-gram F\-score for automatic MT evaluation\.In Ondřej Bojar, Rajan Chatterjee, Christian Federmann, Barry Haddow, Chris Hokamp, Matthias Huck, Varvara Logacheva, and Pavel Pecina, editors,Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal, September 2015\. Association for Computational Linguistics\.
- \[69\]Kishore Papineni, Salim Roukos, Todd Ward, and Wei\-Jing Zhu\.Bleu: a method for automatic evaluation of machine translation\.In Pierre Isabelle, Eugene Charniak, and Dekang Lin, editors,Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA, July 2002\. Association for Computational Linguistics\.
- \[70\]Jessica Ojo, Odunayo Ogundepo, Akintunde Oladipo, Kelechi Ogueji, Jimmy Lin, Pontus Stenetorp, and David Ifeoluwa Adelani\.AfroBench: How good are large language models on African languages?In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Findings of the Association for Computational Linguistics: ACL 2025, pages 19048–19095, Vienna, Austria, July 2025\. Association for Computational Linguistics\.
- \[71\]Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury\.The state and fate of linguistic diversity and inclusion in the NLP world\.In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors,Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online, July 2020\. Association for Computational Linguistics\.
- \[72\]Hellina Hailu Nigatu, Atnafu Lambebo Tonja, Benjamin Rosman, Thamar Solorio, and Monojit Choudhury\.The Zeno’s paradox of ‘low\-resource’ languages\.In Yaser Al\-Onaizan, Mohit Bansal, and Yun\-Nung Chen, editors,Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17753–17774, Miami, Florida, USA, November 2024\. Association for Computational Linguistics\.
- \[73\]Idris Abdulmumin, Sthembiso Mkhwanazi, Mahlatse Mbooi, Shamsuddeen Hassan Muhammad, Ibrahim Said Ahmad, Neo Putini, Miehleketo Mathebula, Matimba Shingange, Tajuddeen Gwadabe, and Vukosi Marivate\.Correcting FLORES evaluation dataset for four African languages\.In Barry Haddow, Tom Kocmi, Philipp Koehn, and Christof Monz, editors,Proceedings of the Ninth Conference on Machine Translation, pages 570–578, Miami, Florida, USA, November 2024\. Association for Computational Linguistics\.
- \[74\]Jesujoba O\. Alabi, Xuechen Liu, Dietrich Klakow, and Junichi Yamagishi\.AfriHuBERT: A self\-supervised speech representation model for African languages\.InInterspeech 2025, pages 4023–4027, 2025\.
- \[75\]Lawyers Hub\.Africa ai privacy report 2025\.[https://www\.aipolicy\.africa/pdfs/report\.pdf](https://www.aipolicy.africa/pdfs/report.pdf), 2025\.Africa AI Policy Lab\.