Open but Incompatible: A License Compatibility Analysis of Corpora for Low-Resource African Languages
Summary
This paper audits license provenance of over twenty African NLP corpus families, identifies compatibility failures like the JW300 violation and hidden NoDerivs clauses, and provides a due diligence checklist for legally clean dataset creation.
View Cached Full Text
Cached at: 06/30/26, 05:28 AM
# Open but Incompatible: A License Compatibility Analysis of Corpora for Low-Resource African Languages
Source: [https://arxiv.org/html/2606.28867](https://arxiv.org/html/2606.28867)
###### Abstract
Creative Commons licenses dominate African NLP corpus releases, but their compatibility rules are rarely applied\. CC\-BY\-SA and CC\-BY\-NC cannot be combined in a single published dataset; a NoDerivs clause silently prohibits tokenisation and annotation\. This paper audits the license provenance of over twenty corpus families used in African NLP, constructs a six\-tier compatibility matrix, and applies it to three case\-study languages: Kituba/Munukutuba, Zarma, and Moore\. Four failure modes are documented with primary\-source evidence: outright prohibition \(JW300, removed from OPUS after a legal audit confirmed Terms of Service violation\); composite license misrepresentation \(WAXAL, whose CC\-BY 4\.0 claim is contradicted by its own HuggingFace dataset card\); a NoDerivs clause hidden behind a CC\-BY label \(Tanzil\); and data persistence failure \(the Congolese Radio Corpus, where 402 of 405 source URLs are now dead\)\. A pre\-annotation due diligence checklist and a survey of legally clean enrichment opportunities close the paper\.
Keywords:corpus licensing, license compatibility, African languages, low\-resource NLP, Creative Commons, reproducibility, data persistence, Kituba, Zarma, Moore
\\NAT@set@cites
Open but Incompatible: A License Compatibility Analysis of Corpora for Low\-Resource African Languages
Ernst van GassenArktos AI LabNetherlandsevg\.gassen@gmail\.comAbstract content
## 1\. Introduction
NLP researchers are not lawyers\. For high\-resource languages, this is ordinarily not a problem\. Corpora in common use have been legally vetted over decades of practice\. For low\-resource African languages, neither condition holds\.
Since 2019, parallel corpora, NER datasets, sentiment benchmarks, and speech resources have been published for dozens of African languages\. This paper argues that much of this output has not received systematic license review\. The consequences are beginning to surface\. JW300\(Agić and Vulić,[2019](https://arxiv.org/html/2606.28867#bib.bib3)\)was a parallel corpus covering 300\+ languages, including dozens with no alternative source\. It was found to have been built in violation of the Jehovah’s Witnesses website Terms of Service, which explicitly prohibit text and data mining\. A legal audit by the Centre for Intellectual Property and Information Technology \(CIPIT\) in Nairobi confirmed the violation\(Centre for Intellectual Property and Information Technology Law \(2020\),[CIPIT](https://arxiv.org/html/2606.28867#bib.bib7)\)\. OPUS removed the corpus\. Every dataset, model, and benchmark that incorporated JW300 data now carries a contaminated provenance chain\.
JW300 is not an isolated case\. GEITje, a Dutch\-language model, was removed from HuggingFace following a copyright enforcement request from Stichting BREIN\(Rijgersberg,[2023](https://arxiv.org/html/2606.28867#bib.bib32); nu\.nl / Tweakers,[2024](https://arxiv.org/html/2606.28867#bib.bib27); RTL Nieuws,[2024](https://arxiv.org/html/2606.28867#bib.bib33); Tweakers,[2024](https://arxiv.org/html/2606.28867#bib.bib37); GoingDutch\.ai,[2024](https://arxiv.org/html/2606.28867#bib.bib12)\); for high\-resource languages these incidents are disruptive but recoverable\. For African languages, losing one corpus may mean losing the only usable source for that language\. Common Corpus\(Langlais et al\.,[2025](https://arxiv.org/html/2606.28867#bib.bib20)\), a two\-trillion\-token corpus explicitly curated for open licensing, illustrates the baseline: an audit of its training table finds 15 native Sub\-Saharan languages accounting for approximately 91 rows combined, against 18,485 for English \(Section[5\.1](https://arxiv.org/html/2606.28867#S5.SS1)\)\.
The stakes are higher for low\-resource languages than for other NLP contexts for reasons that compound: there is no substitution corpus when a source is lost; annotation investment is sunk; one license conflict can block half the available landscape; and tainted sources cascade through multi\-language benchmark releases\.
Three languages anchor the case studies\.Kituba/Munukutuba\(ktu/mkw\), the vehicular language of southern Congo\-Brazzaville and a Kinshasa lingua franca, has 5–8 million speakers and no entry in FLORES\-200, MasakhaNER, or AfriSenti\.Zarma\(dje\), spoken by 4–5 million people in southwestern Niger, has MT and NER resources from the 27Group but is absent from FLORES\-200, NLLB, and all major benchmarks; OCHA and UNHCR list it as a required crisis communication language\.Moore\(mos\), with FLORES\-200 coverage and active research groups in Burkina Faso, serves as contrast: the upper bound of what the open\-license landscape currently offers a relatively well\-studied but still low\-resource language\.
## 2\. Related Work
#### Legal scholarship on open licensing\.
Creative Commons licensing has attracted sustained scholarly critique\.Katz \([2006](https://arxiv.org/html/2606.28867#bib.bib18)\)identifies two structural problems: variant proliferation creates user confusion, and ShareAlike terms create compatibility deadlocks that prevent legal distribution of derivatives\.Boyle \([2003](https://arxiv.org/html/2606.28867#bib.bib6)\)provides the theoretical frame: restrictive IP licensing constitutes a second enclosure movement\.
#### Data provenance and license audits in NLP\.
Gebru et al\. \([2021](https://arxiv.org/html/2606.28867#bib.bib11)\)andBender and Friedman \([2018](https://arxiv.org/html/2606.28867#bib.bib5)\)propose structured documentation standards for datasets and NLP corpora\. Both argue that provenance and licensing must be documented as first\-class metadata\.Dodge et al\. \([2021](https://arxiv.org/html/2606.28867#bib.bib9)\)applies this to large webtext corpora, finding machine\-generated text and benchmark contamination that documentation would have surfaced\.Kreutzer et al\. \([2022](https://arxiv.org/html/2606.28867#bib.bib19)\)audits 205 language\-specific corpora across five multilingual web\-crawl datasets, finding at least 15 contain no usable text and many use ambiguous language codes\. The challenges of web\-mined corpora for LLM pre\-training are surveyed inPerełkiewicz and Poświata \([2024](https://arxiv.org/html/2606.28867#bib.bib31)\)\.
The closest prior work to this paper is the Data Provenance Initiative\(Longpre et al\.,[2023](https://arxiv.org/html/2606.28867#bib.bib22),[2024](https://arxiv.org/html/2606.28867#bib.bib23)\)\. It audits 1,800\+ text datasets used to train LLMs\. License omission rates are above 70%; error rates are above 50%\. The Data Provenance Initiative operates at the general\-purpose LLM level\. It does not focus on African languages\. It does not construct a license compatibility matrix\. It does not address the failure modes documented here\.Mahari and Longpre \([2024](https://arxiv.org/html/2606.28867#bib.bib24)\)extends this line of work into legal analysis, arguing that provenance documentation affects fair use claims for fine\-tuning data\.
#### Legal analysis of AI training data\.
Henderson et al\. \([2023](https://arxiv.org/html/2606.28867#bib.bib16)\)analyses the four US fair use factors as applied to foundation model training, concluding that fair use is plausible but not guaranteed\.Lee et al\. \([2023](https://arxiv.org/html/2606.28867#bib.bib21)\)maps copyright questions across the full generative\-AI supply chain\.Jernite et al\. \([2022](https://arxiv.org/html/2606.28867#bib.bib17)\)proposes a multi\-stakeholder data governance framework addressing licensing at each stage of the data journey\. None of these works applies this framework to low\-resource African languages\.
#### African NLP data licensing\.
Nekoto et al\. \([2020](https://arxiv.org/html/2606.28867#bib.bib25)\)is the founding Masakhane paper\. It is the first African NLP work to address data ownership and licensing governance explicitly\. It brought the JW300 licensing issue to community attention and motivated the subsequent CIPIT legal audit\(Centre for Intellectual Property and Information Technology Law \(2020\),[CIPIT](https://arxiv.org/html/2606.28867#bib.bib7)\)\.Adelani et al\. \([2021](https://arxiv.org/html/2606.28867#bib.bib1),[2022](https://arxiv.org/html/2606.28867#bib.bib2)\)document licensing decisions for MasakhaNER releases\.Okerie and Marivate \([2024](https://arxiv.org/html/2606.28867#bib.bib28)\)surveys the African NLP community on copyright barriers\. It finds that the JW300 withdrawal created downstream disruption for projects with no alternative sources\.Omino \([2025](https://arxiv.org/html/2606.28867#bib.bib30)\)proposes the Nwulite Obodo Open Data License \(NOODL\), a tiered community license designed for African language datasets\.
Tiedemann \([2020](https://arxiv.org/html/2606.28867#bib.bib36)\)demonstrates the value of Tatoeba for low\-resource MT benchmarking; this paper focuses on the African\-language subset and the licensing constraints governing which sources can be legally combined\. The compatibility matrix in Section[4](https://arxiv.org/html/2606.28867#S4)is the practical output of that synthesis\.
## 3\. License Taxonomy
I define six license tiers for African NLP text corpora, ordered from least to most restrictive\. For non\-specialist readers:NC\(Non\-Commercial\) means the resource may not be used for revenue\-generating purposes as defined by the license; what constitutes commercial use is context\-dependent and jurisdiction\-sensitive, but publishing an annotated dataset via a paid service or commercial API is a clear case\.ND\(NoDerivs\) means the license prohibits*sharing*modified, adapted, tokenised, or otherwise derived versions; private use may still be permitted by applicable law \(e\.g\. fair use, TDM exceptions\), but no published annotation dataset derived from an ND source can be legally distributed under the license terms\.
Table 1:License tier taxonomy\. T4a and T4b are separated because their restrictions operate differently and are mutually incompatible with T3\.The key practical distinction is between T3 \(share\-alike propagates but derivatives are permitted\) and T4b \(derivatives not permitted at all\)\. Many practitioners conflate these, treating all non\-T2 sources as merely requiring a more restrictive output license\. This is incorrect: a T4b source cannot legally be incorporated into any published annotation dataset, regardless of the output license chosen\.
## 4\. License Compatibility Matrix
Table[2](https://arxiv.org/html/2606.28867#S4.T2)shows the legally valid output license when two corpus sources are combined\. “×\\times” denotes an incompatible combination: no single license can satisfy both sources’ requirements simultaneously\.
Table 2:License compatibility matrix\. Cell shows the required output license when the row\-source and column\-source are combined\.×\\times= incompatible combination; no valid output license exists\. T1\+ = any license acceptable\. T4b and T5 are incompatible with all other tiers\.A note on provenance quality independent of license tier: the compatibility matrix captures output license requirements, not the trustworthiness of the collection process\. Two datasets can both carry CC0 labels while having very different provenance standing\. ParaCrawl\(Bañón et al\.,[2020](https://arxiv.org/html/2606.28867#bib.bib4)\)is a web\-scale parallel corpus co\-financed by the EU Connecting Europe Facility, with the University of Edinburgh as lead institution, and carries CC0\. Importantly, ParaCrawl explicitly states that it does not own the underlying text; CC0 applies to the packaging and database rights only\. Its institutional context provides a degree of accountability that informal web scrapes do not\. JW300 also presented as open\-access but was built in violation of platform Terms of Service\. The license tier alone does not distinguish these two cases; provenance standing does\.
### 4\.1\. Dataset License versus Redistribution Rights
Several widely\-used corpora are web\-mined, meaning the dataset license reflects packaging or database rights rather than rights in the underlying text\(Perełkiewicz and Poświata,[2024](https://arxiv.org/html/2606.28867#bib.bib31)\)\. CCMatrix\(Schwenk et al\.,[2021](https://arxiv.org/html/2606.28867#bib.bib34)\)is mined from Common Crawl snapshots and carries no stated text license on OPUS or in its paper\. NLLB mined bitext\(NLLB Team et al\.,[2022](https://arxiv.org/html/2606.28867#bib.bib26)\)uses Common Crawl WET files as a primary source; ODC\-BY, the license on NLLB bitext, governs database rights, not rights in the underlying text\. WURA\(Oladipo et al\.,[2023](https://arxiv.org/html/2606.28867#bib.bib29)\)is built by auditing mC4 \(itself Common Crawl\-derived\) plus additional focused crawls\. For these corpora, the dataset\-level license does not clear the underlying text for redistribution or relicensing\.
A use\-case distinction is practically consequential\. Modeltrainingon mC4\-derived text may be defensible under fair use or EU Text and Data Mining \(TDM\) exceptions, depending on jurisdiction\. Publishing anannotated datasetderived from the same text is redistribution of copyrighted content\. The dataset’s Apache 2\.0 or CC packaging label does not change this: it covers the packaging, not the text\. This distinction is not visible in the compatibility matrix\. For the primary output of the African NLP community \(published annotation datasets\), redistribution risk applies to all mC4\-derived sources regardless of their stated license\. Rights\-cleared sources \(UDHR, TICO\-19, FLEURS, SMOL, original speech recordings\) avoid this risk entirely\. Practitioners who use WURA or Leipzig as annotation seeds for a published NER or POS dataset are redistributing copyrighted web text under a license the copyright holders never granted\.
One useful indicator of provenance quality is institutional context\. EU\-funded projects with named university leads, corpora with public ethics disclosures, and datasets with named investigators tend to carry more accountability than anonymous uploads\. This is an indicator of lower risk, not a guarantee\. No institutional label substitutes for verification of the actual collection method\.
For African languages, ParaCrawl’s bonus releases include English–Swahili \(132,517 sentence pairs, CC0\) and English–Somali \(14,879 sentence pairs, CC0\)\.
Three results from this matrix are practically consequential for African NLP:
\(1\) T3×\\timesT4a = incompatible\.Wikipedia \(CC\-BY\-SA, T3\) and the 27Group Feriji Zarma corpus \(CC\-BY\-NC, T4a\) cannot be combined in a published dataset\. A practitioner who annotates Wikipedia sentences alongside Feriji sentences and publishes the result has created a legally invalid dataset\. Wikipedia’s share\-alike requirement demands CC\-BY\-SA output\. Feriji’s non\-commercial restriction demands CC\-BY\-NC output\. No single license satisfies both\.
\(2\) T4b×\\timesanything = blocked\.Any corpus with a NoDerivs clause, including Tanzil, cannot legally be used to create a published annotation dataset\. The annotation itself is a derivative work\. This is not a question of the output license; distributing a derived annotation dataset is not permitted under the license, regardless of jurisdiction\-specific exceptions that may apply to private use\.
\(3\) T3 propagates upward; T4a similarly\.A T2 source combined with a T3 source produces a T3 output\. A T2 or T3 source combined with a T4a source produces a T4a output \(non\-commercial restriction is inherited\)\. Practitioners who use Wikipedia as annotation seed must release under CC\-BY\-SA 4\.0\. MasakhaNER 2\.0\(Adelani et al\.,[2022](https://arxiv.org/html/2606.28867#bib.bib2)\)uses Wikipedia text in its annotation pipeline\. Note that MasakhaNER’s HuggingFace dataset card lists CC BY\-NC 4\.0 for the dataset release; the source\-text licensing is heterogeneous\. Practitioners should verify the specific version they use\. The important point is that license decisions must be madebeforeannotation begins\. Choosing CC\-BY\-SA forecloses certain commercial downstream uses\.
## 5\. African NLP Corpus Survey
Table[4](https://arxiv.org/html/2606.28867#A1.T4)\(Appendix[A](https://arxiv.org/html/2606.28867#A1)\) surveys the corpus families used in African NLP with their tier assignments; an asterisk \(\*\) marks web\-mined corpora where the dataset license covers packaging or database rights, not rights in the underlying text\.
### 5\.1\. Common Corpus: African Language Representation
We streamed the full Common Corpus training split\(Langlais et al\.,[2025](https://arxiv.org/html/2606.28867#bib.bib20)\)and filtered by thelanguagefield\. All 91 native Sub\-Saharan rows carry CC\-BY\-SA licenses;subsetandurlfields arenullthroughout\. Text content identifies the source as Wikipedia 2023 via MediaWiki markup \(e\.g\.\{\{infobox tanàna\}\}in Malagasy rows\)\. The 16 rows labelled “Various open data” \(Lingala 7, Kabyle 5, Wolof 3, Hausa 1\) are language\-identification errors on French archival documents; none contains usable African\-language text\. Common Corpus is not an independent African\-language source: it repackages the same Wikipedia dumps audited in Table[4](https://arxiv.org/html/2606.28867#A1.T4), with worse provenance metadata\. Researchers should not count both as separate entries\. The ratio is approximately 200:1 \(English 18,485 rows; all 15 native Sub\-Saharan languages combined, 91 rows\); Afrikaans alone, with 11 rows, exceeds the native total\. This figure also reflects a structural feedback loop: platforms such as YouTube generate CC\-licensed ASR transcripts only for languages with an existing seed model\. Swahili is the sole Sub\-Saharan language with YouTube ASR support; it accumulates more CC text with each upload while the flywheel never starts for Lingala, Kikongo, or Tshiluba\. Critical mass of labelled speech data is a prerequisite, not merely a goal\.
### 5\.2\. Applying the Compatibility Matrix to Case\-Study Languages
Table[3](https://arxiv.org/html/2606.28867#S5.T3)shows which source combinations are legally valid for the three case\-study languages, and what output license each combination requires\.
Table 3:Compatibility analysis for case\-study language combinations\. “No” entries indicate legally invalid combinations\.For Moore, the license landscape is relatively clean\. The main sources \(MT560, FLORES\-200, WURA, MooreFRCollections\) are all T2 or T3 and compatible\. One caveat applies: the 125,695\-row Moore sentiment dataset \(michsethowusu/mossi\-sentiments\-corpus\) assigns labels via English back\-translation through DistilBERT\. In the NLP annotation literature,silverlabels are automatically generated, whilegoldlabels are human\-annotated directly in the target language\. This dataset should not be used as a gold standard for sentiment benchmarking without human verification of a stratified sample\.
For Zarma, the combination of Feriji \(T4a\) with the 27Group noisy GEC corpus \(T3\) is incompatible\. Both corpora are published by the same research group\. A practitioner who used both in a single annotation pipeline would have created a dataset with no valid output license\.
## 6\. Four Failure Modes
### 6\.1\. Prohibition: JW300
JW300\(Agić and Vulić,[2019](https://arxiv.org/html/2606.28867#bib.bib3)\)was a parallel corpus covering 300\+ languages, built from the Jehovah’s Witnesses websitejw\.org\. It was widely used in African NLP from 2019 onward due to coverage of languages with no other parallel text\.
The legal problem is straightforward\. Thejw\.orgTerms of Service explicitly prohibit text and data mining\. A legal audit by CIPIT Nairobi confirmed the violation\(Centre for Intellectual Property and Information Technology Law \(2020\),[CIPIT](https://arxiv.org/html/2606.28867#bib.bib7)\)\. OPUS removed the corpus following Masakhane’s formal request for permission, which the organisation denied\(Walled Culture,[2020](https://arxiv.org/html/2606.28867#bib.bib38)\)\. This is Tier 5 under the taxonomy above: prohibited regardless of how the corpus was obtained\.
The consequential risk for current practitioners is indirect\. Language models, cross\-lingual embeddings, and benchmark systems trained before 2021 may incorporate JW300\-derived representations\. Derivative datasets built from those models carry a tainted provenance chain\. African NLP papers should include an explicit statement:“This dataset does not include JW300\-derived text or derivatives thereof\.”This is analogous to the “IRB approved” statement in human subjects research: a reproducible declaration that reviewers can verify\.
### 6\.2\. Composite License Misrepresentation: WAXAL
WAXAL\(Diack et al\.,[2026](https://arxiv.org/html/2606.28867#bib.bib8)\)is a 2026 speech dataset covering 19 African languages for ASR and 16 for TTS\. The arXiv paper\(Diack et al\.,[2026](https://arxiv.org/html/2606.28867#bib.bib8)\)claims that the collection is released under a uniform CC\-BY 4\.0 license\. The HuggingFace dataset card\(Google,[2026](https://arxiv.org/html/2606.28867#bib.bib14)\)explicitly contradicts this, listing per\-provider licenses\. Per\-provider attribution, traced from WAXAL’s supplementary tables:
- •CC\-BY 4\.0 \(T2\):University of Ghana contributions only: Akan, Ewe, Dagbani, Dagaare, Ikposo \(ASR\); Fante, Twi \(TTS\)\.
- •CC\-BY\-SA 4\.0 \(T3\):All other contributions: Makerere University \(Acholi, Luganda, Masaaba, Nyankole, Soga\), Digital Umuganda \(Fula,Lingala, Shona, Malagasy, Amharic, Oromo, Sidama, Tigrinya, Wolaytta\), Media Trust \(Fula, Igbo, Hausa, Yoruba, Nigerian Pidgin\), Loud and Clear \(Kikuyu, Luo\)\.
The composite misrepresentation creates a concrete legal failure\. A practitioner who reads the WAXAL arXiv abstract, downloads the Lingala subset, annotates a NER dataset from its transcripts, and publishes under CC\-BY 4\.0 has violated the CC\-BY\-SA 4\.0 share\-alike requirement of the Digital Umuganda contribution\. This occurs despite acting in good faith on the stated license\. Lingala, Hausa, Igbo, Yoruba, and all Makerere\-sourced languages in WAXAL require CC\-BY\-SA 4\.0 output for any published derivative\.
A noteworthy asymmetry exists\. Digital Umuganda’s standaloneAfriVoicedataset, which covers the same Lingala recordings, is released under CC\-BY 4\.0\. The same speech data carries different terms depending on which dataset packaging it is accessed through\. Practitioners cannot resolve this without per\-provider provenance tracing that the arXiv paper does not facilitate\.
Composite dataset papers should include a per\-language provenance and license table as a required metadata artifact\.
### 6\.3\. Hidden NoDerivs Restriction: Tanzil
Tanzil \(tanzil\.net\) provides Quran translations in approximately 40 languages, including several African languages \(Hausa, Swahili, Somali, Amharic, partial Yoruba\)\. Its stated license is CC\-BY 3\.0\. In the NLP literature, CC\-BY is typically treated as Tier 2: permissive, derivatives allowed, attribution required\.
The Tanzil license page explicitly states:“You are not allowed to modify this text in any way”\(Tanzil Project,[2010](https://arxiv.org/html/2606.28867#bib.bib35)\)\. This is a NoDerivs restriction \(Tier 4b\)\. It is not disclosed in the CC\-BY label\. A practitioner who tokenises Tanzil text, aligns it to a parallel target, and publishes the result as a training dataset has violated this restriction\. The derivative prohibition applies regardless of the output license chosen\.
The NoDerivs clause reflects the religious status of the Quran in Islam\. Tanzil’s policy holds that Quranic text may not be altered, in order to preserve the integrity of a text considered holy and inviolable in Islamic tradition\(Tanzil Project,[2010](https://arxiv.org/html/2606.28867#bib.bib35)\)\. This explains why the ND restriction is applied even under an otherwise permissive CC\-BY label\.
An annotation dataset derived from Tanzil text cannot legally be published under any open license\. The NoDerivs clause prohibits the derivative work entirely\. To the extent that Tanzil\-derived text has been incorporated into African NLP pipelines for languages with Quran translation coverage, those pipelines carry this undisclosed legal risk\.
No modern African\-language Quran translation is available in a clearly public\-domain or CC\-BY \(without ND\) machine\-readable format\. The classical English translations of Sale \(1734, Project Gutenberg \#7440\), Rodwell \(1861, \#3434\), and Palmer \(1880, Wikisource\) are public domain\. These English public domain translations provide no African\-language text and are of no direct utility for practitioners building African\-language NLP resources\.
### 6\.4\. Data Persistence Failure: The Congolese Radio Corpus
The Congolese Radio Corpus \(CRC\)Wheatley et al\. \([2020](https://arxiv.org/html/2606.28867#biba.bib1)\)for Lingala was published with a claim of hundreds of hours of broadcast audio sourced from YouTube\. An audit conducted in February 2026 found that402 of 405 YouTube IDs referenced in the CRC are now dead, returning 404 errors due to video removal or channel deletion\. The reproducible resource is approximately 14\.4 hours of elicited LRSC speech and Radio Okapi broadcasts\.
This is not a criticism of the original authors\. It is a structural warning:corpora that depend on third\-party platform URLs are inherently non\-persistent\.A published corpus that cannot be reproduced by a subsequent researcher is not a scientific contribution in the standard sense\. The CRC is not an isolated case\. Common Voice drops recordings when contributors withdraw consent\. HuggingFace datasets are occasionally removed by their owners\. YouTube channels are deleted routinely\.
Corpora distributed via repositories with persistent identifiers, such as Zenodo DOIs, OpenSLR stable IDs, or LDC catalogue numbers, have remained reproducible across years\. I recommend that African NLP publication venues adopt a data availability standard requiring either \(a\) a persistent DOI\-backed deposit for all corpus resources, or \(b\) an explicit statement of which components are platform\-dependent and may become unavailable\.
A related failure is the lack of provenance documentation in community HuggingFace uploads\. Several large parallel corpora for African languages carry labels such as “MT560/OPUS\-derived” with no source URLs, translation pipeline documentation, or quality filter parameters\. For example:michsethowusu/english\-luba\-kasai\_sentence\-pairs\_mt560\(292,000 rows, CC\-BY 4\.0\),michsethowusu/english\-congo\-swahili\_sentence\-pairs\_mt560\(272,000 rows, CC\-BY 4\.0\), andmichsethowusu/english\-zarma\_sentence\-pairs\_mt560\(60,000 rows, CC\-BY 4\.0\) all fall into this category\. These cannot be audited for license provenance\. A practitioner cannot verify whether T5 sources were included in the pipeline, making them legally ambiguous despite carrying permissive license labels\.
#### Data persistence and digital sovereignty\.
The CRC failure is not merely a technical problem; it is also a sovereignty problem\. All four platforms implicated in African NLP data rot, YouTube, HuggingFace, GitHub, and OPUS, are maintained by US or European organisations with no specific accountability to African language communities\. When a corpus disappears from these platforms, no African institution has the mandate or authority to recover it\. This argues for African\-controlled digital infrastructure for language data\. Initiatives such as SADILAR \(South African Centre for Digital Language Resources\) and the ISLRN persistent identifier system point toward a model where African language resources are deposited in African\-managed archives with ISLRN identifiers\(Nekoto et al\.,[2020](https://arxiv.org/html/2606.28867#bib.bib25); Omino,[2025](https://arxiv.org/html/2606.28867#bib.bib30)\)\. The CRC case illustrates what happens in the absence of such infrastructure: a published corpus becomes an unverifiable claim\.
## 7\. Enrichment Opportunities Within the Open\-License Landscape
The foregoing analysis could be read pessimistically\. The legal constraints are severe, several documented corpora contain license problems, and authentic open\-license text for under\-resourced African languages is limited\. The opposite reading is more productive: identifying the legal constraints clearly is what makes enrichment tractable\.
#### Transcribing untranscribed speech\.
WAXAL includes speech subsets for which transcripts are not released\. The University of Ghana subsets \(Akan, Ewe, Dagbani, Dagaare, Ikposo ASR; Fante, Twi TTS\) carry CC\-BY 4\.0 licensing\. Transcribing these recordings with community annotators and releasing under CC\-BY 4\.0 would produce genuinely new, derivative\-safe text corpora requiring no new data collection\.
#### Annotation of existing T2/T3 seeds\.
For each case\-study language, T2 seed text exists and can be annotated for NER, POS, or sentiment\. The key legal decision is whether to include Wikipedia \(T3, forcing CC\-BY\-SA 4\.0 output\) or restrict annotation to T2 sources \(permitting CC\-BY 4\.0 output\)\. This decision must be made before annotation begins, as it affects downstream commercial usability\. A further distinction applies to web\-mined T2 sources such as WURA and Leipzig: their packaging license does not clear the underlying text for redistribution\. Publishing an annotated dataset whose seed sentences come from WURA is redistribution of mC4\-derived content\. Rights\-cleared T2 sources \(FLEURS, SMOL, TICO\-19\) do not carry this risk and are preferable as annotation seeds when they provide sufficient coverage\.
For Kituba, the Leipzig Corpora Collection\(Goldhahn et al\.,[2012](https://arxiv.org/html/2606.28867#bib.bib13)\)mkw\_community\_2017entry \(143,476 sentences, CC\-BY, T2\) is, to the author’s knowledge, the largest available Kituba text corpus and has not appeared in published NLP work\. Combined with the SMOLgatitos\_\_en\_ktupairs \(863 sentences, CC\-BY 4\.0, T2\), it provides an NER annotation seed with known provenance that permits CC\-BY 4\.0 output without share\-alike propagation\.
For Zarma, the MT560 parallel corpus \(60,515 sentences, CC\-BY 4\.0, T2\) is the safe annotation seed for CC\-BY 4\.0 output\. The 27Group noisy GEC corpus \(508,869 sentences, T3\) is available but forces CC\-BY\-SA 4\.0 output and is incompatible with Feriji \(T4a\)\.
#### Parallel and bridged resources\.
For zero\-pivot African–African pairs, the UDHR \(T1, public domain\) provides the same 30 articles across 570\+ language editions in sentence\-aligned form on OPUS; any two editions pair directly with no English or French intermediary\. NTREX\-128\(Federmann et al\.,[2022](https://arxiv.org/html/2606.28867#bib.bib10)\)provides 1,997 professionally translated news sentences in 24 African languages under CC\-BY\-SA 4\.0; the shared source enables direct pairing of any two\. For languages outside these resources, bridge construction via FLORES\-200 \(T3, CC\-BY\-SA\) or TICO\-19 \(T1, CC0\) is possible where both languages have segments against the same pivot\. English, French, Arabic, and Portuguese cover the main regional pivot groups\. Global Voices \(OPUS, CC\-BY 3\.0\) provides human\-translated Swahili \(≈\\approx20K pairs\) and Amharic \(≈\\approx1K\)\. None substitutes for large training corpora, but all are legally clean and available today\.
#### The kgwiki discovery\.
A finding with direct enrichment implications: the Kongo Wikipedia \(kgwiki\), labeled and indexed asKongo, is written in Kituba/Munukutuba\. This is confirmed by article content inspection and theSvngokudataset card, which explicitly invites speakers of Munukutuba, Kituba, and Kikongo ya Leta to contribute\. As a result, 1,200\+ articles of usable Kituba text \(CC\-BY\-SA 4\.0, T3\) may have been missed by practitioners searching for Kituba text under its correct ISO codes \(ktu,mkw\)\. The same mislabeling affects FLORES\-200’skon\_Latnentry\. Researchers building Kongo NLP systems may have trained on Kituba data; researchers building Kituba systems may have missed this resource\. Resolving this ISO code confusion, which involves at least five codes \(kon/kg,ktu,mkw,kwy\), is a prerequisite for any systematic enrichment effort\.
## 8\. A Legal Due Diligence Checklist
Four steps before annotation begins:
Step 1: Inventory sources\.Consult: Wikipedia, Leipzig, UDHR, Tatoeba, FLORES\-200, FLEURS, WAXAL \(per\-provider\), WURA, eBible\.org \(per\-translation\), African Storybook \(per\-story\), TICO\-19, Common Voice \(per\-subset\), MT560/HuggingFace \(with provenance caveat\), OPUS \(excluding JW300\-derived subsets\)\. Avoid or flag: CCMatrix \(no stated license\), TED2020 \(T4b\), JW300 \(T5\), bible\-uedin \(CC0 claimed; verify per\-translation rights\)\.
Step 2: Assign tiers; distinguish training from redistribution\.Use Table[1](https://arxiv.org/html/2606.28867#S3.T1)\. Do not trust aggregator license labels: Tanzil is T4b despite stating CC\-BY; WAXAL majority subsets are T3 despite stating CC\-BY 4\.0\. For web\-mined sources \(WURA, Leipzig, ParaCrawl, NLLB bitext\), distinguish model training \(may be defensible under fair use or TDM exceptions\) from dataset publication \(redistribution, higher risk\)\. Verify against the original source repository\.
Step 3: Run the compatibility matrix\.Use Table[2](https://arxiv.org/html/2606.28867#S4.T2)for each pair of sources\. Any×\\timesmust be resolved before annotation begins—usually by dropping one source\.
Step 4: Verify ISO codes and archive\.For Bantu languages, inspect actual text samples and cross\-reference with Ethnologue and Glottolog; kgwiki is Kituba, not Kongo\. Record dataset name, license version, retrieval date, and checksum or DOI for every source\. Deposit a snapshot in a persistent archive\. Include an explicit statement that no JW300\-derived text is present\.
## 9\. Discussion
None of the errors documented here was wilful; each was a legal assumption that NLP practice gave no reason to question explicitly\. The compatibility matrix \(Table[2](https://arxiv.org/html/2606.28867#S4.T2)\) requires no legal expertise: it is a lookup table\. Tier assignments require a one\-time provenance check per corpus\. The checklist requires discipline\. A common concern is that Wikipedia’s CC\-BY\-SA license propagates share\-alike and is therefore too restrictive\. In practice, Wikipedia has 3–5 named entities per sentence — higher density than religious text or children’s stories — making it a strong NER seed despite the share\-alike constraint\. Practitioners who need CC\-BY 4\.0 output should use WURA or MT560\-derived corpora as seeds and accept somewhat lower entity density\.
## 10\. Conclusion
The four case studies share a pattern: a legal assumption was made implicitly that would not have survived explicit examination\. JW300 was used because it existed and seemed open\. Tanzil was treated as CC\-BY because that is what the label said\. WAXAL’s per\-provider terms were not traced because the arXiv paper did not prompt it\. The CRC’s YouTube dependency was not flagged because URL persistence is not a standard publication criterion\. None of these errors was wilful; all were avoidable with a one\-time provenance check\.
Concrete outcomes from the compatibility analysis: the 27Group Feriji corpus \(T4a\) is incompatible with the 27Group GEC corpus \(T3\) despite coming from the same research group; the Leipzigmkw\_community\_2017entry is, to the author’s knowledge, the largest open Kituba corpus and has not appeared in any published NLP work; and the Kongo Wikipedia \(kgwiki\) contains 1,200\+ articles of Kituba text mislabeled as Kongo, a confusion that extends to FLORES\-200’skon\_Latnentry\. Data logging and persistent archiving should become standard publication requirements for African NLP work\.
## LRE Map
This paper does not introduce new language resources; it audits the license provenance of existing ones\. No new LRE Map entries are created\. All resources cited here are existing catalogued resources; their existing identifiers \(ISLRN, HuggingFace dataset IDs, OPUS corpus IDs, or GitHub repositories\) are referenced in the bibliography\. The LRE Map URL is[http://lremap\.elra\.info](http://lremap.elra.info/)\.
## Ethical Considerations
This paper audits license provenance of existing resources; no new datasets, models, or personal data were collected\. The legal analysis draws on published reports, license page text, and dataset cards retrieved in January–February 2026\. None of the corpora identified as legally problematic \(JW300, Tanzil, TED2020\) were used to produce any output\. Licenses may change after the retrieval date; practitioners should verify independently\. This paper does not constitute legal advice\. Future annotation work on the languages surveyed should follow community consent protocols as outlined byNekoto et al\. \([2020](https://arxiv.org/html/2606.28867#bib.bib25)\)\.
## Limitations
Jurisdiction\-specific law \(e\.g\., EU TDM exceptions under the DSM Directive\) may alter practical conclusions; redistribution of modified corpora would remain restricted regardless\. Tier assignments for MT560/HuggingFace datasets are provisional due to undocumented provenance\.
## 11\. Bibliographical References
\\c@NAT@ctr
- Adelani et al\. \(2021\)David Ifeoluwa Adelani, Jade Abbott, et al\. 2021\.[MasakhaNER: Named entity recognition for African languages](https://aclanthology.org/2021.tacl-1.66)\.In*Transactions of the Association for Computational Linguistics*, volume 9, pages 1116–1131\. MIT Press\.
- Adelani et al\. \(2022\)David Ifeoluwa Adelani, Graham Carr, et al\. 2022\.[MasakhaNER 2\.0: Africa\-centric transfer learning for named entity recognition](https://aclanthology.org/2022.emnlp-main.298)\.In*Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing*, pages 4488–4508\. Association for Computational Linguistics\.
- Agić and Vulić \(2019\)Željko Agić and Ivan Vulić\. 2019\.[JW300: A wide\-coverage parallel corpus for low\-resource languages](https://aclanthology.org/P19-1310)\.In*Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, pages 3204–3210\. Association for Computational Linguistics\.
- Bañón et al\. \(2020\)Marta Bañón, Pinzhen Chen, Barry Haddow, Kenneth Heafield, Hieu Hoang, Miquel Esplà\-Gomis, Marcin Junczys\-Dowmunt, Samuel Ma, Prashant Mathur, Paul Paul, Johann Roturier, and Rico Sennrich\. 2020\.[ParaCrawl: Web\-scale acquisition of parallel corpora](https://aclanthology.org/2020.acl-main.417)\.In*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pages 4555–4567\.
- Bender and Friedman \(2018\)Emily M Bender and Batya Friedman\. 2018\.Data statements for natural language processing: Toward mitigating system bias and enabling better science\.*Transactions of the Association for Computational Linguistics*, 6:587–604\.
- Boyle \(2003\)James Boyle\. 2003\.[The second enclosure movement and the construction of the public domain](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=470983)\.*Law and Contemporary Problems*, 66\(1/2\):33–74\.
- Centre for Intellectual Property and Information Technology Law \(2020\) \(CIPIT\)Centre for Intellectual Property and Information Technology Law \(CIPIT\)\. 2020\.[Masakhane projects’ use of the JW300 dataset for natural language processing: Copyright issues, contract overrides and cross\-border implications](https://knowledgegov.org/masakhane-projects-use-of-the-jw300-dataset-for-natural-language-processing-copyright-issues-contract-overrides-and-cross-border-implications/)\.
- Diack et al\. \(2026\)Thierno Diack et al\. 2026\.[WAXAL: A large\-scale multilingual speech dataset for African languages](https://arxiv.org/abs/2602.02734)\.*arXiv preprint arXiv:2602\.02734*\.
- Dodge et al\. \(2021\)Jesse Dodge, Maarten Sap, Ana Marasovic, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner\. 2021\.[Documenting large webtext corpora: A case study on the colossal clean crawled corpus](https://aclanthology.org/2021.emnlp-main.98)\.In*Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pages 1286–1305\.
- Federmann et al\. \(2022\)Christian Federmann, Tom Kocmi, and Ying Xin\. 2022\.[NTREX\-128 – news test references for MT evaluation of 128 languages](https://aclanthology.org/2022.sumeval-1.4)\.In*Proceedings of the First Workshop on Systematic Biases in MT Research*\.
- Gebru et al\. \(2021\)Timnit Gebru et al\. 2021\.Datasheets for datasets\.*Communications of the ACM*, 64\(12\):86–92\.
- GoingDutch\.ai \(2024\)GoingDutch\.ai\. 2024\.GEITje takedown\.[https://goingdutch\.ai/nl/posts/geitje\-takedown/](https://goingdutch.ai/nl/posts/geitje-takedown/)\.Accessed February 2026\.
- Goldhahn et al\. \(2012\)Dirk Goldhahn, Thomas Eckart, and Uwe Quasthoff\. 2012\.Building large monolingual dictionaries at the Leipzig corpora collection: From 100 to 200 languages\.In*Proceedings of the Eighth International Conference on Language Resources and Evaluation \(LREC 2012\)*, pages 759–765\. European Language Resources Association \(ELRA\)\.
- Google \(2026\)Google\. 2026\.WAXAL: A large\-scale multilingual African language speech corpus – dataset card\.[https://huggingface\.co/datasets/google/WaxalNLP](https://huggingface.co/datasets/google/WaxalNLP)\.Accessed February 2026\. Dataset card specifies per\-provider licenses: University of Ghana contributions are CC\-BY 4\.0; Makerere University, Digital Umuganda, Media Trust, and Loud and Clear contributions are CC\-BY\-SA 4\.0\. Contradicts the uniform CC\-BY 4\.0 claim in the arXiv paper\.
- GoURMET Consortium \(2020\)GoURMET Consortium\. 2020\.[GoURMET: Generalisation of underrepresented languages with modern transformers and evaluation of robustness](https://gourmet-project.eu/)\.EU Horizon 2020 Project 825299; lead institution: University of Sheffield; CC0 parallel corpora available via OPUS\.
- Henderson et al\. \(2023\)Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A\. Lemley, and Percy Liang\. 2023\.[Foundation models and fair use](https://arxiv.org/abs/2303.15715)\.*Journal of Machine Learning Research*, 24\.
- Jernite et al\. \(2022\)Yacine Jernite, Huu Nguyen, Stella Biderman, et al\. 2022\.[Data governance in the age of large\-scale data\-driven language technology](https://doi.org/10.1145/3531146.3534637)\.In*Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency*, pages 2206–2222\.
- Katz \(2006\)Zachary Katz\. 2006\.[Pitfalls of open licensing: An analysis of creative commons licensing](https://ipmall.law.unh.edu/sites/default/files/hosted_resources/IDEA/idea-vol46-no3-katz.pdf)\.*IDEA: The Intellectual Property Law Review*, 46\(3\)\.
- Kreutzer et al\. \(2022\)Julia Kreutzer et al\. 2022\.[Quality at a glance: An audit of web\-crawled multilingual datasets](https://aclanthology.org/2022.tacl-1.4)\.*Transactions of the Association for Computational Linguistics*, 10:50–72\.
- Langlais et al\. \(2025\)Pierre\-Carl Langlais, Carlos Rosas Hinostroza, Mattia Nee, Catherine Arnett, et al\. 2025\.[Common corpus: The largest collection of ethical data for LLM pre\-training](https://arxiv.org/abs/2506.01732)\.*arXiv preprint arXiv:2506\.01732*\.Approximately two trillion tokens; multilingual, with 53% of tokens from non\-Western\-country sources; African languages listed as a future expansion target \(Swahili, Wolof, Bambara\); web\-mined components carry provenance questions analogous to those in WURA and CCMatrix\.
- Lee et al\. \(2023\)Katherine Lee, A\. Feder Cooper, and James Grimmelmann\. 2023\.[Talkin’ ’bout AI generation: Copyright and the generative\-AI supply chain](https://arxiv.org/abs/2309.08133)\.*Journal of the Copyright Society of the USA*\.
- Longpre et al\. \(2023\)Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng\-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, and Sara Hooker\. 2023\.[The data provenance initiative: A large scale audit of dataset licensing and attribution in AI](https://arxiv.org/abs/2310.16787)\.*arXiv preprint arXiv:2310\.16787*\.
- Longpre et al\. \(2024\)Shayne Longpre, Robert Mahari, Anthony Chen, et al\. 2024\.[The data provenance initiative: A large scale audit of dataset licensing and attribution in AI](https://doi.org/10.1038/s42256-024-00878-8)\.*Nature Machine Intelligence*, 6\.
- Mahari and Longpre \(2024\)Robert Mahari and Shayne Longpre\. 2024\.Discit ergo est: Training data provenance and fair use\.*Network Law Review*\.Winter 2024\. Also available at SSRN 4795277\.
- Nekoto et al\. \(2020\)Wilhelmina Nekoto et al\. 2020\.[Participatory research for low\-resourced machine translation: A case study in African languages](https://aclanthology.org/2020.findings-emnlp.195)\.In*Findings of the Association for Computational Linguistics: EMNLP 2020*, pages 2144–2160\. Association for Computational Linguistics\.
- NLLB Team et al\. \(2022\)NLLB Team, Marta R Costa\-jussà, et al\. 2022\.[No language left behind: Scaling human\-centered machine translation](https://arxiv.org/abs/2207.04672)\.*arXiv preprint arXiv:2207\.04672*\.FLORES\-200 benchmark included; 200 languages, CC\-BY\-SA 4\.0\.
- nu\.nl / Tweakers \(2024\)nu\.nl / Tweakers\. 2024\.Ontwikkelaar haalt Nederlands AI\-taalmodel offline na verzoek Stichting BREIN\.[https://www\.nu\.nl/tweakers/6343889/ontwikkelaar\-haalt\-nederlands\-ai\-taalmodel\-offline\-na\-verzoek\-stichting\-brein\.html](https://www.nu.nl/tweakers/6343889/ontwikkelaar-haalt-nederlands-ai-taalmodel-offline-na-verzoek-stichting-brein.html)\.Accessed February 2026\.
- Okerie and Marivate \(2024\)Chijioke Okerie and Vukosi Marivate\. 2024\.[How African NLP experts are navigating the challenges of copyright, innovation, and access](https://carnegieendowment.org/research/2024/04/how-african-nlp-experts-are-navigating-the-challenges-of-copyright-innovation-and-access)\.Carnegie Endowment for International Peace\.
- Oladipo et al\. \(2023\)Ifeoluwa Adeyemi Oladipo, Abdulmumin Idris, Aremu Anuoluwapo, et al\. 2023\.[Better quality pretraining data and T5 models for African languages](https://aclanthology.org/2023.emnlp-main.11)\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 158–168\. Association for Computational Linguistics\.
- Omino \(2025\)Melissa Omino\. 2025\.[The nwulite obodo open data license \(NOODL\): Licensing African datasets to support research and AI in the global south](https://infojustice.org/archives/46434)\.Conference on Copyright and the Public Interest in Africa and the Global South, Johannesburg\. CIPIT, Strathmore University\.
- Perełkiewicz and Poświata \(2024\)Michał Perełkiewicz and Rafał Poświata\. 2024\.[A review of the challenges with massive web\-mined corpora used in large language models pre\-training](https://arxiv.org/abs/2407.07630)\.*arXiv preprint arXiv:2407\.07630*\.ICAISC 2024\. Surveys noise, duplication, bias, and legal issues in web\-mined LLM pre\-training data\.
- Rijgersberg \(2023\)Edwin Rijgersberg\. 2023\.[GEITje: A large open dutch language model](https://github.com/Rijgersberg/GEITje)\.GitHub repository; model subsequently removed from HuggingFace at the request of Stichting BREIN due to copyright concerns over Dutch GigaCorpus training data\.Demonstrates that training data provenance problems are not limited to low\-resource languages; a Dutch \(high\-resource\) language model was taken down following copyright enforcement action by a national rights management foundation\.
- RTL Nieuws \(2024\)RTL Nieuws\. 2024\.Illegale dataset van zinnen uit Nederlandse films en boeken offline\.[https://www\.rtl\.nl/nieuws/tech/artikel/5465687/illegale\-dataset\-van\-zinnen\-uit\-nederlandse\-films\-en\-boeken\-offline](https://www.rtl.nl/nieuws/tech/artikel/5465687/illegale-dataset-van-zinnen-uit-nederlandse-films-en-boeken-offline)\.Accessed February 2026\.
- Schwenk et al\. \(2021\)Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Édouard Grave, Armand Joulin, and Angela Fan\. 2021\.[CCMatrix: Mining billions of high\-quality parallel sentences on the WEB](https://aclanthology.org/2021.acl-long.70)\.In*Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics*, pages 932–944\. Association for Computational Linguistics\.
- Tanzil Project \(2010\)Tanzil Project\. 2010\.[Text license — tanzil documents](https://tanzil.net/docs/text_license)\.Accessed February 2026\. States: “You are not allowed to modify this text in any way\.”\.
- Tiedemann \(2020\)Jörg Tiedemann\. 2020\.[The tatoeba translation challenge – realistic data sets for low resource and multilingual MT](https://aclanthology.org/2020.wmt-1.139)\.In*Proceedings of the Fifth Conference on Machine Translation*, pages 1174–1182\.
- Tweakers \(2024\)Tweakers\. 2024\.BREIN haalt illegale Nederlandstalige dataset voor trainen AI\-modellen offline\.[https://tweakers\.net/nieuws/225340/brein\-haalt\-illegale\-nederlandstalige\-dataset\-voor\-trainen\-ai\-modellen\-offline\.html](https://tweakers.net/nieuws/225340/brein-haalt-illegale-nederlandstalige-dataset-voor-trainen-ai-modellen-offline.html)\.Accessed February 2026\.
- Walled Culture \(2020\)Walled Culture\. 2020\.[A “blatant no” from a copyright holder stops vital linguistic research work in Africa](https://walledculture.org/a-blatant-no-from-a-copyright-holder-stops-vital-linguistic-research-work-in-africa/)\.
## 12\. Language Resource References
\\c@NAT@ctr
- Wheatley et al\. \(2020\)Wheatley, Julian and others\. 2020\.[*Congolese Radio Corpus \(CRC\) for Lingala*](https://github.com/lowerquality/crc)\.Data persistence failure\.Originally claimed hundreds of hours of YouTube broadcast audio\. Audit conducted February 2026 found 402 of 405 YouTube IDs dead \(404 errors\)\. Reproducible content: approximately 8\.3 hours elicited LRSC speech \(IPA\-transcribed\) \+ approximately 6\.1 hours Radio Okapi broadcast audio = approximately 14\.4 hours total\.
## Appendix AAfrican NLP Corpus Survey
Table 4:License tier assignments for major African NLP corpus families\. Asterisk \(\*\) marks web\-mined corpora where the dataset license covers packaging or database rights, not rights in the underlying text\. Italic entries in the Tier column represent cases where the stated license differs from the effective license after review\.Similar Articles
The African Language Tax: Quantifying the Cost, Latency, and Context Penalty of Tokenizing African Languages in Frontier LLMs
This paper systematically quantifies the tokenization penalty for 20 African languages across 11 frontier and open tokenizers, finding up to 8.9× inference cost and latency multipliers and as little as 11% effective context window compared to English, highlighting a structural digital divide encoded in subword vocabularies.
Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research
This paper systematically evaluates the applications of large language models in low-resource language research, analyzing opportunities and challenges across linguistic variation, historical documentation, cultural expressions, and literary analysis. The study emphasizes interdisciplinary collaboration and customized model development to preserve linguistic and cultural heritage while addressing issues of data accessibility, model adaptability, and cultural sensitivity.
LLM Safety Alignment in Low-Resource Languages: A Systematic Literature Review
This systematic literature review examines safety alignment of large language models in low-resource languages, identifying a persistent multilingual safety gap and suggesting future directions such as culturally grounded benchmarks and participatory data collection.
From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languages
This paper evaluates the Mamba state space model for ASR on seven South African languages, finding it matches Conformer accuracy with fewer resources, and explores multilingual training strategies and low-resource settings.
Can Large Language Models Reliably Correct Errors in Low-Resource ASR? A Contamination-Aware Case Study on West Frisian
This paper investigates LLM-based generative error correction (GER) for low-resource West Frisian ASR, using a contamination-aware evaluation with a private dataset to show that GPT-5.1 reduces errors beyond oracle levels.