FineWeb-CLaR:用于基准对齐语料审计的文化、语言和区域标注

arXiv cs.CL 论文

摘要

本文介绍了FineWeb-CLaR,这是一个大规模标注数据集,为网页文档标注文化、语言和区域元数据,以实现对语言模型预训练数据和基准测试中文化表示的审计。

arXiv:2609.25298v1 Announce Type: new Abstract: Cultural evaluation coverage and robustness in language models are difficult to diagnose because pretraining corpora and cultural benchmarks are rarely indexed with comparable metadata. Benchmarks increasingly target culturally situated phenomena at the level of languages, regions, and locale-specific practices, while web-scale corpora are usually organized only by language. A shared culture-language-region layer makes these resources comparable, enabling audits of whether a target cultural phenomenon is represented in pretraining data, evaluated by benchmarks or both. To this end, we introduce FineWeb-CLaR, a large-scale annotated dataset derived from FineWeb and FineWeb-2 that places web documents on a shared culture-language-region axis for corpus auditing and benchmark alignment. FineWeb-CLaR annotates the full 30.9B-document collection from FineWeb and FineWeb-2 with URL-derived region labels and cultural-topic provenance. Our region resolver assigns a non-empty region to 25.61% of documents (7.92B). For cultural-topic analysis, we induce locale-specific topics and project them onto the 14 leaves of the Cultural Taxonomy of Liu et al. (2025), producing Locale Topic Distributions (LTDs) for corpus-side comparison. We also annotate 277 cultural NLP benchmarks with the same taxonomy, language coverage, and region coverage. Together, these resources enable direct comparison between corpus-side pretraining evidence and benchmark-side evaluation coverage.
查看原文
查看缓存全文

缓存时间: 2026/09/23 09:13

# FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing
Source: [https://arxiv.org/html/2609.25298](https://arxiv.org/html/2609.25298)
Yusser Al GhussinAffiliation:Saarland University German Research Center for Artificial Intelligence \(DFKI\)Email:[yusser\.alghussin@dfki\.de](mailto:)Eva GavallerCristina España\-BonetAffiliation:Saarland University German Research Center for Artificial Intelligence \(DFKI\)Affiliation:Barcelona Supercomputing Center \(BSC\-CNS\), Barcelona, Catalonia, SpainJosef van GenabithAffiliation:Saarland University German Research Center for Artificial Intelligence \(DFKI\)Simon OstermannAffiliation:Saarland University German Research Center for Artificial Intelligence \(DFKI\)

###### Abstract

Cultural evaluation coverage and robustness in language models are difficult to diagnose because pretraining corpora and cultural benchmarks are rarely indexed with comparable metadata\. Benchmarks increasingly target culturally situated phenomena at the level of languages, regions, and locale\-specific practices, while web\-scale corpora are usually organized only by language\. A shared culture\-language\-region layer makes these resources comparable, enabling audits of whether a target cultural phenomenon is represented in pretraining data, evaluated by benchmarks or both\. To this end, we introduce FineWeb\-CLaR, a large\-scale annotated dataset derived from FineWeb and FineWeb\-2 that places web documents on a shared culture–language–region axis for corpus auditing and benchmark alignment\.

FineWeb\-CLaR annotates the full 30\.9B\-document collection from FineWeb and FineWeb\-2 with URL\-derived region labels and cultural\-topic provenance\. Our region resolver assigns a non\-empty region to 25\.61% of documents \(7\.92B\)\. For cultural\-topic analysis, we induce locale\-specific topics and project them onto the 14 leaves of the Cultural Taxonomy of[Liu et al\. \(2025\)](https://arxiv.org/html/2609.25298#bib.bib17), producing Locale Topic Distributions \(LTDs\) for corpus\-side comparison\. We also annotate 277 cultural NLP benchmarks with the same taxonomy, language coverage, and region coverage\. Together, these resources enable direct comparison between corpus\-side pretraining evidence and benchmark\-side evaluation coverage\.111Code and data: [https://github\.com/Yusser96/FineWeb\-CLaR](https://github.com/Yusser96/FineWeb-CLaR) [https://huggingface\.co/collections/Yusser/culture\-datasets](https://huggingface.co/collections/Yusser/culture-datasets)

## 1Introduction

Cultural evaluation coverage and robustness in language models are difficult to diagnose because failures may stem from missing cultural evidence in pretraining data, weak benchmark coverage, poor alignment with target communities, or model behaviour that relies on surface associations rather than culturally grounded understanding\([Adilazuarda et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib18);[Chiu et al\., 2024a](https://arxiv.org/html/2609.25298#bib.bib20);[Naous and Xu, 2025](https://arxiv.org/html/2609.25298#bib.bib26);[Vo and Koyejo, 2025](https://arxiv.org/html/2609.25298#bib.bib27)\)\. Diagnosing these cases requires comparing the cultural phenomena targeted by benchmarks with the evidence available in pretraining data\.

Existing web\-scale pretraining corpora are not designed for this comparison\. Corpora such as FineWeb\([Penedo et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib15)\)and FineWeb\-2\([Penedo et al\., 2025](https://arxiv.org/html/2609.25298#bib.bib16)\)are organized primarily by detected language, but language is too coarse for culturally situated analysis: Many languages span multiple regions, institutions, communities, and communicative norms\. Following the Cultural Taxonomy \(CT\) of[Liu et al\. \(2025\)](https://arxiv.org/html/2609.25298#bib.bib17), we treat culture associally shared phenomenaorganized into ideational, linguistic, and social dimensions, covering knowledge, values, beliefs, norms, language varieties, social groups, and relations\. Existing language\-only corpus annotations cannot show which of these phenomena appear in which regional or locale\-specific slices of pretraining data\.

To that end, we introduce FineWeb\-CLaR, a large\-scale annotated dataset derived from FineWeb and FineWeb\-2 that places web documents on a shared language\-region\-culture axis\. The language axis comes from the original corpus; for the region axis we assign URL\-derived ISO 3166\-1 alpha\-2 labels with confidence metadata; and for the cultural\-labels axis we map the corpus to the 14 leaves of the CT via a topic\-based rule set\. Our contributions are:

- •We introduce FineWeb\-CLaR, a large\-scale version of FineWeb and FineWeb\-2 augmented with region metadata and CT\-aligned topic metadata for culture\-, language\-, and region\-aware corpus auditing\.
- •We propose a scalable URL\-based region attribution method for web\-scale data with confidence and resolution\-source metadata\.
- •We derive cultural\-taxonomy\-aligned locale\-level data using topic modeling, embedding\-based ranking and LLM adjudication\.
- •We extend prior taxonomy\-driven cultural benchmark surveys\([Liu et al\., 2025](https://arxiv.org/html/2609.25298#bib.bib17);[Namazifard and Galke, 2025](https://arxiv.org/html/2609.25298#bib.bib28)\)by annotating 212 additional cultural NLP benchmarks with locale metadata\.

## 2Related Work

##### Culture, benchmarks, and corpus audits\.

Recent work has argued that NLP systems and evaluations often conflate language, nationality, ethnicity, geography, and culture, while cultural evaluation resources vary in how they operationalise knowledge, values, norms, social relations, and situated behaviour\([Hershcovich et al\., 2022](https://arxiv.org/html/2609.25298#bib.bib19);[Adilazuarda et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib18);[Liu et al\., 2025](https://arxiv.org/html/2609.25298#bib.bib17)\)\. We build on the Cultural Taxonomy of[Liu et al\. \(2025\)](https://arxiv.org/html/2609.25298#bib.bib17), which organizes cultural phenomena into ideational, linguistic, and social dimensions, and use it as a shared coding scheme for corpus\-side topic metadata and benchmark\-side target phenomena\. Prior benchmark efforts evaluate multicultural factual knowledge, everyday cultural knowledge, values, social bias, stereotypes, and culturally grounded reasoning\([Myung et al\., 2024b](https://arxiv.org/html/2609.25298#bib.bib21);[Chiu et al\., 2024a](https://arxiv.org/html/2609.25298#bib.bib20);[Durmus et al\., 2023](https://arxiv.org/html/2609.25298#bib.bib23);[Pistilli et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib22);[Nangia et al\., 2020b](https://arxiv.org/html/2609.25298#bib.bib24);[Parrish et al\., 2022b](https://arxiv.org/html/2609.25298#bib.bib25)\)\. Taxonomy\-driven resource surveys and curations, including[Liu et al\. \(2025\)](https://arxiv.org/html/2609.25298#bib.bib17)and MUREL\([Namazifard and Galke, 2025](https://arxiv.org/html/2609.25298#bib.bib28)\), make these resources more comparable\. Our benchmark audit follows this line of work, but connects benchmark coverage to corpus\-side evidence on a pretraining scale, by annotating both with the same language–region\-taxonomy metadata\.

##### Web corpora and geographic attribution\.

Large web corpora such as CCNet, OSCAR, mC4, CC100, CulturaX, FineWeb, and FineWeb\-2 improve pretraining data transparency through language identification, filtering, deduplication, and quality\-control pipelines, but they are primarily organized by detected language rather than by region or locale\([Wenzek et al\., 2020](https://arxiv.org/html/2609.25298#bib.bib10);[Ortiz Suárez et al\., 2020](https://arxiv.org/html/2609.25298#bib.bib11);[Xue et al\., 2021](https://arxiv.org/html/2609.25298#bib.bib12);[Conneau et al\., 2020](https://arxiv.org/html/2609.25298#bib.bib13);[Nguyen et al\., 2023a](https://arxiv.org/html/2609.25298#bib.bib14);[Penedo et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib15);[Penedo et al\., 2025](https://arxiv.org/html/2609.25298#bib.bib16)\)\. Geographically organized web corpora are a step closer to our culture–language–region annotation objectives: the Corpus of Global Language Use indexes Common Crawl data by language\-country pairs\([Dunn, 2020](https://arxiv.org/html/2609.25298#bib.bib8)\), and GloWbE organizes web English into country\-specific varieties\([Davies and Fuchs, 2015](https://arxiv.org/html/2609.25298#bib.bib9)\)\. We follow this tradition of URL\-based geographic attribution, using ccTLDs, public\-suffix\-aware domain parsing, host and path tokens, and locale parameters\. Because such signals are noisy and do not directly identify the author’s location, audience, or cultural membership, we release region labels together with confidence and resolution\-source metadata\.

##### Topic modeling and cultural mapping\.

Topic modeling is widely used to summarize large corpora into interpretable thematic distributions, from classical models such as LDA\([Blei et al\., 2003](https://arxiv.org/html/2609.25298#bib.bib33)\)to neural methods such as BERTopic and FASTopic\([Grootendorst, 2022](https://arxiv.org/html/2609.25298#bib.bib34);[Wu et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib29)\)\. Multilingual topic models have also been used to compare culturally or linguistically distinct corpora\([Gutiérrez et al\., 2016](https://arxiv.org/html/2609.25298#bib.bib30)\)\. Our use of topic modeling differs from work that treats discovered topics as the final analysis object, and from supervised label\-aligned topic models that use task labels during training\([Yang et al\., 2025](https://arxiv.org/html/2609.25298#bib.bib31)\)\. Instead, we induce topics independently within each language\-region locale using multilingual document BGE\-M3 embeddings\([Chen et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib32)\), and FASTopic\([Wu et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib29)\), then project topic centroids post hoc onto the 14 leaves of the Cultural Taxonomy to obtain locale topic distributions\.

## 3FineWeb\-CLaR Corpus Annotation

We describe our two\-stage annotation approach for labeling FineWeb\([Penedo et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib15)\)and FineWeb\-2\([Penedo et al\., 2025](https://arxiv.org/html/2609.25298#bib.bib16)\)with region information and mapping the resulting language\-region locales to the Cultural Taxonomy of[Liu et al\. \(2025\)](https://arxiv.org/html/2609.25298#bib.bib17)\.

### 3\.1Region Attribution

Cultural benchmarks often specify evaluation targets at the level of countries, regions, dialect areas, or language–region communities\. Such annotations are usually missing in pretraining data, which are organized primarily by language\. We close this gap and add document\-level region metadata to FineWeb and FineWeb\-2 so that pretraining evidence can be aggregated into units closer to those used by cultural evaluations\.

#### 3\.1\.1Region Labels

A region label can refer to several different concepts, including author location, server location, target audience, domain jurisdiction, or the country conventionally associated with a language\. We use a narrower operational definition: for a URLuu, a region labelrrmeans that URL\-derived evidence supports an association betweenuuand the ISO 3166\-1 alpha\-2 regionrr\([International Organization for Standardization, 2026](https://arxiv.org/html/2609.25298#bib.bib5)\)\. Prior work has shown that web\-scale corpora can be organized geographically using URL\-, domain\-, and TLD\-derived evidence\([Dunn, 2020](https://arxiv.org/html/2609.25298#bib.bib8)\), but ccTLD and public\-web signals provide only partial and source\-dependent coverage of local web spaces\([Sommese et al\., 2023](https://arxiv.org/html/2609.25298#bib.bib7)\)\. We therefore use URL\-derived region labels only as weak auditing metadata, not as author\-location, audience\-location, or cultural\-membership labels\.

This narrow definition is necessary because URL evidence is scalable but noisy\. URLs and crawl metadata are available for web\-scale corpora and are preserved in Common Crawl WARC/WAT records\([Common Crawl, 2023](https://arxiv.org/html/2609.25298#bib.bib6)\)\. Regional signals such as ccTLDs, hostnames, path tokens, query parameters, and language headers can provide evidence of regional association, but they are imperfect: domains may be used outside their nominal jurisdictions, generic TLDs such as\.comare widespread, and hosting or domain choices do not directly identify authorship or cultural affiliation\. We therefore release region labels together with source and confidence metadata\.

Table 1:Search\-time priority chain for region resolution, with lookup\-backed sources inheriting table confidence \(high, medium or low\) and fallback sources receiving weak confidence labels\.
#### 3\.1\.2Resolver Design

Our region resolver separates offline region evidence aggregation from corpus\-scale annotation and emits three fields for each document: a region label, a resolution source, and a confidence value\. The region label is an ISO 3166\-1 alpha\-2 code orXXwhen no region can be resolved; the resolution source records which URL signal produced the label; and the confidence value records the resolver’s confidence in that assignment\.

In the offline stage, we build a frozen URL\-signature lookup table from Common Crawl records sampled across the availableCC\-MAINarchives, ranging fromCC\-MAIN\-2008\-2009toCC\-MAIN\-2026\-08\. Each sampled record contributes a URL and, when available, HTTPContent\-Language, HTMLlang, and language\-identification metadata from FineWeb\. We extract URL signatures at three granularities: domain, host, and host plus first path segment, then aggregate weighted evidence over candidate regions for each signature\. The following example illustrates the terminology:

https://travel\.​example\.co​\.uk⏟ccTLD⏟domain⏞host​/id/⏞first\_path⏟url\_hint⏞host\_path\.\.\.?locale=id\-ID⏟query\\texttt\{https://\}\\overbrace\{\\overbrace\{\\texttt\{travel\.\}\\underbrace\{\\texttt\{example\.co\}\\underbrace\{\\texttt\{\.uk\}\}\_\{\\text\{ccTLD\}\}\}\_\{\\text\{domain\}\}\}^\{\\text\{host\}\}\\underbrace\{\\overbrace\{\\texttt\{/id/\}\}^\{\\text\{first\\\_path\}\}\}\_\{\\text\{url\\\_hint\}\}\}^\{\\text\{host\\\_path\}\}\\texttt\{\.\.\.\}\\underbrace\{\\texttt\{?locale=id\-ID\}\}\_\{\\text\{query\}\}

Each lookup row stores the winning region, confidence tier, and supporting metadata; details of the evidence rules are provided in Appendix[A](https://arxiv.org/html/2609.25298#A1)\.

In the search\-time stage, we apply the frozen lookup table to every FineWeb and FineWeb\-2 document\. Given a URL, the resolver follows the fixed priority chain shown in Table[1](https://arxiv.org/html/2609.25298#S3.T1); the first source that emits a valid label determines the document’s region and resolution source\. Lookup\-backed matches inherit the confidence metadata stored in the frozen lookup table, while fallback sources receive separate weak confidence labels\. Details of the search\-time resolution sources are provided in Appendix[B](https://arxiv.org/html/2609.25298#A2)\. The production pass never inspects document text, which keeps region attribution independent of the cultural\-topic analysis in Section[3\.2](https://arxiv.org/html/2609.25298#S3.SS2)\.

Table 2:Region\-resolution diagnostics on the full FineWeb \+ FineWeb\-2 release, by resolution source \(top\) and released confidence tier \(middle\)\.high\+medium\+low\(15\.02%\) equals lookup\-table coverage plusquery\(highincludesquery\); the weakurl\_hintandccTLDtiers map one\-to\-one onto theurl\_hintandtldsource rows and are not repeated\.
#### 3\.1\.3Confidence and Coverage Settings

Our lookup table contains 1,335,000 URL\-signature rows\. Confidence is assigned after evidence aggregation rather than per individual crawl record, and falls into three tiers \(high,medium,low\) combining vote count, vote share, and dump diversity; exact thresholds are given in Appendix[A](https://arxiv.org/html/2609.25298#A1)\.

We report two coverage settings\. Thebroadsetting retains any non\-XXregion emitted by the resolver, including weak URL\-hint and ccTLD fallback labels\. Thelookup\-table coveragesetting retains only labels whose resolution source ishost\_path,host, ordomain, i\.e\. labels backed by the frozen lookup table\.querylabels are high\-confidence but not lookup\-backed; they are excluded from lookup\-table coverage by definition and reported separately \(0\.05% of documents\)\. This separation allows downstream users to trade off recall and precision according to the requirements of their analysis\. Table[2](https://arxiv.org/html/2609.25298#S3.T2)reports the empirical distribution of resolution sources on the full release\.

#### 3\.1\.4Resolver Evaluation Against Independent References

Defining a single gold\-standard region label for an arbitrary URL is difficult: the legal jurisdiction of the registrant, the physical hosting location, the content\-declared locale, and the intended audience may all disagree \(e\.g\., Egyptian\-recipe content written in Arabic, hosted by a French CDN on a generic\.comdomain, withog:locale=ar\_EG\)\. We therefore do not have a single reference standard for evaluation, and human annotations are impractical at this scale\. Instead, we report*convergence*between the resolver’s output and two independent automatic reference signals, each capturing a different notion of “region”:

Table 3:Resolver evaluation against independent automatic references on stratified random samples of 1,000 URLs per tier, comparing three notions of “region”\. Cells are percentages of the tier’s URLs with the required signal\(s\) available\.All==/All≠\\neq: all three sources coincide / are pairwise distinct\.- •IP geolocation\(MaxMind GeoLite2\([MaxMind, Inc\., 2026](https://arxiv.org/html/2609.25298#bib.bib1)\)\), capturing physical hosting jurisdiction\. For each URL we resolve the hostname to an IP address and look up its country in GeoLite2, yielding the jurisdiction of the server that serves the bytes\.
- •Content\-declared locale, captured from the HTMLlangattribute\([Phillips and Davis, 2009](https://arxiv.org/html/2609.25298#bib.bib2)\)and the Open Graphog:localeproperty\([Facebook, 2014](https://arxiv.org/html/2609.25298#bib.bib3)\)\. For each URL we re\-fetch the page, parse the root<html lang="…"\>attribute and anyog:localemeta tag, and take the region subtag \(e\.g\.en\-GBorar\_EG→\\toregionGBorEG\)\.

For each confidence tier of the resolver, Table[3](https://arxiv.org/html/2609.25298#S3.T3)reports the percentage of URLs whose label agrees with each reference, the percentage where the resolver and both references coincide, and the percentage where all three sources disagree — which we interpret as URLs that are genuinely ambiguous rather than as resolver failures\.

Two patterns are worth highlighting\. First, the all\-three\-disagree fraction increases monotonically with decreasing resolver confidence \(14\.4%, 19\.0%, 22\.6% acrosshigh,medium,low; rising to 33\.2% forbroad\-onlyURLs that have no lookup\-backed evidence at all\), indicating that the confidence tiers correctly identify which URLs are inherently ambiguous across reference signals\. Second, agreement withog:localedrops sharply across tiers \(41\.4%→\\to16\.1%→\\to9\.6%\), confirming that high\-tier URLs are also the ones whose content most frequently declares the same locale\. Agreement with IP geolocation is broadly tier\-invariant \(∼\\sim17–22%\) because IP measures physical hosting, which is largely orthogonal to the resolver’s audit\-unit semantics; the higher 37\.8% forbroad\-onlyreflects ccTLD\- and hint\-based URLs whose region label trivially aligns with hosting jurisdiction\. Both reference signals are themselves imperfect: IP geolocation captures hosting rather than authorship, andog:localeis content\-declared rather than verified\. We therefore treat this analysis as a convergence study rather than a validation against ground truth, and recommend that downstream users rely on the released confidence tiers when interpreting individual region labels\.

### 3\.2Culture\-Taxonomy Annotation

The region axis introduced in Section[3\.1](https://arxiv.org/html/2609.25298#S3.SS1)identifies where a document plausibly originates from, but it does not identify what kind of cultural phenomenon is present in the document or its local corpus context\. We therefore add cultural\-topic labels to the region\-resolved corpus\.

#### 3\.2\.1Taxonomy and Label Space

We operationalize the Cultural Taxonomy \(CT\) of[Liu et al\. \(2025\)](https://arxiv.org/html/2609.25298#bib.bib17)as the cultural label space\. The CT organizes cultural phenomena into ideational, linguistic, and social dimensions\. Figure[1](https://arxiv.org/html/2609.25298#S3.F1)summarizes the label space used for topic alignment\. The Values element of the Ideational branch is further refined into four sub\-leaves, yieldingC=14C=14leaf categories in total\. The goal of our mapping is to estimate which CT categories are evidenced by a document’s topic context\.

Cultural Taxonomy \(CT\)Ideational 8 leaves • Concepts • Knowledge • Norms and Morals • Artifacts • Values→\\rightarrowGeneral; Bias; Hate; Other PerceptionsLinguistic 2 leaves • Dialects • Styles, Registers and GenresSocial 4 leaves • Relationship • Context • Communicative Goals • Demographics

Figure 1:Cultural Taxonomy label space used for corpus\-side topic alignment\.
#### 3\.2\.2Locale\-Specific Topic Induction

Human document\-level CT annotation at the scale of FineWeb and FineWeb\-2 is infeasible\. We therefore use a topic labeling pipeline over language–region locales\. Not all locales contain enough usable text for stable topic modeling: a locale is eligible when it contains at least 500 documents passing a quality filter \(200–50,000 characters, language confidence≥0\.8\\geq 0\.8\), which 6,332 locales meet\. We sample up to 10,000 quality\-filtered documents per locale; discarding locales whose retrieved sample falls below the 500\-document threshold, yielding 5,954 topic\-modeled locales spanning 297 languages \(322 distinct language–script combinations\)\. The resulting culturally annotated resource is therefore a subset of the full region\-annotated corpus\.

For each retained locale, we encode documents using multilingual BGE\-M3\([Chen et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib32)\)and induce topics independently with FASTopic\([Wu et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib29)\)\. Modeling each locale separately prevents high\-resource languages and globally dominant regions from determining the topic space for smaller locales, and allows locale\-specific cultural content to form separate clusters\. We use multilingual BGE\-M3 because the pipeline requires a single shared embedding space in which document embeddings, topic centroids, and CT\-leaf description embeddings are directly comparable by cosine similarity across all locales; we use its dense 1,024\-dimensional embeddings\. The language counts above describe the corpus language codes of the retained locales, not the validated language coverage of BGE\-M3\. Documents are truncated before encoding: the initial embedding pass uses BGE\-M3’s maximum input length of 8,192 tokens, and the consolidation pass that completed the final locale set caps inputs at 1,024 tokens\.

We use a fixed initial maximum number of topicsK=200K=200for every retained locale\. This keeps the topic\-induction hyperparameter constant across locales, which is important for the cross\-locale comparisons in Section[5\.3](https://arxiv.org/html/2609.25298#S5.SS3)\.222The fixed value should be interpreted as an overcomplete topic budget rather than as the final number of topics: the effective topic count is determined after the merge step\.

FASTopic\([Wu et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib29)\)returns a document\-topic distribution and topic descriptors\. For each topic, we compute a centroid as the mean BGE\-M3 embedding of the documents whose highest\-probability topic assignment is that topic\. We use these centroids for CT projection to the BGE\-M3 CT leaf\-description embeddings\.

The fixed topic budget can produce near\-duplicate topics, especially in smaller locales\. We therefore merge topics using agglomerative average\-linkage clustering from scikit\-learn\([Pedregosa et al\., 2011](https://arxiv.org/html/2609.25298#bib.bib4)\)over cosine distance between topic centroids, cutting clusters at cosine similarity≥0\.85\\geq 0\.85\. For each merged topic, we sum the corresponding columns of the document\-topic matrix, compute a document\-count\-weighted centroid, and retain the top unique words from the merged member topics\. The merged topics are then passed to the topic\-to\-taxonomy projection step\. Merging operates within each locale independently; across the 5,954 topic\-modeled locales, theK=200K=200per\-locale budget yields 801,344 merged topics in total, an effective average of≈\\approx135 topics per locale\.

#### 3\.2\.3Topic\-to\-Taxonomy Projection

We project topic centroids onto CT\-leaf embeddings \(both encoded with BGE\-M3\) via a temperature\-scaled cosine softmax withτ=0\.05\\tau=0\.05; full equation and rationale in Appendix[C](https://arxiv.org/html/2609.25298#A3)\.

#### 3\.2\.4Low\-Confidence Adjudication

The embedding\-based projection is used as the default topic\-to\-CT mapping\. Topics whose maximum CT probability falls below 0\.5 receive LLM adjudication\. Because the temperature\-scaled softmax atτ=0\.05\\tau=0\.05is diffused over the 14\-leaf label space \(observedmaxc⁡p⁡\(c∣t\)\\max\_\{c\}p\(c\\mid t\): mean 0\.124, 99th percentile 0\.315\), this criterion selects 99\.97% of merged topics \(801,068 of 801,344\); adjudication is therefore near\-universal, and the 0\.5 cutoff acts as a guardrail that bypasses adjudication only for unambiguous projections\. The adjudicator receives the topic descriptors together with the top\-three CT candidate categories and their softmax probabilities, and either confirms the automatic label or selects a better\-fitting CT leaf; it changed the primary assignment for 7\.1% of adjudicated topics \(56,931\)\. Table[4](https://arxiv.org/html/2609.25298#S3.T4)shows example FASTopic topics and their final CT leaf assignments for selected locales\.

Table 4:Example FASTopic topics and final CT leaf assignments for selected locales\.
#### 3\.2\.5Locale Topic Distributions

The output of the corpus\-side cultural\-topic pipeline is a Locale Topic Distributions \(LTD\) for each topic\-modeled locale\. For a locale withNNdocuments,KKmerged topics, andCCCT categories, letD∈ℝN×KD\\in\\mathbb\{R\}^\{N\\times K\}be the document–topic distribution andA∈ℝK×CA\\in\\mathbb\{R\}^\{K\\times C\}be the topic\-to\-CT assignment matrix\. The locale’s Locale Topic Distribution is:

LTD=1N​∑i=1N\(D​A\)i\.\\mathrm\{LTD\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\(DA\)\_\{i\}\.The resulting vector is L1\-normalized and interpreted as the locale’s distribution over CT leaves\. Across all topic\-modeled locales, this procedure produces a5,954×C5\{,\}954\\times Cmatrix of LTD, withC=14C=14\. We use this matrix for distributional analyses of locale\-level cultural\-topic profiles, including the same\-language regional divergence analysis in Section[5\.3](https://arxiv.org/html/2609.25298#S5.SS3)\.

## 4Benchmark Audit

To compare corpus\-side cultural evidence with evaluation demand, we construct a benchmark\-side audit using the same Cultural Taxonomy label space and locale metadata as FineWeb\-CLaR\. The audit extends prior taxonomy\-driven cultural\-NLP surveys\([Liu et al\., 2025](https://arxiv.org/html/2609.25298#bib.bib17);[Namazifard and Galke, 2025](https://arxiv.org/html/2609.25298#bib.bib28)\)by adding newly identified datasets and by coding each resource for cultural, language, and region coverage, provenance, and usability\.

##### Dataset collection\.

We compile cultural NLP benchmarks from three sources: datasets included in the cultural\-NLP taxonomy and survey of[Liu et al\. \(2025\)](https://arxiv.org/html/2609.25298#bib.bib17), datasets included in MUREL\-style cultural\-resource curation\([Namazifard and Galke, 2025](https://arxiv.org/html/2609.25298#bib.bib28)\), and newly identified resources from systematic searches of the ACL Anthology, arXiv, HuggingFace, GitHub, and dataset repositories\. After deduplication and filtering, the audit contains 277 cultural NLP benchmark datasets \(43 inherited from the Liu et al\. taxonomy survey, 3 from MUREL, 19 shared by both, and 212 newly identified\)\. A resource is included when it provides an accessible dataset, targets at least one cultural phenomenon covered by the CT, and is documented well enough to recover its language, region, task, and provenance metadata\. Figure[2](https://arxiv.org/html/2609.25298#S4.F2)shows the per\-leaf coverage of the expanded audit\. The newly identified datasets increase coverage across most CT leaves, including linguistic and social categories that are comparatively sparse in the inherited survey sources\.

RelationshipCommunicative GoalsValues \- otherArtifactsDemographicsValues \- hateValues \- generalDialectsStyles/Reg\./GenresContextConceptsValues \- biasKnowledgeNorms and Morals771111121212121212131315151515161617172121282832323636Figure 2:Number of audited benchmark datasets per CT leaf \(n=277n=277\)\.
##### Annotation schema\.

Each benchmark is annotated along four groups of fields\. Cultural coverage records the CT branch and one or more CT leaves targeted by the dataset\. Scope records the covered languages, regions and example items when available\. Provenance records the source paper, venue, publication date, source list, and dataset lineage\. Usability records whether the dataset is public, restricted, or unavailable\.

ConceptsKnowledgeValues \(General\)Values \(Bias\)Values \(Hate\)Values \(Other\)Norms & MoralsArtifactsDialectsStyles/Reg\./GenresRelationshipContextComm\. GoalsDemographicsN\. AmericaW\. EuropeE\. AsiaS\. AsiaSE AsiaW\. AsiaS\. EuropeE\. EuropeS\. AmericaN\. EuropeE\. AfricaS\. AfricaW\. AfricaN\. AfricaC\. AsiaOceaniaC\. AmericaMiddle AfricaCaribbeanDouble blind spot \(neither\)FineWeb\-CLaR only \(data, no benchmark\)Survey only \(benchmark, no data\)Both presentFigure 3:Coverage over CT leaf×\\timesregion cells on the full release \(30\.9B documents\)\. A cell is corpus\-covered if at leastk=50k=50documents support the CT\-leaf/region pair, and benchmark\-covered if at least one audited benchmark targets it\. All 266 cells are corpus\-covered; light cells \(56\. 21%\) have corpus evidence but no audited benchmark\.
##### Human annotation and validation\.

Two authors annotated the benchmarks and resolved disagreements through discussion\. CT branch and CT leaf labels were assigned using each dataset’s paper, dataset card, repository, and example items\. A dataset may receive multiple CT leaves when it evaluates more than one cultural phenomenon\. Details of the search protocol, annotation rubric, lineage coding, accessibility coding, and quality\-control procedure are provided in Appendix[D](https://arxiv.org/html/2609.25298#A4)\. Inter\-annotator agreement on the multi\-label Cultural Taxonomy assignments was 0\.43 average pairwise Jaccard, with 0\.89 partial agreement when any shared CT branch/leaf label counted as overlap\.333We release the full benchmark survey in our code repository\.

## 5Experiments and Analysis

We evaluate FineWeb\-CLaR as a corpus\-auditing resource through three analyses: human annotation, corpus\-benchmark mismatch, and same\-language regional divergence\.

### 5\.1Validation of the Topic\-to\-CT Projection

Two annotators independently coded 100 \(topic, CT\-leaf\) pairs sampled stratified across the 14 CT leaves and across high\-, medium\-, and low\-resource locales\. Each pair was presented with the top\-15 FASTopic descriptors, ten randomly drawn documents from the topic, and the natural\-language definition of each CT leaf\. Annotators selected the single best\-fitting CT leaf orNONE\. Annotators did not see the embedding\-softmax or LLM\-adjudicated labels during annotation\.

Cohen’sκ\\kappabetween annotators was 0\.25, indicating only fair agreement on the topic\-to\-CT\-leaf task and confirming that CT\-leaf assignment at the topic level is intrinsically ambiguous\. On 15% of topics the two annotators could not converge on a single leaf; the remainingN=85N=85topics form the consensus reference set\. Against this reference, the embedding\-softmax label agreed on 18\.8% of cases \(Cohen’sκ\\kappa= 0\.13\); the LLM adjudicator, which was triggered on all sampled topics and altered the embedding\-softmax assignment for 35% of them, agreed on 22\.4% of cases \(Cohen’sκ\\kappa= 0\.15\)\. Both automatic methods are well below human consensus, and the adjudicator yields only marginal improvements over the embedding baseline\. We therefore interpret LTDs not as definitive CT\-leaf assignments but as locale\-level distributions over topic\-to\-leaf*prevalences*under substantial labeling uncertainty\. Accordingly, we treat the CT\-level analyses that follow as exploratory corpus audits: the coverage classification in Section[5\.2](https://arxiv.org/html/2609.25298#S5.SS2)depends on document counts per cell rather than on individual leaf assignments, and the divergence analysis in Section[5\.3](https://arxiv.org/html/2609.25298#S5.SS3)compares whole distributions between locales that share a language, but both inherit the label uncertainty quantified here\. Appendix[E](https://arxiv.org/html/2609.25298#A5)quantifies this inheritance by resampling every topic\-to\-leaf assignment at the observed agreement rate \(B=1000B=1000replicates\): the count\-based coverage results and the pairs\-above\-noise\-floor finding hold in every replicate of every condition, while exact pair orderings are only partially preserved\.

### 5\.2Corpus–Benchmark Mismatch

The shared language–region–CT axis allows us to compare corpus evidence with benchmark demand\. On the corpus side, FineWeb\-CLaR provides document counts and Locale Topic Distributions for language–region locales\. On the benchmark side, the human annotation in Section[4](https://arxiv.org/html/2609.25298#S4)provides CT leaves, language coverage, and region coverage for 277 cultural NLP benchmarks\.

We mark a CT leaf–region cell as corpus\-covered when at leastk=50k=50documents are associated with that cell\. Each cell is then classified as both present, corpus only, benchmark only, or neither present, depending on whether it appears in FineWeb\-CLaR, the benchmark inventory, both, or neither\.

Figure[3](https://arxiv.org/html/2609.25298#S4.F3)shows that, on the full release, corpus evidence exceeds thek=50k=50threshold for all 266 CT\-leaf×\\timesregion cells, so the mismatch is benchmark\-side: 56 cells \(21%\) carry corpus evidence but no audited benchmark, concentrated regionally in the Caribbean \(11 of 14 leaves\), Central America \(10\), Central Asia \(7\), and Middle Africa \(6\), and thematically in the Social branch \(Context 10, Communicative Goals 10\) and the Values leaves \(18\)\. Ideational categories such as*Knowledge*,*Concepts*, and*Artifacts*are more often visible in web text and more often converted into benchmark tasks\. Thus, corpus–benchmark mismatch is not only a language\-level problem; it also depends on which cultural phenomenon is being evaluated\.

ConceptsKnowledgeValues \(General\)Values \(Bias\)Values \(Hate\)Values \(Other\)Norms & MoralsArtifactsDialectsStyles/Reg\./GenresRelationshipContextComm\. GoalsDemographicsfra\-WFsms\-FIkua\-NAfra\-RWven\-ZAaeb\-TNnso\-ZAcrs\-SCurd\-PKkal\-GLtso\-ZAche\-RUarz\-SApbt\-PKuzs\-AFgug\-PYssw\-ZAtsn\-ZAfrr\-DEkrc\-RUdan\-GLilo\-PHoci\-ESarb\-SSukr\-MDuzn\-AFmdf\-RUfra\-BLspa\-GItel\-INmfe\-MUarb\-TDwar\-PHvec\-ITuzs\-UZnbl\-ZAsot\-LSarb\-DJukr\-RSkoi\-RUgom\-INdzo\-BTspa\-SXfra\-MFnya\-MWDouble blind spot \(neither\)FineWeb\-CLaR only \(data, no benchmark\)Survey only \(benchmark, no data\)Both presentFigure 4:CT leaf×\\timeslocale coverage for the smallest topic\-modeled main\-language locales on the full release\. Locale\-level analysis reveals the double blind spots and benchmark\-only cells that macro\-region aggregation hides\.Earlier versions of this analysis, computed on analysis corpora in which English was represented through the FineWebsample\-350BTsubset, showed several region\-level double blind spots, including the entire Caribbean column\. Recomputing the analysis on the full release provides a more complete picture: these patterns do not persist at full scale and were primarily driven by the∼\\sim50×\\timesdownsampling of English rather than by an absence of corpus evidence \(see Appendix[F](https://arxiv.org/html/2609.25298#A6)for details on the different sampling experiments\)\. The change is specific to English: English\-locale mass in these regions increases by 46–56×\\timeson the full release, while their non\-English main\-language mass remains unchanged\. The full\-release analysis therefore resolves the apparent macro\-region blind spots observed in earlier versions while revealing more fine\-grained coverage differences at the locale level \(Figure[4](https://arxiv.org/html/2609.25298#S5.F4)\)\. Applying the same classification at locale granularity shows that 26\.5% of cells are covered by both corpus and benchmarks, 63\.9% by the corpus only, 1\.9% by benchmarks only, and 7\.7% by neither\. Chichewa \(nya\-MW; 546 documents\) lacks both corpus mass and benchmarks across all 14 leaves; Dzongkha \(dzo\-BT\) is benchmark\-covered but corpus\-sparse on 6 leaves; and even en\-GB, the largest locale \(1\.33B documents\), has no benchmark coverage on 8 of 14 leaves\. Overall, 863 locales \(14\.8%\) are covered by no region\-scoped benchmark, including 67 main\-language locales in their respective countries \(e\.g\., Bosnian bs\-BA with 3\.0M documents\)\. Figure[4](https://arxiv.org/html/2609.25298#S5.F4)shows the locale\-level coverage matrix for the smallest main\-language locales\.

10−310^\{\-3\}10−210^\{\-2\}10−110^\{\-1\}zh\-CN/zh\-TWpt\-BR/pt\-PTen\-IN/en\-USar\-EG/ar\-MAes\-ES/es\-MX166 languages with≥2\\geq 2locales, ranked by median JSDwithin\-language JSDper\-language median \(IQR bars\)split\-half noise floor \(p95=1\.9×10−3p\_\{95\}=1\.9\\times 10^\{\-3\}\)Figure 5:Within\-language LTD divergence \(JSD\) across all languages with≥2\\geq 2locales: per\-language median \(dots\) and IQR \(bars\), with illustrative same\-language locale pairs labelled\. The shaded band and dashed line show the split\-half noise floor \(p95=1\.9×10−3p\_\{95\}=1\.9\\times 10^\{\-3\}\)\.
### 5\.3Same\-Language Regional Divergence

Language\-level corpus partitions can hide regional variation\. We quantify this by computing the Jensen–Shannon distance \(JSD full definition in Appendix[C](https://arxiv.org/html/2609.25298#A3)\) between Locale Topic Distributions for locales that share a language but differ in region\. Higher JSD indicates greater divergence between the two locale\-level CT distributions\.

To distinguish genuine regional divergence from topic\-modeling noise, we compute a within\-locale split\-half JSD baseline\. For each of the 4,715 locales with at least 1,000 sampled documents, we randomly partition the locale’s documents into two halves, fit an LTD on each half, and compute the JSD between halves; we repeat this 50 times per locale \(235,750 split\-half computations in total, seed 42\)\. The pooled distribution yields a median JSD of4\.9×10−44\.9\\times 10^\{\-4\}\(IQR\[2\.4×10−4,8\.8×10−4\]\[2\.4\\times 10^\{\-4\},\\,8\.8\\times 10^\{\-4\}\],p95=1\.9×10−3p\_\{95\}=1\.9\\times 10^\{\-3\}\), which we use as the topic\-model noise floor\. The floor decreases with locale size \(full details in Appendix[E\.3](https://arxiv.org/html/2609.25298#A5.SS3)\); the five labelled pairs below involve locales at the 10,000\-document cap, where thep95p\_\{95\}floor is roughly half the pooled value, so the pooled floor if anything understates their separation from noise\.

Figure[5](https://arxiv.org/html/2609.25298#S5.F5)shows the distribution of within\-language JSD across every language with at least two locales meeting the 500\-document floor\. The labelled examples all lie well above the split\-half noise floor, indicating that same\-language regional differences in LTDs are substantially larger than topic\-modeling noise\. The largest labelled divergences occur for the Arabic and English locale pairs \(ar\-EG/ar\-MA 0\.079, en\-IN/en\-US 0\.071\), followed by Chinese \(zh\-CN/zh\-TW 0\.047\), while the Spanish and Portuguese pairs \(0\.035, 0\.028\) show smaller but still clearly non\-noise divergence\. We interpret this as evidence that benchmarks targeting a language may correspond to specific regional CT profiles, while noting that the magnitude of the language\-level/regional gap varies substantially across language families\.

## 6Conclusion

We introduced FineWeb\-CLaR, a large\-scale resource that augments FineWeb and FineWeb\-2 with region metadata and Cultural Taxonomy\-aligned topic annotations to support culture\-, language\-, and region\-aware corpus auditing\. By combining URL\-derived region attribution, locale\-specific topic modeling, and an expanded audit of cultural NLP benchmarks, FineWeb\-CLaR enables direct comparison between pretraining evidence and evaluation demand\. Our analyses show that pretraining evidence at web scale exists for every cultural\-topic×\\timesregion cell we audit, but is unevenly distributed across regions by more than two orders of magnitude, while benchmark coverage is absent for 21% of region\-level cells; at locale granularity, coverage gaps run in both directions, with 7\.7% of locale×\\timescultural\-topic cells lacking both corpus evidence and benchmarks\. Corpus–benchmark mismatch is therefore structured by region and cultural phenomenon, and must be audited at locale level, beyond language\-only metadata\.

## Limitations

FineWeb\-CLaR is an auditing resource, not demographic or cultural ground truth\. Its language, region, and CT axes make corpora and benchmarks comparable, but they do not define cultures or communities\. Region labels are derived from URL evidence and should not be interpreted as author location, user location, national identity, or cultural membership\. They may be affected by multi\-region websites, generic or vanity domains, CDNs, URL parsing errors, branded ccTLDs, and the use of a sampled Common Crawl lookup table\. False positives and false negatives are therefore unavoidable\.

The CT annotation pipeline is also coarse\. CT labels support corpus\-level comparison, but they cannot capture the full complexity of cultural practice, identity, power, tacit knowledge, interactional norms, or intra\-community variation\. Locale\-specific topics may reflect domain composition, crawl artefacts, or institutional web presence rather than cultural salience alone\. Embedding\-based CT ranking and LLM\-assisted adjudication reduce annotation cost, but can introduce errors, especially for low\-resource languages, mixed\-language documents, and culturally specific concepts that are weakly represented in the embedding model or adjudicator\. Human projection quality is validated only in aggregate \(Section[5\.1](https://arxiv.org/html/2609.25298#S5.SS1)\), and BGE\-M3’s validated language coverage is narrower than the corpus language inventory, so CT\-level results should be read as exploratory; computationally, 67% of merged topics in locales outside BGE\-M3’s documented coverage project onto the taxonomy with near\-uniform confidence, against 46% inside \(Appendix[E\.2](https://arxiv.org/html/2609.25298#A5.SS2)\), so LTDs for out\-of\-coverage languages carry additional uncertainty\. Empirically, locale evidence is often domain\-concentrated: the median topic\-modeled locale draws on 116 distinct base domains, and 34% of locales draw more than half of their documents from a single domain \(Figure[6](https://arxiv.org/html/2609.25298#A6.F6)\); 83 locales \(1\.4%\) rest on a single base domain \(verified exhaustively against the full release\), typically a multilingual portal or machine\-translated content farm attributed to one region\. Locale\-level distributions for such locales should be read as summaries of a few dominant sites rather than of a broad regional web\.

Locale Topic Distributions are available only for locales with sufficient document support\. Locales below the 500 quality\-filtered\-document threshold remain region\-annotated, but do not receive LTDs because their topic distributions would be dominated by sampling noise\. Even among the 5,954 topic\-modeled locales, smaller locales should be interpreted with greater uncertainty\.

The benchmark survey is time\-bounded and depends on the availability and documentation of existing datasets\. Human CT coding reduces noise, but does not establish that a benchmark validly represents the community or cultural phenomenon it claims to test\. The coverage matrix should therefore be read as a diagnostic of available resources, not as a normative statement about which cultures or cultural dimensions matter\.

## Ethics Statement

This work annotates public web\-corpus metadata for auditing and analysis\. Because region labels can be misused if treated as demographic labels, we release confidence and source metadata and recommend against using weak labels for claims about people, identity, or community representation\. Cultural\-taxonomy annotations should likewise be used to identify broad coverage patterns, not to essentialise cultures or reduce communities to fixed attributes\. Downstream use for data filtering, benchmark construction, or model evaluation should document the selected confidence threshold, the intended interpretation of region labels, and the limitations of URL\-derived evidence\.

## Acknowledgments

This research was supported by the German Federal Ministry for Economic Affairs and Energy \(BMWE\) as part of the project“Souveräne KI für Europa \(SOOFI\)”\(13IPC040H\), and by the German Federal Ministry of Research, Technology and Space \(BMFTR\) as part of the project TRAILS \(01IW24005\)\. We thank Julia Kreutzer for her valuable feedback on an early draft of the paper\. We also thank AriaRay Brown for helping with a part of the human annotations on the topic assignment\.

## References

- Aakankshaet al\.\(2024\)Aakanksha, A\. Ahmadian, B\. Ermis, S\. Goldfarb\-Tarrant, J\. Kreutzer, M\. Fadaee, and S\. HookerThe multilingual alignment prism: aligning global and local preferences to reduce harm\.External Links:2406\.18682,[Link](https://arxiv.org/abs/2406.18682)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1)\.
- Abdul\-Mageedet al\.\(2023\)M\. Abdul\-Mageed, A\. Elmadany, C\. Zhang, E\. M\. B\. Nagoudi, H\. Bouamor, and N\. HabashNADI 2023: the fourth nuanced Arabic dialect identification shared task\.InProceedings of ArabicNLP 2023,Singapore \(Hybrid\),pp\. 600–613\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.arabicnlp-1.62),[Link](https://aclanthology.org/2023.arabicnlp-1.62/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1)\.
- Adilazuardaet al\.\(2024\)M\. F\. Adilazuarda, S\. Mukherjee, P\. Lavania, S\. S\. Singh, A\. F\. Aji, J\. O’Neill, A\. Modi, and M\. ChoudhuryTowards measuring and modeling “culture” in LLMs: a survey\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 15763–15784\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.882),[Link](https://aclanthology.org/2024.emnlp-main.882/)Cited by:[§1](https://arxiv.org/html/2609.25298#S1.p1.1),[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px1.p1.1)\.
- Al Kautsaret al\.\(2025\)M\. D\. Al Kautsar, A\. Candra, M\. A\. Al Hakim, M\. S\. Kahfi, F\. Koto, A\. F\. Aji,et al\.SEADIALOGUES: a multilingual culturally grounded multi\-turn dialogue dataset on southeast asian languages\.External Links:2508\.07069,[Link](https://arxiv.org/abs/2508.07069)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Ayashet al\.\(2025\)L\. Ayash, H\. Alhuzali, A\. Alasmari, and S\. AloufiSaudiCulture: a benchmark for evaluating large language models cultural competence within Saudi Arabia\.External Links:2503\.17485,[Link](https://arxiv.org/abs/2503.17485)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Barriereet al\.\(2025\)V\. Barriere, N\. Gomez, L\. Hemamou, S\. Callejas, and B\. RavenetStandUp4AI: a new multilingual dataset for humor detection in stand\-up comedy videos\.External Links:2505\.18903,[Link](https://arxiv.org/abs/2505.18903)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Bassignanaet al\.\(2018\)E\. Bassignana, V\. Basile, and V\. PattiHurtlex: a multilingual lexicon of words to hurt\.InProceedings of the Fifth Italian Conference on Computational Linguistics \(CLiC\-it 2018\),Turin, Italy,pp\. 52–57\.External Links:[Link](https://aclanthology.org/2018.clicit-1.11/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1)\.
- Becattiniet al\.\(2023\)F\. Becattini, P\. Bongini, L\. Bulla, M\. Bertini, L\. Marinucci, M\. Mongiovì, and A\. GangemiVISCOUNTH: a large\-scale multilingual visual question answering dataset for cultural heritage\.ACM Transactions on Multimedia Computing, Communications, and Applications19\(6\),pp\. 1–20\.External Links:[Document](https://dx.doi.org/10.1145/3590773),[Link](https://dl.acm.org/doi/10.1145/3590773)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Belayet al\.\(2025\)T\. D\. Belay, A\. H\. Ahmed, A\. Grissom II, I\. Ameer, G\. Sidorov, O\. Kolesnikova, and S\. M\. YimamCULEMO: cultural lenses on emotion – benchmarking LLMs for cross\-cultural emotion understanding\.External Links:2503\.10688,[Link](https://arxiv.org/abs/2503.10688)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Bhutaniet al\.\(2024\)M\. Bhutani, K\. C\. Robinson, V\. Prabhakaran, S\. Dave, and S\. DevSeeGULL multilingual: a dataset of geo\-culturally situated stereotypes\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),Bangkok, Thailand,pp\. 842–854\.External Links:[Link](https://aclanthology.org/2024.acl-short.75/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Bleiet al\.\(2003\)D\. M\. Blei, A\. Y\. Ng, and M\. I\. JordanLatent Dirichlet allocation\.Journal of Machine Learning Research3,pp\. 993–1022\.Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px3.p1.1)\.
- Briakouet al\.\(2021\)E\. Briakou, D\. Lu, K\. Zhang, and J\. TetreaultOlá, bonjour, salve\! XFORMAL: a benchmark for multilingual formality style transfer\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,External Links:[Link](https://aclanthology.org/2021.naacl-main.256/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1)\.
- Buiet al\.\(2025\)M\. D\. Bui, K\. v\. d\. W\. F\. D\. Schmid, J\. Plepi,et al\.Multi3Hate: multimodal, multilingual, and multicultural hate speech detection with vision\-language models\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies,External Links:[Link](https://aclanthology.org/2025.naacl-long.490/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1)\.
- Cahyawijayaet al\.\(2025\)S\. Cahyawijaya, D\. Chen, Y\. Bang, L\. Khalatbari, B\. Wilie, Z\. Ji, E\. Ishii, and P\. FungHigh\-dimension human value representation in large language models\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),Albuquerque, New Mexico,pp\. 5303–5330\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.274),[Link](https://aclanthology.org/2025.naacl-long.274/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Caoet al\.\(2023\)Y\. Cao, Y\. Kementchedjhieva, R\. Cui, A\. Karamolegkou, L\. Zhou, M\. Dare, L\. Donatelli, and D\. HershcovichCultural adaptation of recipes\.External Links:2310\.17353,[Link](https://arxiv.org/abs/2310.17353)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Casolaet al\.\(2024\)S\. Casola, S\. Frenda, S\. M\. Lo, E\. Sezerer, A\. Uva, V\. Basile, C\. Bosco, A\. Pedrani, C\. Rubagotti, V\. Patti, and D\. BernardiMultiPICo: multilingual perspectivist irony corpus\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),External Links:[Link](https://aclanthology.org/2024.acl-long.849/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- CH\-Wanget al\.\(2023\)S\. CH\-Wang, A\. Saakyan, O\. Li, Z\. Yu, and S\. MuresanSociocultural norm similarities and differences via situational alignment and explainable textual entailment\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 3548–3564\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.215/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Chakravarthi \(2020\)B\. R\. ChakravarthiHopeEDI: a multilingual hope speech detection dataset for equality, diversity, and inclusion\.InProceedings of the Third Workshop on Computational Modeling of People’s Opinions, Personality, and Emotion’s in Social Media,Barcelona, Spain \(Online\),pp\. 41–53\.External Links:[Link](https://aclanthology.org/2020.peoples-1.5/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Chenet al\.\(2023\)C\. Chenet al\.C\-MFD2\.0: developing a Chinese moral foundation dictionary\.Computational Communication Research5\(2\)\.External Links:[Document](https://dx.doi.org/10.5117/CCR2023.2.10.CHEN),[Link](https://www.aup-online.com/content/journals/10.5117/CCR2023.2.10.CHEN)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Chenet al\.\(2024\)J\. Chen, S\. Xiao, P\. Zhang, K\. Luo, D\. Lian, and Z\. LiuM3\-embedding: multi\-linguality, multi\-functionality, multi\-granularity text embeddings through self\-knowledge distillation\.External Links:2402\.03216,[Link](https://arxiv.org/abs/2402.03216)Cited by:[Appendix G](https://arxiv.org/html/2609.25298#A7.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px3.p1.1),[§3\.2\.2](https://arxiv.org/html/2609.25298#S3.SS2.SSS2.p2.1)\.
- Chiuet al\.\(2024a\)Y\. Y\. Chiu, L\. Jiang, B\. Y\. Lin, C\. Y\. Park, S\. S\. Li, S\. Ravi, M\. Bhatia, M\. Antoniak, Y\. Tsvetkov, V\. Shwartz, and Y\. ChoiCulturalBench: a robust, diverse, and challenging cultural benchmark by human\-ai cultural teaming\.External Links:2410\.02677,[Link](https://arxiv.org/abs/2410.02677)Cited by:[§1](https://arxiv.org/html/2609.25298#S1.p1.1),[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px1.p1.1)\.
- Chiuet al\.\(2024b\)Y\. Y\. Chiu, L\. Jiang, B\. Y\. Lin, C\. Y\. Park, S\. S\. Li, S\. Ravi, M\. Bhatia, M\. Antoniak, Y\. Tsvetkov, V\. Shwartz, and Y\. ChoiCulturalBench: a robust, diverse, and challenging cultural benchmark by human\-AI CulturalTeaming\.External Links:2410\.02677,[Link](https://arxiv.org/abs/2410.02677)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.15.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Clarket al\.\(2020\)J\. H\. Clark, E\. Choi, M\. Collins, D\. Garrette, T\. Kwiatkowski, V\. Nikolaev, and J\. PalomakiTyDi qa: a benchmark for information\-seeking question answering in typologically diverse languages\.External Links:2003\.05002,[Link](https://arxiv.org/abs/2003.05002)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Common Crawl \(2023\)Common CrawlWeb archiving file formats explained\.Note:[https://commoncrawl\.org/blog/web\-archiving\-file\-formats\-explained](https://commoncrawl.org/blog/web-archiving-file-formats-explained)Accessed 2026\-05\-22Cited by:[§3\.1\.1](https://arxiv.org/html/2609.25298#S3.SS1.SSS1.p2.1)\.
- Conneauet al\.\(2020\)A\. Conneau, K\. Khandelwal, N\. Goyal, V\. Chaudhary, G\. Wenzek, F\. Guzmán, E\. Grave, M\. Ott, L\. Zettlemoyer, and V\. StoyanovUnsupervised cross\-lingual representation learning at scale\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,Online,pp\. 8440–8451\.Note:Introduces the CC100 corpus and XLM\-RExternal Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.747),[Link](https://aclanthology.org/2020.acl-main.747/)Cited by:[§E\.2](https://arxiv.org/html/2609.25298#A5.SS2.p1.1),[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px2.p1.1)\.
- Curreyet al\.\(2022\)A\. Currey, M\. Nadejde, R\. R\. Pappagari, M\. Mayer, S\. Lauly, X\. Niu, B\. Hsu, and G\. DinuMT\-GenEval: a counterfactual and contextual dataset for evaluating gender accuracy in machine translation\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2022.emnlp-main.288/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1)\.
- Davies and Fuchs \(2015\)M\. Davies and R\. FuchsExpanding horizons in the study of world englishes with the 1\.9 billion word global web\-based english corpus \(GloWbE\)\.English World\-Wide36\(1\),pp\. 1–28\.External Links:[Document](https://dx.doi.org/10.1075/eww.36.1.01dav),[Link](https://doi.org/10.1075/eww.36.1.01dav)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px2.p1.1)\.
- Deaset al\.\(2024\)N\. Deas, E\. Turcan, I\. Pérez Mejía, and K\. McKeownMASIVE: open\-ended affective state identification in English and Spanish\.External Links:2407\.12196,[Link](https://arxiv.org/abs/2407.12196)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Dementievaet al\.\(2025\)D\. Dementieva, N\. Babakov, A\. Ronen, A\. A\. Ayele, N\. Rizwan, F\. Schneider, X\. Wang, S\. M\. Yimam, D\. Moskovskiy, E\. Stakovskii, E\. Kaufman, A\. Elnagar, A\. Mukherjee, and A\. PanchenkoMultilingual and explainable text detoxification with parallel corpora\.Association for Computational Linguistics\.Note:See also CLEF 2025 PAN shared task on multilingual text detoxificationCited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1)\.
- Dhanwalet al\.\(2020\)S\. Dhanwal, H\. Dutta, H\. Nankani, N\. Shrivastava, Y\. Kumar, J\. J\. Li,et al\.An annotated dataset of discourse modes in Hindi stories\.InProceedings of the Twelfth Language Resources and Evaluation Conference,Marseille, France,pp\. 1191–1196\.External Links:ISBN 979\-10\-95546\-34\-4,[Link](https://aclanthology.org/2020.lrec-1.149/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1)\.
- Dunn \(2020\)J\. DunnMapping languages: the corpus of global language use\.Language Resources and Evaluation54\(4\),pp\. 999–1018\.External Links:[Document](https://dx.doi.org/10.1007/s10579-020-09489-2),[Link](https://doi.org/10.1007/s10579-020-09489-2)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px2.p1.1),[§3\.1\.1](https://arxiv.org/html/2609.25298#S3.SS1.SSS1.p1.1)\.
- Durmuset al\.\(2024\)E\. Durmus, K\. Nguyen, T\. I\. Liao, N\. Schiefer, A\. Askell, A\. Bakhtin, C\. Chen, Z\. Hatfield\-Dodds, D\. Hernandez, N\. Joseph, L\. Lovitt, S\. McCandlish, O\. Sikder, A\. Tamkin, J\. Thamkul, J\. Kaplan, J\. Clark, and D\. GanguliTowards measuring the representation of subjective global opinions in language models\.Note:Conference on Language Modeling \(COLM\) 2024External Links:2306\.16388,[Link](https://arxiv.org/abs/2306.16388)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1)\.
- Durmuset al\.\(2023\)E\. Durmus, K\. Nyugen, T\. I\. Liao, N\. Schiefer, A\. Askell, A\. Bakhtin, C\. Chen, Z\. Hatfield\-Dodds, D\. Hernandez, N\. Joseph, L\. Lovitt, S\. McCandlish, O\. Sikder, A\. Tamkin, J\. Thamkul, J\. Kaplan, J\. Clark, and D\. GanguliTowards measuring the representation of subjective global opinions in language models\.External Links:2306\.16388,[Link](https://arxiv.org/abs/2306.16388)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px1.p1.1)\.
- Dwivediet al\.\(2023\)A\. Dwivedi, P\. Lavania, and A\. ModiEtiCor: corpus for analyzing LLMs for etiquettes\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 6921–6931\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.428/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Dwivediet al\.\(2025\)A\. Dwivedi, S\. S\. Singh, and A\. ModiEtiCor\+\+: towards understanding etiquettical bias in LLMs\.InFindings of the Association for Computational Linguistics: ACL 2025,Vienna, Austria,pp\. 9355–9376\.External Links:[Link](https://aclanthology.org/2025.findings-acl.488/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Emelinet al\.\(2021\)D\. Emelin, R\. Le Bras, J\. D\. Hwang, M\. Forbes, and Y\. ChoiMoral stories: situated reasoning about norms, intents, actions, and their consequences\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2021.emnlp-main.54/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Epureet al\.\(2020\)E\. V\. Epure, G\. Salha, M\. Moussallam, and R\. HennequinModeling the music genre perception across language\-bound cultures\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:[Link](https://aclanthology.org/2020.emnlp-main.386/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1)\.
- Facebook \(2014\)FacebookThe Open Graph protocol\.Note:[https://ogp\.me](https://ogp.me/)Accessed: 2026\-05\-22Cited by:[2nd item](https://arxiv.org/html/2609.25298#S3.I1.i2.p1.1)\.
- Faisalet al\.\(2024\)F\. Faisal, O\. Ahia, A\. Srivastava, K\. Ahuja, D\. Chiang, Y\. Tsvetkov,et al\.DIALECTBENCH: a NLP benchmark for dialects, varieties, and closely\-related languages\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 14412–14454\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.777),[Link](https://aclanthology.org/2024.acl-long.777/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1)\.
- FitzGeraldet al\.\(2022\)J\. FitzGerald, C\. Hench, C\. Peris, S\. Mackie, K\. Rottmann, A\. Sanchez, A\. Nash, L\. Urbach, V\. Kakarala, R\. Singh, S\. Ranganath, L\. Crist, M\. Britan, W\. Leeuwis, G\. Tur, and P\. NatarajanMASSIVE: a 1M\-example multilingual natural language understanding dataset with 51 typologically\-diverse languages\.External Links:2204\.08582,[Link](https://arxiv.org/abs/2204.08582)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Forbeset al\.\(2020\)M\. Forbes, J\. D\. Hwang, V\. Shwartz, M\. Sap, and Y\. ChoiSocial chemistry 101: learning to reason about social and moral norms\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:[Link](https://aclanthology.org/2020.emnlp-main.48/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Frendaet al\.\(2023\)S\. Frenda, A\. Pedrani, V\. Basile, S\. M\. Lo, A\. T\. Cignarella, R\. Panizzon, C\. Marco, B\. Scarlini, V\. Patti, C\. Bosco, and D\. BernardiEPIC: multi\-perspective annotation of a corpus of irony\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Toronto, Canada,pp\. 13844–13857\.External Links:[Link](https://aclanthology.org/2023.acl-long.774/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Funget al\.\(2023\)Y\. Fung, T\. Chakrabarty, H\. Guo, O\. Rambow, S\. Muresan, and H\. JiNORMSAGE: multi\-lingual multi\-cultural norm discovery from conversations on\-the\-fly\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2023.emnlp-main.941/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Gebruet al\.\(2021\)T\. Gebru, J\. Morgenstern, B\. Vecchione, J\. W\. Vaughan, H\. Wallach, H\. D\. Iii, and K\. CrawfordDatasheets for datasets\.Communications of the ACM64\(12\),pp\. 86–92\.External Links:[Document](https://dx.doi.org/10.1145/3458723)Cited by:[Appendix F](https://arxiv.org/html/2609.25298#A6.p1.1)\.
- Goelet al\.\(2023\)R\. Goel, W\. Ammar, A\. Gupta, S\. Vashishtha, M\. Sano, F\. Surani, M\. Chang, H\. Choe, D\. Greene, K\. He, R\. Nielsen, J\. Robichaud, K\. Sethi, N\. Sun, B\. Tan, and J\. ZhangPRESTO: a multilingual dataset for parsing realistic task\-oriented dialogs\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2023.emnlp-main.667/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1)\.
- Grootendorst \(2022\)M\. GrootendorstBERTopic: neural topic modeling with a class\-based TF\-IDF procedure\.External Links:2203\.05794,[Link](https://arxiv.org/abs/2203.05794)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px3.p1.1)\.
- Guoet al\.\(2025\)G\. Guo, T\. Naous, H\. Wakaki, Y\. Nishimura, Y\. Mitsufuji, A\. Ritter, and W\. XuCARE: multilingual human preference learning for cultural awareness\.External Links:2504\.05154,[Link](https://arxiv.org/abs/2504.05154)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Gutiérrezet al\.\(2016\)E\. D\. Gutiérrez, E\. Shutova, P\. Lichtenstein, G\. de Melo, and L\. GilardiDetecting cross\-cultural differences using a multilingual topic model\.Transactions of the Association for Computational Linguistics4,pp\. 47–60\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00082),[Link](https://aclanthology.org/Q16-1004/)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px3.p1.1)\.
- Handeet al\.\(2021\)A\. Hande, R\. Priyadharshini, A\. Sampath, K\. P\. Thamburaj, P\. Chandran, and B\. R\. ChakravarthiHope speech detection in under\-resourced kannada language\.External Links:2108\.04616,[Link](https://arxiv.org/abs/2108.04616)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Hanges and Dickson \(2004\)P\. J\. Hanges and M\. W\. DicksonThe development and validation of the GLOBE culture and leadership scales\.InLeadership, Culture, and Organizations: The GLOBE Study of 62 Societies,R\. J\. House, P\. J\. Hanges, M\. Javidan, P\. W\. Dorfman, and V\. Gupta \(Eds\.\),pp\. 122–151\.External Links:[Link](https://www.researchgate.net/publication/305397723_The_Development_and_Validation_of_the_GLOBE_Culture_and_Leadership_Scales)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.15.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1)\.
- Hardalovet al\.\(2020\)M\. Hardalov, T\. Mihaylov, D\. Zlatkova, Y\. Dinkov, I\. Koychev, and P\. NakovEXAMS: a multi\-subject high school examinations dataset for cross\-lingual and multilingual question answering\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.438),[Link](https://aclanthology.org/2020.emnlp-main.438/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Havaldaret al\.\(2024\)S\. Havaldar, S\. Giorgi, S\. Rai, T\. Talhelm, S\. C\. Guntuku, and L\. UngarBuilding knowledge\-guided lexica to model cultural variation\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),Mexico City, Mexico,pp\. 211–226\.External Links:[Link](https://aclanthology.org/2024.naacl-long.12/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Havaldaret al\.\(2023\)S\. Havaldar, M\. Pressimone, E\. Wong, and L\. UngarComparing styles across languages\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 6775–6791\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.419),[Link](https://aclanthology.org/2023.emnlp-main.419/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1)\.
- Hershcovichet al\.\(2022\)D\. Hershcovich, S\. Frank, H\. Lent, M\. de Lhoneux, M\. Abdou, S\. Brandl, E\. Bugliarello, L\. Cabello Piqueras, I\. Chalkidis, R\. Cheng, A\. Doucet,et al\.Challenges and strategies in cross\-cultural NLP\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Dublin, Ireland,pp\. 6997–7013\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.482),[Link](https://aclanthology.org/2022.acl-long.482/)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px1.p1.1)\.
- Hobsonet al\.\(2024\)D\. G\. Hobson, H\. Zhou, D\. Ruths, and A\. PiperStory morals: surfacing value\-driven narrative schemas using large language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2024.emnlp-main.723/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Hooveret al\.\(2020\)J\. Hoover, G\. Portillo\-Wightman, L\. Yeh, S\. Havaldar, A\. M\. Davani, Y\. Lin, B\. Kennedy, M\. Atari, Z\. Kamel, M\. Mendlen, G\. Moreno, C\. Park, T\. E\. Chang, J\. Chin, C\. Leong, J\. Y\. Leung, A\. Mirinjian, and M\. DehghaniMoral foundations Twitter corpus: a collection of 35k tweets annotated for moral sentiment\.Social Psychological and Personality Science11\(8\),pp\. 1057–1071\.External Links:[Document](https://dx.doi.org/10.1177/1948550619876629)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Hoppet al\.\(2021\)F\. R\. Hopp, J\. T\. Fisher, D\. Cornell, R\. Huskey, and R\. WeberThe extended moral foundations dictionary \(eMFD\): development and applications of a crowd\-sourced approach to extracting moral intuitions from text\.Behavior Research Methods53\(1\),pp\. 232–246\.External Links:[Document](https://dx.doi.org/10.3758/s13428-020-01433-0)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Hossainet al\.\(2022\)E\. Hossain, O\. Sharif, and M\. M\. HoqueMemoSen: a multimodal dataset for sentiment analysis of memes\.InProceedings of the Thirteenth Language Resources and Evaluation Conference,Marseille, France\.External Links:[Link](https://aclanthology.org/2022.lrec-1.165/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1)\.
- Huet al\.\(2023\)S\. Hu, H\. Zhou, M\. Hergul, M\. Gritta, G\. Zhang, I\. Iacobacci, I\. Vulić, and A\. KorhonenMulti3\{\}^\{3\}WOZ: a multilingual, multi\-domain, multi\-parallel dataset for training and evaluating culturally adapted task\-oriented dialog systems\.Transactions of the Association for Computational Linguistics11,pp\. 1396–1415\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00609),[Link](https://aclanthology.org/2023.tacl-1.79/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Huet al\.\(2024\)T\. Hu, M\. Maistro, and D\. HershcovichBridging cultures in the kitchen: a framework and benchmark for cross\-cultural recipe retrieval\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 1068–1080\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.61),[Link](https://aclanthology.org/2024.emnlp-main.61/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Huang and Yang \(2023\)J\. Huang and D\. YangCulturally aware natural language inference\.InFindings of the Association for Computational Linguistics: EMNLP 2023,Singapore,pp\. 7591–7609\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.509/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- in the Wild \(2024\)B\. in the WildAlign Via Actions \(AVA\): learning behavior aligns LLMs with human opinions in zero\-shot\.Note:[https://behavior\-in\-the\-wild\.github\.io/align\-via\-actions\.html](https://behavior-in-the-wild.github.io/align-via-actions.html)Project page; full author list pending publicationCited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1)\.
- International Organization for Standardization \(2026\)International Organization for StandardizationISO 3166: country codes\.Note:[https://www\.iso\.org/iso\-3166\-country\-codes\.html](https://www.iso.org/iso-3166-country-codes.html)Accessed 2026\-05\-22Cited by:[§3\.1\.1](https://arxiv.org/html/2609.25298#S3.SS1.SSS1.p1.1)\.
- Jianget al\.\(2023\)Y\. E\. Jiang, T\. Liu, S\. Ma, D\. Zhang, M\. Sachan, and R\. CotterellDiscourse\-centric evaluation of document\-level machine translation with a new densely annotated parallel corpus of novels\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Toronto, Canada,pp\. 7853–7872\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.435),[Link](https://aclanthology.org/2023.acl-long.435/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1)\.
- Jianget al\.\(2020\)Z\. Jiang, A\. Anastasopoulos, J\. Araki, H\. Ding, and G\. NeubigX\-FACTR: multilingual factual knowledge retrieval from pretrained language models\.Association for Computational Linguistics\.External Links:[Link](https://aclanthology.org/2020.emnlp-main.479/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Jinet al\.\(2024\)C\. Jin, K\. Ren, L\. Kong, X\. Wang, R\. Song, and H\. ChenPersuading across diverse domains: a dataset and persuasion large language model\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand\.External Links:[Link](https://aclanthology.org/2024.acl-long.92/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1)\.
- Jinet al\.\(2023\)J\. Jin, J\. Kim, N\. Lee, H\. Yoo, A\. Oh, and H\. LeeKoBBQ: Korean bias benchmark for question answering\.External Links:2307\.16778,[Link](https://arxiv.org/abs/2307.16778)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Joshiet al\.\(2025\)R\. Joshi, R\. Paul, K\. Singla, A\. Kamath, M\. Evans, K\. Luna, S\. Ghosh, U\. Vaidya, E\. Long, S\. S\. Chauhan, and N\. WartikarCultureGuard: towards culturally\-aware dataset and guard model for multilingual safety applications\.External Links:2508\.01710,[Link](https://arxiv.org/abs/2508.01710)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Joy \(2025\)S\. S\. JoyBnMMLU: measuring massive multitask language understanding in Bengali\.Note:Findings of ACL 2026External Links:2505\.18951,[Link](https://arxiv.org/abs/2505.18951)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Kappl \(2025\)M\. KapplAre all Spanish doctors male? evaluating gender bias in German machine translation\.External Links:2502\.19104,[Link](https://arxiv.org/abs/2502.19104)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Karinshaket al\.\(2024\)E\. Karinshak, A\. Hu, K\. Kong, V\. Rao, J\. Wang, J\. Wang, and Y\. ZengLLM\-GLOBE: a benchmark evaluating the cultural values embedded in LLM output\.External Links:2411\.06032,[Link](https://arxiv.org/abs/2411.06032)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1)\.
- Kassneret al\.\(2021\)N\. Kassner, P\. Dufter, and H\. SchützeMultilingual LAMA: investigating knowledge in multilingual pretrained language models\.InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume,External Links:[Link](https://aclanthology.org/2021.eacl-main.284/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Keleg and Magdy \(2023\)A\. Keleg and W\. MagdyDLAMA: a framework for curating culturally diverse facts for probing the knowledge of pretrained language models\.InFindings of the Association for Computational Linguistics: ACL 2023,Toronto, Canada\.External Links:[Link](https://aclanthology.org/2023.findings-acl.389/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Khoshtabet al\.\(2025\)P\. Khoshtab, D\. Namazifard, M\. Masoudi, A\. Akhgary, S\. Mahdizadeh Sani, and Y\. YaghoobzadehComparative study of multilingual idioms and similes in large language models\.InProceedings of the 31st International Conference on Computational Linguistics,External Links:[Link](https://aclanthology.org/2025.coling-main.580/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1)\.
- Kimet al\.\(2022\)H\. Kim, Y\. Yu, L\. Jiang, X\. Lu, D\. Khashabi, G\. Kim, Y\. Choi, and M\. SapProsocialDialog: a prosocial backbone for conversational agents\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2022.emnlp-main.267/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Kirket al\.\(2024\)H\. R\. Kirk, A\. Whitefield, P\. Röttger, A\. Bean, K\. Margatina, J\. Ciro, R\. Mosquera, M\. Bartolo, A\. Williams, H\. He, B\. Vidgen, and S\. A\. HaleThe PRISM alignment dataset: what participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models\.Note:NeurIPS 2024 Datasets and BenchmarksExternal Links:2404\.16019,[Link](https://arxiv.org/abs/2404.16019)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1)\.
- Kotoet al\.\(2023\)F\. Koto, N\. Aisyah, H\. Li, and T\. BaldwinLarge language models only pass primary school exams in Indonesia: a comprehensive test on IndoMMLU\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2023.emnlp-main.760/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Kotoet al\.\(2024a\)F\. Koto, H\. Li, S\. Shatnawi, J\. Doughman, A\. B\. Sadallah, A\. Alraeesi, K\. Almubarak, Z\. Alyafeai, N\. Sengupta, S\. Shehata, N\. Habash, P\. Nakov, and T\. BaldwinArabicMMLU: assessing massive multitask language understanding in Arabic\.External Links:2402\.12840,[Link](https://arxiv.org/abs/2402.12840)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Kotoet al\.\(2024b\)F\. Koto, R\. Mahendra, N\. Aisyah, and T\. BaldwinIndoCulture: exploring geographically influenced cultural commonsense reasoning across eleven Indonesian provinces\.Transactions of the Association for Computational Linguistics12,pp\. 1703–1719\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00726),[Link](https://aclanthology.org/2024.tacl-1.92/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Krishnaet al\.\(2022\)K\. Krishna, D\. Nathani, X\. Garcia, B\. Samanta, and P\. TalukdarFew\-shot controllable style transfer for low\-resource multilingual settings\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Dublin, Ireland,pp\. 7439–7468\.External Links:[Link](https://aclanthology.org/2022.acl-long.514/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1)\.
- Kuparinenet al\.\(2023\)O\. Kuparinen, A\. Miletić, and Y\. ScherrerDialect\-to\-standard normalization: a large\-scale multilingual evaluation\.InFindings of the Association for Computational Linguistics: EMNLP 2023,Singapore,pp\. 13814–13828\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.923),[Link](https://aclanthology.org/2023.findings-emnlp.923/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1)\.
- Kuzmanet al\.\(2023\)T\. Kuzman, I\. Mozetič, and N\. LjubešićAutomatic genre identification for robust enrichment of massive text collections: investigation of classification methods in the era of large language models\.External Links:2306\.04298,[Link](https://arxiv.org/abs/2306.04298)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1)\.
- Laippalaet al\.\(2022\)V\. Laippala, A\. Salmela, S\. Rönnqvist, A\. F\. Aji, L\. Chang, A\. Dhifallah, L\. Goulart, H\. Kortelainen, M\. Krajňc, D\. Krevelj, M\. Aigro, L\. Repo, V\. Skórska, A\. Sahala, Y\. Yang, and S\. PyysaloTowards better structured and less noisy web data: OSCAR with register annotations\.InProceedings of the Eighth Workshop on Noisy User\-generated Text \(W\-NUT 2022\),External Links:[Link](https://aclanthology.org/2022.wnut-1.23/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1)\.
- Lauscher and Glavaš \(2019\)A\. Lauscher and G\. GlavašAre we consistently biased? multidimensional analysis of biases in distributional word vectors\.External Links:1904\.11783,[Link](https://arxiv.org/abs/1904.11783)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Lauscheret al\.\(2020\)A\. Lauscher, R\. Takieddin, S\. P\. Ponzetto, and G\. GlavašAraWEAT: multidimensional analysis of biases in Arabic word embeddings\.InProceedings of the Fifth Arabic Natural Language Processing Workshop,External Links:[Link](https://aclanthology.org/2020.wanlp-1.17/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Leeet al\.\(2023\)N\. Lee, C\. Jung, J\. Myung, J\. Jin, J\. Camacho\-Collados, J\. Kim, and A\. OhExploring cross\-cultural differences in English hate speech annotations: from dataset construction to analysis\.External Links:2308\.16705,[Link](https://arxiv.org/abs/2308.16705)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, B\. Oğuz, R\. Rinott, S\. Riedel, and H\. SchwenkMLQA: evaluating cross\-lingual extractive question answering\.External Links:1910\.07475,[Link](https://arxiv.org/abs/1910.07475)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Liet al\.\(2024a\)C\. Li, M\. Chen, J\. Wang, S\. Sitaram, and X\. XieCultureLLM: incorporating cultural differences into large language models\.External Links:2402\.10946,[Link](https://arxiv.org/abs/2402.10946)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1)\.
- Liet al\.\(2024b\)C\. Li, D\. Teney, L\. Yang, Q\. Wen, X\. Xie, and J\. WangCulturePark: boosting cross\-cultural understanding in large language models\.External Links:2405\.15145,[Link](https://arxiv.org/abs/2405.15145)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Liet al\.\(2024c\)H\. Li, Y\. Zhang, F\. Koto, Y\. Yang, H\. Zhao, Y\. Gong, N\. Duan, and T\. BaldwinCMMLU: measuring massive multitask language understanding in Chinese\.InFindings of the Association for Computational Linguistics: ACL 2024,Bangkok, Thailand,pp\. 11260–11285\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.671),[Link](https://aclanthology.org/2024.findings-acl.671/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Liet al\.\(2024d\)H\. Li, L\. Jiang, N\. Dziri, X\. Ren, and Y\. ChoiCULTURE\-GEN: revealing global cultural perception in language models through natural language prompting\.Note:Conference on Language Modeling \(COLM\) 2024External Links:2404\.10199,[Link](https://arxiv.org/abs/2404.10199)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Liet al\.\(2024e\)J\. Li, J\. Wang, J\. Hu, and M\. JiangHow well do LLMs identify cultural unity in diversity?\.External Links:2408\.05102,[Link](https://arxiv.org/abs/2408.05102)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Liet al\.\(2023a\)O\. Li, M\. Subramaniam, A\. Saakyan, S\. CH\-Wang, and S\. MuresanNormDial: a comparable bilingual synthetic dialog dataset for modeling social norm adherence and violation\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2023.emnlp-main.974/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.15.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Liet al\.\(2023b\)Y\. Li, B\. Hui, Z\. Yin, M\. Yang, F\. Huang, and Y\. LiPaCE: unified multi\-modal dialogue pre\-training with progressive and compositional experts\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Toronto, Canada,pp\. 13402–13416\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.749),[Link](https://aclanthology.org/2023.acl-long.749/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Linet al\.\(2021\)B\. Y\. Lin, S\. Lee, X\. Qiao, and X\. RenCommon sense beyond English: evaluating and improving multilingual language models for commonsense reasoning\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),pp\. 1274–1287\.External Links:[Link](https://aclanthology.org/2021.acl-long.102/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Liuet al\.\(2025\)C\. C\. Liu, I\. Gurevych, and A\. KorhonenCulturally aware and adapted NLP: a taxonomy and a survey of the state of the art\.Transactions of the Association for Computational Linguistics13,pp\. 652–689\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00760),[Link](https://aclanthology.org/2025.tacl-1.31/)Cited by:[item 1](https://arxiv.org/html/2609.25298#A4.I1.i1.p1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13),[4th item](https://arxiv.org/html/2609.25298#S1.I1.i4.p1.1),[§1](https://arxiv.org/html/2609.25298#S1.p2.1),[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px1.p1.1),[§3\.2\.1](https://arxiv.org/html/2609.25298#S3.SS2.SSS1.p1.1),[§3](https://arxiv.org/html/2609.25298#S3.p1.1),[§4](https://arxiv.org/html/2609.25298#S4.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2609.25298#S4.p1.1),[Abstract](https://arxiv.org/html/2609.25298#abstract1.2)\.
- Liuet al\.\(2024\)C\. C\. Liu, F\. Koto, T\. Baldwin, and I\. GurevychAre multilingual LLMs culturally\-diverse reasoners? an investigation into multicultural proverbs and sayings\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),Mexico City, Mexico,pp\. 2016–2039\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.112),[Link](https://aclanthology.org/2024.naacl-long.112/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Liuet al\.\(2021\)F\. Liu, E\. Bugliarello, E\. M\. Ponti, S\. Reddy, N\. Collier, and D\. ElliottVisually grounded reasoning across languages and cultures\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,Online and Punta Cana, Dominican Republic,pp\. 10467–10485\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.818),[Link](https://aclanthology.org/2021.emnlp-main.818/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Luuet al\.\(2021\)S\. T\. Luu, K\. Van Nguyen, and N\. L\. NguyenA large\-scale dataset for hate speech detection on Vietnamese social media texts\.External Links:2103\.11528,[Link](https://arxiv.org/abs/2103.11528)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1)\.
- Majewskaet al\.\(2022\)O\. Majewska, E\. Razumovskaia, E\. M\. Ponti, I\. Vulić, and A\. KorhonenCross\-lingual dialogue dataset creation via outline\-based generation\.External Links:2201\.13405,[Link](https://arxiv.org/abs/2201.13405)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Mandlet al\.\(2021\)T\. Mandl, S\. Modha, G\. K\. Shahi, A\. K\. Jaiswal, D\. Nandini, D\. Patel, P\. Majumder, and J\. SchäferOverview of the HASOC track at FIRE 2020: hate speech and offensive content identification in Indo\-European languages\.External Links:2108\.05927,[Link](https://arxiv.org/abs/2108.05927)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1)\.
- Mathewet al\.\(2020\)B\. Mathew, P\. Saha, S\. M\. Yimam, C\. Biemann, P\. Goyal, and A\. MukherjeeHateXplain: a benchmark dataset for explainable hate speech detection\.External Links:2012\.10289,[Link](https://arxiv.org/abs/2012.10289)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1)\.
- MaxMind, Inc\. \(2026\)MaxMind, Inc\.GeoLite2 free geolocation data\.Note:[https://dev\.maxmind\.com/geoip/geolite2\-free\-geolocation\-data](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data)Accessed: 2026\-05\-22Cited by:[1st item](https://arxiv.org/html/2609.25298#S3.I1.i1.p1.1)\.
- Mitchellet al\.\(2025\)M\. Mitchell, S\. Bhattacharyya, S\. Dev, Y\. Jernite, S\. Hassan, Z\. Talat, L\. Cabello,et al\.SHADES: towards a multilingual assessment of stereotypes in large language models\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies,Albuquerque, New Mexico,pp\. 11995–12041\.External Links:[Link](https://aclanthology.org/2025.naacl-long.600/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Mohamedet al\.\(2022\)Y\. Mohamed, M\. Abdelfattah, S\. Alhuwaider, F\. Li, X\. Zhang, K\. Church, and M\. ElhoseinyArtELingo: a million emotion annotations of WikiArt with emphasis on diversity over language and culture\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2022.emnlp-main.600/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Mohamedet al\.\(2024\)Y\. Mohamed, R\. Khan, K\. Almazrouei, K\. Khan, M\. A\. Khattak, E\. Bonnah, D\. Adelani, M\. Khattak, M\. Elhoseiny, and K\. ChurchNo culture left behind: ArtELingo\-28, a benchmark of WikiArt with captions in 28 languages\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2024.emnlp-main.1165/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Mukherjeeet al\.\(2023a\)A\. Mukherjee, C\. Raj, Z\. Zhu, and A\. AnastasopoulosGlobal voices, local biases: socio\-cultural prejudices across languages\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 15828–15845\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.981/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Mukherjeeet al\.\(2023b\)S\. Mukherjee, A\. Bansal, P\. Majumdar, A\. Kr\. Ojha, and O\. DušekLow\-resource text style transfer for Bangla: data & models\.InProceedings of the First Workshop on Bangla Language Processing \(BLP\-2023\),Singapore,pp\. 34–47\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.banglalp-1.5),[Link](https://aclanthology.org/2023.banglalp-1.5/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1)\.
- Myunget al\.\(2024a\)J\. Myung, N\. Lee, Y\. Zhou, J\. Jin, R\. A\. Putri, D\. Antypas, H\. Borkakoty, E\. Kim, C\. Perez\-Almendros, A\. A\. Ayele, V\. Gutiérrez\-Basulto, Y\. Ibáñez\-García, H\. Lee, S\. H\. Muhammad, K\. Park, A\. S\. Rzayev, N\. White, S\. M\. Yimam, M\. T\. Pilehvar, N\. Ousidhoum, J\. Camacho\-Collados, and A\. OhBLEnD: a benchmark for LLMs on everyday knowledge in diverse cultures and languages\.External Links:2406\.09948,[Link](https://arxiv.org/abs/2406.09948)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Myunget al\.\(2024b\)J\. Myung, N\. Lee, Y\. Zhou, J\. Jin, R\. A\. Putri, D\. Antypas, H\. Borkakoty, E\. Kim, C\. Perez\-Almendros, A\. A\. Ayele, V\. Gutiérrez\-Basulto, Y\. Ibáñez\-García, H\. Lee, S\. H\. Muhammad, K\. Park, A\. S\. Rzayev, N\. White, S\. M\. Yimam, M\. T\. Pilehvar, N\. Ousidhoum, J\. Camacho\-Collados, and A\. OhBLEnD: a benchmark for LLMs on everyday knowledge in diverse cultures and languages\.External Links:2406\.09948,[Link](https://arxiv.org/abs/2406.09948)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px1.p1.1)\.
- Nadeemet al\.\(2021\)M\. Nadeem, A\. Bethke, and S\. ReddyStereoSet: measuring stereotypical bias in pretrained language models\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),External Links:[Link](https://aclanthology.org/2021.acl-long.416/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Nadejdeet al\.\(2022\)M\. Nadejde, A\. Currey, B\. Hsu, X\. Niu, M\. Federico, and G\. DinuCoCoA\-MT: a dataset and benchmark for contrastive controlled MT with application to formality\.InFindings of the Association for Computational Linguistics: NAACL 2022,External Links:[Link](https://aclanthology.org/2022.findings-naacl.47/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1)\.
- Namazifard and Galke \(2025\)D\. Namazifard and L\. GalkeIsolating culture neurons in multilingual large language models\.Note:Introduces the MUREL cultural datasetExternal Links:2508\.02241,[Link](https://arxiv.org/abs/2508.02241)Cited by:[item 2](https://arxiv.org/html/2609.25298#A4.I1.i2.p1.1),[4th item](https://arxiv.org/html/2609.25298#S1.I1.i4.p1.1),[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2609.25298#S4.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2609.25298#S4.p1.1)\.
- Nangiaet al\.\(2020a\)N\. Nangia, C\. Vania, R\. Bhalerao, and S\. R\. BowmanCrowS\-Pairs: a challenge dataset for measuring social biases in masked language models\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 1953–1967\.External Links:[Link](https://aclanthology.org/2020.emnlp-main.154/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Nangiaet al\.\(2020b\)N\. Nangia, C\. Vania, R\. Bhasin, and S\. R\. BowmanCrowS\-Pairs: a challenge dataset for measuring social biases in masked language models\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Online,pp\. 1953–1967\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.154),[Link](https://aclanthology.org/2020.emnlp-main.154/)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px1.p1.1)\.
- Naous and Xu \(2025\)T\. Naous and W\. XuOn the origin of cultural biases in language models: from pre\-training data to linguistic phenomena\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),Albuquerque, New Mexico,pp\. 6423–6443\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.326),[Link](https://aclanthology.org/2025.naacl-long.326/)Cited by:[§1](https://arxiv.org/html/2609.25298#S1.p1.1)\.
- Nayaket al\.\(2024\)S\. Nayak, K\. Jain, R\. Awal, S\. Reddy, S\. V\. Steenkiste, L\. A\. Hendricks, K\. Stanczak, and A\. AgrawalBenchmarking vision language models for cultural understanding\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 5769–5790\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.329),[Link](https://aclanthology.org/2024.emnlp-main.329/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Neplenbroeket al\.\(2024\)V\. Neplenbroek, A\. Bisazza, and R\. FernándezMBBQ: a dataset for cross\-lingual comparison of stereotypes in generative LLMs\.External Links:2406\.07243,[Link](https://arxiv.org/abs/2406.07243)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Neudecker \(2016\)C\. NeudeckerAn open corpus for named entity recognition in historic newspapers\.InProceedings of the Tenth International Conference on Language Resources and Evaluation \(LREC’16\),Portorož, Slovenia,pp\. 4348–4352\.External Links:[Link](https://aclanthology.org/L16-1689/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1)\.
- Névéolet al\.\(2022\)A\. Névéol, Y\. Dupont, J\. Bezançon, and K\. FortFrench CrowS\-Pairs: extending a challenge dataset for measuring social bias in masked language models to a language other than English\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Dublin, Ireland,pp\. 8521–8531\.External Links:[Link](https://aclanthology.org/2022.acl-long.583/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Nguyenet al\.\(2023a\)T\. Nguyen, C\. V\. Nguyen, V\. D\. Lai, H\. Man, N\. T\. Ngo, F\. Dernoncourt, R\. A\. Rossi, and T\. H\. NguyenCulturaX: a cleaned, enormous, and multilingual dataset for large language models in 167 languages\.External Links:2309\.09400,[Link](https://arxiv.org/abs/2309.09400)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px2.p1.1)\.
- Nguyenet al\.\(2023b\)T\. Nguyen, S\. Razniewski, A\. Varde, and G\. WeikumExtracting cultural commonsense knowledge at scale\.InProceedings of the ACM Web Conference 2023,pp\. 1907–1917\.External Links:[Document](https://dx.doi.org/10.1145/3543507.3583535),[Link](https://arxiv.org/abs/2210.07763)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Nozzaet al\.\(2021\)D\. Nozza, F\. Bianchi, and D\. HovyHONEST: measuring hurtful sentence completion in language models\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 2398–2406\.External Links:[Link](https://aclanthology.org/2021.naacl-main.191/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Ohashiet al\.\(2024\)T\. Ohashi, T\. Nakagawa, and H\. IyatomiExtended Japanese commonsense morality dataset with masked token and label enhancement\.External Links:2410\.09564,[Link](https://arxiv.org/abs/2410.09564)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Olabisiet al\.\(2022\)O\. Olabisi, A\. Hudson, A\. Jetter, and A\. AgrawalAnalyzing the dialect diversity in multi\-document summaries\.InProceedings of the 29th International Conference on Computational Linguistics,Gyeongju, Republic of Korea,pp\. 6208–6221\.External Links:[Link](https://aclanthology.org/2022.coling-1.542/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1)\.
- Ortiz Suárezet al\.\(2020\)P\. J\. Ortiz Suárez, L\. Romary, and B\. SagotA monolingual approach to contextualized word embeddings for mid\-resource languages\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,Online,pp\. 1703–1714\.Note:Introduces the OSCAR corpusExternal Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.156),[Link](https://aclanthology.org/2020.acl-main.156/)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px2.p1.1)\.
- Öztürket al\.\(2023\)I\. T\. Öztürk, R\. Nedelchev, C\. Heumann, E\. Garces Arias, M\. Roger, B\. Bischl, and M\. AßenmacherHow different is stereotypical bias across languages?\.External Links:2307\.07331,[Link](https://arxiv.org/abs/2307.07331)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Palta and Rudinger \(2023\)S\. Palta and R\. RudingerFORK: a bite\-sized test set for probing culinary cultural biases in commonsense reasoning models\.InFindings of the Association for Computational Linguistics: ACL 2023,Toronto, Canada,pp\. 9952–9962\.External Links:[Link](https://aclanthology.org/2023.findings-acl.631/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Parrishet al\.\(2022a\)A\. Parrish, A\. Chen, N\. Nangia, V\. Padmakumar, J\. Phang, J\. Thompson, P\. M\. Htut, and S\. R\. BowmanBBQ: a hand\-built bias benchmark for question answering\.InFindings of the Association for Computational Linguistics: ACL 2022,Dublin, Ireland,pp\. 2086–2105\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.findings-acl.165),[Link](https://aclanthology.org/2022.findings-acl.165/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Parrishet al\.\(2022b\)A\. Parrish, A\. Chen, N\. Nangia, V\. Padmakumar, J\. Phang, J\. Thompson, P\. M\. Htut, and S\. R\. BowmanBBQ: a hand\-built bias benchmark for question answering\.InFindings of the Association for Computational Linguistics: ACL 2022,Dublin, Ireland,pp\. 2086–2105\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.findings-acl.165),[Link](https://aclanthology.org/2022.findings-acl.165/)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px1.p1.1)\.
- Pedregosaet al\.\(2011\)F\. Pedregosa, G\. Varoquaux, A\. Gramfort, V\. Michel, B\. Thirion, O\. Grisel, M\. Blondel, P\. Prettenhofer, R\. Weiss, V\. Dubourg,et al\.Scikit\-learn: machine learning in python\.the Journal of machine Learning research12,pp\. 2825–2830\.Cited by:[§3\.2\.2](https://arxiv.org/html/2609.25298#S3.SS2.SSS2.p5.1)\.
- Peiet al\.\(2023\)J\. Pei, V\. Silva, M\. Bos, Y\. Liu, L\. Neves, D\. Jurgens, and F\. BarbieriSemEval\-2023 task 9: multilingual tweet intimacy analysis\.InProceedings of the The 17th International Workshop on Semantic Evaluation \(SemEval\-2023\),External Links:[Link](https://aclanthology.org/2023.semeval-1.309/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Penedoet al\.\(2024\)G\. Penedo, H\. Kydlíček, L\. Ben Allal, A\. Lozhkov, M\. Mitchell, C\. Raffel, L\. Von Werra, and T\. WolfThe FineWeb datasets: decanting the web for the finest text data at scale\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/370df50ccfdf8bde18f8f9c2d9151bda-Abstract-Datasets_and_Benchmarks_Track.html)Cited by:[Appendix F](https://arxiv.org/html/2609.25298#A6.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.25298#S1.p2.1),[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2609.25298#S3.p1.1)\.
- Penedoet al\.\(2025\)G\. Penedo, H\. Kydlíček, V\. Sabolčec, B\. Messmer, N\. Foroutan, A\. H\. Kargaran, C\. Raffel, M\. Jaggi, L\. Von Werra, and T\. WolfFineWeb2: one pipeline to scale them all – adapting pre\-training data processing to every language\.External Links:2506\.20920,[Link](https://arxiv.org/abs/2506.20920)Cited by:[Appendix F](https://arxiv.org/html/2609.25298#A6.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.25298#S1.p2.1),[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2609.25298#S3.p1.1)\.
- Phillips and Davis \(2009\)A\. Phillips and M\. DavisTags for identifying languages\.Internet Engineering Task Force\.Note:RFC 5646External Links:[Document](https://dx.doi.org/10.17487/RFC5646),[Link](https://www.rfc-editor.org/rfc/rfc5646)Cited by:[2nd item](https://arxiv.org/html/2609.25298#S3.I1.i2.p1.1)\.
- Pistilliet al\.\(2024\)G\. Pistilli, A\. Leidinger, Y\. Jernite, A\. Kasirzadeh, A\. S\. Luccioni, and M\. MitchellCIVICS: building a dataset for examining culturally\-informed values in large language models\.InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society,Vol\.7,pp\. 1132–1144\.External Links:[Document](https://dx.doi.org/10.1609/aies.v7i1.31710),[Link](https://ojs.aaai.org/index.php/AIES/article/view/31710)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1),[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px1.p1.1)\.
- Plüsset al\.\(2023\)M\. Plüss, J\. Deriu, Y\. Schraner, C\. Paonessa, J\. Hartmann, L\. Schmidt,et al\.STT4SG\-350: a speech corpus for all Swiss German dialect regions\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),Toronto, Canada,pp\. 1763–1772\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.acl-short.150),[Link](https://aclanthology.org/2023.acl-short.150/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1)\.
- Plüsset al\.\(2022\)M\. Plüss, M\. Hürlimann, M\. Cuny, A\. Stöckli, N\. Kapotis, J\. Hartmann,et al\.SDS\-200: a Swiss German speech to Standard German text corpus\.InProceedings of the Thirteenth Language Resources and Evaluation Conference,Marseille, France,pp\. 3250–3256\.External Links:[Link](https://aclanthology.org/2022.lrec-1.347/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1)\.
- Pontiet al\.\(2020\)E\. M\. Ponti, G\. Glavaš, O\. Majewska, Q\. Liu, I\. Vulić, and A\. KorhonenXCOPA: a multilingual dataset for causal commonsense reasoning\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 2362–2376\.External Links:[Link](https://aclanthology.org/2020.emnlp-main.185/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Pramodyaet al\.\(2025\)A\. Pramodya, Y\. Sakai, J\. Vasselli, H\. Kamigaito, and T\. WatanabeTranslating movie subtitles by large language models using movie\-meta information\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 4: Student Research Workshop\),External Links:[Link](https://aclanthology.org/2025.acl-srw.22/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Pyatkinet al\.\(2022\)V\. Pyatkin, J\. D\. Hwang, V\. Srikumar, X\. Lu, L\. Jiang, Y\. Choi, and C\. BhagavatulaClarifyDelphi: reinforced clarification questions with defeasibility rewards for social and moral situations\.External Links:2212\.10409,[Link](https://arxiv.org/abs/2212.10409)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Raoet al\.\(2024\)A\. Rao, A\. Yerukola, V\. Shah, K\. Reinecke, and M\. SapNormAd: a framework for measuring the cultural adaptability of large language models\.External Links:2404\.12464,[Link](https://arxiv.org/abs/2404.12464)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Ravikiran and Annamalai \(2021\)M\. Ravikiran and S\. AnnamalaiDOSA: Dravidian code\-mixed offensive span identification dataset\.InProceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages,External Links:[Link](https://aclanthology.org/2021.dravidianlangtech-1.2/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1)\.
- Reiet al\.\(2023\)L\. Rei, D\. Mladenić, M\. Dorozynski, F\. Rottensteiner, T\. Schleider, R\. Troncy, J\. Sebastián Lozano, and M\. Gaitán SalvatellaMultimodal metadata assignment for cultural heritage artifacts\.Multimedia Systems\.External Links:[Document](https://dx.doi.org/10.1007/s00530-022-01025-2),[Link](https://link.springer.com/article/10.1007/s00530-022-01025-2)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Romeroet al\.\(2024\)D\. Romero, C\. Lyu, H\. A\. Wibowo, T\. Lynn, I\. Hamed, A\. N\. Kishore, A\. Mandal, A\. Dragonetti, A\. Abzaliev, A\. L\. Tonja, B\. F\. Balcha, C\. Whitehouse, C\. Salamea, D\. J\. Velasco, D\. I\. Adelani, D\. Le Meur, E\. Villa\-Cueva, F\. Koto, F\. Farooqui, F\. Belcavello, G\. Batnasan, G\. Vallejo, G\. Caulfield, G\. Ivetta, H\. Song, H\. B\. Ademtew, H\. Maina, H\. Lovenia, I\. A\. Azime, J\. C\. B\. Cruz, J\. Gala, J\. Geng, J\. Ortiz\-Barajas, J\. Baek, J\. Dunstan, L\. A\. Alemany, K\. R\. Y\. Nagasinghe, L\. Benotti, L\. F\. D’Haro, M\. Viridiano, M\. Estecha\-Garitagoitia, M\. C\. B\. Cabrera, M\. Rodríguez\-Cantelar, M\. Jouitteau, M\. Mihaylov, M\. F\. M\. Imam, M\. F\. Adilazuarda, M\. Gochoo, M\. Otgonbold, N\. Etori, O\. Niyomugisha, P\. M\. Silva, P\. Chitale, R\. Dabre, R\. Chevi, R\. Zhang, R\. Diandaru, S\. Cahyawijaya, S\. Góngora, S\. Jeong, S\. Purkayastha, T\. Kuribayashi, T\. Clifford, T\. Jayakumar, T\. T\. Torrent, T\. Ehsan, V\. Araujo, Y\. Kementchedjhieva, Z\. Burzo, Z\. W\. Lim, Z\. X\. Yong, O\. Ignat, J\. Nwatu, R\. Mihalcea, T\. Solorio, and A\. F\. AjiCVQA: culturally\-diverse multilingual visual question answering benchmark\.External Links:2406\.05967,[Link](https://arxiv.org/abs/2406.05967)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Röttgeret al\.\(2024\)P\. Röttger, H\. R\. Kirk, B\. Vidgen, G\. Attanasio, F\. Bianchi, and D\. HovyXSTEST: a test suite for identifying exaggerated safety behaviours in large language models\.External Links:2308\.01263,[Link](https://arxiv.org/abs/2308.01263)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Rudingeret al\.\(2018\)R\. Rudinger, J\. Naradowsky, B\. Leonard, and B\. Van DurmeGender bias in coreference resolution\.External Links:1804\.09301,[Link](https://arxiv.org/abs/1804.09301)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Saffariet al\.\(2024\)H\. Saffari, M\. Shafiei, D\. Rooein, F\. Pierri, and D\. NozzaCan i introduce my boyfriend to my grandmother? evaluating large language models capabilities on Iranian social norm classification\.External Links:2406\.09123,[Link](https://arxiv.org/abs/2406.09123)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Sahooet al\.\(2024\)N\. R\. Sahoo, P\. Kulkarni, N\. Asad, A\. Ahmad, T\. Goyal, A\. Garimella, and P\. BhattacharyyaIndiBias: a benchmark dataset to measure social biases in language models for Indian context\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),External Links:[Link](https://aclanthology.org/2024.naacl-long.487/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Sahuet al\.\(2025\)P\. Sahu, A\. Som, D\. Vergyri, and A\. DivakaranMINDS: a cross\-cultural dialogue corpus for social norm classification and adherence detection\.InFindings of the Association for Computational Linguistics: IJCNLP 2025,External Links:[Link](https://aclanthology.org/2025.findings-ijcnlp.128/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Sakaiet al\.\(2024\)Y\. Sakai, H\. Kamigaito, and T\. WatanabeMCSQA: multilingual commonsense reasoning dataset with unified creation strategy by language models and humans\.InFindings of the Association for Computational Linguistics: ACL 2024,External Links:[Link](https://aclanthology.org/2024.findings-acl.844/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Scariaet al\.\(2024\)K\. Scaria, H\. Gupta, S\. Goyal, S\. A\. Sawant, S\. Mishra, and C\. BaralInstructABSA: instruction learning for aspect based sentiment analysis\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 2: Short Papers\),External Links:[Link](https://aclanthology.org/2024.naacl-short.63/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1)\.
- Semenov and Sennrich \(2025\)K\. Semenov and R\. SennrichMeasuring the effect of disfluency in multilingual knowledge probing benchmarks\.External Links:2510\.15115,[Link](https://arxiv.org/abs/2510.15115)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Shaikhet al\.\(2023\)O\. Shaikh, C\. Ziems, W\. Held, A\. Pariani, F\. Morstatter, and D\. YangModeling cross\-cultural pragmatic inference with codenames duet\.InFindings of the Association for Computational Linguistics: ACL 2023,Toronto, Canada,pp\. 6550–6569\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.410),[Link](https://aclanthology.org/2023.findings-acl.410/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.15.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1)\.
- Shiet al\.\(2024\)W\. Shi, R\. Li, Y\. Zhang, C\. Ziems, R\. Horesh, R\. A\. d\. C\. Riemer, and D\. YangCultureBank: an online community\-driven knowledge base towards culturally aware language technologies\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Miami, Florida, USA\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.288/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Singhet al\.\(2025\)S\. Singh, A\. Romanou, C\. Fourrier, D\. I\. Adelani, J\. G\. Ngui, D\. Vila\-Suero, P\. Limkonchotiwat, K\. Marchisio, W\. Q\. Leong, Y\. Susanto, R\. Ng, S\. Longpre, W\. Ko, A\. Bosselut, A\. O\. Smith, L\. C\. Foerster, R\. Vassilieva, S\. Hooker, and M\. FadaeeGlobal MMLU: understanding and addressing cultural and linguistic biases in multilingual evaluation\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),External Links:[Link](https://aclanthology.org/2025.acl-long.919/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Singhet al\.\(2024\)S\. Singh, F\. Vargus, D\. Dsouza, B\. F\. Karlsson, A\. Mahendiran, W\. Ko,et al\.Aya dataset: an open\-access collection for multilingual instruction tuning\.External Links:2402\.06619,[Link](https://arxiv.org/abs/2402.06619)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Sommeseet al\.\(2023\)R\. Sommese, R\. van Rijswijk\-Deij, and M\. JonkerThis is a local domain: on amassing country\-code top\-level domains from public data\.External Links:2309\.01441,[Link](https://arxiv.org/abs/2309.01441)Cited by:[§3\.1\.1](https://arxiv.org/html/2609.25298#S3.SS1.SSS1.p1.1)\.
- Sonet al\.\(2024\)G\. Son, H\. Lee, S\. Kim, S\. Kim, N\. Muennighoff, T\. Choi, C\. Park, K\. M\. Yoo, and S\. BidermanKMMLU: measuring massive multitask language understanding in Korean\.External Links:2402\.11548,[Link](https://arxiv.org/abs/2402.11548)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Sorensenet al\.\(2024\)T\. Sorensen, L\. Jiang, J\. D\. Hwang, S\. Levine, V\. Pyatkin, P\. West, N\. Dziri, X\. Lu, K\. Rao, C\. Bhagavatula, M\. Sap, J\. Tasioulas, and Y\. ChoiValue kaleidoscope: engaging AI with pluralistic human values, rights, and duties\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38\.External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/29970)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Srinivasan and Choi \(2022\)A\. Srinivasan and E\. ChoiTyDiP: a dataset for politeness classification in nine typologically diverse languages\.InFindings of the Association for Computational Linguistics: EMNLP 2022,External Links:[Link](https://aclanthology.org/2022.findings-emnlp.420/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1)\.
- Stanovskyet al\.\(2019\)G\. Stanovsky, N\. A\. Smith, and L\. ZettlemoyerEvaluating gender bias in machine translation\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,External Links:[Link](https://aclanthology.org/P19-1164/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Stranisciet al\.\(2021\)M\. Stranisci, M\. De Leonardis, C\. Bosco, and V\. PattiThe expression of moral values in the Twitter debate: a corpus of conversations\.Italian Journal of Computational Linguistics \(IJCoL\)7\(1\-2\),pp\. 113–132\.External Links:[Link](https://journals.openedition.org/ijcol/880)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Strazda and Spanakis \(2025\)E\. Strazda and G\. SpanakisDutch CrowS\-Pairs: adapting a challenge dataset for measuring social biases in language models for Dutch\.External Links:2507\.16442,[Link](https://arxiv.org/abs/2507.16442)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Sun and Xu \(2022\)Z\. Sun and Y\. XuTracing semantic variation in slang\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Abu Dhabi, United Arab Emirates,pp\. 1299–1313\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.84/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1)\.
- Sweed and Shahaf \(2021\)N\. Sweed and D\. ShahafCatchphrase: automatic detection of cultural references\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 2: Short Papers\),External Links:[Link](https://aclanthology.org/2021.acl-short.1/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1)\.
- Thapliyalet al\.\(2022\)A\. V\. Thapliyal, J\. Pont Tuset, X\. Chen, and R\. SoricutCrossmodal\-3600: a massively multilingual multimodal evaluation dataset\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Abu Dhabi, United Arab Emirates,pp\. 715–729\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.45),[Link](https://aclanthology.org/2022.emnlp-main.45/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Tikhonovet al\.\(2021\)A\. Tikhonov, I\. Samenko, and I\. P\. YamshchikovStoryDB: broad multi\-language narrative dataset\.InProceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems,External Links:[Link](https://aclanthology.org/2021.eval4nlp-1.4/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.10.2.1.1)\.
- Tomaret al\.\(2025\)A\. Tomar, N\. R\. Sahoo, and P\. BhattacharyyaBharatBBQ: a multilingual bias benchmark for question answering in the Indian context\.External Links:2508\.07090,[Link](https://arxiv.org/abs/2508.07090)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Trageret al\.\(2025\)J\. Trager, F\. Vargas, D\. Alves, M\. Guida, M\. K\. Ngueajio, A\. Agrawal, Y\. Daryani, F\. Karimi Malekabadi, and F\. M\. Plaza\-del\-ArcoMFTCXplain: a multilingual benchmark dataset for evaluating the moral reasoning of LLMs through multi\-hop hate speech explanation\.InFindings of the Association for Computational Linguistics: EMNLP 2025,External Links:[Link](https://aclanthology.org/2025.findings-emnlp.851/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Troles and Schmid \(2021\)J\. Troles and U\. SchmidExtending challenge sets to uncover gender bias in machine translation: impact of stereotypical verbs and adjectives\.InProceedings of the Sixth Conference on Machine Translation,External Links:[Link](https://aclanthology.org/2021.wmt-1.61/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Vargaset al\.\(2022\)F\. Vargas, I\. Carvalho, F\. R\. de Góes, T\. Pardo, and F\. BenevenutoHateBR: a large expert annotated corpus of Brazilian Instagram comments for offensive language and hate speech detection\.InProceedings of the Thirteenth Language Resources and Evaluation Conference,External Links:[Link](https://aclanthology.org/2022.lrec-1.777/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1)\.
- Vargaset al\.\(2026\)F\. Vargas, J\. Trager, D\. Alves, S\. Thapa, M\. Guida, B\. Atil, D\. Dementieva, A\. Smart, and A\. AgrawalSelf\-explaining hate speech detection with moral rationales\.External Links:2601\.03481,[Link](https://arxiv.org/abs/2601.03481)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Vo and Koyejo \(2025\)T\. Vo and S\. KoyejoCURE: cultural understanding and reasoning evaluation – a framework for “thick” culture alignment evaluation in LLMs\.External Links:2511\.12014,[Link](https://arxiv.org/abs/2511.12014)Cited by:[§1](https://arxiv.org/html/2609.25298#S1.p1.1)\.
- Wanget al\.\(2020\)H\. Wang, M\. Li, Z\. Zhou, G\. P\. C\. Fung, and K\. WongKddRES: a multi\-level knowledge\-driven dialogue dataset for restaurant towards customized dialogue system\.External Links:2011\.08772,[Link](https://arxiv.org/abs/2011.08772)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1)\.
- Wanget al\.\(2021\)J\. Wang, P\. Madhyastha, J\. Figueiredo, C\. Lala, and L\. SpeciaMultiSubs: a large\-scale multimodal and multilingual dataset\.External Links:2103\.01910,[Link](https://arxiv.org/abs/2103.01910)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Wanget al\.\(2025\)T\. Wang, S\. Liu, Y\. Wang, B\. Wang, Y\. Lin, J\. Wang, L\. Wang, and H\. YuMcBE: a multi\-task chinese bias evaluation benchmark for large language models\.External Links:2507\.02088,[Link](https://arxiv.org/abs/2507.02088)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Wanget al\.\(2024\)Y\. Wang, Y\. Zhu, C\. Kong, S\. Wei, X\. Yi, X\. Xie, and J\. SangCDEval: a benchmark for measuring the cultural dimensions of large language models\.InProceedings of the 2nd Workshop on Cross\-Cultural Considerations in NLP,External Links:[Link](https://aclanthology.org/2024.c3nlp-1.1/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.15.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Wenzeket al\.\(2020\)G\. Wenzek, M\. Lachaux, A\. Conneau, V\. Chaudhary, F\. Guzmán, A\. Joulin, and E\. GraveCCNet: extracting high quality monolingual datasets from web crawl data\.InProceedings of the Twelfth Language Resources and Evaluation Conference,Marseille, France,pp\. 4003–4012\.External Links:[Link](https://aclanthology.org/2020.lrec-1.494/)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px2.p1.1)\.
- Wibowoet al\.\(2024\)H\. A\. Wibowo, E\. H\. Fuadi, M\. N\. Nityasya, R\. E\. Prasojo, and A\. F\. AjiCOPAL\-ID: indonesian language reasoning with local culture and nuances\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),Mexico City, Mexico\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.77),[Link](https://aclanthology.org/2024.naacl-long.77/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Winataet al\.\(2025\)G\. I\. Winata, F\. Hudi, P\. A\. Irawan, D\. Anugraha, R\. A\. Putri, W\. Yutong, A\. Nohejl, U\. A\. Prathama, N\. Ousidhoum, A\. Amriani, A\. Rzayev, A\. Das, A\. Pramodya, A\. Adila, B\. Wilie, C\. O\. Mawalim, C\. C\. Lam, D\. Abolade, E\. Chersoni, E\. Santus, F\. Ikhwantri, G\. Kuwanto, H\. Zhao, H\. A\. Wibowo, H\. Lovenia, J\. C\. B\. Cruz, J\. W\. G\. Putra, J\. Myung, L\. Susanto, M\. A\. R\. Machin, M\. Zhukova, M\. Anugraha, M\. F\. Adilazuarda, N\. C\. Santosa, P\. Limkonchotiwat, R\. Dabre, R\. A\. Audino, S\. Cahyawijaya, S\. Zhang, S\. Y\. Salim, Y\. Zhou, Y\. Gui, D\. I\. Adelani, E\. A\. Lee, S\. Okada, A\. Purwarianti, A\. F\. Aji, T\. Watanabe, D\. T\. Wijaya, A\. Oh, and C\. NgoWorldCuisines: a massive\-scale benchmark for multilingual and multicultural visual question answering on global cuisines\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),Albuquerque, New Mexico,pp\. 3242–3264\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.167),[Link](https://aclanthology.org/2025.naacl-long.167/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Wuet al\.\(2024\)X\. Wu, T\. Nguyen, D\. C\. Zhang, W\. Y\. Wang, and A\. T\. LuuFASTopic: pretrained transformer is a fast, adaptive, stable, and transferable topic model\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/998f8d8ff2697da2eae5f87143668754-Abstract-Conference.html)Cited by:[Appendix G](https://arxiv.org/html/2609.25298#A7.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px3.p1.1),[§3\.2\.2](https://arxiv.org/html/2609.25298#S3.SS2.SSS2.p2.1),[§3\.2\.2](https://arxiv.org/html/2609.25298#S3.SS2.SSS2.p4.1)\.
- Wulczynet al\.\(2017\)E\. Wulczyn, N\. Thain, and L\. DixonEx machina: personal attacks seen at scale\.External Links:1610\.08914,[Link](https://arxiv.org/abs/1610.08914)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.7.2.1.1)\.
- Xia and Monti \(2021\)M\. Xia and E\. MontiMultilingual neural semantic parsing for low\-resourced languages\.InProceedings of \*SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics,pp\. 185–194\.External Links:[Link](https://aclanthology.org/2021.starsem-1.17/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Xuet al\.\(2024\)S\. Xu, W\. Dong, Z\. Guo, X\. Wu, and D\. XiongExploring multilingual concepts of human values in large language models: is value alignment consistent, transferable and controllable across languages?\.InFindings of the Association for Computational Linguistics: EMNLP 2024,External Links:[Link](https://aclanthology.org/2024.findings-emnlp.96/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Xueet al\.\(2021\)L\. Xue, N\. Constant, A\. Roberts, M\. Kale, R\. Al\-Rfou, A\. Siddhant, A\. Barua, and C\. RaffelmT5: a massively multilingual pre\-trained text\-to\-text transformer\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,Online,pp\. 483–498\.Note:Introduces the mC4 corpusExternal Links:[Document](https://dx.doi.org/10.18653/v1/2021.naacl-main.41),[Link](https://aclanthology.org/2021.naacl-main.41/)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px2.p1.1)\.
- Yanget al\.\(2025\)H\. Yang, Y\. Li, X\. Zhang, and J\. LinSupervised neural topic modeling with label alignment\.Transactions of the Association for Computational Linguistics13,pp\. 249–263\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00738),[Link](https://direct.mit.edu/tacl/article/13/249/128712/Supervised-Neural-Topic-Modeling-with-Label)Cited by:[§2](https://arxiv.org/html/2609.25298#S2.SS0.SSS0.Px3.p1.1)\.
- Yinet al\.\(2022\)D\. Yin, H\. Bansal, M\. Monajatipoor, L\. H\. Li, and K\. ChangGeoMLAMA: geo\-diverse commonsense probing on multilingual pre\-trained language models\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Abu Dhabi, United Arab Emirates\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.132/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1)\.
- Yinet al\.\(2021\)D\. Yin, L\. H\. Li, Z\. Hu, N\. Peng, and K\. ChangBroaden the vision: geo\-diverse visual commonsense reasoning\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,Online and Punta Cana, Dominican Republic,pp\. 2115–2129\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.162),[Link](https://aclanthology.org/2021.emnlp-main.162/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.3.2.1.1)\.
- Yuet al\.\(2024\)L\. Yu, Y\. Leng, Y\. Huang, S\. Wu, H\. Liu, X\. Ji, J\. Zhao, J\. Song, T\. Cui, X\. Cheng, T\. Liu, and D\. XiongCMoralEval: a moral evaluation benchmark for Chinese large language models\.InFindings of the Association for Computational Linguistics: ACL 2024,External Links:[Link](https://aclanthology.org/2024.findings-acl.703/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Yuanet al\.\(2024\)Y\. Yuan, K\. Tang, J\. Shen, M\. Zhang, and C\. WangMeasuring social norms of large language models\.InFindings of the Association for Computational Linguistics: NAACL 2024,External Links:[Link](https://aclanthology.org/2024.findings-naacl.43/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.4.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Yueet al\.\(2024\)T\. Yue, X\. Shi, R\. Mao, Z\. Hu, and E\. CambriaSarcNet: a multilingual multimodal sarcasm detection dataset\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\),Torino, Italia,pp\. 14325–14335\.External Links:[Link](https://aclanthology.org/2024.lrec-main.1248/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.8.2.1.1)\.
- Zahraei and Asgari \(2025\)P\. S\. Zahraei and E\. AsgariI am aligned, but with whom? MENA values benchmark for evaluating cultural alignment and multilingual bias in LLMs\.External Links:2510\.13154,[Link](https://arxiv.org/abs/2510.13154)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1)\.
- Zhanet al\.\(2024\)H\. Zhan, Z\. Li, X\. Kang, T\. Feng, Y\. Hua, L\. Qu, Y\. Ying, M\. R\. Chandra, K\. Rosalind, J\. Jiang, S\. Sharma, S\. Q\. L\. L\. L\. S\. Z\. L\. G\. H\. S\. S\. Y\. R\. Stevens, and G\. HaffariRENOVI: a benchmark towards remediating norm violations in socio\-cultural conversations\.InFindings of the Association for Computational Linguistics: NAACL 2024,Mexico City, Mexico,pp\. 3104–3117\.External Links:[Link](https://aclanthology.org/2024.findings-naacl.196/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.15.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Zhanet al\.\(2023\)H\. Zhan, Z\. Li, Y\. Wang, L\. Luo, T\. Feng, X\. Kang, Y\. Hua, L\. Qu, L\. Soon, S\. Sharma, and G\. HaffariSocialDial: a benchmark for socially\-aware dialogue systems\.InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval \(SIGIR ’23\),External Links:[Document](https://dx.doi.org/10.1145/3539618.3591877),[Link](https://github.com/zhanhl316/socialdial)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.13.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.15.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Zhaoet al\.\(2018\)J\. Zhao, T\. Wang, M\. Yatskar, V\. Ordonez, and K\. ChangGender bias in coreference resolution: evaluation and debiasing methods\.External Links:1804\.06876,[Link](https://arxiv.org/abs/1804.06876)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.
- Zhaoet al\.\(2024\)W\. Zhao, D\. Mondal, N\. Tandon, D\. Dillion, K\. Gray, and Y\. GuWorldValuesBench: a large\-scale benchmark dataset for multi\-cultural value awareness of language models\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\),External Links:[Link](https://aclanthology.org/2024.lrec-main.1539/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.5.2.1.1)\.
- Ziemset al\.\(2022a\)C\. Ziems, J\. Chen, C\. Harris, J\. Anderson, and D\. YangVALUE: understanding dialect disparity in nlu\.External Links:2204\.03031,[Link](https://arxiv.org/abs/2204.03031)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1)\.
- Ziemset al\.\(2023a\)C\. Ziems, J\. Dwivedi\-Yu, Y\. Wang, A\. Halevy, and D\. YangNormBank: a knowledge bank of situational social norms\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Toronto, Canada\.External Links:[Link](https://aclanthology.org/2023.acl-long.429/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.16.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.17.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.18.2.1.1),[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Ziemset al\.\(2023b\)C\. Ziems, W\. Held, J\. Yang, J\. Dhamala, R\. Gupta, and D\. YangMulti\-VALUE: a framework for cross\-dialectal English NLP\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Toronto, Canada\.External Links:[Link](https://aclanthology.org/2023.acl-long.44/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.12.2.1.1)\.
- Ziemset al\.\(2022b\)C\. Ziems, J\. Yu, Y\. Wang, A\. Halevy, and D\. YangThe moral integrity corpus: a benchmark for ethical dialogue systems\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Dublin, Ireland\.External Links:[Link](https://aclanthology.org/2022.acl-long.261/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.9.2.1.1)\.
- Zulaika and Saralegi \(2025\)M\. Zulaika and X\. SaralegiBasqBBQ: a QA benchmark for assessing social biases in LLMs for Basque, a low\-resource language\.InProceedings of the 31st International Conference on Computational Linguistics,External Links:[Link](https://aclanthology.org/2025.coling-main.318/)Cited by:[Table 13](https://arxiv.org/html/2609.25298#A7.T13.2.6.2.1.1)\.

## Appendix ARegion Resolver Details

During lookup construction, each Common Crawl sample record is assigned a weighted vote according to the agreement pattern among URL evidence, HTTPContent\-Language, HTMLlang, and language\-identification metadata\. The rules are used only during offline lookup construction; they do not fire during FineWeb/FineWeb\-2 annotation\.

##### Confidence\-tier thresholds\.

After evidence aggregation, each URL\-signature row is assigned a confidence tier based on the following thresholds\. A row is labelledhighif the winning region has at least 20 weighted votes, at least 90% vote share, and appears across at least three Common Crawl dumps\. It is labelledmediumif it has at least seven weighted votes, at least 75% vote share, and appears across at least two dumps\. It is labelledlowif it has at least three weighted votes and at least 70% vote share\. These thresholds favour URL signatures that are stable across time and not merely frequent in a single crawl; we selected them empirically by manually inspecting samples of candidate lookup rows at different cutoff values before freezing the table, and no downstream corpus statistics or analyses were consulted during threshold selection\.

##### Real ccTLD evidence\.

When the URL contains a non\-branded country\-code top\-level domain, the ccTLD supplies the primary region candidate\. Three\-way agreement among ccTLD, HTMLlang, and HTTPContent\-Languagereceives the highest weight\. Agreement between the ccTLD and one header signal receives slightly lower weight\. When the ccTLD is contradicted by a likely CDN\-default header, the ccTLD is still trusted but downweighted\. When the ccTLD is contradicted by a country\-specific header, the vote is retained with weaker weight\. If no header signal is available, the ccTLD alone still contributes a strong but not maximal vote\.

##### Path and subdomain evidence\.

For generic or branded domains, the system searches for locale\-bearing path or subdomain tokens, such as/uk/,/jp/,de\.example\.com, oren\-gb\. These hints receive higher weight when corroborated by HTTP or HTML language metadata and lower weight when they appear alone\.

##### Header\-only evidence\.

If the URL contains no region cue, low\-resolution header evidence may contribute a weak vote only when it is consistent with language identification\. High\-resolution headers on otherwise generic URLs are not treated as positive region evidence because such values are often injected by server or CDN defaults\.

##### Fallback and unresolved records\.

If no reliable rule supplies evidence during lookup construction, the record contributes no vote\. During search\-time resolution, URLs that do not match the lookup table may still receive weak labels from curated URL hints or non\-branded ccTLD fallback; otherwise the resolver emitsXX\.

##### Per\-rule vote weights\.

Each rule contributes a per\-record vote with a fixed weight, calibrated to reflect the strength of evidence it provides\. The weights used during offline lookup construction are summarized in Table[5](https://arxiv.org/html/2609.25298#A1.T5)\.

Table 5:Vote weights applied by each evidence rule during offline lookup\-table construction\. Higher weights correspond to stronger multi\-signal agreement\. Rules A9 and A10 are recorded for diagnostic completeness but do not contribute to weighted aggregation\.
##### Lookup\-Table Schema

The frozen URL\-signature lookup table is released as a CSV with 1,335,000 rows and the columns described in Table[6](https://arxiv.org/html/2609.25298#A1.T6)\.

Table 6:Schema of the released URL\-signature lookup table\.Rows are emitted only when the winning region passes the confidence thresholds in Section[3\.1\.3](https://arxiv.org/html/2609.25298#S3.SS1.SSS3)\. Null encodings:top\_regionandsecondary\_regionuse the empty string when no candidate region is available;n\_content\_votesandcontent\_shareare zero for all rows in the URL\-only release\.

## Appendix BSearch\-Time Resolution Sources

At search time, the resolver applies the priority chain in Table[1](https://arxiv.org/html/2609.25298#S3.T1)\. The first source that emits a valid region label determines the document’s region, resolution source, and confidence value\. The sources are interpreted as follows\.

1. 1\.Query parameters\.Explicit region parameters such asgl=US,country=ca,region=GB, orlocale=ja\-JPreceive the highest priority\. These cases are rare, but they are strong signals because the site operator is explicitly encoding a locale choice\.
2. 2\.Lookup\-backed keys\.The next three sources consult the frozen lookup table\.host\_pathcaptures multinational websites that organize regional sites under the first path segment, such aspwc\.com/uk\.hostcaptures full hostnames such aswww\.lemonde\.fr\.domaincollapses subdomains to a registered domain, allowingnews\.bbc\.co\.ukandsport\.bbc\.co\.ukto inherit evidence frombbc\.co\.ukwhen appropriate\.
3. 3\.URL hints\.If the lookup table does not contain a matching key, the resolver searches for curated country tokens in structurally informative URL positions\. Examples includeuk\.airbnb\.com,airbnb\.com/jp/rooms, andredbubble\.com/en\-gb/\. Ambiguous tokens such asco,it, andmeare excluded unless the URL structure makes the regional interpretation unambiguous\.
4. 4\.ccTLD fallback\.Finally, the resolver falls back to the URL’s ccTLD when it is not on a branded\-ccTLD denylist\. The denylist includes ccTLDs widely used outside their nominal jurisdictions, such as\.tv,\.ly,\.io,\.me,\.ai,\.co,\.cm, and\.gg\. Non\-branded ccTLDs such as\.de,\.fr,\.jp,\.ru, and\.brare retained as weak fallback evidence\.
5. 5\.No signal\.When all sources fail, the resolver emitsXXwith sourcenoneand confidencenone\. These records remain in the intermediate annotated corpus; filtering is left to downstream users\.

## Appendix CTopic\-to\-CT Softmax and JSD

##### Temperature\-scaled softmax for topic\-to\-CT projection \(Sec\.[3\.2](https://arxiv.org/html/2609.25298#S3.SS2)\)\.

After topic induction, each merged topic is mapped to the CT label space\. Each CT category is embedded once using BGE\-M3 from its natural\-language description\. For each merged topicttand CT categorycc, we compute cosine similarity between the topic centroidete\_\{t\}and the CT category embeddingece\_\{c\}, and convert similarities into a probability distribution over CT categories using a temperature\-scaled softmax:

p⁡\(c∣t\)=exp⁡\(cos⁡\(et,ec\)/τ\)∑c′exp⁡\(cos⁡\(et,ec′\)/τ\),τ=0\.05\.p\(c\\mid t\)=\\frac\{\\exp\(\\cos\(e\_\{t\},e\_\{c\}\)/\\tau\)\}\{\\sum\_\{c^\{\\prime\}\}\\exp\(\\cos\(e\_\{t\},e\_\{c^\{\\prime\}\}\)/\\tau\)\},\\quad\\tau=0\.05\.The low temperature makes the mapping sharp enough for the highest\-scoring category to be interpretable, while still preserving non\-zero mass on plausible runner\-up categories\.

##### Jensen–Shannon distance over LTDs \(Sec\.[5\.3](https://arxiv.org/html/2609.25298#S5.SS3)\)\.

For two locale\-level Locale Topic DistributionsP,QP,Qover the 14 CT leaves, we report the Jensen–Shannon distance computed withlog2\\log\_\{2\}:

JSD⁡\(P,Q\)\\displaystyle\\mathrm\{JSD\}\(P,Q\)=12KL\(P∥M\)\+12KL\(Q∥M\),\\displaystyle=\\sqrt\{\\tfrac\{1\}\{2\}\\,\\mathrm\{KL\}\(P\\,\\\|\\,M\)\+\\tfrac\{1\}\{2\}\\,\\mathrm\{KL\}\(Q\\,\\\|\\,M\)\},M\\displaystyle M=12​\(P\+Q\)\.\\displaystyle=\\tfrac\{1\}\{2\}\(P\+Q\)\.withKL\\mathrm\{KL\}inlog2\\log\_\{2\}, soJSD⁡\(P,Q\)∈\[0,1\]\\mathrm\{JSD\}\(P,Q\)\\in\[0,1\]\. Because per\-locale leaf mass is produced by the temperature\-scaled softmax above, every LTD has strictly positive entries on every leaf and no smoothing is applied; theKL\\mathrm\{KL\}terms are finite without an additive prior\. Implementation:scipy\.spatial\.distance\.jensenshannon\(p, q, base=2\)\.

## Appendix DBenchmark Audit Protocol

This appendix describes the construction and annotation protocol for the benchmark audit summarized in Section[4](https://arxiv.org/html/2609.25298#S4)\. The audit is designed to support corpus–benchmark comparison by coding each dataset along cultural coverage, scope, provenance, and usability dimensions\.

### D\.1Dataset Sources

We compile candidate datasets from three sources:

1. 1\.Datasets included in the cultural\-NLP taxonomy and survey of[Liu et al\. \(2025\)](https://arxiv.org/html/2609.25298#bib.bib17)\.
2. 2\.Datasets included in MUREL\-style cultural\-resource curation\([Namazifard and Galke, 2025](https://arxiv.org/html/2609.25298#bib.bib28)\)\.
3. 3\.Newly identified datasets from systematic searches of the ACL Anthology, arXiv, HuggingFace, GitHub, and dataset repositories\.

The systematic searches combine cultural terms with resource terms\. Cultural terms include*culture*,*cultural*,*cross\-cultural*,*multilingual*,*norms*,*values*,*bias*,*dialect*,*stereotype*,*social norms*, and*cultural knowledge*\. Resource terms include*dataset*,*benchmark*,*corpus*,*evaluation*, and*test set*\. Candidate records are deduplicated across source lists and benchmark families\.

### D\.2Inclusion and Exclusion Criteria

A resource is included if it satisfies all of the following criteria:

1. 1\.It provides a downloadable, inspectable, or otherwise accessible dataset\.
2. 2\.It targets at least one cultural phenomenon covered by the CT, such as values, norms, dialects, communicative practices, culturally grounded knowledge, or culturally grounded bias\.
3. 3\.It is documented in a paper, preprint, dataset card, or repository with sufficient metadata for annotation\.
4. 4\.It is usable as an evaluation dataset\.

We exclude resources that do not provide dataset access, papers that discuss cultural evaluation without releasing or identifying a dataset, duplicate entries already represented by another variant, and resources whose cultural target cannot be mapped to any CT leaf\.

When a benchmark family contains multiple variants, we count a variant separately only if it introduces a distinct language set, region set, task format, annotation schema, or culturally adapted item set\. For example, a translated or culturally localised derivative is counted as a separate dataset but linked to its parent through the lineage field\.

### D\.3Annotation Dimensions

Each included dataset is annotated along four groups of dimensions\.

##### Cultural coverage\.

These fields describe what cultural phenomena the dataset tests:

- •CT branch;
- •CT leaf or leaves;

##### Scope\.

These fields describe the populations, languages, and task format covered by the dataset:

- •dataset name;
- •geographical diversity;
- •linguistic typology;
- •languages covered;
- •regions covered;
- •dataset format;
- •example item, when available\.

##### Provenance\.

These fields describe where the dataset comes from and how it was constructed:

- •paper link;
- •publication venue;
- •publication date;
- •generation method: human, hybrid human–LLM, LLM\-only, web\-scraped or automatically extracted, or unclear;
- •source list;
- •dataset lineage\.

##### Usability\.

These fields describe whether and how the dataset can be used for cultural evaluation:

- •data link;
- •accessibility status;

### D\.4CT Leaf Annotation

CT branch and leaf assignments are coded from each dataset’s paper, dataset card, repository, and example items\. A dataset may receive multiple CT leaves if it evaluates more than one cultural phenomenon\. For example, a benchmark containing culturally situated dialogue about politeness norms may be coded as both*Norms & Morals*and*Communicative Goals*; a multilingual stereotype benchmark may be coded as*Values–Bias*; and a dialect\-identification benchmark is coded as*Dialects*\.

Assignments are made at the dataset level rather than the individual item level\. This choice allows broad survey coverage across the benchmark landscape, but it means that fine\-grained within\-dataset variation is not fully captured\. Multi\-category datasets are counted once per applicable CT leaf in the coverage matrix\.

### D\.5Lineage Coding

The lineage field records whether a dataset is original or derived from one or more parent datasets\. We mark a dataset as derived if the paper or repository explicitly states that it is a translation, adaptation, localisation, extension, reannotation, or culturally modified version of an existing benchmark\. Lineage is recorded using a parent\-to\-child notation, for example:

> BBQ \> MBBQ \> KoBBQ

A dataset is coded as original if we find no known parent benchmark or source dataset beyond generic data sources such as surveys, web pages, or human\-authored prompts\. Lineage coding is conservative: if derivation is plausible but not documented, the dataset is not marked as derived\.

### D\.6Accessibility and Adaptation Coding

Accessibility is coded according to whether the dataset can be obtained and used for cultural evaluation:

- •Public: the dataset is available through a repository, dataset hub, project page, or supplementary material\.
- •Restricted: the dataset requires an application, institutional access, or approval\.
- •Unavailable: the dataset link is broken, missing, or no longer accessible\.

Adaptation feasibility records whether the dataset is directly usable as cultural assertions or evaluation items, or whether transformation is required\. Common transformations include converting QA formats, extracting text from multimodal examples, converting classification labels into assertions, or filtering non\-cultural items from broader multilingual datasets\.

### D\.7Quality Control

Factual metadata such as languages, regions, venue, publication date, dataset link, and format is extracted directly from source papers, dataset cards, or repositories\. CT leaf assignments are coded by two annotators, with disagreements resolved through discussion\. The final annotated table is released with dataset names, CT labels, language and region coverage, lineage fields, accessibility status, and annotation notes\.

## Appendix ERobustness of the CT\-Based Analyses

This appendix reports the robustness analyses promised in the discussion phase for the exploratory CT\-level results: a label\-perturbation study of the Section[5\.2](https://arxiv.org/html/2609.25298#S5.SS2)coverage classifications and the Section[5\.3](https://arxiv.org/html/2609.25298#S5.SS3)divergence ordering, projection behaviour inside vs\. outside BGE\-M3’s documented language coverage, a noise floor conditioned on locale size, and the decomposition behind the change in the region\-level blind\-spot pattern\.

### E\.1Label\-Perturbation Study

The validation in Section[5\.1](https://arxiv.org/html/2609.25298#S5.SS1)measures topic\-level agreement with the human consensus of 22\.4% for the adjudicated labels \(18\.8% for the embedding softmax\); per\-pair annotations were recorded only in aggregate, so a full leaf\-confusion matrix cannot be estimated from them\. We therefore perturb at the observed rate: each of the 801,344 merged topics keeps its adjudicated leaf with probability0\.2240\.224and is otherwise reassigned, drawing the replacement either uniformly over the other 13 leaves \(*uniform*kernel\) or proportionally to the topic’s own softmax mass over the other leaves \(*softmax*kernel, concentrating errors on the leaves the projection itself confuses\)\. Perturbed assignments enter the pipeline in two variants:*swap\-soft*exchanges the softmax mass between the original and resampled leaf within the topic’s soft assignment row, preserving the scale of the released LTDs;*hard*moves the topic’s entire mass to the resampled leaf \(the harshest reading, baseline: one\-hot at the adjudicated leaf\)\. For each condition we drawB=1000B=1000replicates and recompute the region\-level 266\-cell coverage, the locale\-level four\-class percentages \(n=5,826n=5\{,\}826locales\), and the within\-language JSDs \(334,896 same\-language locale pairs over 182 languages\)\.

Table[7](https://arxiv.org/html/2609.25298#A5.T7)summarizes the four conditions\. The region\-level result of Section[5\.2](https://arxiv.org/html/2609.25298#S5.SS2)is invariant: every replicate of every condition keeps all 266 CT\-leaf×\\timesregion cells corpus\-covered atk=50k=50, with the 1st\-percentile smallest cell support across replicates at2\.6×1052\.6\\times 10^\{5\}documents, more than three orders of magnitude above the threshold\. Under the released soft assignment \(swap\-soft\), the locale\-level four\-class percentages are equally stable and in every replicate all five labelled pairs of Section[5\.3](https://arxiv.org/html/2609.25298#S5.SS3)stay above the pooled noise floor, the smallest by a factor of at least 11 at the 2\.5th percentile\. Fine\-grained structure is weaker: the strict five\-pair ordering survives in 44% \(uniform\) and 59% \(softmax kernel\) of swap\-soft replicates; the fragile comparisons are the two close ones \(en–zh preserved in 73%/91%, es–pt in 69%/71%\), while the coarse structure is stable, and per\-language median JSDs correlate with the unperturbed ranking atρ=0\.77\\rho=0\.77–0\.830\.83\. The hard variant additionally shows that the locale\-level percentages are tied to the soft\-assignment representation itself: replacing each topic’s soft row with a one\-hot at its adjudicated leaf already shifts the unperturbed percentages to 14\.5/34\.0/13\.9/37\.6 \(both / corpus\-only / benchmark\-only / neither\), and perturbation then moves them by up to 26 points, while the region\-level coverage and the pairs\-above\-floor findings continue to hold in every replicate\. We therefore read the count\-based coverage results and the pairs\-above\-noise finding as robust; locale\-level percentages and exact pair orderings are descriptive of the released soft\-LTD pipeline, consistent with the exploratory framing of Section[5\.1](https://arxiv.org/html/2609.25298#S5.SS1)\.

Table 7:Label\-perturbation study \(B=1000B=1000replicates per condition\): every merged topic keeps its adjudicated leaf with the observed human\-agreement probability 0\.224 and is otherwise resampled \(uniform or softmax\-confusable kernel\), entering the pipeline as a soft swap or a hard reassignment\. Columns: fraction of replicates in which all 266 region\-level cells stay corpus\-covered atk=50k=50; fraction in which any locale\-level four\-class percentage moves by more than 2 points \(largest observed move, in points\); fraction in which all five labelled pairs stay above the pooledp95p\_\{95\}noise floor; fraction preserving the full five\-pair JSD ordering; Spearman correlation of per\-language median JSDs \(182 languages\) with the unperturbed baseline\.
### E\.2Projection Behaviour by BGE\-M3 Coverage Regime

Table[8](https://arxiv.org/html/2609.25298#A5.T8)splits the topic\-modeled locales by whether their language is inside BGE\-M3’s documented coverage, operationalized as the CC\-100/XLM\-R inventory of its backbone\([Conneau et al\., 2020](https://arxiv.org/html/2609.25298#bib.bib13)\), with macrolanguages mapped to their standard varieties \(so, e\.g\., Egyptian Arabicarzand Cantoneseyuecount as outside\)\. Outside documented coverage the embedding projection is measurably less informative: the medianmaxc⁡p⁡\(c∣t\)\\max\_\{c\}p\(c\\mid t\)equals the uniform value1/14≈0\.0711/14\\approx 0\.071, and 67% of projections are within 2% of uniform \(vs\. 46% inside\)\. The*lower*adjudicator change rate outside coverage \(3\.5% vs\. 7\.5%\) should therefore not be read as better projections: near\-uniform softmax rows give the adjudicator weaker candidate rankings to correct\. This quantifies the Limitations statement that CT\-level results for languages outside BGE\-M3’s validated coverage carry additional uncertainty\.

Table 8:Topic\-to\-CT behaviour inside vs\. outside BGE\-M3’s documented language coverage \(CC\-100/XLM\-R inventory of its backbone; standard varieties for macrolanguages\)\. Near\-uniform:maxc⁡p⁡\(c∣t\)≤1\.02/14\\max\_\{c\}p\(c\\mid t\)\\leq 1\.02/14\. The lower adjudicator change rate outside coverage is not evidence of better projections there — the median outside\-coverage projection carries no discriminative signal, which also weakens the candidate rankings shown to the adjudicator\.
### E\.3Size\-Conditioned Noise Floor

Reviewer discussion noted that the pooled split\-half noise floor of Section[5\.3](https://arxiv.org/html/2609.25298#S5.SS3)is not conditioned on locale size\. Table[9](https://arxiv.org/html/2609.25298#A5.T9)reports the same 235,750 split\-half computations binned by locale sample size\. The floor decreases monotonically with size; the five labelled pairs of Figure[5](https://arxiv.org/html/2609.25298#S5.F5)all involve locales at the 10,000\-document cap, where thep95p\_\{95\}floor is8\.7×10−48\.7\\times 10^\{\-4\}\(roughly half the pooled value\) so the pooled floor understates, rather than overstates, how far those pairs sit above topic\-modeling noise\.

Table 9:Split\-half JSD noise floor conditioned on locale sample size \(50 splits per locale, seed 42\)\. The floor decreases with locale size; the five labelled pairs of Figure[5](https://arxiv.org/html/2609.25298#S5.F5)all lie in the 10k bin, so the pooled floor used in the main text if anything understates their separation from noise\.
### E\.4English vs\. Main\-Language Evidence in the Formerly Blind Regions

Table[10](https://arxiv.org/html/2609.25298#A5.T10)decomposes corpus mass in the regions the submitted analysis showed as blind spots\. Moving from the sample\-350BT analysis corpus to the full release multiplies English\-locale mass by 46–56×\\timesin every such region \(the direct reversal of the∼\\sim50×\\timesEnglish downsampling\) while the mass of the regions’ own non\-English main languages is unchanged, because FineWeb\-2 was already included at full scale\. The region\-level gap therefore closes through English\-language evidence; in the Caribbean, Oceania, and Middle Africa the regions’ own main languages contribute at most 5% of full\-release mass, and locale\-level double blind spots persist among main\-language locales \(e\.g\., French for Saint\-Martin,fra\-MF, 562 documents, 10 of 14 leaves uncovered on both sides\)\.

English\-locale massNon\-English main\-language mass \(full\)Main\-lang\. localesRegionsamplefullgrowthdocsshare of regionw/ double\-blind cellsCaribbean1\.4M66\.5M×\\times48\.03\.6M5%3Oceania13\.8M719\.0M×\\times52\.1309k<<1%0C\. America372k18\.8M×\\times50\.533\.9M60%0Middle Africa449k20\.5M×\\times45\.81\.6M5%0C\. Asia85k4\.7M×\\times55\.615\.7M76%1Table 10:Why the submitted region\-level double\-blind\-spot pattern changed on the full release, for the regions the submitted figure showed as blind\. English\-locale document mass grows by∼\\sim50×\\timesfrom the sample\-350BT analysis corpus to the full release, erasing the region\-level gaps, while non\-English main\-language mass is unchanged \(FineWeb\-2 was already at full scale\)\. In the Caribbean, Oceania, and Middle Africa the regions’ own main languages contribute at most 5% of full\-release mass; in Central America and Central Asia they dominate\. Locale\-level double blind spots \(k=50k=50\) persist among main\-language locales \(last column\)\.

## Appendix FDatasheet for FineWeb\-CLaR

Following[Gebru et al\. \(2021\)](https://arxiv.org/html/2609.25298#bib.bib35), we release a datasheet covering the six artefacts in our resource bundle\. All artefacts share the same release URL and version\-control history\.

##### Motivation, composition, collection\.

The corpus is annotated from the full FineWeb\([Penedo et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib15)\)and FineWeb\-2\([Penedo et al\., 2025](https://arxiv.org/html/2609.25298#bib.bib16)\)release; we do not redistribute the source text and reproduce only the metadata columns required for region attribution and topic\-modeling\.

##### Provenance of reported quantities\.

All region\-layer statistics in this paper \(abstract, Section[1](https://arxiv.org/html/2609.25298#S1), Table[2](https://arxiv.org/html/2609.25298#S3.T2)\) are computed on the full FineWeb \+ FineWeb\-2 release\. All cultural\-topic statistics \(Sections[3\.2](https://arxiv.org/html/2609.25298#S3.SS2)and[5](https://arxiv.org/html/2609.25298#S5)\) come from the released consolidated topic run over this corpus\. Table[11](https://arxiv.org/html/2609.25298#A6.T11)reconciles the corpus snapshots from our experiments; the revised paper reports full\-release values only\.

Table 11:Reconciliation and provenance of corpus snapshots\. The submitted manuscript inadvertently mixed statistics from the first two snapshots; every number in the revised paper comes from the full\-release column\. Each column combines the stated FineWeb subset with the full FineWeb\-2\.10010^\{0\}10110^\{1\}10210^\{2\}10310^\{3\}000\.20\.20\.40\.40\.60\.60\.80\.811median116distinct base domainsper 10k\-URL sampleshare of locales000\.50\.511000\.20\.20\.40\.40\.60\.60\.80\.811median 0\.3534% of locales\>0\.5\>0\.5top\-1 domain shareof locale documentsFigure 6:URL base\-domain diversity per topic\-modeled locale \(up to 10,000 URLs sampled per locale from the full release\)\. Left: CDF of distinct registered base domains per locale \(median 116\)\. Right: CDF of the top\-1 domain’s share of locale documents \(median 0\.35\); 34% of locales draw more than half of their documents from a single domain\.
##### PII and query\-string filtering\.

The lookup table is keyed onkey\_type∈\{\\in\\\{domain,host,host\_path\}\\\}; full URLs and query strings are never persisted in the lookup table\. At search time, the resolver inspects query strings only to extract explicit locale parameters \(gl,country,region,locale\); other query parameters are discarded before annotation columns are written\. No personal data, session tokens, or user identifiers are emitted in the released annotations\.

Table 12:Released artifacts, formats, and licenses\. The region\-annotated corpus covers the full FineWeb \+ FineWeb\-2 release \(30\.9B documents\) and inherits their ODC\-By 1\.0 license\. Locale Topic Distributions cover 5,954 locales, and the benchmark survey contains 277 entries\.
##### Distribution, maintenance\.

Artefacts are released under the licences in Table[12](https://arxiv.org/html/2609.25298#A6.T12)with a fixed version tag; future versions will be released alongside a changelog\. The region\-annotated corpus is distributed asYusser/FineWeb\-CLaR\-region\(locale\-partitioned, id\-joinable to the source corpora\) and the culture\-annotated locale samples asYusser/FineWeb\-CLaR\-culture\(with per\-document topic and CT metadata\), both on the Hugging Face Hub\.

## Appendix GCompute and Wall\-Clock Disclosure

All region\-attribution runs were single\-machine, CPU\-bound, with 64 worker processes against the frozen lookup table\.

##### Topic modeling \(BGE\-M3 \+ FASTopic\)\.

Per\-locale topic induction over the 5,954 topic\-modeled locales used multilingual BGE\-M3\([Chen et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib32)\)for document encoding and FASTopic\([Wu et al\., 2024](https://arxiv.org/html/2609.25298#bib.bib29)\)for topic induction withK=200K=200initial topics per locale\. Document embedding ran in two passes: an initial pass on 6×\\timesNVIDIA A100 80 GB GPUs \(≈\\approx65 GPU\-h, 8,192\-token inputs\), whose per\-locale embeddings were reused, and a consolidation pass on single NVIDIA H100 GPUs \(≈\\approx1\.0 GPU\-h, 1,024\-token cap\) covering the locales fetched or re\-sampled for the final locale set\. Topic induction for the final run took≈\\approx0\.7 GPU\-h with per\-locale crash\-resume checkpointing, and CT projection with LLM adjudication dominated the cost at≈\\approx273 GPU\-h \(split across two parallel single\-GPU jobs\), plus≈\\approx0\.8 CPU\-h for the aggregate analysis\. Figure[7](https://arxiv.org/html/2609.25298#A7.F7)shows the distribution of merged\-topic counts per locale under theK=200K=200budget \(mean 135, median 140, 5th–95th percentile 70–187; e\.g\. ar\-EG 106, en\-US 84, nya\-MW 154\)\.

005050100100150150200200005050100100150150200200250250300300merged topics per locale \(K=200 budget\)localesmean 135median 140Figure 7:Merged topics per locale across the 5,954 topic\-modeled locales \(801,344 in total\)\. Per\-locale merging of theK=200K=200FASTopic budget at cosine0\.850\.85retains an effective average of≈\\approx135 topics per locale\.
##### LLM adjudication\.

Low\-confidence topic\-to\-CT projections \(maxc⁡p⁡\(c∣t\)<0\.5\\max\_\{c\}p\(c\\mid t\)<0\.5\) were adjudicated by Qwen2\.5\-32B\-Instruct \(AWQ\-quantised, served via vLLM 0\.19\.0 withenforce\_eager=True\) at temperatureT=0T=0\. Because the temperature\-scaled softmax atτ=0\.05\\tau=0\.05over the CT label space produces a flat maximum in practice \(observedmaxc⁡p⁡\(c∣t\)\\max\_\{c\}p\(c\\mid t\): mean0\.1240\.124, 99th percentile0\.3150\.315\), 801,068 of the 801,344 merged topics across the 5,954 locales \(99\.97%\) fell below the0\.50\.5cutoff and were adjudicated; the adjudicator changed the primary CT\-leaf assignment for 7\.1% of adjudicated topics \(56,931\)\.

Table 13:Datasets organized by taxonomy element, following[Liu et al\. \(2025\)](https://arxiv.org/html/2609.25298#bib.bib17)\. Numbers in parentheses indicate dataset count per category\. Multi\-category datasets appear under each applicable element\.

相似文章

ArtECulture:多模态大语言模型中文化条件视觉情感理解的基准测试

arXiv cs.CL

本文介绍了ArtECulture,这是一个用于多模态大语言模型中文化条件视觉情感理解的基准测试,涵盖英语、中文和阿拉伯文化,并包含均衡的西方和非西方艺术作品。评估表明该任务仍具挑战性,作者提出了一种检索增强框架,将文化知识注入多模态大语言模型(MLLMs)。

迈向LLM的下一个前沿:私有数据训练——联邦微调的跨域基准

arXiv cs.LG

本文提出了一个在私有数据上对大型语言模型进行联邦微调的跨域基准,评估了LoRA、QLoRA和IA3策略在医疗和金融数据集上的表现。结果表明,联邦微调接近集中式训练的性能,并优于孤立学习,证明了在数据无法共享时通过联邦微调适配LLM的可行性。

语言模型水印中的跨语言公平性审计

arXiv cs.CL

本文提出了一种针对语言模型水印跨语言公平性的评估框架,揭示了语言间的差异是语言类型学结构所致,而非特定语言的个别现象。

文化漏斗:无法对齐数据中缺失的内容

arXiv cs.CL

本文介绍了'文化漏斗'概念,展示了LLM训练数据中的文化信号在后训练阶段急剧下降。作者发布了一个包含5.6M样本的标记数据集,以帮助在模型对齐中保留文化基础。