Figurative and Cultural Knowledge in LLMs: Investigating Cross-Domain Transfer through Fine-Tuning

# Figurative and Cultural Knowledge in LLMs: Investigating Cross-Domain Transfer through Fine-Tuning
Source: [https://arxiv.org/html/2608.18361](https://arxiv.org/html/2608.18361)
Mona DiabAffiliation:MBZUAI, Carnegie Mellon University\{mena\.attia, thamar\.solorio\}@mbzuai\.ac\.aeThamar Solorio

###### Abstract

Figurative language is deeply culturally embedded; fluent use requires not just linguistic competence but cultural immersion\. We ask whether LLMs can learn this link: does fine\-tuning on cultural data improve figurative language understanding, and vice versa? We conduct a systematic study across four models \(ALLaM\-7B, Fanar\-1\-9B, Qwen3\-8B, Llama\-3\.1\-8B\) and six Arabic datasets spanning cultural commonsense, proverbs, and poetry across diverse dialects and regions\. Fine\-tuning on poetry improves idiom comprehension \(\+2\.33%\+2\.33\\%,p<0\.05p<0\.05\), a gain our ArabicMMLU control does not reproduce, indicating that it stems from figurative content rather than Arabic language adaptation and pointing to a sensitivity to non\-literal meaning that transfers across figurative types\. Cultural fine\-tuning, by contrast, lowers proverb\-interpretation accuracy in both Arabic\-centric models\. Transfer between the two domains is otherwise indistinguishable from noise, with Arabic models frequently regressing after fine\-tuning, suggesting prior saturation of relevant knowledge, while multilingual models show greater adaptation headroom\. Error analysis further reveals that fine\-tuning reinforces experiential cultural knowledge while destabilizing historically grounded factual knowledge\. Our findings suggest that the relationship between culture and figurative language, though conceptually natural, is not straightforwardly captured through fine\-tuning alone\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2608.18361v1/figures/nelly.png)Figure 1:Scene from the Egyptian TV show Nelly and Sherihan\.Figurative language is deeply embedded in culture\. Proverbs and idioms, and poetry do more than convey non\-literal meaning; they encode shared values, social norms, and historical experience, so that interpreting them draws on familiarity with the community that produced them rather than on linguistic competence alone\. This premise is well established outside Natural Language Processing \(NLP\)\. Cognitive linguistics documents extensive cross\-cultural variation in figurative meaning: even conceptual metaphors grounded in shared bodily experience are elaborated and conventionalized differently across speech communities\(Kovecses2005\), and cultural conceptualizations are encoded in and reproduced through its language\(sharifian2017cultural\)\. Conventional Figurative Language Theory likewise treats idioms as units whose interpretation draws on culture\-specific connotations, so an idiom may be structurally transparent yet opaque to an outsider\(Dobrovolskij1756694\), and paremiology makes the same point for proverbs, which derive their force from a community’s shared stock of experience rather than compositional meaning\(mieder2004proverbs;mieder1993proverbs\)\. Cognitive science converges on this: figurative interpretation recruits conceptual and background knowledge directly rather than extending literal decoding\(gibbs1994poetics\)\. Together, this literature motivates treating figurative competence and cultural knowledge as coupled rather than independent capacities\.

The 2016 Egyptian television series Nelly and Sherihan offers a concrete instantiation\. Nelly, wealthy and westernized, repeatedly attempts Egyptian proverbs and misuses them, drawing corrections that she dismisses with “whatever,” while her working\-class cousin wields the same expressions fluently \(Figure[1](https://arxiv.org/html/2608.18361#S1.F1)\)\. Its comedic device works only because the audience already shares the norm that misusing proverbs signals cultural distance, which is itself an attestation of the coupling\.

In NLP, however, the two are studied separately\. Cultural benchmarks evaluate factual knowledge, commonsense reasoning, social practices, and regional variation; figurative\-language benchmarks evaluate the interpretation of proverbs, idioms, poetry, and metaphor\. Whether the two forms of knowledge support one another in large language models remains untested: we do not know whether culturally grounded supervision improves figurative interpretation, or whether figurative supervision builds more general cultural knowledge\.

We address this gap by investigating the relationship between cultural and figurative knowledge through controlled fine\-tuning experiments in Arabic\. We study two directions of cross\-domain transfer\. First, we ask whether fine\-tuning on Arabic cultural data improves the interpretation of Arabic idioms and proverbs\. Second, we ask whether fine\-tuning on Arabic figurative\-language data improves performance on culturally grounded question answering\. We additionally examine transfer across figurative forms, asking whether supervision on proverbs or poetry improves understanding of unseen idioms and proverbs\. Third, because both kinds of supervision are also exposure to Arabic, we ask whether any observed transfer reflects the content of the training data or Arabic\-language adaptation alone\. These experiments allow us to distinguish transfer between culture and figurative language from cross\-form figurative transfer that may generalize across non\-literal forms\.

Our contributions are as follows\. \(1\) We present a controlled framework for studying bidirectional transfer between Arabic cultural knowledge and figurative\-language understanding\. \(2\) We provide model\-, dialect\-, and topic\-level analyses showing that fine\-tuning effects are limited, unevenly distributed, and dependent on both model family and knowledge type\. \(3\) We identify poetry fine\-tuning as the only reliable source of positive figurative transfer, supporting the possibility of cross\-form figurative transfer not restricted to a single non\-literal category\.

Taken together, our results show that the relationship between cultural and figurative knowledge is not straightforwardly captured through fine\-tuning\. Transfer effects are generally small, inconsistent, and highly model\-dependent\.

## 2Related Work

### 2\.1Benchmarks for Figurative Language and Culture

Global figurative benchmarks\.In English, Fig\-QAliu\-etal\-2022\-testingframes figurative interpretation as multiple\-choice question answering, while FLUTEchakrabarty\-etal\-2022\-fluteprovides human\-written explanations for idioms alongside other figurative categories\. Multilingual coverage is thinner\. MABLkabra\-etal\-2023\-multievaluates figurative understanding across eight languages, MAPSliu\-etal\-2024\-multilingualevaluates proverb understanding across multiple languages, ProverbEvalazime\-etal\-2025\-proverbevalcovers cultural proverbs in Ethiopian languages and English, and MIDIalmheiri\-etal\-2026\-multilingualspans idioms in 18 languages\. The AdMIRe 2 shared taskarslan\-etal\-2026\-mwechallenges models to interpret idiomatic expressions across multiple languages\.

Arabic figurative benchmarks\.Three recent benchmarks target Arabic: Jawahermagdy\-etal\-2025\-jawaher, a large\-scale collection of proverbs across many dialects; Kinayatattia\-etal\-2026\-beyond, a benchmark of Egyptian Arabic idioms; and FannOrFlopalghallabi2025fannflopmultigenremultiera, a benchmark for poetry interpretation spanning multiple genres and eras\. We draw our training and evaluation data from these resources \(Section[3\.1](https://arxiv.org/html/2608.18361#S3.SS1)\)\.

Global cultural benchmarks\.BLEnDmyung2025blendbenchmarkllmseverydayspans 16 regions and 13 languages with 52\.6K QA pairs, and CulturalBenchchiu\-etal\-2025\-culturalbenchcovers 45 regions through human–AI red\-teaming, but both emphasize factual cultural knowledge\. ALM\-Benchvayani2025languagesmatterevaluatinglmmsand CVQAromero2024cvqaculturallydiversemultilingualvisualextend coverage to 100 and 31 languages respectively, yet target visual and commonsense knowledge rather than figurative interpretation\.

Arabic cultural benchmarks\.AraDiCEmousi2024aradicebenchmarksdialectalcultural, CAMEL\-Benchghaboura2024camelbenchcomprehensivearabiclmm, and CIDARalyafeai\-etal\-2024\-cidarcover dialectal variation, cultural knowledge, and reasoning, but focus on factual or task\-oriented understanding\. Figurative language appears only marginally elsewhere: ArabCulturesadallah2025commonsensereasoningarabculturecovers 13 countries and 12 topics but allocates five idiom samples per country, and Palmalwajih\-etal\-2025\-palmtreats proverbs as one of 20 topics across 22 countries without in\-depth evaluation\. Across both Arabic and global benchmarks, figurative language remains peripheral, motivating dedicated evaluation frameworks for culturally grounded figurative understanding\.

### 2\.2Improving Figurative Interpretation

Prior work falls into two main directions: task\-specific fine\-tuning and structured prompting\. Supervised fine\-tuning remains dominant:yayavaram\-etal\-2024\-berttrain a BERT\-based model with objectives capturing word cohesion and cross\-lingual translation, yielding gains on idiom detection, andpragmatic\_sarcasmshow that fine\-tuned encoder models outperform decoder\-only LLMs on multilingual sarcasm detection, underscoring the difficulty of capturing implicit, culturally grounded meaning through pretraining alone\. A second line elicits figurative reasoning without parameter updates:prystawski2023psychologicallyinformedchainofthoughtpromptsmetaphoruse psychologically informed chain\-of\-thought prompts for metaphor interpretation, andlee\-etal\-2025\-pragmaticpropose Pragmatic Metacognitive Prompting, which scaffolds reasoning over implied meaning and literal–intended discrepancies\. Beyond these, a smaller body of work injects explicit knowledge or human\-inspired strategies:chakrabarty\-etal\-2022\-rocketmodel contextual inference and compositional reasoning for idiom and simile interpretation in narratives, showing that pretrained representations alone fall well short of human performance\.

### 2\.3Cultural Adaptation and Cross\-Domain Transfer

A parallel literature adapts models to specific cultures, typically by fine\-tuning on culturally grounded datali2024culturellm;pham\-etal\-2025\-cultureinstructor by training separate adapters per cultureadilazuarda\-etal\-2025\-surveys\. For Arabic, the PalmX shared taskalwajih\-etal\-2025\-palmxhas driven a set of cultural fine\-tuning systems, andalmheiri\-etal\-2025\-crossshow that cultural commonsense transfers*across*Arab countries under lightweight alignment\.

In all of these, the target of adaptation is cultural knowledge itself, just as figurative benchmarks evaluate figurative competence on its own terms\. We investigate instead whether supervision in either domain transfers to the other, and additionally whether it transfers across figurative forms, from poetry to both proverbs and idioms\.

## 3Experimental Setup

To examine the connection between culture and figurative language in LLMs, we design a controlled fine\-tuning and evaluation framework that isolates the direction of transfer between the two domains\. We design three complementary experiments to investigate the bidirectional transfer between cultural and figurative language understanding in Arabic LLMs\.

1. 1\.The Impact of Cultural Fine\-tuning:We fine\-tune on Arabic cultural datasets and evaluate the impact on LLM understanding of idioms and proverbs\.
2. 2\.The Impact of Figurative Language Fine\-tuning:We fine\-tune on Arabic poetry and proverbs and evaluate the impact on LLM understanding of idioms and proverbs\.
3. 3\.The Contribution of Language Content \(Baseline\):We fine\-tune on general Arabic data and evaluate on the same benchmarks, to test whether transfer reflects cultural content specifically or Arabic exposure generally\.

DatasetDescriptionSizeCoverageUsageAraDiCE\-Culturemousi2024aradicebenchmarksdialectalculturalMultiple\-choice cultural commonsense reasoning benchmark1806 Arab countriesTestArabCulturesadallah2025commonsensereasoningarabcultureCultural knowledge and practices across the Arab world3,48213 Arab countriesTrainPalmalwajih\-etal\-2025\-palmCultural commonsense and social reasoning in Arabic15\.5k \(train\), 1\.93k \(test\)22 Arab countriesTrainFannOrFlopalghallabi2025fannflopmultigenremultieraPoem–explanation pairs capturing poetic preference and aesthetic judgment across 14 genres, 12 eras6,984Arabic poetryTrainJawahermagdy\-etal\-2025\-jawaherArabic proverb understanding and interpretation800 \(train\), 198 \(test\)20 Arabic varietiesTrain, testKinayatattia\-etal\-2026\-beyondEgyptian Arabic idiom–explanation pairs150Egyptian ArabicTestArabicMMLUkoto\-etal\-2024\-arabicmmluArabic language and grammar MCQs \(control\)980Modern Standard ArabicTrainTable 1:Datasets used for training and evaluation, covering figurative language and cultural knowledge across Arabic varieties and regions\.Culturaldatasets \(AraDiCE\-Culture, ArabCulture, Palm\) andfigurative languagedatasets \(FannOrFlop, Jawaher, Kinayat\) are visually distinguished; ArabicMMLU serves as a control\.### 3\.1Datasets

#### Training and Evaluation Data\.

Table[1](https://arxiv.org/html/2608.18361#S3.T1)summarizes all resources used; all are publicly available\. Figurative supervision comes from the Jawahermagdy\-etal\-2025\-jawahertraining split and FannOrFlopalghallabi2025fannflopmultigenremultiera, cultural supervision from ArabCulturesadallah2025commonsensereasoningarabcultureand the Palmalwajih\-etal\-2025\-palmtraining split\. We evaluate cultural understanding on AraDiCEmousi2024aradicebenchmarksdialectalculturaland figurative understanding on the Jawaher test split and Kinayatattia\-etal\-2026\-beyond\.

#### Design Choices\.

First, because Palm includes proverbs among its topic categories, we remove all proverb\-topic items, so that neither cultural condition carries figurative supervision\. Second, each condition is trained on a 1,000\-sample subset, except Jawaher which only consists of 800 samples for training\. Full\-corpus training for FannOrFlop and ArabCulture is reported as an ablation in Appendix[D](https://arxiv.org/html/2608.18361#A4)\. Third, because every fine\-tuning condition also constitutes exposure to Arabic text, gains on any benchmark could reflect language adaptation rather than the content of the training data\. We therefore add a control condition trained on the Arabic Language \(General\) and Arabic Language \(Grammar\) subsets of ArabicMMLUkoto\-etal\-2024\-arabicmmlu, 980 examples in total\.

#### Data Preparation\.

All training data is cast as open\-ended generation\. For multiple\-choice datasets, the question serves as the input and the correct answer as the target, with incorrect options discarded; for generative datasets, the input expression serves as the input and its explanation or interpretation as the target\. Prompt templates are given in Appendix[A](https://arxiv.org/html/2608.18361#A1)\. For Jawaher evaluation we use the distractors ofattia\-etal\-2026\-beyond; Kinayat and AraDiCE are used as released\. Jawaher and Kinayat have two options per item \(50% chance baseline\) and AraDiCE three \(33\.3%\)\.

### 3\.2Models and Evaluation

We experiment with both multilingual and Arabic\-focused instruction\-tuned models\. All models are fine\-tuned using parameter\-efficient Low\-Rank Adaptation \(LoRA\)hu2021loralowrankadaptationlarge\. Fine\-tuning setup is in Appendix[B](https://arxiv.org/html/2608.18361#A2)\. Our multilingual models are LLaMA 3\.1–8B Instructgrattafiori2024llama3herdmodelsand Qwen 3–8Byang2025qwen3technicalreport\. Our Arabic\-centric models are Fanar\-1\-9B–Instructfanarteam2025fanararabiccentricmultimodalgenerativeand ALLaM–7B–Instructbari2025allam\. All models are evaluated under identical training and inference conditions to ensure comparability\. We evaluate models under the zero\-shot setting, and each experiment is run three times with different random seeds for shuffling the multiple\-choice options to account for sensitivity to answer ordering\. All evaluations are run through thelm\-evaleval\-harnessframework, applied identically to base and fine\-tuned models\. Multiple\-choice items are scored by comparing the log\-likelihood the model assigns to each answer option, with an item counted correct when the gold option receives the highest log\-likelihood\. We report accuracy, averaged across the three runs\.

## 4Results

The one fine\-tuning condition that produces a statistically reliable effect is poetry fine\-tuning evaluated on idiom interpretation: FannOrFlop\-finetuned models average\+2\.33%\+2\.33\\%on Kinayat, the only pooled effect in our study whose 95% confidence interval excludes zero \(\[\+0\.39,\+4\.28\]\[\+0\.39,\+4\.28\],p=0\.021p=0\.021; Table[11](https://arxiv.org/html/2608.18361#A5.T11)\)\. Importantly, this is a within\-figurative transfer effect rather than evidence of culture\-to\-figurative transfer\. This effect is driven primarily by LLaMA\-3\.1\-8B, whose largest single\-seed gain of\+10\.00%\+10\.00\\%is significant under an exact McNemar test \(p=0\.008p=0\.008\); the remaining two seeds gain\+8\.00%\+8\.00\\%and\+2\.67%\+2\.67\\%, neither individually significant\. Jawaher\-finetuned models average\+1\.83%\+1\.83\\%on Kinayat, in the same direction but not distinguishable from zero\.

The most pronounced regressions occur under Palm fine\-tuning, and these are the only negative effects with statistical support\. ALLaM\-7B fine\-tuned on Palm degrades by3\.70%3\.70\\%on Jawaher, with one seed reaching significance \(−5\.56%\-5\.56\\%,p=0\.013p=0\.013\), and Fanar\-1\-9B declines by3\.03%3\.03\\%on Jawaher under the same condition, likewise with one significant seed \(−3\.54%\-3\.54\\%,p=0\.039p=0\.039\)\. Both regressions are consistent in sign across all three seeds\. Other regressions we observe—LLaMA\-3\.1\-8B fine\-tuned on ArabCulture dropping2\.78%2\.78\\%on AraDiCE \(Figure[9](https://arxiv.org/html/2608.18361#A3.F9)\), and Qwen3\-8B fine\-tuned on Jawaher declining2\.36%2\.36\\%on Jawaher itself—fall within the range of seed\-to\-seed variation and should be treated as suggestive rather than established\. Ablations on fine\-tuning data size and hyperparameter choices were also conducted; full results are reported in Appendix[D](https://arxiv.org/html/2608.18361#A4)\.

![Refer to caption](https://arxiv.org/html/2608.18361v1/average_performance_difference_heatmap.png)Figure 2:Average performance difference\.### 4\.1Transfer between Cultural and Figurative Language Fine\-tuning

Table[2](https://arxiv.org/html/2608.18361#S4.T2)and Figure[2](https://arxiv.org/html/2608.18361#S4.F2)summarize the performance differences \(fine\-tuned−\-base\) across all fine\-tuning and evaluation dataset combinations, averaged over four models: ALLaM\-7B, Fanar\-1\-9B, Qwen3\-8B, and LLaMA\-3\.1\-8B \(See Appendix[C](https://arxiv.org/html/2608.18361#A3)for detailed results of individual runs and model heatmaps\)\. Given the size of our evaluation sets \(150–198 items\), we quantify uncertainty for every reported effect using paired bootstrap confidence intervals over test items and exact McNemar tests, with a cluster bootstrap for effects pooled across models\. Full per\-seed intervals and test results appear in Appendix[E](https://arxiv.org/html/2608.18361#A5)\. Unless noted, differences discussed below are not statistically significant atα=0\.05\\alpha=0\.05\.

ModelJawaherKinayatAraDiCEBaseALLaM\-7B\-Instruct0\.89900\.84000\.7537Qwen3\-8B0\.81310\.68000\.5037Fanar\-1\-9B\-Instruct0\.89060\.76670\.6870Llama\-3\.1\-8B\-Instruct0\.68860\.59560\.5204Fine\-tuned onPalm SubsetALLaM\-7B\-Instruct0\.86200\.83330\.7167Qwen3\-8B0\.83000\.71780\.5204Fanar\-1\-9B\-Instruct0\.86030\.74670\.6926Llama\-3\.1\-8B\-Instruct0\.72050\.63330\.5481Fine\-tuned onArabCulturesubsetALLaM\-7B\-Instruct0\.89060\.82670\.7500Qwen3\-8B0\.81480\.71330\.5185Fanar\-1\-9B\-Instruct0\.88050\.75330\.6870Llama\-3\.1\-8B\-Instruct0\.70200\.62000\.4926Fine\-tuned onJawaherALLaM\-7B\-Instruct0\.89900\.85330\.7407Qwen3\-8B0\.78960\.67560\.5130Fanar\-1\-9B\-Instruct0\.89060\.74890\.6907Llama\-3\.1\-8B\-Instruct0\.71210\.67780\.5130Fine\-tuned onFannOrFlopALLaM\-7B\-Instruct0\.88890\.82890\.7352Qwen3\-8B0\.83330\.70000\.5111Fanar\-1\-9B\-Instruct0\.89730\.78220\.6944Llama\-3\.1\-8B\-Instruct0\.71210\.66440\.5241Fine\-tuned onArabicMMLU\(control\)ALLaM\-7B\-Instruct0\.89060\.78970\.7833Qwen3\-8B0\.77950\.63690\.5389Fanar\-1\-9B\-Instruct0\.88380\.74360\.6926Llama\-3\.1\-8B\-Instruct0\.70370\.61640\.5463

Table 2:Average evaluation results across three runs on Jawaher, Kinayat, and AraDiCE datasets\. Results show accuracy scores \(↑\\uparrow\) for base models and models fine\-tuned on different subsets\.#### Effect of Cultural Fine\-tuning on Figurative Language Benchmarks\.

Fine\-tuning on ArabCulture and Palm—both cultural datasets—does not reliably improve performance on the figurative language benchmarks Jawaher and Kinayat\. ArabCulture produces near\-zero change on Jawaher \(−0\.08%\-0\.08\\%,p=0\.92p=0\.92\) and a small positive point estimate on Kinayat \(\+0\.78%\+0\.78\\%,p=0\.44p=0\.44\), while Palm yields\+1\.22%\+1\.22\\%on Kinayat \(p=0\.29p=0\.29\) and−0\.46%\-0\.46\\%on Jawaher \(p=0\.58p=0\.58\)\. All four confidence intervals span zero\. Cultural knowledge alone therefore appears to provide no measurable transfer to tasks requiring understanding of idioms and proverbs\.

The pooled estimates, however, average over sharply divergent per\-model behavior \(Figure[12](https://arxiv.org/html/2608.18361#A3.F12)\)\. Under Palm fine\-tuning, ALLaM\-7B and Fanar\-1\-9B both regress across all three evaluation datasets, and as noted above their Jawaher regressions are the only negative effects in our study with statistical support\. Qwen3\-8B and LLaMA\-3\.1\-8B move in the opposite direction, with LLaMA\-3\.1\-8B posting positive point estimates of\+3\.20%\+3\.20\\%,\+3\.78%\+3\.78\\%, and\+2\.78%\+2\.78\\%on Jawaher, Kinayat, and AraDiCE respectively; none of the underlying per\-seed comparisons is individually significant, so we read this as a consistent sign pattern rather than a demonstrated gain\. Qwen3\-8B records one significant seed under Palm fine\-tuning \(\+4\.67%\+4\.67\\%on Kinayat,p=0\.016p=0\.016\)\. That the two Arabic\-centric models and the two multilingual models respond in opposite directions, with significance on both sides, is the clearest evidence in our results that the effect of cultural fine\-tuning is model\-dependent\.

#### Effect of Figurative Language Fine\-tuning on AraDiCE\.

Fine\-tuning on Jawaher \(proverbs\) and FannOrFlop \(poetry\) produces no detectable improvements on AraDiCE\. The pooled effects are−0\.19%\-0\.19\\%\(p=0\.81p=0\.81\) and\+0\.00%\+0\.00\\%\(p=0\.99p=0\.99\) respectively, both tightly centered on zero\. At the model level \(Figures[10](https://arxiv.org/html/2608.18361#A3.F10)and[11](https://arxiv.org/html/2608.18361#A3.F11)\), Qwen3\-8B and LLaMA\-3\.1\-8B show small positive point estimates while ALLaM\-7B shows negative ones; the ALLaM\-7B average of−1\.85%\-1\.85\\%under FannOrFlop rests on a single significant seed \(−5\.00%\-5\.00\\%,p=0\.022p=0\.022\) against seeds of\+1\.11%\+1\.11\\%and−1\.67%\-1\.67\\%, so the size of this regression should be read with caution even though its direction is supported\. Overall, fine\-tuning on proverb and poetry datasets does not improve performance on more general Arabic cultural commonsense tasks\.

#### Control Fine\-tuning on ArabicMMLU\.

The ArabicMMLU control does not reproduce the pattern observed under content\-matched fine\-tuning\. Averaged across models, it yields−2\.39%\-2\.39\\%on Kinayat and−0\.84%\-0\.84\\%on Jawaher \(Figure[13](https://arxiv.org/html/2608.18361#A3.F13)\), with three of four models regressing on each\. On Kinayat the control thus moves in the opposite direction from poetry fine\-tuning, a divergence of nearly five percentage points, indicating that the reliable gain reported above is not explained by adaptation to the fine\-tuning format\. ALLaM\-7B records the single largest regression in our study under this condition \(−5\.03%\-5\.03\\%on Kinayat\), consistent with the broader pattern that the Arabic\-centric models respond poorly to fine\-tuning on data whose content they have likely already absorbed\.

The control does, however, produce the only unanimous effect we observe: all four models improve on AraDiCE, averaging\+2\.41%\+2\.41\\%\. AraDiCE does not move under every other condition, including both cultural sets, so the one condition that moves it targets general language ability rather than cultural knowledge\. ArabicMMLU is not culturally neutral: as natively produced rather than translated Arabic, it carries cultural content incidentally, so our contrast is between curated and incidental cultural exposure rather than cultural and non\-cultural data\. With that qualification, we hypothesize that AraDiCE performance is limited less by cultural knowledge than by general Arabic competence, inverting the assumption that motivates cultural fine\-tuning for cultural commonsense benchmarks\.

### 4\.2General Observations

Several patterns emerge across fine\-tuning conditions\. First, the Kinayat benchmark is the most responsive to fine\-tuning and is the only benchmark on which any fine\-tuning condition produces a reliable aggregate gain\. It also shows the widest spread of individual outcomes, ranging from LLaMA\-3\.1\-8B’s\+8\.22%\+8\.22\\%under Jawaher fine\-tuning to small regressions for ALLaM\-7B and Fanar\-1\-9B under the same conditions\.

Second, the Arabic\-centric models benefit less from fine\-tuning\. ALLaM\-7B and Fanar\-1\-9B start from substantially higher baselines and record negative deltas in the majority of fine\-tuning and evaluation combinations, including the two significant regressions on Jawaher under Palm fine\-tuning discussed above\. Their consistently higher baselines, combined with the direction of these effects, suggest that these models already capture much of the cultural and figurative knowledge present in the fine\-tuning corpora\. The multilingual models Qwen3\-8B and LLaMA\-3\.1\-8B show positive point estimates more often, and account for the two significant positive cells in our results, though most of their individual gains are not distinguishable from zero\.

Third, and most consistently, the average performance differences across all fine\-tuning datasets \(Figure[2](https://arxiv.org/html/2608.18361#S4.F2)\) are uniformly small in magnitude \(ranging from−0\.46%\-0\.46\\%to\+2\.33%\+2\.33\\%\), and mostly statistically indistinguishable from zero\. Cross\-task transfer between cultural knowledge and figurative language understanding in Arabic is therefore limited, model\-dependent, and—with the single exception of poetry fine\-tuning on idiom interpretation—not reliably positive\. The one reliable effect we observe is not reproduced by our ArabicMMLU control, which regresses on Kinayat where poetry fine\-tuning gains, indicating that the improvement stems from the figurative content of the poetry data rather than from adaptation to the fine\-tuning format\.

## 5Error Analysis

### 5\.1Figurative Language and Cultural Understanding

We analyze the impact of figurative language fine\-tuning \(Jawaher proverbs and FannOrFlop poetry\) on culturally grounded QA across models, countries, and topic categories\. Improvements \(185\) and regressions \(189\) are nearly balanced, yielding a slight net degradation \(−4\-4\) and negligible average change \(−0\.11%\-0\.11\\%\), consistent with the null pooled effects reported in Section[4](https://arxiv.org/html/2608.18361#S4)\. The analysis below therefore characterizes the item\-level churn underlying that null result rather than a net gain or loss; because counts aggregate over models and seeds, and several categories contain few items, the patterns are descriptive and we do not test them individually\.

#### Model\-Level Trends

Table[3](https://arxiv.org/html/2608.18361#S5.T3)summarizes performance changes across models and datasets\. Fanar and Qwen post small positive point estimates under both fine\-tuning sets, while ALLaM shows the largest degradation, particularly under poetry \(−1\.87%\-1\.87\\%\); this last figure rests on a single significant seed \(−5\.00%\-5\.00\\%,p=0\.022p=0\.022\) against seeds of\+1\.11%\+1\.11\\%and−1\.67%\-1\.67\\%, so its direction is better supported than its magnitude\. LLaMA is mixed across conditions\.

ModelTrain SetAvgΔ\\Delta%ImprRegrNetALLaMJawaher\-1\.302128\-7FannOrFlop\-1\.871525\-10FanarJawaher\+0\.372220\+2FannOrFlop\+0\.732319\+4LlamaJawaher\-0\.773842\-4FannOrFlop\+0\.374341\+2QwenJawaher\+0\.87105\+5FannOrFlop\+0\.70139\+4

Table 3:Aggregate performance changes on culture by model and fine\-tuning dataset\.
#### Geographic Variation

Performance changes vary across countries \(Table[4](https://arxiv.org/html/2608.18361#S5.T4)\)\. These counts reflect aggregated improvement and regression instances across all models and fine\-tuning datasets, such that a single question may contribute multiple counts if it consistently improved or regressed across configurations\. Jordan accumulates the strongest net improvement \(\+32\+32\), while most other countries experience net regressions, with the largest degradations observed for Syria \(−18\-18\) and Egypt \(−12\-12\)\. Since the aggregate effect is null, this dispersion indicates that fine\-tuning redistributes performance across cultural contexts rather than improving it uniformly\.

ImprRegrNetBy CountryJordan5018\+32\+32Lebanon3029\+1\+1Qatar2628−2\-2Palestine3035−5\-5Egypt2234−12\-12Syria2745−18\-18By TopicFood/Cuisine183\+15\+15Traditional Games185\+13\+13Other6665\+1\+1Holidays/Occasions4046−6\-6History/Civilization311−8\-8Religion210−8\-8Traditional Clothing1731−14\-14Table 4:Performance changes on cultural evaluation by country and topic category, aggregated across all models and fine\-tuning configurations\.
#### Topic\-Level Analysis

We further analyze performance across cultural topics \(Table[4](https://arxiv.org/html/2608.18361#S5.T4)\)\. The strongest net gains appear in Food/Cuisine \(\+15\+15\) and Traditional Games \(\+13\+13\), while the largest net losses occur in Traditional Clothing \(−14\-14\), History/Civilization \(−8\-8\), and Religion \(−8\-8\); the latter two rest on few items and should be read with caution\. At the question level, the most improved items cluster around popular culture: traditional games, sweets, marriage customs, and celebrations such as Ramadan\. Regressions concentrate on historical and political content—battles, the Islamic conquest of Damascus, independence dates—as well as traditional clothing and infrastructural knowledge \(Table[13](https://arxiv.org/html/2608.18361#A6.T13)\)\. This suggests that where figurative fine\-tuning does shift cultural predictions, it tends to favor culturally embedded, experiential knowledge over historically grounded or domain\-specific factual knowledge—a topical sensitivity that goes beyond model or dialect differences\.

### 5\.2Culture and Poetry as Signals for Figurative Language

The positive effect of poetry fine\-tuning on idiom interpretation is noteworthy\. While one might intuitively sense a connection—poetry, idioms, and proverbs all operate in non\-literal registers—this relationship is rarely foregrounded in NLP research, which tends to treat these as distinct figurative categories with separate datasets, tasks, and benchmarks\. Poetry draws on imagery, meter, and rhetorical devices, whereas idioms and proverbs rely on fixed, conventionalized meanings; the overlap is conceptual rather than structural\. The results here suggest, however, that fine\-tuning on poetry develops a broader figurative competence—a sensitivity to non\-literal meaning that is not tied to any one figurative type but transfers across them\. We note that the same effect on proverbs points in the same direction but does not reach significance \(\+1\.01%\+1\.01\\%,p=0\.23p=0\.23\), so the evidence for such competence rests on the poetry\-to\-idiom pairing rather than on transfer to figurative language generally\.

#### Overall Fine\-Tuning Effect

Table[5](https://arxiv.org/html/2608.18361#S5.T5)compares fine\-tuning outcomes across both figurative tasks\. Cultural fine\-tuning shifts idiom accuracy by \+1\.00% and proverb accuracy by \-0\.27%, neither distinguishable from zero, whereas poetry fine\-tuning is positive on both and reliably so on idioms \(\+2\.33% vs\. \+1\.01%\)\. Effects are smaller on proverbs under both conditions, which may reflect the greater dialectal diversity of the proverb set\. Per\-model breakdowns are given in Tables[19](https://arxiv.org/html/2608.18361#A6.T19)and[20](https://arxiv.org/html/2608.18361#A6.T20)\(Appendix[F](https://arxiv.org/html/2608.18361#A6)\)\.

TaskImprRegrNetAvgΔ%\\Delta\\%Cultural Fine\-tuningIdioms195159\+36\+36\+1\.00\+1\.00Proverbs149162−13\-13−0\.27\-0\.27Poetry Fine\-tuningIdioms10058\+42\+42\+2\.33\+2\.33Proverbs9773\+24\+24\+1\.01\+1\.01Table 5:Overall fine\-tuning effect on idiom and proverb interpretation, aggregated across all models\. Cultural fine\-tuning uses ArabCulture and Palm; Poetry fine\-tuning uses FannOrFlop\.
#### Model\-Level Trends

Table[6](https://arxiv.org/html/2608.18361#S5.T6)aggregates results by model and evaluation dataset for both tasks\. The pattern is consistent across fine\-tuning conditions: ALLaM and Fanar regress while LLaMA and Qwen improve, with the magnitude of harm greater on proverbs\. ALLaM on Palm proverbs loses−3\.70%\-3\.70\\%\(net−22\-22\) against only−0\.67%\-0\.67\\%\(net−3\-3\) for idioms, and its worst single run is a−5\.56%\-5\.56\\%drop on proverbs \(Palm, seed 21,p=0\.013p=0\.013\)\. This amplified degradation for the stronger Arabic\-centric models is consistent with their having already internalized proverb knowledge during pre\-training on large Arabic web corpora, so that fine\-tuning on a curated, narrower corpus introduces noise relative to that broader prior\. LLaMA and Qwen gain on both tasks but less on proverbs; Qwen accumulates only 4 regressions on idioms against 18 on proverbs, though its net remains positive\. Poetry fine\-tuning broadly mirrors this pattern, with one exception: Fanar now benefits on both idioms \(\+1\.56%\+1\.56\\%, net\+7\+7\) and proverbs \(\+0\.67%\+0\.67\\%, net\+4\+4\), whereas cultural fine\-tuning yielded only losses\. Poetry fine\-tuning produces the largest single\-seed improvement in LLaMA’s idiom performance \(\+10\.00%\+10\.00\\%, seed 21,p=0\.008p=0\.008\), with a net improvement of\+31\+31predictions across the three seeds\.

IdiomsProverbsModelTrain SetAvgΔ\\Delta%NetAvgΔ\\Delta%NetALLaMArabCulture−1\.33\-1\.33−6\-6−0\.84\-0\.84−5\-5Palm−0\.67\-0\.67−3\-3−3\.70\-3\.70−22\-22Poetry−1\.11\-1\.11−5\-5−1\.01\-1\.01−6\-6FanarArabCulture−1\.33\-1\.33−6\-6−1\.01\-1\.01−6\-6Palm−2\.00\-2\.00−9\-9−3\.03\-3\.03−18\-18Poetry\+1\.56\+1\.56\+7\+7\+0\.67\+0\.67\+4\+4LLaMAArabCulture\+2\.44\+2\.44\+11\+11\+1\.35\+1\.35\+8\+8Palm\+3\.78\+3\.78\+17\+17\+3\.20\+3\.20\+19\+19Poetry\+6\.89\+6\.89\+31\+31\+2\.36\+2\.36\+14\+14QwenArabCulture\+3\.33\+3\.33\+15\+15\+0\.17\+0\.17\+1\+1Palm\+3\.78\+3\.78\+17\+17\+1\.68\+1\.68\+10\+10Poetry\+2\.02\+2\.02\+9\+9\+2\.00\+2\.00\+12\+12

Table 6:Aggregate fine\-tuning results by model and fine\-tuning dataset for both idiom and proverb interpretation\.Culturalfine\-tuning datasets \(ArabCulture, Palm\) andPoetryfine\-tuning \(FannOrFlop\) are visually distinguished\. Net refers to the total net improved predictions across all three seeds\.

## 6Conclusion

This work examined whether fine\-tuning on cultural and figurative language can induce transfer across the two domains\. Across four models and six datasets spanning diverse Arabic dialects and regions, we find that this transfer is limited, inconsistent, and highly model\-dependent\. When cultural datasets serve as the training signal, pooled gains on figurative benchmarks are small and unreliable \(\+0\.78%\+0\.78\\%for ArabCulture and\+1\.22%\+1\.22\\%for Palm on Kinayat\), and the underlying per\-model effects are polarized in sign; Palm fine\-tuning produces the only statistically supported regressions in our study, lowering the accuracy of both Arabic\-centric models on Jawaher\. In the opposite direction, figurative fine\-tuning produces no detectable change on cultural benchmarks\.

The one reliable effect runs within the figurative domain rather than across the cultural–figurative divide: poetry fine\-tuning on FannOrFlop improves idiom interpretation \(\+2\.33%\+2\.33\\%,p=0\.021p=0\.021\), and our ArabicMMLU control regresses on the same benchmark, indicating that this gain stems from figurative content rather than additional language exposure\. A consistent divide also emerges between model families: the Arabic\-centric models start from higher baselines and account for both supported regressions, suggesting they have already internalized much of the relevant knowledge during pre\-training, whereas the multilingual models show greater headroom\. Error analysis further reveals that fine\-tuning tends to favor culturally embedded, experiential knowledge \(such as food, customs, and celebrations\) over historically grounded or domain\-specific factual knowledge\. Given our small evaluation set sizes, these findings are best read as a constraint on what current Arabic resources can support rather than evidence that no such relationship exists\.

## Limitations

Our work is subject to several limitations\. First, our fine\-tuning experiments are limited to LoRA due to constraints in computational resources\. While LoRA has proven effective in many settings, full fine\-tuning or alternative adaptation methods may yield different results, and we were unable to explore this space\. Second, hyperparameter search was limited in scope; factors such as learning rate, rank, and training duration were not exhaustively explored, and it is possible that more carefully tuned configurations would produce stronger or more consistent results\. Third, our experiments are restricted to models in the 7–9 billion parameter range\. Larger models may exhibit different sensitivity to cultural and poetic fine\-tuning data, and our findings may not generalize to models at greater scales\. We leave broader exploration of model sizes, fine\-tuning strategies, and hyperparameter configurations to future work\.

A limitation of our cross\-domain analysis is that the two transfer directions are not directly comparable\. They involve different source datasets, different target benchmarks, and different task formulations, so the observed difference in magnitude between directions cannot be cleanly attributed to a directional asymmetry in the underlying relationship — it may instead reflect variation in dataset difficulty, domain specificity, or evaluation sensitivity\. We therefore avoid drawing strong conclusions about which direction of transfer is inherently stronger\. We also find that neither domain reliably generalizes to the other across all models and conditions, which further limits the generality of any cross\-domain claims we can make from these experiments\.

## References

## Appendix APrompt Templates

Figure[3](https://arxiv.org/html/2608.18361#A1.F3)shows the prompt used for the MCQ understanding task, which presents the model with a question and a set of candidate answers and elicits a single\-letter selection\. Figure[4](https://arxiv.org/html/2608.18361#A1.F4)shows the corresponding template for AraDiCE, which follows the same structure with three candidate options\. Figures[5](https://arxiv.org/html/2608.18361#A1.F5),[6](https://arxiv.org/html/2608.18361#A1.F6), and[7](https://arxiv.org/html/2608.18361#A1.F7)present the prompts used for fine\-tuning on the FannOrFlop poetry explanation, Jawaher proverb explanation, and ArabCulture open\-ended completion tasks, respectively\. Each of these prompts pairs a short task\-specific preamble with the input field corresponding to that task \(a poem, a proverb, or a cultural prompt\) and instructs the model to produce a free\-form explanation or completion\. Finally, Figure[8](https://arxiv.org/html/2608.18361#A1.F8)shows the prompt used for fine\-tuning on the Palm instruction\-following task; in contrast to the other tasks, no task\-specific preamble is used, and the question field alone carries the full context required for the model to generate a response\.

You are tasked with selecting the correct explanation for the following proverb\.Choose the correct explanation from the options provided\. Only output the letter corresponding to the correct answer and nothing else\.
Proverb:\[PROVERB\]
Options:A\.\[OPTION 1\]
B\.\[OPTION 2\]
Answer:Figure 3:Prompt used for the MCQ understanding task\.You are tasked with selecting the correct answer to the following question\. Choose the correct answer from the options provided\. Only output the letter corresponding to the correct answer and nothing else\.
Question:\[QUESTION\]
Options:A\.\[OPTION 1\]
B\.\[OPTION 2\]
C\.\[OPTION 3\]
Answer:Figure 4:Prompt template used for AraDiCE\-Culture evaluation\.Explain the following poetry verse\.
Verse:\[VERSE\]
Explanation:\[EXPLANATION\]Figure 5:Prompt used for fine\-tuning on FannOrFlop poetry explanation\.Explain the following Arabic proverb\.
Proverb:\[PROVERB\]
Explanation:\[AR\_EXPLANATION\]Figure 6:Prompt used for fine\-tuning on the Jawaher proverb explanation task\.Based on the following cultural context, provide an appropriate completion\.
Scenario:\[SCENARIO\]
Answer:\[EXPECTED\_ANSWER\]Figure 7:Prompt used for fine\-tuning on ArabCulture open\-ended completion\.\[INSTRUCTION\]
\[OUTPUT\]Figure 8:Prompt used for fine\-tuning on the Palm instruction\-following task\. No task\-specific preamble is used; the instruction field carries full context\.
## Appendix BFine\-tuning Setup

We apply LoRA to the attention and feed\-forward projection layers with rankr=4r=4, scaling factorα=8\\alpha=8, and dropout0\.10\.1\. Fine\-tuning is performed for33epochs using a learning rate of5​e−55e\-5and a batch size of11\. A validation set is used to monitor convergence\. This setup enables efficient adaptation while keeping the base model parameters frozen\. All models were fine\-tuned on a single NVIDIA RTX 5000 Ada Generation GPU\. Additional hyperparameters were also explored\.

## Appendix CAdditional Results

Table[7](https://arxiv.org/html/2608.18361#A3.T7)presents evaluation results across three runs with different random seeds on the Jawaher, Kinayat, and AraDiCE datasets, reporting accuracy scores \(↑\) for both the base models and the models fine\-tuned on different subsets; reporting multiple runs allows us to assess the stability of the observed trends under random initialization\. Figures[9](https://arxiv.org/html/2608.18361#A3.F9),[10](https://arxiv.org/html/2608.18361#A3.F10),[11](https://arxiv.org/html/2608.18361#A3.F11), and[12](https://arxiv.org/html/2608.18361#A3.F12)visualize, as heatmaps, the per\-dataset performance differences \(diff = fine\-tuned accuracy−\-base accuracy\) for models fine\-tuned on the ArabCulture, FannOrFlop, Jawaher, and Palm datasets, respectively\. Each heatmap reports the change in accuracy on the held\-out evaluation datasets\.

Run 1 \(seed=0\)Run 2 \(seed=42\)Run 3 \(seed=21\)ModelJawaherKinayatAraDiCEJawaherKinayatAraDiCEJawaherKinayatAraDiCEBaseALLaM\-7B\-Instruct0\.88890\.86000\.76670\.90910\.82670\.76670\.89900\.83330\.7278Qwen3\-8B0\.80300\.67330\.52780\.83330\.68670\.50560\.80300\.68000\.4778Fanar\-1\-9B\-Instruct0\.87880\.77330\.72220\.90400\.72670\.70000\.88890\.80000\.6389Llama\-3\.1\-8B\-Instruct0\.70200\.59330\.50560\.69190\.59330\.51670\.67170\.60000\.5389Fine\-tuned onPalm SubsetALLaM\-7B\-Instruct0\.86870\.85330\.73330\.87370\.81330\.71110\.84340\.83330\.7056Qwen3\-8B0\.82320\.70000\.55000\.85350\.73330\.51110\.81310\.72000\.5000Fanar\-1\-9B\-Instruct0\.85860\.74670\.72780\.86870\.71330\.70000\.85350\.78000\.6500Llama\-3\.1\-8B\-Instruct0\.74750\.62670\.50000\.70200\.62670\.55000\.71210\.64670\.5944Fine\-tuned onArabCulturesubsetALLaM\-7B\-Instruct0\.88890\.84670\.77780\.90910\.81330\.73890\.87370\.82000\.7333Qwen3\-8B0\.81310\.70670\.57780\.83330\.72000\.50000\.79800\.71330\.4778Fanar\-1\-9B\-Instruct0\.88380\.75330\.69440\.87880\.72000\.72220\.87880\.78670\.6444Llama\-3\.1\-8B\-Instruct0\.71210\.62000\.43330\.70710\.59330\.50000\.68690\.64670\.5444Fine\-tuned onJawaherALLaM\-7B\-Instruct0\.89900\.87330\.77780\.91920\.83330\.72780\.87880\.85330\.7167Qwen3\-8B0\.77270\.68000\.54440\.81820\.68000\.50000\.77780\.66670\.4944Fanar\-1\-9B\-Instruct0\.88890\.74670\.72780\.89900\.72000\.70000\.88380\.78000\.6444Llama\-3\.1\-8B\-Instruct0\.73230\.67330\.50000\.71720\.67330\.50560\.68690\.68670\.5333Fine\-tuned onFannOrFlopALLaM\-7B\-Instruct0\.88380\.86670\.77780\.90400\.81330\.71670\.87880\.80670\.7111Qwen3\-8B0\.81820\.70000\.54440\.84850\.70670\.50000\.83330\.69330\.4889Fanar\-1\-9B\-Instruct0\.89390\.78670\.72220\.89390\.75330\.72220\.90400\.80670\.6389Llama\-3\.1\-8B\-Instruct0\.73740\.67330\.52780\.70200\.62000\.50560\.69700\.70000\.5389Fine\-tuned onArabicMMLU\(control\)ALLaM\-7B\-Instruct0\.89900\.80310\.80000\.89900\.78460\.77780\.87370\.78150\.7722Qwen3\-8B0\.76770\.63690\.56110\.79800\.60310\.54440\.77270\.67080\.5111Fanar\-1\-9B\-Instruct0\.88380\.73230\.71670\.89390\.74150\.72220\.87370\.75690\.6389Llama\-3\.1\-8B\-Instruct0\.72220\.60000\.50560\.71720\.62770\.55000\.67170\.62150\.5833Table 7:Zero\-shot evaluation results across three runs with different random seeds on Jawaher, Kinayat, and AraDiCE datasets\. Results show accuracy scores \(↑\\uparrow\) for base models and models fine\-tuned on different subsets\.![Refer to caption](https://arxiv.org/html/2608.18361v1/heatmap_ArabCulture.png)Figure 9:Performance difference on different datasets for models fine\-tuned on the ArabCulture dataset \(diff = fine\-tuned accuracy \- base accuracy\)\.![Refer to caption](https://arxiv.org/html/2608.18361v1/heatmap_FannOrFlop.png)Figure 10:Performance difference on different datasets for models fine\-tuned on the FannOrFlop dataset \(diff = fine\-tuned accuracy \- base accuracy\)\.![Refer to caption](https://arxiv.org/html/2608.18361v1/heatmap_Jawaher.png)Figure 11:Performance difference on different datasets for models fine\-tuned on the Jawaher dataset \(diff = fine\-tuned accuracy \- base accuracy\)\.![Refer to caption](https://arxiv.org/html/2608.18361v1/heatmap_PALM.png)Figure 12:Performance difference on different datasets for models fine\-tuned on the Palm dataset \(diff = fine\-tuned accuracy \- base accuracy\)\.![Refer to caption](https://arxiv.org/html/2608.18361v1/arabicmmlu_performance_difference_heatmap.png)Figure 13:Performance difference on different datasets for models fine\-tuned on the ArabicMMLU baseline dataset \(diff = fine\-tuned accuracy \- base accuracy\)\.
## Appendix DAblations

### D\.1Increasing the size of the fine\-tuning data

Table[8](https://arxiv.org/html/2608.18361#A4.T8)displays the results of fine\-tuning on the full FannOrFlop \(6,984 samples\) and ArabCulture \(3,482 samples\) datasets, as opposed to the default subset of 1,000 samples\. When comparing models fine\-tuned on the full dataset versus a subset, the results vary across models and evaluation sets\. For models fine\-tuned on FannOrFlop \(Figure[14](https://arxiv.org/html/2608.18361#A4.F14)\), ALLaM and Qwen consistently benefited from full\-dataset training, posting positive differences across all three evaluation benchmarks \(Jawaher, Kinayat, and AraDiCE\)\. In contrast, Fanar suffered clear degradation, with losses across all benchmarks, most prominently on Jawaher \(−3\.37%\-3\.37\\%\), while LLaMA showed the largest drop on Kinayat \(−8\.44%\-8\.44\\%\)\. For models fine\-tuned on ArabCulture \(Figure[15](https://arxiv.org/html/2608.18361#A4.F15)\), the picture is more mixed: ALLaM again benefited modestly across all benchmarks, and Fanar showed a marginal gain on AraDiCE \(\+1\.48%\+1\.48\\%\)\. However, Qwen and LLaMA both declined in performance, with LLaMA showing particularly sharp drops on Jawaher \(−3\.20%\-3\.20\\%\) and Kinayat \(−4\.22%\-4\.22\\%\)\. Overall, ALLaM is the most consistent beneficiary of full\-dataset fine\-tuning regardless of domain, while for other models, more data does not uniformly translate to better performance\.

![Refer to caption](https://arxiv.org/html/2608.18361v1/heatmap_FannOrFlop_full.png)Figure 14:Average performance difference between models fine\-tuned on the full FannOrFlop dataset vs\. FannOrFlop subset\.![Refer to caption](https://arxiv.org/html/2608.18361v1/heatmap_ArabCulture_full.png)Figure 15:Average performance difference between models fine\-tuned on the full ArabCulture dataset vs\. ArabCulture subset\.Run 1 \(seed=0\)Run 2 \(seed=42\)Run 3 \(seed=21\)ModelJawaherKinayatAraDiCEJawaherKinayatAraDiCEJawaherKinayatAraDiCEFine\-tuned onFannOrFlopALLaM\-7B\-Instruct0\.89900\.86000\.78890\.93430\.83330\.71670\.89900\.84670\.7222Qwen3\-8B0\.83330\.68670\.55560\.85860\.73330\.49440\.85860\.72000\.5278Fanar\-1\-9B\-Instruct0\.86870\.75330\.71670\.86360\.72670\.68890\.85860\.78670\.6278Llama\-3\.1\-8B\-Instruct0\.74240\.57330\.51670\.69190\.54670\.48890\.69700\.62000\.5556Fine\-tuned onArabCultureALLaM\-7B\-Instruct0\.89900\.84670\.77220\.90400\.83330\.74440\.88380\.83330\.7389Qwen3\-8B0\.78790\.68000\.55000\.82830\.68000\.51110\.80300\.66670\.4889Fanar\-1\-9B\-Instruct0\.87370\.76670\.70560\.89390\.72000\.73890\.88380\.76670\.6611Llama\-3\.1\-8B\-Instruct0\.69190\.58670\.47220\.65150\.53330\.52220\.66670\.61330\.5222Table 8:Zero\-shot evaluation results across three runs with different random seeds on Jawaher, Kinayat, and AraDiCE datasets\. Results show accuracy scores \(↑\\uparrow\) for models finetuned on FannOrFlop and ArabCulture full datasets\.ConfigurationSubsetSeedJawaherKinayatAraDiCEDefaultPalm00\.86870\.85330\.7333420\.87370\.81330\.7111210\.84340\.83330\.7056FannOrFlop00\.88380\.86670\.7778420\.90400\.81330\.7167210\.87880\.80670\.7111r=16,α\\alpha=32,lr=1e\-5Palm00\.87370\.84670\.7556420\.88380\.80000\.7278210\.86360\.82000\.7278r=16,α\\alpha=32,lr=5e\-4Palm00\.81310\.82000\.7389420\.83330\.82670\.7278210\.80810\.82000\.7611FannOrFlop00\.88380\.79330\.7111420\.88380\.74670\.6889210\.86870\.78670\.6611Table 9:Zero\-shot evaluation results for ALLaM\-7B\-Instruct on Jawaher, Kinayat, and AraDiCE datasets\. Results show accuracy scores \(↑\\uparrow\) comparing the default LoRA configuration against two ablation settings\.ConfigurationDatasetJawaherΔ\\DeltaJawaherKinayatΔ\\DeltaKinayatAraDiCEΔ\\DeltaAraDiCEr=4,α\\alpha=8, lr=5e\-5Palm Subset0\.8620—0\.8333—0\.7167—FannOrFlop0\.8889—0\.8289—0\.7352—r=16,α\\alpha=32, lr=1e\-5Palm Subset0\.8737\+0\.01180\.8222−\-0\.01110\.7370\+0\.0204r=16,α\\alpha=32, lr=5e\-4Palm Subset0\.8182−\-0\.04380\.8222−\-0\.01110\.7426\+0\.0259FannOrFlop0\.8788−\-0\.01010\.7756−\-0\.05330\.6870−\-0\.0481Table 10:Average zero\-shot evaluation results for ALLaM\-7B\-Instruct across three runs on Jawaher, Kinayat, and AraDiCE datasets\.Δ\\Deltavalues indicate the difference relative to the default configuration for the corresponding finetuning dataset\. Results show accuracy scores \(↑\\uparrow\)\.
### D\.2Changing LoRA hyperparameters

Table[9](https://arxiv.org/html/2608.18361#A4.T9)shows the results of fine\-tuning ALLaM with different LoRA rank, alpha and learning rate\. Compared to the default configuration, the alternative LoRA settings largely failed to improve performance\. The only notable gains were observed with the configuration using rankr=16r=16,α=32\\alpha=32, and lr=1e\-5 fine\-tuned on Palm Subset, which achieved a 1\.18% improvement on Jawaher and a 2\.04% improvement on AraDiCE\. The configuration with rankr=16r=16,α=32\\alpha=32, and lr=5e\-4 yielded a marginal 2\.59% gain on AraDiCE alone, as shown in Table[10](https://arxiv.org/html/2608.18361#A4.T10)\. Crucially, despite these partial gains over the default fine\-tuned configuration, all alternative configurations still fell short of the base ALLaM model’s performance, indicating that the gains from hyperparameter tuning remain limited in comparison to the pretrained model’s capabilities\.

## Appendix EConfidence Intervals and Significance Testing\.

Because our evaluation sets are small \(198 Jawaher, 150 Kinayat, 180 AraDiCE\), point estimates of accuracy differences are unstable, and we therefore quantify uncertainty for every reported effect\. For each \(model, training set, benchmark, seed\) condition we compute a 95% confidence interval on the accuracy difference by paired bootstrap over test items\(koehn\-2004\-statistical\), drawing 20,000 resamples of item indices and applying each resampled index set jointly to the base and fine\-tuned predictions so that the pairing between conditions is preserved\. Because the precision of a paired comparison is governed by the number of items whose prediction changes rather than by the size of the evaluation set, we additionally report exact McNemar testsMcNemar1947over the discordant predictions\. Where fewer than ten predictions differ, the empirical difference distribution is too sparse for the percentile bootstrap to be reliable, and we treat the exact test as authoritative\. To aggregate across models we use a cluster bootstrap in which items rather than predictions are the resampling unit, so that all model\-by\-seed measurements for a resampled item are carried together; this remains valid whatever the degree of dependence between measurements of the same item, and so does not require us to assume that repeated measurements are independent\.

Tables[19](https://arxiv.org/html/2608.18361#A6.T19),[20](https://arxiv.org/html/2608.18361#A6.T20), and[21](https://arxiv.org/html/2608.18361#A6.T21)report these per\-seed intervals alongside the improvement and regression counts for Kinayat idioms, Jawaher proverbs, and AraDiCE respectively\. The counts make the sparsity concern concrete: across most conditions fewer than twenty predictions change in either direction, and in several Fanar and Qwen seeds fewer than ten do, which is precisely the case in which we defer to the exact test\. The intervals are correspondingly wide and, with few exceptions, straddle zero\.

Table[11](https://arxiv.org/html/2608.18361#A5.T11)pools these effects across all four models under the cluster bootstrap described above\. Only poetry fine\-tuning on Kinayat idioms attains significance \(\+2\.33\+2\.33,p=0\.021p=0\.021\); every other pooled effect is within noise of zero, including all three AraDiCE conditions, where the pooled estimates are essentially null\. The intraclass correlation coefficients are small throughout \(all\|ICC\|<0\.08\|\\text\{ICC\}\|<0\.08\), indicating that item\-level agreement between models is weak and that the clustering correction has only a modest effect on the width of the intervals\.

Table[12](https://arxiv.org/html/2608.18361#A5.T12)collects the five individual runs reachingp<0\.05p<0\.05out of the 72 per\-seed tests conducted\. This is close to the number expected under the global null\. We therefore treat the per\-cell results as descriptive and base our substantive claims on the pooled estimates in Table[11](https://arxiv.org/html/2608.18361#A5.T11)\.

TaskFinetuned onΔ\\Delta95% CIppICCSig\.AraDiCEJawaher−0\.19\-0\.19\[−1\.76,\+1\.39\]\[\-1\.76,\+1\.39\]0\.81010\.018\\phantom\{\-\}0\.018AraDiCEPoetry\+0\.00\+0\.00\[−1\.48,\+1\.48\]\[\-1\.48,\+1\.48\]0\.9940−0\.006\-0\.006KinayatArabCulture\+0\.78\+0\.78\[−1\.22,\+2\.72\]\[\-1\.22,\+2\.72\]0\.4353−0\.011\-0\.011KinayatPalm\+1\.22\+1\.22\[−1\.00,\+3\.44\]\[\-1\.00,\+3\.44\]0\.28810\.027\\phantom\{\-\}0\.027KinayatPoetry\+2\.33\+2\.33\[\+0\.39,\+4\.28\]\[\+0\.39,\+4\.28\]0\.02070\.009\\phantom\{\-\}0\.009∗\*JawaherArabCulture−0\.08\-0\.08\[−1\.56,\+1\.39\]\[\-1\.56,\+1\.39\]0\.91950\.078\\phantom\{\-\}0\.078JawaherPalm−0\.46\-0\.46\[−2\.06,\+1\.18\]\[\-2\.06,\+1\.18\]0\.58150\.040\\phantom\{\-\}0\.040JawaherPoetry\+1\.01\+1\.01\[−0\.63,\+2\.61\]\[\-0\.63,\+2\.61\]0\.23120\.046\\phantom\{\-\}0\.046Table 11:Clustered aggregate effects, pooled across all four models \(items resampled jointly\)\. ICC is the intraclass correlation coefficient\.Δ\\Deltaand CI bounds in percentage points\.∗\*denotesp<0\.05p<0\.05\.ModelFinetuned onTaskSeedΔ\\Delta95% CIppALLaMPoetryAraDiCE42−5\.00\-5\.00\[−8\.89,−1\.11\]\[\-8\.89,\-1\.11\]0\.0225QwenPalmIdioms42\+4\.67\+4\.67\[\+1\.33,\+8\.00\]\[\+1\.33,\+8\.00\]0\.0156LlamaPoetryIdioms21\+10\.00\+10\.00\[\+3\.33,\+16\.67\]\[\+3\.33,\+16\.67\]0\.0081ALLaMPalmProverbs21−5\.56\-5\.56\[−9\.60,−1\.52\]\[\-9\.60,\-1\.52\]0\.0127FanarPalmProverbs42−3\.54\-3\.54\[−6\.57,−1\.01\]\[\-6\.57,\-1\.01\]0\.0391Table 12:Per\-run results significant by exact McNemar’s test \(p<0\.05p<0\.05\)\.
## Appendix FError Analysis

### F\.1Using Figurative Language to Improve Cultural Understanding

Table[13](https://arxiv.org/html/2608.18361#A6.T13)lists the top 10 most frequently improved and regressed questions from the AraDiCE cultural benchmark, aggregated across all models fine\-tuned on figurative language data \(proverbs and poetry\)\. These tables shed light on which cultural knowledge categories are most receptive to transfer from figurative language fine\-tuning, and which are most prone to degradation, providing a granular view of the asymmetric and selective nature of cross\-domain transfer in this direction\.

CountCountryQuestionTop 10 Most Frequently Improved9Jordan؟ندرألابناضمرراهنبراطفإلابرهجلاةبوقعوش7Jordan؟ندرألابجاوزلاديلاقتوتاداعيهوش6Syria؟ايروسلهأاهيفلفتحيبيللادايعألايهوش6Jordan؟ندرألابلافطأللةيبعشلاباعلألايهوش6Jordan؟ندرألابةروهشمراجشأوتاتابنرتكأوش5Qatar؟رطقيفةروهشملاتايولحلاونش5Palestine؟نيطسلفبةروهشملاتايولحلايهوش5Qatar؟ناصرقلاةياورفلأونم5Lebanon؟نانبلبةيمسرلالطعلامايأوش4Lebanon؟نانبلبناضمرراهنبراطفإلابرهجلاةبوقعوشTop 10 Most Frequently Regressed7Egypt؟رصميفةيمسرلالطعلامايأهيإ7Syria؟لاجرلليروسلايديلقتلاسبللاوش7Syria؟ناوسنلليروسلايديلقتلاسبللاوش6Palestine؟نيطسلفبتراصةيخيراتكراعم٣رهشأوش6Lebanonنانبلبنوجسلانعيكحتبنيتياورنيوانعيدب5Syria؟قشمدحتفداقيللايباحصلانيم5Syria؟ايروساهيلعلطتبيللاراحبلايهوش4Qatar؟رطقيفلالقتسالاديعىتم4Egypt؟رصميفجاوزلاديلاقتوتاداعهيإ4Qatar؟رطقيفةماعلاتالصاومللنيتليسورهشأونش

Table 13:Top 10 most frequently improved and regressed questions from the AraDiCE dataset, with counts across all models fine\-tuned on figurative language \(proverbs and poetry\)\.Idiom \(Arabic\)ImprRegrْهشِوْلَكَأ93ْهَدْلِجْلاىَلَع84نِرِيْهاَّلَخ27ْهَضاَخَمْلاِّسَج25ْبلْقِتِوْبَرْضِتِباَيْنُّدلا45بياسهفك62ْرْحَبْلاِتْحَف41

Table 14:Representative subset of the 21 unstable idioms \(improved in some seeds, regressed in others\), with counts across all fine\-tuned models\.Proverb \(Arabic\)DialectImprRegrلمجعسويبابلاKuwaiti18اهنبنمهسفنلماعEgyptian27اودعابيجتامنطبلاLibyan16صوردلاوارعنيامژاژغتلاتعاسMauritanian53مادكنمديأوهرونمديأعجرIraqi42اشابنايمعلاىلعروعألاOmani12

Table 15:Representative subset of the 15 unstable proverbs \(improved in some seeds, regressed in others\)\.Idiom \(Arabic\)ImprRegrْهشِوْلَكَأ33نِرِيْهاَّلَخ14ْهَدْلِجْلاىَلَع42ْبلْقِتِوْبَرْضِتِباَيْنُّدلا41ْهَضاَخَمْلاِّسَج12يِدْنهْرْمَتْنَبَلْكَمَس12اهاَطَغّْدَرْوهَمْلِك12بياسهفك21شِوْشوُلاَم21

Table 16:Representative subset of the 15 unstable idioms \(improved in some seeds, regressed in others\) under poetry fine\-tuning, with counts across all models\.Proverb \(Arabic\)DialectImprRegrحازمةودعلاAlgerian32اهنبنمهسفنلماعEgyptian13لبجلوقتتناولمجلوقااناOmani11نيَعلايحَتسِتمَفلامَعطِاPalestinian21يراوشلازنختةدحوةتوحMoroccan21مادكنمديأوهرونمديأعجرIraqi21اهصنبىضريةزببخىضراميللاAlgerian21

Table 17:The 7 unstable proverbs \(improved in some seeds, regressed in others\) under poetry fine\-tuning\.
### F\.2Using Culture and Poetry to Improve Figurative Language Understanding

Tables[19](https://arxiv.org/html/2608.18361#A6.T19)and[20](https://arxiv.org/html/2608.18361#A6.T20)present the number of improvements and regressions on idiom and proverb comprehension for each model and seed, under both cultural fine\-tuning \(ArabCulture, Palm\) and poetry fine\-tuning \(FannOrFlop\)\. Tables[18](https://arxiv.org/html/2608.18361#A6.T18)and[24](https://arxiv.org/html/2608.18361#A6.T24)complement these per\-model counts by reporting the net fine\-tuning effect on the proverb task broken down by Arabic dialect variety, allowing us to see whether transfer is uniform across dialects or disproportionately benefits \(or harms\) particular varieties\.

At the sample level, Tables[22](https://arxiv.org/html/2608.18361#A6.T22)and[23](https://arxiv.org/html/2608.18361#A6.T23)list the top 15 most frequently improved and regressed idioms and proverbs under cultural fine\-tuning, with Tables[25](https://arxiv.org/html/2608.18361#A6.T25)and[26](https://arxiv.org/html/2608.18361#A6.T26)giving the corresponding lists for poetry fine\-tuning\. The improved items highlight which expressions benefited most consistently; the regressed items reveal which exhibited the most persistent degradation, along with their dialect variety for proverbs\. Tables[14](https://arxiv.org/html/2608.18361#A6.T14),[15](https://arxiv.org/html/2608.18361#A6.T15),[16](https://arxiv.org/html/2608.18361#A6.T16), and[17](https://arxiv.org/html/2608.18361#A6.T17)further identify items that behave inconsistently across fine\-tuning conditions, improving for some models while regressing for others, underscoring the selectivity and instability of transfer at the level of individual items\.

#### Dialect\-Level Analysis\.

Unlike the idiom dataset, which is drawn exclusively from Egyptian Arabic, the proverb dataset spans 19 Arabic dialect varieties and Modern Standard Arabic \(MSA\)\. Under cultural fine\-tuning \(Table[18](https://arxiv.org/html/2608.18361#A6.T18)\), the results are geographically split: proverbs from Algeria \(−11\-11\), Sudan \(−10\-10\), and Libya \(−8\-8\) suffer the most severe regressions, while Mauritanian \(\+20\+20\), Yemeni \(\+14\+14\), Iraqi and Tunisian \(\+5\+5\) varieties benefit most\. It is not clear why this is the case, as Algeria, Sudan, and Libya regress despite being represented in both fine\-tuning datasets, while Mauritania and Iraq improve despite not being represented in ArabCulture\.

This geographic split largely persists under poetry fine\-tuning \(Table[24](https://arxiv.org/html/2608.18361#A6.T24)\)\. Sudanese \(−8\-8\), Qatari \(−8\-8\), and Algerian \(−5\-5\) varieties suffer the most severe regressions, while Yemeni \(\+8\+8\), Mauritanian \(\+8\+8\), and Jordanian \(\+7\+7\) benefit most\. Two varieties diverge between conditions: Qatari emerges as a clear regression category under poetry fine\-tuning—a decline largely driven by the single most\-regressed proverb \(ىمعلاالويدجتلا, 6 regressions\)—whereas it was only mildly negative under cultural fine\-tuning, and Egyptian proverbs move from−5\-5under cultural fine\-tuning to\+3\+3under poetry fine\-tuning, possibly because the latter reinforces Egyptian Arabic figurative patterns more directly\. The persistence of Mauritanian and Yemeni gains alongside continued Algerian and Sudanese losses suggests that the underlying difficulty is linked to intrinsic properties of those dialects in the evaluation set rather than to the choice of fine\-tuning data\.

DialectImprRegrNetMauritanian3010\+20\+20Yemeni151\+14\+14Iraqi72\+5\+5Tunisian72\+5\+5Jordanian117\+4\+4Kuwaiti128\+4\+4MSA40\+4\+4Moroccan76\+1\+1Qatari43\+1\+1Egyptian813−5\-5Syrian38−5\-5Lebanese712−5\-5Palestinian410−6\-6Saudi613−7\-7Omani310−7\-7Libyan513−8\-8Sudanese616−10\-10Algerian1021−11\-11

Table 18:Net fine\-tuning effect by Arabic dialect variety on the proverb task\.Model / DatasetBase%FT%Δ%\\Delta\\%95% CIppImprRegrALLaMArabCulture \(seed 0\)86\.084\.7−1\.33\-1\.33\[−5\.33,\+2\.67\]\[\-5\.33,\+2\.67\]0\.75446ArabCulture \(seed 21\)83\.382\.0−1\.33\-1\.33\[−6\.00,\+3\.33\]\[\-6\.00,\+3\.33\]0\.77457ArabCulture \(seed 42\)82\.781\.3−1\.33\-1\.33\[−5\.33,\+2\.67\]\[\-5\.33,\+2\.67\]0\.75446Palm \(seed 0\)86\.085\.3−0\.67\-0\.67\[−5\.33,\+4\.00\]\[\-5\.33,\+4\.00\]1\.00067Palm \(seed 21\)83\.383\.3±0\.00\\pm 0\.00\[−4\.67,\+4\.67\]\[\-4\.67,\+4\.67\]1\.00066Palm \(seed 42\)82\.781\.3−1\.33\-1\.33\[−6\.67,\+4\.00\]\[\-6\.67,\+4\.00\]0\.80479FannOrFlop \(seed 0\)86\.086\.7\+0\.67\+0\.67\[−3\.33,\+4\.67\]\[\-3\.33,\+4\.67\]1\.00054FannOrFlop \(seed 21\)83\.380\.7−2\.67\-2\.67\[−7\.33,\+2\.00\]\[\-7\.33,\+2\.00\]0\.38848FannOrFlop \(seed 42\)82\.781\.3−1\.33\-1\.33\[−5\.33,\+2\.67\]\[\-5\.33,\+2\.67\]0\.75446FanarArabCulture \(seed 0\)77\.375\.3−2\.00\-2\.00\[−5\.33,\+1\.33\]\[\-5\.33,\+1\.33\]0\.45325ArabCulture \(seed 21\)80\.078\.7−1\.33\-1\.33\[−4\.67,\+2\.00\]\[\-4\.67,\+2\.00\]0\.68824ArabCulture \(seed 42\)72\.772\.0−0\.67\-0\.67\[−4\.00,\+2\.67\]\[\-4\.00,\+2\.67\]1\.00034Palm \(seed 0\)77\.374\.7−2\.67\-2\.67\[−6\.67,\+0\.67\]\[\-6\.67,\+0\.67\]0\.28926Palm \(seed 21\)80\.078\.0−2\.00\-2\.00\[−5\.33,\+0\.67\]\[\-5\.33,\+0\.67\]0\.37514Palm \(seed 42\)72\.771\.3−1\.33\-1\.33\[−4\.67,\+2\.00\]\[\-4\.67,\+2\.00\]0\.68824FannOrFlop \(seed 0\)77\.378\.7\+1\.33\+1\.33\[−2\.00,\+4\.67\]\[\-2\.00,\+4\.67\]0\.68842FannOrFlop \(seed 21\)80\.080\.7\+0\.67\+0\.67\[−2\.00,\+3\.33\]\[\-2\.00,\+3\.33\]1\.00032FannOrFlop \(seed 42\)72\.775\.3\+2\.67\+2\.67\[−0\.67,\+6\.67\]\[\-0\.67,\+6\.67\]0\.28962LLaMAArabCulture \(seed 0\)59\.362\.0\+2\.67\+2\.67\[−5\.33,\+10\.67\]\[\-5\.33,\+10\.67\]0\.6272117ArabCulture \(seed 21\)60\.064\.7\+4\.67\+4\.67\[−2\.00,\+11\.33\]\[\-2\.00,\+11\.33\]0\.2481710ArabCulture \(seed 42\)59\.359\.3±0\.00\\pm 0\.00\[−7\.33,\+6\.67\]\[\-7\.33,\+6\.67\]1\.0001414Palm \(seed 0\)59\.362\.7\+3\.33\+3\.33\[−4\.67,\+12\.00\]\[\-4\.67,\+12\.00\]0\.5332318Palm \(seed 21\)60\.064\.7\+4\.67\+4\.67\[−2\.67,\+12\.67\]\[\-2\.67,\+12\.67\]0\.3102114Palm \(seed 42\)59\.362\.7\+3\.33\+3\.33\[−4\.00,\+10\.67\]\[\-4\.00,\+10\.67\]0\.4871914FannOrFlop \(seed 0\)59\.367\.3\+8\.00\+8\.00\[\+0\.00,\+16\.00\]\[\+0\.00,\+16\.00\]0\.0732513FannOrFlop \(seed 21\)60\.070\.0\+10\.00∗\+10\.00^\{\*\}\[\+3\.33,\+16\.67\]\[\+3\.33,\+16\.67\]0\.008227FannOrFlop \(seed 42\)59\.362\.0\+2\.67\+2\.67\[−4\.00,\+9\.33\]\[\-4\.00,\+9\.33\]0\.5571511QwenArabCulture \(seed 0\)67\.370\.7\+3\.33\+3\.33\[\+0\.00,\+6\.67\]\[\+0\.00,\+6\.67\]0\.12561ArabCulture \(seed 21\)68\.071\.3\+3\.33\+3\.33\[\+0\.67,\+6\.67\]\[\+0\.67,\+6\.67\]0\.06250ArabCulture \(seed 42\)68\.772\.0\+3\.33\+3\.33\[\+0\.67,\+6\.67\]\[\+0\.67,\+6\.67\]0\.06250Palm \(seed 0\)67\.370\.0\+2\.67\+2\.67\[−0\.67,\+6\.67\]\[\-0\.67,\+6\.67\]0\.28962Palm \(seed 21\)68\.072\.0\+4\.00\+4\.00\[\+0\.67,\+8\.00\]\[\+0\.67,\+8\.00\]0\.07071Palm \(seed 42\)68\.773\.3\+4\.67∗\+4\.67^\{\*\}\[\+1\.33,\+8\.00\]\[\+1\.33,\+8\.00\]0\.01670FannOrFlop \(seed 0\)67\.370\.0\+2\.67\+2\.67\[\+0\.00,\+6\.00\]\[\+0\.00,\+6\.00\]0\.21951FannOrFlop \(seed 21\)68\.069\.3\+1\.33\+1\.33\[−1\.33,\+4\.00\]\[\-1\.33,\+4\.00\]0\.62531FannOrFlop \(seed 42\)68\.770\.7\+2\.00\+2\.00\[−0\.67,\+5\.33\]\[\-0\.67,\+5\.33\]0\.37541Table 19:Improvement and regression breakdown on Kinayat idioms across cultural fine\-tuning \(ArabCulture and Palm\) and poetry fine\-tuning \(FannOrFlop\)\. Base% and FT% are accuracy before and after fine\-tuning;Δ%\\Delta\\%is the percentage\-point change, reported with 95% paired\-bootstrap confidence intervals over test items and exact McNemarpp\-values; Impr and Regr are the number of individual predictions improved or worsened\.∗marksp<0\.05p<0\.05\.Model / DatasetBase%FT%Δ%\\Delta\\%95% CIppImprRegrALLaMArabCulture \(seed 0\)88\.988\.9±0\.00\\pm 0\.00\[−2\.53,\+2\.53\]\[\-2\.53,\+2\.53\]1\.00033ArabCulture \(seed 21\)89\.987\.4−2\.53\-2\.53\[−5\.05,\+0\.00\]\[\-5\.05,\+0\.00\]0\.12516ArabCulture \(seed 42\)90\.990\.9±0\.00\\pm 0\.00\[−3\.03,\+3\.03\]\[\-3\.03,\+3\.03\]1\.00055Palm \(seed 0\)88\.986\.9−2\.02\-2\.02\[−5\.56,\+1\.52\]\[\-5\.56,\+1\.52\]0\.38848Palm \(seed 21\)89\.984\.3−5\.56∗\-5\.56^\{\*\}\[−9\.60,−1\.52\]\[\-9\.60,\-1\.52\]0\.013314Palm \(seed 42\)90\.987\.4−3\.54\-3\.54\[−7\.58,\+0\.51\]\[\-7\.58,\+0\.51\]0\.143512FannOrFlop \(seed 0\)88\.988\.4−0\.51\-0\.51\[−3\.54,\+2\.53\]\[\-3\.54,\+2\.53\]1\.00045FannOrFlop \(seed 21\)89\.987\.9−2\.02\-2\.02\[−5\.05,\+0\.51\]\[\-5\.05,\+0\.51\]0\.28926FannOrFlop \(seed 42\)90\.990\.4−0\.51\-0\.51\[−4\.04,\+3\.03\]\[\-4\.04,\+3\.03\]1\.00056FanarArabCulture \(seed 0\)87\.988\.4\+0\.51\+0\.51\[−2\.02,\+3\.03\]\[\-2\.02,\+3\.03\]1\.00043ArabCulture \(seed 21\)88\.987\.9−1\.01\-1\.01\[−4\.04,\+2\.02\]\[\-4\.04,\+2\.02\]0\.75446ArabCulture \(seed 42\)90\.487\.9−2\.53\-2\.53\[−6\.06,\+0\.51\]\[\-6\.06,\+0\.51\]0\.22738Palm \(seed 0\)87\.985\.9−2\.02\-2\.02\[−5\.56,\+1\.52\]\[\-5\.56,\+1\.52\]0\.38848Palm \(seed 21\)88\.985\.4−3\.54\-3\.54\[−7\.07,−0\.51\]\[\-7\.07,\-0\.51\]0\.06529Palm \(seed 42\)90\.486\.9−3\.54∗\-3\.54^\{\*\}\[−6\.57,−1\.01\]\[\-6\.57,\-1\.01\]0\.03918FannOrFlop \(seed 0\)87\.989\.4\+1\.52\+1\.52\[−2\.02,\+5\.05\]\[\-2\.02,\+5\.05\]0\.58185FannOrFlop \(seed 21\)88\.990\.4\+1\.52\+1\.52\[−1\.52,\+5\.05\]\[\-1\.52,\+5\.05\]0\.54974FannOrFlop \(seed 42\)90\.489\.4−1\.01\-1\.01\[−4\.55,\+2\.53\]\[\-4\.55,\+2\.53\]0\.77457LLaMAArabCulture \(seed 0\)70\.271\.2\+1\.01\+1\.01\[−3\.54,\+5\.56\]\[\-3\.54,\+5\.56\]0\.824119ArabCulture \(seed 21\)67\.268\.7\+1\.52\+1\.52\[−3\.03,\+6\.06\]\[\-3\.03,\+6\.06\]0\.6781310ArabCulture \(seed 42\)69\.270\.7\+1\.52\+1\.52\[−3\.54,\+6\.57\]\[\-3\.54,\+6\.57\]0\.6901411Palm \(seed 0\)70\.274\.7\+4\.55\+4\.55\[\+0\.00,\+9\.09\]\[\+0\.00,\+9\.09\]0\.078156Palm \(seed 21\)67\.271\.2\+4\.04\+4\.04\[−0\.51,\+9\.09\]\[\-0\.51,\+9\.09\]0\.152168Palm \(seed 42\)69\.270\.2\+1\.01\+1\.01\[−3\.54,\+5\.56\]\[\-3\.54,\+5\.56\]0\.8321210FannOrFlop \(seed 0\)70\.273\.7\+3\.54\+3\.54\[−1\.01,\+8\.08\]\[\-1\.01,\+8\.08\]0\.210158FannOrFlop \(seed 21\)67\.269\.7\+2\.53\+2\.53\[−2\.53,\+7\.58\]\[\-2\.53,\+7\.58\]0\.4421611FannOrFlop \(seed 42\)69\.270\.2\+1\.01\+1\.01\[−4\.04,\+6\.06\]\[\-4\.04,\+6\.06\]0\.8451412QwenArabCulture \(seed 0\)80\.381\.3\+1\.01\+1\.01\[−2\.02,\+4\.04\]\[\-2\.02,\+4\.04\]0\.72753ArabCulture \(seed 21\)80\.379\.8−0\.51\-0\.51\[−3\.03,\+2\.02\]\[\-3\.03,\+2\.02\]1\.00034ArabCulture \(seed 42\)83\.383\.3±0\.00\\pm 0\.00\[−2\.53,\+2\.53\]\[\-2\.53,\+2\.53\]1\.00033Palm \(seed 0\)80\.382\.3\+2\.02\+2\.02\[−0\.51,\+5\.05\]\[\-0\.51,\+5\.05\]0\.28962Palm \(seed 21\)80\.381\.3\+1\.01\+1\.01\[−2\.02,\+4\.04\]\[\-2\.02,\+4\.04\]0\.75464Palm \(seed 42\)83\.385\.4\+2\.02\+2\.02\[−0\.51,\+5\.05\]\[\-0\.51,\+5\.05\]0\.28962FannOrFlop \(seed 0\)80\.381\.8\+1\.52\+1\.52\[−1\.52,\+5\.05\]\[\-1\.52,\+5\.05\]0\.54974FannOrFlop \(seed 21\)80\.383\.3\+3\.03\+3\.03\[\+0\.00,\+6\.06\]\[\+0\.00,\+6\.06\]0\.10982FannOrFlop \(seed 42\)83\.384\.8\+1\.52\+1\.52\[−1\.52,\+4\.55\]\[\-1\.52,\+4\.55\]0\.50863Table 20:Improvement and regression breakdown on Jawaher proverbs across cultural fine\-tuning \(ArabCulture and Palm\) and poetry fine\-tuning \(FannOrFlop\)\. Base% and FT% are accuracy before and after fine\-tuning;Δ%\\Delta\\%is the percentage\-point change, reported with 95% paired\-bootstrap confidence intervals over test items and exact McNemarpp\-values; Impr and Regr are the number of individual predictions improved or worsened\.∗marksp<0\.05p<0\.05\.Model / DatasetΔ%\\Delta\\%95% CIImprRegrALLaMJawaher \(seed 0\)\+1\.11\+1\.11\[−3\.33,\+5\.56\]\[\-3\.33,\+5\.56\]108Jawaher \(seed 21\)−1\.11\-1\.11\[−5\.56,\+3\.33\]\[\-5\.56,\+3\.33\]810Jawaher \(seed 42\)−3\.89\-3\.89\[−7\.78,\+0\.00\]\[\-7\.78,\+0\.00\]310FannOrFlop \(seed 0\)\+1\.11\+1\.11\[−3\.33,\+5\.56\]\[\-3\.33,\+5\.56\]97FannOrFlop \(seed 21\)−1\.67\-1\.67\[−5\.56,\+1\.67\]\[\-5\.56,\+1\.67\]47FannOrFlop \(seed 42\)−5\.00∗\-5\.00^\{\*\}\[−8\.89,−1\.11\]\[\-8\.89,\-1\.11\]211FanarJawaher \(seed 0\)\+0\.56\+0\.56\[−3\.89,\+5\.00\]\[\-3\.89,\+5\.00\]98Jawaher \(seed 21\)\+0\.56\+0\.56\[−3\.89,\+5\.00\]\[\-3\.89,\+5\.00\]87Jawaher \(seed 42\)±0\.00\\pm 0\.00\[−3\.33,\+3\.33\]\[\-3\.33,\+3\.33\]55FannOrFlop \(seed 0\)±0\.00\\pm 0\.00\[−3\.89,\+3\.89\]\[\-3\.89,\+3\.89\]77FannOrFlop \(seed 21\)±0\.00\\pm 0\.00\[−4\.44,\+4\.44\]\[\-4\.44,\+4\.44\]88FannOrFlop \(seed 42\)\+2\.22\+2\.22\[−1\.67,\+6\.11\]\[\-1\.67,\+6\.11\]84LLaMAJawaher \(seed 0\)−0\.56\-0\.56\[−6\.11,\+5\.00\]\[\-6\.11,\+5\.00\]1213Jawaher \(seed 21\)−0\.56\-0\.56\[−5\.56,\+4\.44\]\[\-5\.56,\+4\.44\]1112Jawaher \(seed 42\)−1\.11\-1\.11\[−7\.22,\+5\.00\]\[\-7\.22,\+5\.00\]1517FannOrFlop \(seed 0\)\+2\.22\+2\.22\[−3\.33,\+7\.78\]\[\-3\.33,\+7\.78\]1612FannOrFlop \(seed 21\)±0\.00\\pm 0\.00\[−5\.56,\+5\.56\]\[\-5\.56,\+5\.56\]1313FannOrFlop \(seed 42\)−1\.11\-1\.11\[−7\.22,\+5\.00\]\[\-7\.22,\+5\.00\]1416QwenJawaher \(seed 0\)\+1\.67\+1\.67\[−1\.11,\+4\.44\]\[\-1\.11,\+4\.44\]52Jawaher \(seed 21\)\+1\.67\+1\.67\[\+0\.00,\+3\.89\]\[\+0\.00,\+3\.89\]30Jawaher \(seed 42\)−0\.56\-0\.56\[−2\.78,\+1\.67\]\[\-2\.78,\+1\.67\]23FannOrFlop \(seed 0\)\+1\.67\+1\.67\[−1\.67,\+5\.00\]\[\-1\.67,\+5\.00\]63FannOrFlop \(seed 21\)\+1\.11\+1\.11\[−1\.67,\+3\.89\]\[\-1\.67,\+3\.89\]42FannOrFlop \(seed 42\)−0\.56\-0\.56\[−3\.33,\+2\.22\]\[\-3\.33,\+2\.22\]34

Table 21:Improvement and regression breakdown on AraDiCE\-Culture across figurative fine\-tuning \(Jawaher\) and poetry fine\-tuning \(FannOrFlop\)\.Δ%\\Delta\\%is the percentage\-point change from base to fine\-tuned accuracy, reported with 95% paired\-bootstrap confidence intervals over test items; Impr and Regr are the number of individual predictions improved or worsened\.∗marksp<0\.05p<0\.05under an exact McNemar test\.CountIdiomTop 15 Most Frequently Improved16ْدوُعىَلَعْدوُد9ْهَديِدَحْلاىَلَع9ْهشِوْلَكَأ8ْهَدْلِجْلاىَلَع6ْعاَرِّدلاوْعاَبْلاِب6ْماَّدُقْنِمْديإِواَرَوْنِمْديإ6بياسهفك6ْهَفْشاَنْهُديإ6ّْبَدْنِمْوِّبَهْنِم6ْتيبلاِنِمْهَريِمَخْلاِعَطْقِيْهُّشِو6ْداَّدَحْلاِعَنَصاَمْهُنيبْوهُنيب6ْروُّزلاِنِمْشْلِزْنِياَم5ْهُنيِعْنِمْعِلِط4ْقِطاَّنلاْقِلاَخْلا4ىَراَصَنْةِزاَوَجTop 15 Most Frequently Regressed9ْساَّنلاِّيَز7نِرِيْهاَّلَخ6ْهُبوُتْنِمْشوُم6ْهُغاَدْباَج6ْلاَخْلُخْلاِباَيْنُّدلاهُدْنَع6ْلِجْنِمْلاِوْلِجْنِحْلاِب6ْهُتِّبُقيِفاَهْباَج6يفيِصْهَخيِّطَبْهُنْطَبيفطَح6رَبْلااَهْبِياَجْشوُم5ْبلْقِتِوْبَرْضِتِباَيْنُّدلا5هَنيِحْطِرْحَبلاِلَمَع5ْهَضاَخَمْلاِّسَج5ْهُعاَبُصِّلِضيِفىَراَّدإ4ْهَتاَلَّتلاِتْلِت4ْهُقيِرْعَلَبTable 22:Top 15 most frequently improved and regressed idioms across all models fine\-tuned on cultural data \(ArabCulture and Palm\)\.CountDialectProverbTop 15 Most Frequently Improved10Mauritanian\.ْكلْكَّلَعْيالْكاعْماْدِراوُهاَمْلا9Mauritanianلازغاومانيعفاسونخبلك6Yemeniقزريكبروقزخ6Lebaneseسبايلاورَضْخألالكأ6Kuwaiti\.رحبهطقوريخوس6Sudaneseنينسباىلعكحضينسبا5Mauritanianصوردلاوارعنيامژاژغتلاتعاس4Algerianاهصنبىضريةزببخىضراميللا4Iraqiمادكنمديأوهرونمديأعجر4Jordanianهِشيِرُهُسارْبإ4Jordanianلوقيهنعضماحبنعلالوطياميللا4MSAادرمتميئللاتمركأتنأنإوهتكلمميركلاتمركأتنأاذإ4Yemeniهنمثيفلاعلا4Egyptianاهيلَعسِّسَحِيةَحطَبهُسارىلَعيللإ4AlgerianهرادقملقهرادنمجرخيذلاTop 15 Most Frequently Regressed12Algerianةلوتخمةأرملاوةلولحمرادلا8Kuwaitiلمجعسويبابلا8Lebanese؟رحببكمسيرتشيبادَح7Egyptianاهنبنمهسفنلماع6Sudaneseناكولوريبكلامالكعمسا6Libyan\.اودعابيجتامنطبلا6Omaniدمحمنويعلجانمدمحيباحا5Sudanese\.توفيلمجلاوحبنيبلكلا4Sudanese\.نيرفناهوليشبنينضاماهفُقلا4Libyanهنامزهيلعدقرُيدقرُيىللا4Saudi\.ونمنسحأتاكسلاوحيلميدلا4Jordanian\.اقلبللاقرزلاباجوش4Egyptianْناَطْلُسْهُتيِبيِفْناَرْعُجوُبَأ4Moroccanلخاّدنمكرابخشآاّربنمقّوزملاآ3Palestinianهيّبَرتالودبَعلاِرَتشا

Table 23:Top 15 most frequently improved and regressed proverbs across all fine\-tuned models, with counts and dialect variety\.DialectImprRegrNetYemeni102\+8\+8Mauritanian113\+8\+8Jordanian81\+7\+7Kuwaiti60\+6\+6Tunisian51\+4\+4MSA40\+4\+4Egyptian74\+3\+3Bahraini20\+2\+2Moroccan53\+2\+2Saudi20\+2\+2Palestinian76\+1\+1Syrian54\+1\+1Iraqi21\+1\+1Lebanese54\+1\+1Libyan2200Emirati01−1\-1Omani15−4\-4Algerian712−5\-5Sudanese513−8\-8Qatari311−8\-8

Table 24:Net poetry fine\-tuning effect by Arabic dialect variety on the proverb task\.CountIdiomTop 15 Most Frequently Improved8ْدوُعىَلَعْدوُد6ّْبَدْنِمْوِّبَهْنِم4ْهَدْلِجْلاىَلَع4ْبلْقِتِوْبَرْضِتِباَيْنُّدلا3ْهَديِدَحْلاىَلَع3ْماَّدُقْنِمْديإِواَرَوْنِمْديإ3ْهُغاَدْباَج3ْسوُماَّنلاَعبِواَّتإ3ْداَّدَحْلاِعَنَصاَمْهُنيبْوهُنيب3ْروُّزلاِنِمْشْلِزْنِياَم3ْتيبلاِنِمْهَريِمَخْلاِعَطْقِيْهُّشِو3ْهشِوْلَكَأ3ْهَفْشاَنْهُديإ3ْهُنيِعْنِمْعِلِط2قَّشلايِفْهُعاَبْصِّطَحTop 15 Most Frequently Regressed5يفيِصْهَخيِّطَبْهُنْطَبيفطَح4نِرِيْهاَّلَخ4هَنيِحْطِرْحَبلاِلَمَع3ْهُبوُتْنِمْشوُم3ْهشِوْلَكَأ3ْساَّنلاِّيَز3ْهُتِّبُقيِفاَهْباَج3ْلاَخْلُخْلاِباَيْنُّدلاهُدْنَع2يِدْنهْرْمَتْنَبَلْكَمَس2ْهَدْلِجْلاىَلَع2ْرَجحْلاِوْهضيبْلاِبْبعِل2ْرَبَخْهَركْشَأ2ْلْطَّرلاِب2ْهَضاَخَمْلاِّسَج2اهاَطَغّْدَرْوهَمْلِك

Table 25:Top 15 most frequently improved and regressed idioms across all models fine\-tuned on poetry data \(FannOrFlop\)\.CountDialectProverbTop 15 Most Frequently Improved4MSAادرمتميئللاتمركأتنأنإوهتكلمميركلاتمركأتنأاذإ4Egyptianاهيلَعسِّسَحِيةَحطَبهُسارىلَعيللإ3Mauritanian\.ْكلْكَّلَعْيالْكاعْماْدِراوُهاَمْلا3Algerian\.حازمةودعلا3Yemeniقزريكبروقزخ3Syrian\.ريطلاهكرتناكام،ريخيفاذإ3Palestinian\.حَرطِماَهلشَتَقَالامحَرفِتةنيِزَحلاتَجَأ3Moroccanىمعلاالوشمعلامهللا3Lebaneseسبايلاورَضْخألالكأ3Kuwaiti\.رحبهطقوريخوس3Yemeniساسدقرعلا3Mauritanianلازغاومانيعفاسونخبلك3Sudaneseنينسباىلعكحضينسبا2Jordanianعوجيامةِزاَبَخْلاِنِبِإ2AlgerianاهصنبىضريةزببخىضراميللاTop 15 Most Frequently Regressed6Qatariىمعلاالويدجتلا6Algerianةلوتخمةأرملاوةلولحمرادلا3Palestinianةَسيِرَفلاَتنُكاًسِراَفنُكَتمَلاذا3Sudaneseفوشيمعلادلبيفشاشطلا3Egyptianاهنبنمهسفنلماع3Qatariنوختاالونمأ3Mauritanianةبحصاهاروةقيدلك3Syrianليوطنمزلاوكاّيوانأ3Omaniدمحمنويعلجانمدمحيباحا2Sudaneseناكولوريبكلامالكعمسا2Algerian\.حازمةودعلا2Libyan\.اودعابيجتامنطبلا2Sudanese\.توفيلمجلاوحبنيبلكلا2Lebaneseحيرتسملادرابَعالكأ2Moroccanلخاّدنمكرابخشآاّربنمقّوزملاآ

Table 26:Top 15 most frequently improved and regressed proverbs across all models fine\-tuned on poetry data \(FannOrFlop\), with counts and dialect variety\.

## Appendix GUse of AI Assistants

LLM\-based writing assistants were used to support writing and editing of the manuscript, including suggestions for rephrasing and refinement\. All research design, analysis, interpretation, and final wording decisions were made by the authors\.