Retrieval-Augmented Long-Context Translation for Cultural Image Captioning: Gators submission for AmericasNLP 2026 shared task

# Retrieval-Augmented Long-Context Translation for Cultural Image Captioning: Gators submission for AmericasNLP 2026 shared task
Source: [https://arxiv.org/html/2605.20626](https://arxiv.org/html/2605.20626)
Christopher Driggers\-Ellis University of Florida driggersellis\.cw@ufl\.eduDzmitry Kasinets University of Florida dkasinets@ufl\.eduChristan Grant University of Florida christan@ufl\.eduDaisy Wang University of Florida daisyw@cise\.ufl\.edu

###### Abstract

We present the University of Florida Gators submission to the AmericasNLP 2026 shared task on cultural image captioning for Indigenous languages\. Our two\-stage pipeline generates a Spanish intermediate caption with Qwen2\.5\-VL, then produces the target\-language caption using retrieval\-augmented many\-shot prompting with Gemini 2\.5 Flash\. We achieve 164\.1%, 131\.7%, and 122\.6% improvements over the shared task baseline for Bribri, Guaraní, and Orizaba Nahuatl captioning, respectively, in our dev set evaluation and maintain \>150% improvements for the Bribri and Orizaba Nahuatl languages in the test set evaluation\. We find retrieval is highly language\-dependent, beneficial only for large, in\-domain corpora, and that synthetic data augmentation accounts for around 28 chrF\+\+ of the dev set Guaraní performance gain\. Our submission is the overall winner of the shared task, placing second out of five finalist submissions in human evaluations of target\-language captions\. Code and prompts are available on GitHub\.111[https://github\.com/dhawan98/AmericasNLP2026\-Gators\-Submission](https://github.com/dhawan98/AmericasNLP2026-Gators-Submission)

## 1Introduction

The AmericasNLP 2026Buiet al\.\([2026](https://arxiv.org/html/2605.20626#bib.bib29)\)consists of generating culturally grounded captions for images in Indigenous languages of the Americas\. We identify three major challenges\. First, the target languages are low\-resource\. Second, the captions are culturally specific rather than generic visual descriptions\. Third, the task requires not only lexical transfer, but also stylistic control\. Successful systems must produce short, natural captions that match the reference register expected by the organizers\.

While the cultural image captioning task is new to AmericasNLP 2026, we find that current vision\-language models \(VLMs\) are not able to directly caption images in the target languages\. This is unsurprising given the limited training data available for the target languages\. We therefore formulate the problem as a compound of image captioning in a high\-resource language followed by machine translation from that language into one of the shared task targets, which mirrors the shared task’s baseline approach\.

We first attempt to build on the work ofDhawanet al\.\([2026](https://arxiv.org/html/2605.20626#bib.bib18)\)by providing intermediate Spanish \(Es\) VLM captions to an mBART\-based machine translation \(MT\) model\.Dhawanet al\.\([2026](https://arxiv.org/html/2605.20626#bib.bib18)\)show that synthetic parallel data and language\-specific preprocessing improve low\-resource Indigenous MT, including Es\-Guaraní \(Grn\) translation\. In the 2026 shared task, however, a standard neural machine translation pipeline proves insufficient\. When we apply the existing Es\-Grn translation model to Spanish intermediate captions produced from the dev images, the resulting captions are often fluent enough at the sentence level but do not match the target caption register\. In other words, the performance bottleneck shifts from generic translation quality to domain adaptation and culturally grounded caption style\.

To address this mismatch, we move from sequence\-to\-sequence translation toward retrieval\-augmented in\-context translation with Large Language Models \(LLMs\)\. The core idea is simple\. Instead of relying on a single model fine\-tuned on mixed\-domain low\-resource language corpora, we retrieve Es\-low\-resource \(LoRes\) examples that are similar to the current caption and provide them as in\-context exemplars\. This design lets the decoder adapt its lexical choices and stylistic register at inference time\. We evaluate multiple OpenAI and Gemini models, vary the number of retrieved training examples and development exemplars, and test prompt variants and retrieval heuristics\.

Our experiments lead to three main findings\. First, among OpenAI models, many\-shot direct translation outperforms both our mBART baseline and LLM post\-correction, but gains are modest and highly sensitive to prompt composition\. Second, Gemini 2\.5 FlashComaniciet al\.\([2025](https://arxiv.org/html/2605.20626#bib.bib25)\)is dramatically stronger in Es\-LoRes in\-context translation than the GPT\-family models\. Third, development references are useful as in\-context exemplars for matching the expected caption register, but they must be interpreted carefully because same\-pool development prompting can inflate development\-set scores\.

Our primary contributions are as follows\. We present a retrieval\-augmented LLM caption translation pipeline for low\-resource cultural image captioning\. We document an extensive set of negative and positive ablations, including model substitutions, prompt revisions, and development\-exemplar hyperparameter searches\. Finally, we highlight evaluation design as a central methodological issue for dev\-conditioned in\-context learning in low\-resource captioning\.

## 2Background

Machine translation for Indigenous languages of the Americas has advanced largely through the AmericasNLP shared tasks, which have established evaluation benchmarks and made parallel corpora available for many low\-resource languagesMageret al\.\([2021](https://arxiv.org/html/2605.20626#bib.bib8)\); Ebrahimiet al\.\([2023](https://arxiv.org/html/2605.20626#bib.bib23)\); Ebrahimi and others \([2024](https://arxiv.org/html/2605.20626#bib.bib10)\)\. Strong systems in these shared tasks have typically relied on multilingual pretrained models such as mBART, M2M\-100, or NLLB\-200, often combined with synthetic data generation, multilingual transfer, and additional corpus collectionGow\-Smith and Sánchez Villegas \([2023](https://arxiv.org/html/2605.20626#bib.bib11)\); Tonjaet al\.\([2023](https://arxiv.org/html/2605.20626#bib.bib12)\); Costa\-Jussàet al\.\([2022](https://arxiv.org/html/2605.20626#bib.bib7)\)\. A recurring pattern in this literature is that performance improvements come not only from larger models, but also from better alignment between model capacity, augmentation strategy, and the target domain\.

Character\-based evaluation metrics are also especially relevant in this area\. chrF and chrF\+\+ are widely used for morphologically rich languages because they are more robust than BLEU to inflectional variation and spelling differencesPopović \([2015](https://arxiv.org/html/2605.20626#bib.bib21),[2017](https://arxiv.org/html/2605.20626#bib.bib22)\)\. The AmericasNLP 2026 organizers likewise use chrF\+\+ as the first\-stage ranking metric for the shared task\.

Finally, our work relates to the broader use of in\-context learning for low\-resource generation\. Instead of relying exclusively on fixed model parameters after fine\-tuning, retrieval\-augmented prompting can adapt the decoder to the current example at inference time\. In our setting, this is especially attractive because the target outputs are short and stylistically constrained: parallel demonstrations can serve not only as semantic guides, but also as direct evidence of the desired caption register\.

## 3Dataset and Methodology

### 3\.1Baseline System

We compare our retrieval\-augmented LLM pipeline against the Qwen3VL\-8BQwen Team \([2025](https://arxiv.org/html/2605.20626#bib.bib27)\)Captioning and Sheffield 2023Gow\-Smith and Sánchez Villegas \([2023](https://arxiv.org/html/2605.20626#bib.bib11)\)MT method, which is the stated baseline for the current shared task on image captioning\. We apply this baseline to our submission results for dev set captioning in each of the applicable target languages and report percent improvement over the baseline scores\.

For MT in Guaraní captioning, we also compare against the mBART\-based Es\-Grn model fromDhawanet al\.\([2026](https://arxiv.org/html/2605.20626#bib.bib18)\), trained on curated and synthetic parallel data\. This serves as the strongest conventional translation baseline in our pipeline and allows us to assess whether retrieval\-augmented in\-context generation improves over standard sequence\-to\-sequence translation\. We evaluate several retrieval\-augmented GPT\-4\-family\(OpenAIet al\.,[2024](https://arxiv.org/html/2605.20626#bib.bib26)\)model variants, summarized in Table[2](https://arxiv.org/html/2605.20626#S4.T2): GPT\-4o\-mini, GPT\-4\.1\-mini, GPT\-4\.1, along with the competing Gemini 2\.5 Flash\.

Across these models, we vary the number of retrieved training examplesrr, development exemplarsdd, and prompting strategy\.

### 3\.2Task Data

The official development set𝒟\\mathcal\{D\}\(dev set\) contains 50 examples, and the organizers release the data in JSONL format paired with images\. The pilot set includes Spanish captions for reference, but the organizers explicitly note that these are pilot\-only and will not be present in development or test\.

Our final submission pipeline is two\-stage\. Stage 1 produces a Spanish caption from the image using Qwen 2\.5BBaiet al\.\([2025b](https://arxiv.org/html/2605.20626#bib.bib16)\)\. Stage 2 utilizes Gemini 2\.5 FlashComaniciet al\.\([2025](https://arxiv.org/html/2605.20626#bib.bib25)\)for Es\-LoRes translation\.

### 3\.3Datasets by Language

The shared task covers five target languages: Guaraní, Yucatec Maya, Orizaba Nahuatl, Bribri, and Wixárika\. As shown in[Table˜1](https://arxiv.org/html/2605.20626#S3.T1), the available retrieval dataℛ\\mathcal\{R\}differs substantially across languages, which motivates treating the retrieval sizerrand the number of development exemplarsddas inference\-time hyperparameters that vary across target language submissions\.

#### Guaraní

We use the largest retrieval bank in our setup\. The retrieval Es\-Grn bank of 53,183 pairs contains AmericasNLP 2023Ebrahimiet al\.\([2023](https://arxiv.org/html/2605.20626#bib.bib23)\)training data augmented with synthetic examples from the MultiScript30k projectDriggers\-Elliset al\.\([2025](https://arxiv.org/html/2605.20626#bib.bib15)\)\. The Guaraní retrieval dataset is relatively well aligned with the captioning task, as it contains culturally specific terms, proper nouns, and short descriptive examples useful for visual caption generation\.

#### Yucatec Maya

We do not have a comparable parallel training corpus for retrieval\. The development set𝒟\\mathcal\{D\}is the only retrieval source, so we rely on dev exemplars and the pretrained knowledge of the LLM and we fixr=0r=0in all experiments\.

#### Other Target Languages

We use the available Es\-LoRes parallel data as retrieval banks\. Their retrieval banks vary in size and domain match: Orizaba Nahuatl has 16,145 pairs, Bribri has 7,508 pairs, and Wixárika has 8,966 pairs\. The Wixárika retrieval data is notably less caption\-like, since much of the available corpus is narrative or literary rather than visual\-description oriented\. These differences motivate the language\-specific hyperparameter choices reported in[Table˜1](https://arxiv.org/html/2605.20626#S3.T1)\.

### 3\.4Retrieval Bank and Prompt Construction

For each target language, we construct a language\-specific Es\-LoRes retrieval bankℛ\\mathcal\{R\}from the available parallel data described above\. The Spanish side of each retrieval bank is indexed with BM25Robertson and Zaragoza \([2009](https://arxiv.org/html/2605.20626#bib.bib24)\), a TF\-IDF\-style lexical retrieval method that ranks candidate examples by query\-term overlap while accounting for term importance and document length normalization\. At inference time, the Spanish caption generated in Stage 1 is used as a queryqq, and the top\-rrretrieved pairs are selected asℛr​\(q\)⊂ℛ\\mathcal\{R\}\_\{r\}\(q\)\\subset\\mathcal\{R\}\.

In addition to retrieved training pairs, some configurations include development exemplars\. Let𝒟\\mathcal\{D\}denote the shared\-task development set for a target language, and let𝒟d⊂𝒟\\mathcal\{D\}\_\{d\}\\subset\\mathcal\{D\}denote thedddevelopment examples included in the prompt\. For a query captionqq, the full prompt context is defined as

P​\(q\)=𝒟d∪ℛr​\(q\),P\(q\)=\\mathcal\{D\}\_\{d\}\\cup\\mathcal\{R\}\_\{r\}\(q\),whereℛr​\(q\)\\mathcal\{R\}\_\{r\}\(q\)provides retrieval\-based semantic and lexical grounding, and𝒟d\\mathcal\{D\}\_\{d\}provides examples of the caption style expected by the task\. The model then generates the target\-language caption conditioned onP​\(q\)P\(q\)\.

We sweeprrandddwhere applicable\. The values selected for submissions are in[Table˜1](https://arxiv.org/html/2605.20626#S3.T1), and detailed grid\-search results appear in our appendix\.

### 3\.5Prompting Strategy

The MT prompt structure is deliberately simple\. The system prompt instructs the model to translate from Spanish into the target language, match the example style, stay concise, preserve culturally specific nouns when appropriate, and produce exactly one line\. The user prompt contains two evidence blocks, development exemplars𝒟d\\mathcal\{D\}\_\{d\}and retrieved Spanish–target\-language pairsℛr​\(q\)\\mathcal\{R\}\_\{r\}\(q\), followed by the current Spanish caption\.

We use the same general prompt structure across target languages, adding language\-specific modifications only when required\. After several prompt\-engineering attempts, we found that more aggressive instructions, such as suppressing generic lead\-ins or forcing noun\-first phrasing, reduced performance\. The final prompts therefore keep the system instruction minimal and rely on in\-context examples for lexical and stylistic guidance\. We summarize the final and ablation prompt files in[Table˜5](https://arxiv.org/html/2605.20626#Sx2.T5)of the appendix\.

### 3\.6University of Florida Gators Submission

Our submission system features a retrieval\-augmented long\-context translator embedded in a two\-stage image captioning pipeline\. Figure[1](https://arxiv.org/html/2605.20626#S3.F1)illustrates the full system, which is organized into five steps\. Stage 1 corresponds to Step 1 in the diagram\. A vision\-language model generates one Spanish captionqqfor each target image\. We use either Qwen2\.5\-VL\-72B\-Instruct in 4\-bit precision or Qwen3\-VL\-8BBaiet al\.\([2025a](https://arxiv.org/html/2605.20626#bib.bib17)\)for this stage\. The prompt is culturally aware; follows a noun\-first style; and encourages concise descriptions of visible entities, objects, clothing, actions, and scene context\. We treat this stage as fixed and do not optimize it extensively in this paper\.

![Refer to caption](https://arxiv.org/html/2605.20626v1/x1.png)Figure 1:Overview of the proposed two\-stage image captioning pipeline\. Here,ℛ\\mathcal\{R\}denotes the full Es\-LoRes retrieval bank,ℛr​\(q\)⊂ℛ\\mathcal\{R\}\_\{r\}\(q\)\\subset\\mathcal\{R\}denotes therrnearest Es\-LoRes training pairs retrieved for Spanish captionqq,𝒟\\mathcal\{D\}denotes the full development set, and𝒟d⊂𝒟\\mathcal\{D\}\_\{d\}\\subset\\mathcal\{D\}denotes theddgold dev pairs used in the many\-shot prompt\.Stage 2 corresponds to Steps 2–5 in Figure[1](https://arxiv.org/html/2605.20626#S3.F1)\. It transforms the generated Spanish captionqqinto the final target\-language caption using retrieval\-augmented many\-shot MT\. Theqqis used as a BM25 query over the Spanish side of the available Es\-LoRes retrieval bankℛ\\mathcal\{R\}\. In Step 3, the retrieved subsetℛr​\(q\)\\mathcal\{R\}\_\{r\}\(q\)and development subset𝒟d\\mathcal\{D\}\_\{d\}are assembled into a many\-shot prompt with an instruction to translate from Spanish to the target language while matching the example style\. Each prompt contains approximately 3K–5K tokens\. Therrnearest Es\-LoRes training pairs provide semantic and lexical grounding, while thedddevelopment pairs provide direct evidence of the target caption register\. In Step 4 of Figure[1](https://arxiv.org/html/2605.20626#S3.F1), Gemini 2\.5 Flash performs the final target\-language generation with temperature set to 0\.0, thinking disabled, and a maximum output length of 120 tokens\. We then strip prefixes in Step 5 and normalize whitespace to produce the final JSONL submission\. The system is evaluated using chrF\+\+, followed by human judgment for the top\-ranked submissions\.

Table 1:Submission chrF\+\+ results across languages for the proposed system\. Here,rris the number of retrieved training pairs included in each prompt,ddis the number of development exemplars included in each prompt, andRRdenotes the total size of the available retrieval bank for each language\. Baseline results are those provided for the shared taskGow\-Smith and Sánchez Villegas \([2023](https://arxiv.org/html/2605.20626#bib.bib11)\)\.Boldandunderlinedentries in the right\-side columns indicate the best and second\-best results, respectively\. \* Includes parallel Es\-Grn MultiScript30kDriggers\-Elliset al\.\([2025](https://arxiv.org/html/2605.20626#bib.bib15)\)synthetic exemplars\.†\\daggerNo external retrieval bank is used; only development exemplars are included\.

## 4Results

Table 2:Dev chrF\+\+ Guaraní captioning results across baseline machine translation and RAG\-based prompting configurations\.Boldindicates the best result, andUnderlineindicates the second\-best result\.[Table˜1](https://arxiv.org/html/2605.20626#S3.T1)gives our final submission’s performance for each of the target languages in the shared task while[Table˜2](https://arxiv.org/html/2605.20626#S4.T2)summarizes the main progression of Es\-Grn dev set experiments from an mBART translation model to in\-context MT with Gemini 2\.5 Flash, which is competent in all target languages\. Several trends emerge immediately\. First, direct many\-shot translation is better than post\-correction, confirming that it is more effective to generate the caption in one step than to repair the mBART output after the fact\. Second, among the OpenAI models we tested, GPT\-4o\-mini remains the strongest, but the gains over the mBART baseline are modest\. Third, Gemini 2\.5 Flash yields much larger improvements, even before adding any development exemplars\.

In our final dev set results, the Guaraní target achieves the highest absolute chrF\+\+ score among the five target languages by at least 10 chrF\+\+ and more than doubles the Bribri and Wixárika target performances\. For Guaraní we achieve a remarkable 131\.7% improvement over the baseline methodGow\-Smith and Sánchez Villegas \([2023](https://arxiv.org/html/2605.20626#bib.bib11)\)\. In[Section˜5](https://arxiv.org/html/2605.20626#S5), we investigate the effect that synthetic exemplars may have had on Guaraní target language performance versus the other languages for which no MultiScript30k data exists\. However, as the final column in[Table˜1](https://arxiv.org/html/2605.20626#S3.T1)attests, each language target outperforms the baseline method whenever the baseline is available\. In particular, for the Bribri and Orizaba Nahuatl targets, respectively, we achieve 164\.1% and 122\.6% improvements over the baseline\. As we highlight in[Table˜1](https://arxiv.org/html/2605.20626#S3.T1), our improvement for Bribri is the greatest relative improvements over the baseline method for any target language\.

The test results are slightly different\. Bribri is replaced by Orizaba Nahuatl as most improved language\. The languages achieve 155\.2% and 166\.9% improvement over the testing baseline, respectivelyBuiet al\.\([2026](https://arxiv.org/html/2605.20626#bib.bib29)\)\. Guaraní performance drops by more than half in the test evaluation and most of the dev set performance gain is not reproduced in the official evaluation as a result\. Wixárika performance improvement falls by a similar amount\.

## 5Ablations

In addition to the results we list in the previous section, we provide a number of ablations to demonstrate the superiority of the final submission over numerous alternative approaches to various facets of the captioning pipeline\. In the experiments reported here, we hold the Spanish captions fixed and optimize only the Guaraní translation stage\. This lets us analyze ablation effects independent of the image captioner and makes the ablations directly comparable to our dev set results\. We focus on the dev set evaluation here because ablations are all performed on the dev set before the release of final shared task results\.

### 5\.1Machine Translation Architecture

We allude in the first section to how the initial approach for the machine translation step in Guaraní utilizes an mBART based MT modelDhawanet al\.\([2026](https://arxiv.org/html/2605.20626#bib.bib18)\)\.[Table˜2](https://arxiv.org/html/2605.20626#S4.T2)shows Guaraní captioning performance across several MT architectures, including mBART, GPT\-4o, and Gemini 2\.5 Flash\. Where appropriate, we adopt the retrieval\-augmented approach in our final submission, and we vary the values ofrrandddwithin architectures\.

The results show that the best configuration for Guaraní translation is the Gemini 2\.5 Flash LLM prompted withr=80r=80andd=49d=49exemplars\. We traverse much of ther,dr,dsearch grid with OpenAI models\. The best OpenAI setting used GPT\-4o\-mini with 28 retrieved examples and 4 development exemplars, reaching 23\.60 chrF\+\+\. This performance is reflected in[Table˜2](https://arxiv.org/html/2605.20626#S4.T2)\. Removing or increasing development exemplars reduced performance\. GPT\-4\.1\-mini and GPT\-4\.1 were both competitive but did not surpass GPT\-4o\-mini\. This pattern suggests that, for the OpenAI models we tested, the in\-context regime has a narrow optimum\. Too little context leaves the model underconstrained; too much context appears to add noise or dilute the style signal\.

For its superior performance over GPT models and its positive receptivity to context, we adopt Gemini 2\.5 Flash for Es\-LoRes MT in our final submission\.

### 5\.2Hyperparameter Search

For each language, we sweep values ofrrandddthat we list in[Section˜3](https://arxiv.org/html/2605.20626#S3)\.[Table˜1](https://arxiv.org/html/2605.20626#S3.T1)of final submission results shows performance at the submittal configuration featuring itsr,dr,dpair\. To be thorough,[Table˜6](https://arxiv.org/html/2605.20626#Sx2.T6)reports the results of a partial grid search for each target language in our appendix\.

Though this ablation shows that therrandddexemplar counts from[Table˜1](https://arxiv.org/html/2605.20626#S3.T1)are optimal within our search grid, there is no accounting for values outside of it\. Additionally, performance changes as one scans the grid indicate differing impacts of therrandddhyperparameters for different languages and data sources\. Thus, we stress in this ablation the importance of a thorough search for the optimalr,dr,dpair for any new target language or data configuration in retrieval\-augmented Es\-LoRes MT\.

### 5\.3Synthetic Exemplars

We notice in[Section˜4](https://arxiv.org/html/2605.20626#S4)that our pipeline performs Guaraní image captioning much more effectively in absolute terms versus any of the other target languages, even though we achieve similar percent improvement over the applicable baseline for the Bribri and Orizaba Nahuatl targets\. We also note that Guaraní is the only target language for which we include synthetic exemplars from MultiScript30k\. Testing for the effect of this synthetic data, we ablate the original Guaraní captioning submission by only using AmericasNLP 2023Ebrahimiet al\.\([2023](https://arxiv.org/html/2605.20626#bib.bib23)\)training data for retrieval exemplars \(rr\)\.[Table˜3](https://arxiv.org/html/2605.20626#S5.T3)compares performance with and without MultiScript30kDriggers\-Elliset al\.\([2025](https://arxiv.org/html/2605.20626#bib.bib15)\)synthetic exemplars for threer,dr,dpairs and provides the greatest performance overall without synthetic exemplars\.

The results are clear\. Controlling for our retrieval hyperparameters by fixingrrandddto three pairs, we observe that the original configuration with both genuine AmericasNLP 2023 training data and synthetic MultiScript30k exemplars outperforms the ablation with AmericasNLP 2023 alone by more than 100% relative improvement in chrF\+\+ in each case\. Additionally, the best Guaraní captioning performance without synthetic exemplars is 55\.9% less than the best performance with them\. These results mirror the comparison of our submission’s Guaraní performance to the other target languages in the shared task and attribute much of our improvement in Guaraní captioning to synthetic exemplars in the retrieval\-augmented Es\-Grn translation step\.

DatarrddRet\. PairschrF\+\+ANLP2023 \+ MS30k404953,18351\.34ANLP2023 \+ MS30k402053,18348\.38ANLP2023 \+ MS30k804953,18348\.24ANLP202380026,03222\.65ANLP2023404926,03221\.03ANLP2023402026,03220\.50ANLP2023804926,03220\.75Table 3:Dev chrF\+\+ results for Guaraní captioning with differing retrieval exemplar sets\. Data includes AmericasNLP2023 \(ANLP2023\) training dataEbrahimiet al\.\([2023](https://arxiv.org/html/2605.20626#bib.bib23)\)and/or MultiScript30k \(MS30k\)Driggers\-Elliset al\.\([2025](https://arxiv.org/html/2605.20626#bib.bib15)\)synthetic exemplars as noted in the columnData\.
### 5\.4Alternative Prompting and Reranking

We also test several prompting ablations that looked reasonable but are consistently negative in their impact on performance, with the exception of a specific strategy we give additional attention to in[Section˜5\.5](https://arxiv.org/html/2605.20626#S5.SS5)\. In this section, we quickly summarize other alternative prompts, and in[Table˜5](https://arxiv.org/html/2605.20626#Sx2.T5), we produce all of the relevant prompts for completeness\.

A prompt rewrite aimed at suppressing generic scene\-introduction phrases reduces performance substantially, and a retrieval reranker that attempts to prefer short caption\-like pairs also reduces performance, both for OpenAI and Gemini\. Likewise, using 49 same\-pool development exemplars with GPT\-4o\-mini degraded performance instead of improving it\. These results matter because they show that this task does not respond well to aggressive heuristic control\. The most effective systems are built by keeping the instruction stable and varying only model choice and amount of context supplied\.

For Wixárika in particular, we devise special prompts with cultural context in the form of a glossary\. These glossaries contain the names of common objects from the Wixárika culture and their definitions in Spanish, with the hope that this additional context will help the LLM MT architectures translate intermediate Spanish captions into the Wixárika target language\. We utilize two versions of the cultural glossary, but the results do not improve for either one\. We include the additional prompts in[Table˜5](https://arxiv.org/html/2605.20626#Sx2.T5)to document this alternative prompting\.

### 5\.5Morphological Considerations for Bribri

We ablate our submission for the Bribri \(Bzd\) language by considering performance with and without morphological prompting and post\-processing for the Es\-Bzd MT\. Bribri has complex tonal marking in orthography \(circumflexes, underlines, multiple diacritics\) which makes the deduplication logic harderCoto\-Solano \([2021](https://arxiv.org/html/2605.20626#bib.bib28)\)\. The tokenizer splits differently on tonal characters\.

In our final submission, we utilize a complex postprocessing and additional prompting for the Gemini MT model to improve performance\.[Table˜4](https://arxiv.org/html/2605.20626#S5.T4)shows the evolution of our strategies to account for the morphological complexity of the Bribri language\. First, careful examination of our pipeline’s output reveals that Bribri captions are initially in NFC encoding, which combines base letter and diacritic encodings into one character\. This does not match the dev set examples, which use NFD encoding\. For instance, our model outputsëas a single unit, but the dev set’s NFD encoding would separate the letterefrom the umlaut above it\. Because chrF\+\+ is a character\-level lexical metric, this mismatch depresses the score\. We therefore perform NFD\-Normalization \(NFD\-Norm\.\) to account for the difference\. Secondly, we believe the morphological considerations of Es\-Bzd translation significant enough that they deserve special prompting\. To this end, we formulate a Morphological Prompt \(Morph\.\) for Bribri that includes special instructions about Subject\-Object\-Verb \(SOV\) word order, tonal diacritics, common consonant clusters, verb\-final clauses, and possessive noun prefixes\.

As in previous ablations, the results in[Table˜4](https://arxiv.org/html/2605.20626#S5.T4)largely speak for themselves\. NFD\-normalization overcomes the encoding mismatch, and morphological prompting provides additional improvements\.

Table 4:Morphological ablations for our submission in the Bribri captioning task\. We either utilize NFD\-Normalization \(NFD\-Norm\.\) as a postprocessing step, both NFD\-Norm\. and Morphological Prompting \(Morph\.\), or we apply the same prompting and postprocessing as other target languages \(Standard\)\. Rightmost column shows Dev ChrF\+\+\.

## 6Discussion

We now proceed to a discussion of our method and the results achieved\. We analyze the available data and investigate the potential sources of performance improvement over the baseline method\.

### 6\.1In\-Context Retrieval

Though limited to the Guaraní target, we show in our ablation of MT architectures that long\-context retrieval\-based MT using state\-of\-the\-art LLMs greatly improves image captioning performance at the MT stage versus the dedicated mBART modelDhawanet al\.\([2026](https://arxiv.org/html/2605.20626#bib.bib18)\)\. While we cannot say for certain whether this relationship holds for other language targets, from comparisons to baseline performances, it appears that additional context from real and synthetic exemplars significantly boosts LLM translation in Es\-LoRes tasks\.

### 6\.2Synthetic Exemplars

For the Guaraní language, we include approximately 30k synthetic exemplars for in\-context retrieval from the MultiScript30kDriggers\-Elliset al\.\([2025](https://arxiv.org/html/2605.20626#bib.bib15)\)dataset\. We observe in Section 4 that Guaraní achieves substantially higher absolute dev chrF\+\+ than the other target languages\.

Remembering that synthetic exemplars apply only to Guaraní and its significantly positive effect on Es\-Grn MT in related workDhawanet al\.\([2026](https://arxiv.org/html/2605.20626#bib.bib18)\), we ablate for the synthetic exemplars’ effect on Guaraní captioning\.

For Guaraní captioning without the MultiScript30k synthetic retrieval pairs, the results we elaborate in[Section˜5\.3](https://arxiv.org/html/2605.20626#S5.SS3)and[Table˜3](https://arxiv.org/html/2605.20626#S5.T3)show a comparison similar to Guaraní captioning versus the other target languages\. Results improve over 100%, approximately 28 total chrF\+\+ or more, for Guaraní captioning with the synthetic exemplars for the samer,dr,dpair\. These results also mirror observations fromDhawanet al\.\([2026](https://arxiv.org/html/2605.20626#bib.bib18)\)on the effect of synthetic exemplars on Es\-Grn MT\. We therefore conclude that synthetic exemplars are a driver of Guaraní performance and speculate that synthetic retrieval pairs may further improve captioning performance for other target languages\.

### 6\.3Morphology

Despite favorable results in the Guaraní target, we acknowledge in[Section˜4](https://arxiv.org/html/2605.20626#S4)that our greatest relative improvement is for the Bribri language\. The ablation in[Section˜5\.5](https://arxiv.org/html/2605.20626#S5.SS5)clarifies that much of this performance improvement stems from the morphological considerations we take for the Bribri language target via specialized prompting\. We observe that explicitly prompting for Bribri’s morphological features accounts for over 10% of the performance gain in our final submission over the shared task’s baselineGow\-Smith and Sánchez Villegas \([2023](https://arxiv.org/html/2605.20626#bib.bib11)\)\. We conclude that morphological prompting has potential for application in other low\-resource, morphologically complex languages\.

## 7Future Work

The most impactful next step is improving the visual captioning stage\. Error analysis indicates that approximately 54% of remaining Guaraní errors originate in the vision model rather than the translator\. A stronger VLM will likely yield larger gains than further translation tuning\. Beyond captioning, we may extend retrieval to incorporate visual similarity rather than Spanish text overlap alone, which would help when the intermediate caption is noisy or culturally ambiguous\. The lowest\-resource languages \(Wixárika and Bribri languages\) are bottle\-necked by corpus domain mismatch rather than model choice\. Even small caption\-style parallel data corpora for these languages would likely produce larger gains than any prompting improvement\.

## 8Conclusion

We present the University of Florida Gators team system for the AmericasNLP 2026 shared task on cultural image captioning for Indigenous languagesBuiet al\.\([2026](https://arxiv.org/html/2605.20626#bib.bib29)\)\. Our two\-stage pipeline, VLM captioning in Spanish followed by retrieval\-augmented many\-shot translation with Gemini 2\.5 Flash, substantially outperforms fine\-tuned baseline models across all five target languages, culminating at 48\.24 chrF\+\+ for Guaraní in the submission\. We find that retrieval behavior is highly language\-dependent\. Large retrieval windows help Guaraní but hurt Yucatec Maya, where Gemini’s pre\-training knowledge is sufficient and BM25 retrieval adds noise\. We also find that development exemplars are useful for matching caption register, but their use requires careful interpretation because they can inflate development\-set scores when drawn from the same evaluation pool\. For the lowest\-resource languages, the ceiling is domain mismatch, not model capacity\.

Finally, we hope our submission to the AmericasNLP 2026 shared task on cultural image captioning and our ablative analyses will guide future efforts in LoRes image captioning\.

## Limitations

Our system is a cascade: errors in the Spanish captions propagate into translation with no recovery mechanism\. Because most ablations hold the Spanish captions fixed and vary only the translation stage, they likely underestimate the contribution of the visual captioning model to final performance\.

A second limitation is the use of development examples as in\-context exemplars\. These examples are useful for matching the expected caption register, especially when little caption\-style target\-language data is available, but they can also inflate development\-set scores when exemplar selection and evaluation draw from the same small pool\. We therefore treat dev\-assisted results primarily as model\-selection and submission\-configuration evidence rather than as a fully independent estimate of generalization\. A more rigorous evaluation would report held\-out or cross\-split dev results for each language\.

Finally, evaluation relies primarily on chrF\+\+, which is useful for low\-resource and morphologically rich languages but cannot fully capture fluency, cultural appropriateness, or naturalness for native speakers\. Although the shared task includes human evaluation for finalist systems, our own ablations do not include additional native\-speaker evaluation\.

## References

- S\. Bai, Y\. Cai, R\. Chen, K\. Chen, X\. Chen, Z\. Cheng, L\. Deng, W\. Ding, C\. Gao, C\. Ge,et al\.\(2025a\)Qwen3\-vl technical report\.arXiv preprint arXiv:2511\.21631\.Cited by:[§3\.6](https://arxiv.org/html/2605.20626#S3.SS6.p1.1)\.
- S\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge, S\. Song, K\. Dang, P\. Wang, S\. Wang, J\. Tang,et al\.\(2025b\)Qwen2\.5\-vl technical report\.arXiv preprint arXiv:2502\.13923\.Cited by:[§3\.2](https://arxiv.org/html/2605.20626#S3.SS2.p2.1)\.
- M\. D\. Bui, D\. Guzmán, A\. Ebrahimi, F\. Morales, M\. Agüero\-Torales, R\. Insfrán, C\. González, R\. Araujo, L\. Cernuzzi, C\. R\. Noh Chi, C\. E\. Tec Cahun, S\. E\. Poot Cohuo, D\. R\. Benítez Chi, S\. N\. Palomo Arévalo, J\. E\. Canul Canche, D\. A\. Poot Poot, W\. M\. Dzib Dzib, E\. J\. Ake Pool, R\. A\. Couoh Martin, S\. Fernandez Sabido, L\. S\. Santiago Melchor, S\. Silverio, R\. Pugh, R\. Vázquez, J\. E\. Ortega, A\. Oncevay, R\. Manrique, L\. Chiruzzo, R\. Coto\-Solano, E\. Mager, S\. Rijhwani, D\. I\. Adelani, M\. Mager, and K\. von der Wense \(2026\)Findings of the AmericasNLP 2026 shared task on cultural image captioning for Indigenous languages\.InProceedings of the Sixth Workshop on NLP for Indigenous Languages of the Americas \(AmericasNLP\),San Diego, California\.Cited by:[§1](https://arxiv.org/html/2605.20626#S1.p1.1),[§4](https://arxiv.org/html/2605.20626#S4.p3.1),[§8](https://arxiv.org/html/2605.20626#S8.p1.1)\.
- G\. Comanici, E\. Bieber, M\. Schaekermann, I\. Pasupat, N\. Sachdeva, I\. Dhillon, M\. Blistein, O\. Ram, D\. Zhang, E\. Rosen,et al\.\(2025\)Gemini 2\.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.arXiv preprint arXiv:2507\.06261\.Cited by:[§1](https://arxiv.org/html/2605.20626#S1.p5.1),[§3\.2](https://arxiv.org/html/2605.20626#S3.SS2.p2.1)\.
- M\. R\. Costa\-Jussà, J\. Cross, O\. Çelebi, M\. Elbayad, K\. Heafield, K\. Heffernan, E\. Kalbassi, J\. Lam, D\. Licht, J\. Maillard,et al\.\(2022\)No language left behind: scaling human\-centered machine translation\.arXiv preprint arXiv:2207\.04672\.Cited by:[§2](https://arxiv.org/html/2605.20626#S2.p1.1)\.
- R\. Coto\-Solano \(2021\)Explicit tone transcription improves ASR performance in extremely low\-resource languages: a case study in Bribri\.InProceedings of the First Workshop on Natural Language Processing for Indigenous Languages of the Americas,M\. Mager, A\. Oncevay, A\. Rios, I\. V\. M\. Ruiz, A\. Palmer, G\. Neubig, and K\. Kann \(Eds\.\),Online,pp\. 173–184\.External Links:[Link](https://aclanthology.org/2021.americasnlp-1.20/),[Document](https://dx.doi.org/10.18653/v1/2021.americasnlp-1.20)Cited by:[§5\.5](https://arxiv.org/html/2605.20626#S5.SS5.p1.1)\.
- A\. Dhawan, C\. Driggers\-Ellis, C\. Grant, and D\. Z\. Wang \(2026\)Improving indigenous language machine translation with synthetic data and language\-specific preprocessing\.InProceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages \(LoResMT 2026\),A\. Kr\. Ojha, C\. Liu, E\. Vylomova, F\. Pirinen, J\. Washington, N\. Oco, and X\. Zhao \(Eds\.\),Rabat, Morocco,pp\. 119–126\.External Links:[Link](https://aclanthology.org/2026.loresmt-1.10/),[Document](https://dx.doi.org/10.18653/v1/2026.loresmt-1.10),ISBN 979\-8\-89176\-366\-1Cited by:[§1](https://arxiv.org/html/2605.20626#S1.p3.1),[§3\.1](https://arxiv.org/html/2605.20626#S3.SS1.p2.1),[§5\.1](https://arxiv.org/html/2605.20626#S5.SS1.p1.2),[§6\.1](https://arxiv.org/html/2605.20626#S6.SS1.p1.1),[§6\.2](https://arxiv.org/html/2605.20626#S6.SS2.p2.1),[§6\.2](https://arxiv.org/html/2605.20626#S6.SS2.p3.1)\.
- C\. Driggers\-Ellis, D\. Brinkley, R\. Chen, A\. Dhawan, D\. Z\. Wang, and C\. Grant \(2025\)MultiScript30k: leveraging multilingual embeddings to extend cross script parallel data\.External Links:2512\.11074,[Link](https://arxiv.org/abs/2512.11074)Cited by:[§3\.3](https://arxiv.org/html/2605.20626#S3.SS3.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2605.20626#S3.T1),[§5\.3](https://arxiv.org/html/2605.20626#S5.SS3.p1.2),[Table 3](https://arxiv.org/html/2605.20626#S5.T3),[§6\.2](https://arxiv.org/html/2605.20626#S6.SS2.p1.1)\.
- A\. Ebrahimi, M\. Mager, S\. Rijhwani, E\. Rice, A\. Oncevay, C\. Baltazar, M\. Cortés, C\. Montaño, J\. E\. Ortega, R\. Coto\-solano, H\. Cruz, A\. Palmer, and K\. Kann \(2023\)Findings of the AmericasNLP 2023 shared task on machine translation into indigenous languages\.InProceedings of the Workshop on Natural Language Processing for Indigenous Languages of the Americas \(AmericasNLP\),M\. Mager, A\. Ebrahimi, A\. Oncevay, E\. Rice, S\. Rijhwani, A\. Palmer, and K\. Kann \(Eds\.\),Toronto, Canada,pp\. 206–219\.External Links:[Link](https://aclanthology.org/2023.americasnlp-1.23/),[Document](https://dx.doi.org/10.18653/v1/2023.americasnlp-1.23)Cited by:[§2](https://arxiv.org/html/2605.20626#S2.p1.1),[§3\.3](https://arxiv.org/html/2605.20626#S3.SS3.SSS0.Px1.p1.1),[§5\.3](https://arxiv.org/html/2605.20626#S5.SS3.p1.2),[Table 3](https://arxiv.org/html/2605.20626#S5.T3)\.
- A\. Ebrahimiet al\.\(2024\)Findings of the americasnlp 2024 shared task on machine translation into indigenous languages\.InProceedings of the AmericasNLP Workshop,Cited by:[§2](https://arxiv.org/html/2605.20626#S2.p1.1)\.
- E\. Gow\-Smith and D\. Sánchez Villegas \(2023\)Sheffield’s submission to the americasnlp shared task on machine translation into indigenous languages\.InProceedings of the Third Workshop on NLP for Indigenous Languages of the Americas \(AmericasNLP\),External Links:[Link](https://arxiv.org/abs/2306.09830)Cited by:[§2](https://arxiv.org/html/2605.20626#S2.p1.1),[§3\.1](https://arxiv.org/html/2605.20626#S3.SS1.p1.1),[Table 1](https://arxiv.org/html/2605.20626#S3.T1),[§4](https://arxiv.org/html/2605.20626#S4.p2.1),[§6\.3](https://arxiv.org/html/2605.20626#S6.SS3.p1.1)\.
- M\. Mager, A\. Oncevay,et al\.\(2021\)Findings of the americasnlp 2021 shared task on open machine translation for indigenous languages of the americas\.InProceedings of the AmericasNLP Workshop,Cited by:[§2](https://arxiv.org/html/2605.20626#S2.p1.1)\.
- OpenAI, J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman,et al\.\(2024\)GPT\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§3\.1](https://arxiv.org/html/2605.20626#S3.SS1.p2.1)\.
- M\. Popović \(2015\)ChrF: character n\-gram F\-score for automatic MT evaluation\.InProceedings of the Tenth Workshop on Statistical Machine Translation,External Links:[Link](https://aclanthology.org/W15-3049/),[Document](https://dx.doi.org/10.18653/v1/W15-3049)Cited by:[§2](https://arxiv.org/html/2605.20626#S2.p2.1)\.
- M\. Popović \(2017\)ChrF\+\+: words helping character n\-grams\.InProceedings of the Second Conference on Machine Translation,External Links:[Link](https://aclanthology.org/W17-4770/),[Document](https://dx.doi.org/10.18653/v1/W17-4770)Cited by:[§2](https://arxiv.org/html/2605.20626#S2.p2.1)\.
- Qwen Team \(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§3\.1](https://arxiv.org/html/2605.20626#S3.SS1.p1.1)\.
- S\. Robertson and H\. Zaragoza \(2009\)The probabilistic relevance framework: bm25 and beyond\.Foundations and Trends in Information Retrieval3\(4\),pp\. 333–389\.External Links:ISSN 1554\-0669,[Document](https://dx.doi.org/10.1561/1500000019)Cited by:[§3\.4](https://arxiv.org/html/2605.20626#S3.SS4.p1.4)\.
- A\. L\. Tonja, H\. H\. Nigatu, O\. Kolesnikova, G\. Sidorov, A\. Gelbukh, and J\. Kalita \(2023\)Enhancing translation for indigenous languages: experiments with multilingual models\.InProceedings of the Third Workshop on NLP for Indigenous Languages of the Americas \(AmericasNLP\),External Links:[Link](https://arxiv.org/abs/2305.17406)Cited by:[§2](https://arxiv.org/html/2605.20626#S2.p1.1)\.

## Appendix

Here, we provide additional data for validation of our method versus various ablations with particular focus on alternative prompts for various target languages and ther,dr,dgrid search\. In the following sections, we give tables of alternative prompts and performance at differentr,dr,dpairs for each target language\.

### Alternative Prompts

[Table˜5](https://arxiv.org/html/2605.20626#Sx2.T5)yields the prompts that we utilize in our ablations and the prompts present in our final submission for each target language as indicated in theStandingcolumn\. TheLanguage\(s\)column shows which languages the prompt applies to\.

### r,dr,dHyperparameter Search

We frequently refer to a search for optimalrrandddretrieval hyperparameters in the main body of this paper but reserve detailed communication of the sweep for this appendix due to the number of configurations we consider\. For the Gemini 2\.5 Flash MT architecture and for each target language, we partially sweep a grid consisting ofr,dr,dpairs such thatr∈\{0,10,20,40,80\}r\\in\\\{0,10,20,40,80\\\}andd∈\{0,10,20,30,40,49\}d\\in\\\{0,10,20,30,40,49\\\}\. Table[6](https://arxiv.org/html/2605.20626#Sx2.T6)reports the chrF\+\+ scores for the final captioning pipeline for each combination ofrrandddtested for each target language\.

Table 5:Prompts we utilize in our ablations and final submission for each of the target languages\. TheStandingcolumn indicates whether a prompt is an Ablation or part of ourFinalsubmission\. The finalLanguage\(s\)column indicates what languages the prompt applies to\.BribriLanguageRetrieval \(r\)Dev Exemplars \(d\)NoteschrF\+\+Bribri8020Uses Morphological Bribri prompting and NFD\-normalization\.19\.99Bribri8020Uses Regular Prompting\.11\.50Bribri4020…11\.41Bribri1010…11\.41Bribri2020…11\.16Bribri2010…10\.95Bribri4010…10\.95Bribri1010…10\.63Bribri8010…10\.17Bribri800…6\.53Bribri400…5\.93Bribri200…5\.30Bribri100…4\.75GuaraníLanguageRetrieval \(r\)Dev Exemplars \(d\)NoteschrF\+\+Guaraní4049Includes synthetic exemplars\. Not submitted because ablation was incomplete at submission deadline\.51\.34Guaraní4020…48\.38Guaraní8049Includes synthetic exemplars\.48\.24Guaraní8020…42\.61Guaraní049…20\.80Orizaba NahuatlLanguageRetrieval \(r\)Dev Exemplars \(d\)NoteschrF\+\+Orizaba Nahuatl4020–25\.67Orizaba Nahuatl4010–25\.59Orizaba Nahuatl8020–25\.25Orizaba Nahuatl8010–25\.16Orizaba Nahuatl2020–24\.16Orizaba Nahuatl2010–23\.91Orizaba Nahuatl400–16\.56Orizaba Nahuatl800–16\.37Orizaba Nahuatl200–15\.61WixárikaLanguageRetrieval \(r\)Dev Exemplars \(d\)NoteschrF\+\+Wixárika4020–18\.99Wixárika4010–17\.74Wixárika2020–17\.56Wixárika8010–17\.48Wixárika8020–17\.13Wixárika2010–16\.81Wixárika400–16\.59Wixárika800–16\.25Wixárika200–13\.80Yucatec MayaLanguageRetrieval \(r\)Dev Exemplars \(d\)NoteschrF\+\+Yucatec Maya–49Fixesr=0r=0for lack of retrieval exemplars\.26\.29Yucatec Maya–20…26\.29Yucatec Maya–40…25\.07Yucatec Maya–30…25\.05Yucatec Maya–0Fixesr=0r=0for lack of retrieval exemplars\. Atd=0d=0, the model receives no signal from the target language\.20\.25Table 6:Dev chrF\+\+ results across languages for the proposed system\. \(…\) Indicates previousNotescolumn entry applies\. \(–\) Indicates noNotes\.BoldentriesChrF\+\+column are submission scores for each target language\.