Teaching LLMs a Low-Resource Language: Enhancing Code Completion in Pharo
Summary
This paper presents an end-to-end pipeline for adapting LLMs to low-resource programming languages, using Pharo as a case study, achieving superior code completion with small models suitable for real-time IDE support.
View Cached Full Text
Cached at: 07/09/26, 07:50 AM
Paper page - Teaching LLMs a Low-Resource Language: Enhancing Code Completion in Pharo
Source: https://huggingface.co/papers/2607.04939
Abstract
Large language models can be adapted for low-resource programming languages through specialized training pipelines and benchmarks, achieving superior code completion performance compared to general-purpose models.
Large Language Models(LLMs) unlocked new possibilities in automated code writing, becoming the backbone of mostcode completiontools. While LLMs excel in mainstream languages, they often lack support for the so-called low-resource languages where training data is scarce. As a result, these languages lag behind in the quality ofcode completiontooling available to their communities. A concrete example is Pharo, a Smalltalk-inspired language whose IDE currently offers only single-token completion. In this work, we report on our experience bringing LLM-basedcode completionto Pharo. First, we describe an end-to-end pipeline that combines Pharo-specific data curation,continued pre-trainingandfine-tuningof open code LLMs. Second, we introduce a set of Pharocode completion benchmarksdesigned to evaluate whether models (i) learn Pharo’s syntax and (ii) accurately complete masked Pharo code from real-world GitHub repositories. Third, we show empirically that Pharo-specialized models substantially outperform their original base checkpoints and also exceed the accuracy of substantially larger code LLMs on Pharo completion. Overall, our case study demonstrates the feasibility of bringing strong LLM-basedcode completionto low-resource programming languages, with models small enough to provide ``real-time’’ in-IDE support.
View arXiv pageView PDFGitHub0Add to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.04939 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.04939 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.04939 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages
This tutorial paper provides an overview of building multilingual and multimodal LLMs for low-resource languages, covering data creation, model alignment, fine-tuning, and evaluation, with a focus on practical recipes and hands-on resources.
No Code LLM Finetuning
This article discusses making LLM finetuning accessible to non-coders, highlighting a video demonstration of how anyone can customize LLMs without coding.
HPC-LLM: Practical Domain Adaptation and Retrieval-Augmented Generation for HPC Support
This paper presents HPC-LLM, a retrieval-augmented and domain-adapted assistant for HPC workflows, fine-tuning Llama 3.1 8B with QLoRA on HPC documentation. It demonstrates performance comparable to larger general-purpose models with significantly lower resource requirements.
LLM Safety Alignment in Low-Resource Languages: A Systematic Literature Review
This systematic literature review examines safety alignment of large language models in low-resource languages, identifying a persistent multilingual safety gap and suggesting future directions such as culturally grounded benchmarks and participatory data collection.
No Resource, No Benchmarks, No Problem? Evaluating and Improving LLMs for Code Generation in No-Resource Languages
This paper tackles code generation for no-resource programming languages by building benchmarks and proposing a method that combines further pre-training with weight difference transfer to create specialized instruction-following models at reduced cost.