Tag
This paper quantifies cross-lingual skill inconsistencies in large language models through multilingual self-play in text-based games, revealing significant variations in performance across languages that can be partially mitigated by altering intermediate reasoning language.
HealMed is an expert-reviewed benchmark for evaluating large language models in medicine across nine languages, developed by medical experts to assess multilingual performance in clinical tasks.
This paper evaluates retrieval quality in RAG systems for Bengali agricultural advisory, finding performance varies by query type and language conditions, and introduces a benchmark dataset to highlight the need for disaggregated evaluation in low-resource settings.
Introduces UA-Legal-Bench, a five-task benchmark for evaluating large language models on Ukrainian legal reasoning, built from the Unified State Register of Court Decisions. Evaluates 11 LLMs, revealing task-dependent few-shot effects and the misleading nature of accuracy on imbalanced legal tasks.
GaoYao introduces a 182k-sample benchmark across 26 languages and 51 regions to systematically evaluate LLMs’ multilingual and multicultural capabilities, revealing large geographical performance gaps.