Tag
Introduces CCBench, a framework for evaluating LLMs' cultural competence via health queries with personas across six cultures, finding that even top models achieve only 20-30% culturally appropriate responses.
This paper introduces CAPRI, a dataset to evaluate whether LLMs can infer a user's cultural background from conversational cues and adapt their responses (e.g., using appropriate measurement units). Experiments show LLMs can infer cultural context but often fail to apply it unless explicitly prompted.
CulturALL introduces a 2,610-sample benchmark across 14 languages and 51 regions to evaluate LLMs on real-world, culturally grounded tasks; top model scores only 44.48%, highlighting large room for improvement.
Researchers introduce x1, a family of reasoning models that adaptively select optimal languages for reasoning on a per-instance basis, demonstrating that language choice impacts reasoning quality in multilingual and cultural tasks.