CulturalRiddles: A Multicultural Benchmark of Riddles
Summary
We propose CulturalRiddles, a benchmark of 62 cultures (45 countries, 51 languages) of human-created, culturally grounded riddles requiring abductive and compositional reasoning. Even frontier LLMs reach at most 22% accuracy on some communities, highlighting gaps in cultural understanding.
View Cached Full Text
Cached at: 10/22/26, 03:42 PM
Paper page - CulturalRiddles: A Multicultural Benchmark of Riddles
Source: https://huggingface.co/papers/2610.29968 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
AsLLMsexpandtheirlanguagecoverage,evaluatinghowwelltheyunderstandandrepresentdiverseculturesbecomesincreasinglyimportant.However,manybenchmarksremainEnglish-centric,withlimitedculturalcoverageandafocusonfactualknowledgeoveritsapplicationinreal-worldculturalcontexts.WeproposeCulturalRiddles,abenchmarkoforiginal,culturallygroundedriddlescreatedentirelybyhumansthroughaparticipatorycommunityeffort,spanning62cultures(45countries,51languages).Theseriddlesaredeeplyrootedintheirrespectiveculturesandrequireabductiveandcompositionalreasoning,alongsidedetailedfactualandsocialknowledge,languageskills,andanunderstandingofculturallyspecificassociations.Webenchmark28LLMconfigurationsusingbothautomaticandlarge-scalehumanevaluation.Evenfrontiermodelsachieveatmost22%accuracyinsomecommunities.Wepubliclyreleaseallartifactsalongsideanopencontributionprocesstoexpandthebenchmark’slinguisticandculturalcoverage.
View arXiv pageView PDFProject pageGitHub4Add to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.29968 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.29968 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.29968 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning
This paper introduces CulturalMenuBench, a benchmark with 4,870 items across 10 languages and 18 regions, to test multimodal models' ability to apply culinary knowledge beyond mere recognition. Evaluations reveal a knowledge-application gap where models drop in performance when requiring cultural attribution despite high recognition scores.
CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks
CulturALL introduces a 2,610-sample benchmark across 14 languages and 51 regions to evaluate LLMs on real-world, culturally grounded tasks; top model scores only 44.48%, highlighting large room for improvement.
NSMQ Riddles: A Benchmark of Scientific and Mathematical Riddles for Quizzing Large Language Models
This paper introduces NSMQ Riddles, a novel benchmark using scientific and mathematical riddles from Ghana's National Science and Maths Quiz to evaluate Large Language Models, addressing the underrepresentation of Global South datasets in AI research.
CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queries
Introduces CCBench, a framework for evaluating LLMs' cultural competence via health queries with personas across six cultures, finding that even top models achieve only 20-30% culturally appropriate responses.
The Culture Within: Towards Mechanistic Understanding of Cultural Reasoning in LLMs
The paper introduces CultureLens, a reference-grounded benchmark and mechanistic study of how LLMs use cultural information, showing that stated contradictory norms sharply reduce aligned answers and that supervised fine-tuning can largely mitigate this vulnerability.