Tag
Introduces Inspect India Evals, an open-source framework for evaluating LLMs in Indian linguistic and cultural contexts, with six benchmarks testing multilingual ability, bias, safety, and cultural knowledge. Tests on five models show Sarvam-M 24B and Gemma 2 27B lead.
This paper introduces Ex-ToxiCN-MM, the first Chinese harmful meme explanation dataset, along with a knowledge base C-HarmKB and an attribution analysis framework RIKE, to improve interpretable detection of harmful memes by considering cultural context and ambiguity.