Tag
Q2D-Web is a large-scale benchmark and leaderboard for evaluating retrieval models on web search, featuring 190 million documents and queries in ten languages with methods to minimize bias and reduce evaluation costs.
Introduces ALEE, a framework that uses Abstract Meaning Representations to generate English minimal pairs with controlled semantic shifts and translates them for evaluating text embeddings across 275+ languages, revealing persistent gaps in cross-lingual semantic representation.
A study evaluating six commercial AI chatbots on factual questions derived from BBC News across six languages, finding high multiple-choice accuracy but significant drops in free-response, with retrieval errors driving over 70% of failures and revealing regional biases.