Tag
The paper introduces BLiMP-RO, a Romanian minimal-pair benchmark covering 50 grammatical phenomena with 493 linguist-constructed, human-validated pairs plus 5,942 semi-automatically generated ones, and studies how training data and architecture (GPT-2 vs GPT-BERT) affect grammatical generalization in Romanian.
A Lancaster University study benchmarks summarisation methods on pre-trained transformers for long UK financial annual reports (FNS 2020 dataset), compares evaluation metrics, proposes the novel BRUGEscore (harmonic mean of ROUGE-2 and BERTscore), and validates results with statistical significance testing and adversarial corruption analysis.
This paper presents HistoriQA-ThirdRepublic, a French-language multi-hop question answering dataset derived from historical documents of the French Third Republic, designed to evaluate retrieval-augmented and LLM systems in historical research contexts.
Introduces LAUKIN, a dataset of clause pairs from Australia, UK, and India contracts labeled for legal equivalence, and evaluates 12 models achieving 65.11% macro-F1, establishing a challenging benchmark.