nlp-benchmark

Tag

Cards List
#nlp-benchmark

BLiMP-RO: A Benchmark of Linguistic Minimal Pairs for Romanian

arXiv cs.CL ↗ · 5d ago Cached

The paper introduces BLiMP-RO, a Romanian minimal-pair benchmark covering 50 grammatical phenomena with 493 linguist-constructed, human-validated pairs plus 5,942 semi-automatically generated ones, and studies how training data and architecture (GPT-2 vs GPT-BERT) affect grammatical generalization in Romanian.

0 favorites 0 likes
#nlp-benchmark

A Comparative Study of Evaluation Metrics for Long-Document Financial Narrative Summarization with Transformers

arXiv cs.CL ↗ · 2026-10-08 Cached

A Lancaster University study benchmarks summarisation methods on pre-trained transformers for long UK financial annual reports (FNS 2020 dataset), compares evaluation metrics, proposes the novel BRUGEscore (harmonic mean of ROUGE-2 and BERTscore), and validates results with statistical significance testing and adversarial corruption analysis.

0 favorites 0 likes
#nlp-benchmark

HistoriQA-ThirdRepublic: Multi-Hop Question Answering Corpus for Historical Research, Parliamentary Debates from the French Third Republic (1870-1940)

arXiv cs.AI ↗ · 2026-07-01 Cached

This paper presents HistoriQA-ThirdRepublic, a French-language multi-hop question answering dataset derived from historical documents of the French Third Republic, designed to evaluate retrieval-augmented and LLM systems in historical research contexts.

0 favorites 0 likes
#nlp-benchmark

LAUKIN: A Multi-jurisdictional Common Law Contract Dataset

arXiv cs.CL ↗ · 2026-06-12 Cached

Introduces LAUKIN, a dataset of clause pairs from Australia, UK, and India contracts labeled for legal equivalence, and evaluates 12 models achieving 65.11% macro-F1, establishing a challenging benchmark.

0 favorites 0 likes
← Back to home

Submit Feedback