Tag
A new paper builds and releases a corpus of 2.2 million city and county laws from across the United States, making publicly accessible legal texts that were technically public but hard to find.
Jacob Li introduces 'Machine Studying' as a new problem in continual learning: how AI systems develop expertise in an unfamiliar domain given only a corpus of documents, distinct from avoiding catastrophic forgetting.
This paper introduces Darshana Graph, a parallel commentary corpus for comparative Indian philosophy, and presents stylometric and exploratory graph analyses.
Introduces LOCUS, a comprehensive corpus of U.S. local ordinance codes designed to enable machine-readable legal AI research, covering codes from 9,239 cities and counties with ModernBERT-based classifiers for analysis.
AAbAAC is a manually annotated corpus of 115 PubMed abstracts for autoimmunity information extraction, focusing on entities like autoimmune diseases and autoantibodies. The study demonstrates improved NER performance after fine-tuning on this corpus.
HKJudge is the first sentence-level expert-annotated legal discourse corpus for Hong Kong criminal judgments, featuring a two-tier discourse schema and benchmark evaluations of BERT-based and LLM models.
This paper introduces TypewriterLM, a 7.24B parameter language model trained exclusively on English text predating 1913, along with TypewriterCorpus (a 54B-token cleaned historical corpus) and instruction-tuning datasets to avoid temporal leakage and lookahead bias. It also presents a benchmark suite, History-Event, for evaluating temporal grounding and leakage.
KletterMix is a high-quality German pretraining corpus built by translating a state-of-the-art English pretraining dataset into German while preserving structure and diversity. Controlled experiments show models trained on KletterMix achieve measurable improvements on German-language benchmarks.
This paper introduces BOUTEF, a large-scale multilingual corpus for studying fake news in Algeria and Tunisia, covering Arabic dialects, Arabizi, French, English, and code-switching. It includes empirical analysis of linguistic strategies and engagement dynamics.
This paper presents the Arabic Women and Society Corpus, a ten-year collection of over 250,000 Arabic Facebook posts related to women's empowerment and social wellbeing, with engagement metrics for analyzing gender discourse and sentiment.
ArabDiscrim is a decade-long lexical resource and corpus of 293K Arabic Facebook posts about racism and discrimination, with engagement signals, morphological regex families, and discrimination axes, supporting fairness-oriented Arabic NLP research.
This paper presents the development of parallel and monolingual corpora for scientific machine translation across Spanish-English, French-English, and Portuguese-English, targeting four domains: Cancer Research, Energy Research, Neuroscience, and Transportation. The corpora are used to fine-tune neural machine translation systems, addressing challenges of specialized vocabulary and syntax in scientific text.
Released a free 9.8 million document multilingual Indic corpus (11 languages, CC0 license) on HuggingFace, containing approximately 8.4 billion tokens, built for multilingual research.