Tag
This paper replicates a distributional-semantics extractive summarization method for Hindi and evaluates it on standard corpora, finding that sentence position is the only contributing feature and current Hindi benchmarks fail to incentivize advanced content selection.
This study replicates the finding that emotion representations in AI models have a geometry mirroring human affect psychology, using Google's Gemma-2-27b model, and extends the analysis to locate key layers and assess token-level effects.
A replication study fails to reproduce findings from a 2002 paper on procrastination and deadlines, leading to allegations of data fraud in the original influential research.
A study on GPT-5.4 demonstrates that adding a diacritic to a system prompt significantly increases the rate of exact output artifacts from 47% to 94%, with a paper provided for replication.
This paper replicates a study on AI efficiency assessment using FLOPs, validates that raw FLOPs are not a suitable metric for execution time on newer hardware, and emphasizes the need for complete replication packages in research.
This replication study independently validates the Natural Language Tools (NLT) framework across 14 models and 8,560 trials, showing that NLT improves tool-calling accuracy by 14.9 percentage points and reduces critical errors by 93% compared to structured tool calling, with the benefits most pronounced for smaller and reasoning models.
This paper replicates the finding of 'emotion vectors' in open-weight LLMs Apertus-8B and Gemma-4-E4B, showing that valence geometry is recoverable across models with differences in layer emergence. The study also finds that arousal encoding is sensitive to the story corpus used for extraction.
This paper empirically tests the psychometric reliability of LLM-based user state classification, finding that only 31 of 213 metrics met reliability criteria, questioning trust in real-time adaptive systems.
This replication study evaluates DExperts for mitigating toxicity in LLMs, finding near-perfect safety against explicit toxicity but reduced effectiveness against implicit hate speech and a significant latency trade-off.