measurement-validity

Tag

Cards List
#measurement-validity

When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability

arXiv cs.CL · 2026-07-10 Cached

This paper audits the reliability of LLM-as-judge evaluation by showing that changing the evaluator model can shift scores even when candidate responses are fixed, and it examines scaling and upgrade paths for Qwen3 and MiniMax models, concluding that judge upgrades are not interchangeable and proposing best practices for reporting.

0 favorites 0 likes
#measurement-validity

At-Grok Is Not Converged:A Measurement-Validity Audit for Grokking Representation Metrics

arXiv cs.LG · 2026-07-09 Cached

This paper audits the measurement validity of representation metrics in grokking, showing that values at the grokking transition overstate converged circuit complexity and that compression lags generalization. It provides tooling to separate onset from compression and reports negative results on generality.

0 favorites 0 likes
← Back to home

Submit Feedback