minimum-detectable-effect

Tag

Cards List
#minimum-detectable-effect

Pre-Registering the Detectable Effect: A Paired-MDE Budget for 4-bit Quantization Benchmarks, with a Pilot Audit

arXiv cs.LG · 2026-05-29 Cached

This paper adapts paired binary sample-size calculations to 4-bit quantization benchmarks, providing a conservative minimum detectable effect (MDE) bound that helps benchmark designers determine reliability before running experiments. A pilot audit shows that much of the observed variance across small subsamples is binomial sampling noise, not true model unreliability.

0 favorites 0 likes
← Back to home

Submit Feedback