Tag
This paper investigates how different evaluation setup choices affect base language model benchmarking by examining 8 design aspects across 26 2B-parameter models and larger LLMs in four languages, proposing monotonicity, non-randomness, and ranking consistency as meta-measures to guide reliable evaluation setup selection during pretraining.