test-set-leakage

#test-set-leakage

@MathewShen42: Read an ICLR 2026 paper, couldn't figure out why the results were so good. Coincidentally the code was open-sourced, and it turned out they used test set labels to select parameters... Is this what a top conference is? :)

X AI KOLs Timeline ↗ · 2026-08-04

A tweet criticizing an ICLR 2026 paper, pointing out that an open-sourced paper achieved excellent results but actually used test set labels to select parameters, questioning the rigor of top conference peer review.

0 favorites 0 likes

test-set-leakage

@MathewShen42: Read an ICLR 2026 paper, couldn't figure out why the results were so good. Coincidentally the code was open-sourced, and it turned out they used test set labels to select parameters... Is this what a top conference is? :)

Submit Feedback