Matthias Endler-style (matklad) blog post arguing micro benchmarks should be tuned to run around 300ms — fast enough to iterate quickly, slow enough to avoid startup overhead and to give humans an intuitive sense of speed — with the aim that benchmarking provides decision-making intuition rather than precise measurement.
<header>
<h1>Benchmark In Milliseconds</h1>
<time class="meta" datetime="2026-10-05">Oct 5, 2026</time>
</header>
<p>How long should a micro benchmark run? My rule of thumb is to tweak the input
size until the benchmark takes about 300ms, for the following reasons:</p>
<ul>
<li>
Milliseconds are integers ranging from 1 to 999. Enough precision to notice
even a small improvement, and easy to scan visually. No need for different
units or floating points (compare <code>1.31s</code> with <code>239ms</code>).
</li>
<li>
Anything faster than, say, <code>10ms</code> risks being skewed by fixed costs (e.g,
interpreter startup). Hundreds of milliseconds is an eternity for a computer,
usually enough to make one-off overheads irrelevant without using fancier (=
less robust) techniques to explicitly account for them.
</li>
<li>
For a human, hundreds of milliseconds is fast, but noticeable. Pushing numbers
into human-perceptible range allows me to use my intuitive sense of time and
speed, it doesn’t rely exclusively on numeracy. It’s plain fun to see, as a
result of optimization work, how a previously lagging CLI command becomes
“instant”.
</li>
<li>
But anything longer than a second makes <em>iterating</em> on the benchmark slower
than it needs to be. Running a benchmark 10 times in a row to eyeball variance
should be fast!
</li>
</ul>
<p>The imminently-to-be-stated assumption here is that the purpose of benchmarking
isn’t so much a precise measurement of performance, but rather providing the
author with enough intuition to make a correct decision.</p>
# Benchmark In Milliseconds
Source: [https://matklad.github.io/2026/10/05/benchmark-milliseconds.html](https://matklad.github.io/2026/10/05/benchmark-milliseconds.html)
Oct 5, 2026How long should a micro benchmark run? My rule of thumb is to tweak the input size until the benchmark takes about 300ms, for the following reasons:
- Milliseconds are integers ranging from 1 to 999\. Enough precision to notice even a small improvement, and easy to scan visually\. No need for different units or floating points \(compare`1\.31s`with`239ms`\)\.
- Anything faster than, say,`10ms`risks being skewed by fixed costs \(e\.g, interpreter startup\)\. Hundreds of milliseconds is an eternity for a computer, usually enough to make one\-off overheads irrelevant without using fancier \(= less robust\) techniques to explicitly account for them\.
- For a human, hundreds of milliseconds is fast, but noticeable\. Pushing numbers into human\-perceptible range allows me to use my intuitive sense of time and speed, it doesn’t rely exclusively on numeracy\. It’s plain fun to see, as a result of optimization work, how a previously lagging CLI command becomes “instant”\.
- But anything longer than a second makes*iterating*on the benchmark slower than it needs to be\. Running a benchmark 10 times in a row to eyeball variance should be fast\!
The imminently\-to\-be\-stated assumption here is that the purpose of benchmarking isn’t so much a precise measurement of performance, but rather providing the author with enough intuition to make a correct decision\.
The article provides a step-by-step guide to tuning a server for benchmarking to reduce run-to-run noise and ensure measurements are repeatable, with techniques like hardware inspection and core pinning.
A community rant urging realistic AI model benchmarks that account for context size, multimodal features, hardware specifics, and parallel processing, rather than just raw speed.
Discusses 200 milliseconds as a critical threshold for response time, highlighting its role in perceived performance and user experience in tech systems.