Releasing smolbenchmark: Helps you choose the best model for your hardware!

Reddit r/LocalLLaMA Tools
benchmarking ai-models hardware-performance small-models edge-computing open-source

Summary

smolbenchmark is a new resource that benchmarks small AI models on consumer hardware devices, providing metrics like speed, efficiency, and thermals to help users choose the best model for their specific hardware.

Most model leaderboards assume a server with powerful GPUs to run models that people daily use. However, my smolbenchmark is the other column: models that fit in 8GB, ranked by: decode speed, tokens per joule, and heat, and all of this on your OWN hardware ranging from: tablets phones macs jetsons raspberry pis Currently, 13 families on the chart right now, ~1000 configs for the Jetson nano Orin Super 8GB. One device is live measuring: tok/s tok/J ITL latency power metrics thermals and battery Models that are small enough to actually fit on a device that you own. All the performance benchmarking I did, will be released here for anyone to look at and decide what exact model they would wanna use on their choice of hardware. Well currently, the Pi, phones, and Mac minis still in the oven, cooking and not filled in yet, but will soon be filled in! You will now you know which model is BEST for your own hardware with all the raw data available and details reports available to you https://yuvrajsingh-mist.github.io/smolbenchmark/ (still in heavy development; would love to hear feedback/suggestions on what can be improved!)
Original Article

Similar Articles

Benchmarking became easy

Reddit r/AI_Agents

The author created Any-Bench, a tool to benchmark AI models on personal codebases, addressing shortcomings in existing benchmarks like SWE-Bench.

New benchmark dropped

Reddit r/singularity

A new benchmark has been released, likely for evaluating AI or software performance.

(Rant ;)) Make your benchmarks realistic

Reddit r/LocalLLaMA

A community rant urging realistic AI model benchmarks that account for context size, multimodal features, hardware specifics, and parallel processing, rather than just raw speed.