tiny-llms

MicroLLM Lab is a browser-based tool that allows users to test and benchmark 7 tiny language models, using JavaScript for objective checks and providing speed and accuracy metrics.

A deep benchmark of 8 tiny LLMs (135M to 1B parameters) on a $250 Jetson Orin Nano Super across four power modes finds 25W to be Pareto-optimal, with SmolLM2-135M achieving 165.1 tok/s and best efficiency.