Tag
PrismML's Bonsai 27B model runs on the Jetson Orin Nano 8GB with 4.31 tokens/s and 27 t/s prompt processing, using 6.2GB RAM and about 25W power. It indicates surprisingly usable edge AI performance.
A deep benchmark of 8 tiny LLMs (135M to 1B parameters) on a $250 Jetson Orin Nano Super across four power modes finds 25W to be Pareto-optimal, with SmolLM2-135M achieving 165.1 tok/s and best efficiency.