small-model

Agents-A1-4B, a compact 4B-parameter model from InternScience, achieves state-of-the-art results across multiple agentic and reasoning benchmarks, outperforming larger models by scaling capabilities rather than parameters.

This paper investigates what makes a small 4B parameter model's citations faithful when used as an on-device research agent, finding that exposure to more source text improves faithfulness while retrieval recall limits coverage.

OvisOCR2 is a 0.8B parameter end-to-end document parsing model that converts document page images to Markdown, achieving state-of-the-art scores on public benchmarks through a combination of supervised fine-tuning, reinforcement learning, and model fusion.

PrismML announces Bonsai 27B, the first 27-billion parameter model able to run on a phone, with ternary and 1-bit variants (5.9 GB and 3.9 GB respectively) that enable multi-step reasoning and agentic workflows on local devices, all open-sourced under Apache 2.0.

Rilasciata un'interfaccia per gestire due piccoli modelli (4B e 1.7B) che girano localmente sullo smartphone. Il 4B funziona bene su telefoni di fascia alta; il 1.7B ha problemi di stabilità con il reasoning, in fase di miglioramento con fine-tuning approfondito usando 130k esempi e distillazione da un teacher 32B.

This tweet explains how to use dlt to collect traces and replace a 744B parameter agent with a much smaller 1.7B model in four steps.

QLLM introduces a novel architecture without transformer or mamba, achieving O(1) inference and no KV cache. A 100M parameter model is released as a proof of concept, with open-source code.

A 117M parameter Silia model was trained on an H100 GPU in 5 hours using 82M tokens. The model is severely under-trained, but comparisons with nanoGPT are provided.

This paper proposes replacing the proprietary GPT-3.5 metadata extractor in Multi-Meta-RAG with a lightweight, deterministic probe trained on hidden states of a small open-source model. The probe achieves 90.9% accuracy, outperforming GPT-3.5 (80.9%) and a substring baseline (88.0%), while avoiding allow-list drift and API costs.

Ternlight is a 7 MB embedding model that runs in the browser via WASM, enabling fast semantic search without any API calls or servers. It ships as an npm package and can be used in three lines of code.

SupraLabs releases Supra-Router-51M, a tiny 51M parameter model for routing prompts to appropriate larger or smaller models, enabling low-latency orchestration. A companion dataset is also released.

Competence Gate is a LoRA adapter for Qwen3.5-4B that uses internal confidence signals to gate tool use (answer, web search, local retrieval), improving error detection and privacy.

A user announced they trained a custom 78M parameter AI architecture from scratch using 1B tokens on an 8x3090 rig over 12 hours, claiming the model can now speak.

A user praises the Gemma 4 e2b model for its speed and output quality on low-end hardware, comparing it favorably to ChatGPT 3.5 and 4, and asks for recommendations on other small models that work well on older computers.

A new AI research paper describes a tiny model that acts as a manager to route tasks to larger models, outperforming frontier models like ChatGPT, Gemini, and Claude on a hard coding benchmark by orchestrating a team of models instead of relying on a single one.

A 5M-parameter model outperforms frontier LLMs on hard logical puzzles at a fraction of the inference cost by using continuous latent space test-time compute.

This blog post explores using reinforcement learning to train a small model to detect writing styles of different LLMs, providing code and traces.

Meta FAIR's latest paper proposes the Autodata method, which uses an intelligent data scientist Agent to autonomously generate and optimize high-quality data, enabling a 4B small model to defeat a 397B large model on legal reasoning tasks. This indicates that data quality can bridge the gap in parameter count, providing new insights for data pipelines and scaling.

LiquidAI releases LFM2.5-230M, a 230M parameter language model designed to run on limited hardware, with support for transformers, vLLM, and SGLang.

Liquid AI releases LFM2.5-230M, a lightweight foundation model that runs on devices from cloud GPUs to CPUs and Raspberry Pi, with strong performance on tool use and data extraction tasks.