Tag
A user reflects on Microsoft's Phi small model family, noting the last major release was in December 2024 and speculating whether Phi 5 will ever be released.
A Reddit user shares results from testing proven orchestration techniques on small local LLMs, finding that 90% failed but the surviving 10% roughly doubled task completion across models like LFM 1.2B and Gemma 4 26B-A4B.
This paper evaluates how small open-weight vision-language models (Qwen2-VL-2B and SmolVLM) handle realistic image degradations, finding that their verbalized confidence is unreliable while internal token probability provides much better error detection, though both fail under severe low-light conditions.
The author benchmarks local Qwen models of various sizes on a 4GB RTX 3050 Ti laptop GPU within the Bike4Mind workspace, finding the 2B model at Q4_K_M quantization is the sweet spot for fitting in VRAM, achieving 96 tok/s. Smaller models struggle with tool selection, artifact generation requiring multiple models, and RAG embeddings causing model swap overhead.
MiniCache is a program caching framework that reuses computation across similar requests by parameterizing Program-of-Thought programs, using small models for semantic variable extraction and speculative drafting to improve LLM inference efficiency.
A tweet highlights PoolsideAI's unusual openness, praising their release of a small coding model, publication of papers, and full evaluation datasets, setting a standard for transparency in AI.
This post benchmarks six AI models from 0.6B to 8B parameters running CPU-only inference on a Celeron N5095 single-board computer, providing performance comparisons.
Microsoft AI Frontiers open-sources MagenticLite, including the MagenticBrain 14B orchestrator and Fara 1.5 computer-use models, now available on Hugging Face with open weights and the full application stack.
This article argues that vLLM and similar serving frameworks are inefficient for running multiple small AI models on a single GPU due to design limitations. It introduces the SIE open-source inference engine as a solution for serving many models together to reduce costs.
A user tests various small AI models for coding tasks, finding Qwen3.6-27B-NVFP4 to be the best balance of speed and accuracy, and notes poor Java performance in these models.
A.L.F.R.E.D. proposes a system that distills knowledge from large models into small ones and routes simple tasks to the small models, achieving performance of 35B models with 2B models while reducing inference cost by 4x.
The author trained a Qwen3.6-35B-A3B model using reinforcement learning to then RL-train small task-specific Qwen models, and has released everything fully open source.
A thread introducing the Atomic Task Graph (ATG), a DAG-based execution substrate that allows a small 8B model to outperform GPT-4 on complex agent tasks by storing plan structure in a graph instead of linear text, reducing context bloat and enabling localized failure recovery.
A practitioner shares surprising findings from fine-tuning a small open model to be genuinely better in practical use, not just on benchmarks.
A fine-tuning practitioner recounts discovering that a small open model's weakness wasn't intelligence but a people-pleasing 'backbone' that caused it to cave under pressure, and how training to correct that inadvertently broke formatting ability, requiring a additive balancing approach rather than subtraction.
A CPU TTS benchmark compares Kokoro, Supertonic, Inflect-Nano, and Kyutai's Pocket TTS using UTMOS MOS scores, revealing interesting findings about RTF scaling, UTMOS limitations with small vocoders, and undocumented output caps. Pocket TTS offers unique zero-shot voice cloning on CPU.
A discussion on using applications to enhance the effectiveness of smaller AI models on larger tasks, balancing efficiency and performance.
A paper analyzing the Parameter Golf open challenge for training language models under strict size and time constraints, finding that individual techniques rarely improve BPB by more than 1% but collectively achieved a 13.6% reduction.
IFStruct, an instruction-following benchmark for structured output by Liquid AI, now has a leaderboard on Hugging Face, aimed at improving small models for local execution and correct tool use.
ifstruct is an instruction-following benchmark for structured output by Liquid AI, designed to push the field toward better small models that can run locally.