on-device

Yubi is a lightweight 6 MB Mac app that lets users dictate text at the cursor via voice while keeping all audio processing local to the device, offering a free tier with email sign-in and no credit card required.

ClementDelangue announces that decision models can now run on-device in llama.cpp, highlighting a free, fast, and private workflow using the Kev-4B GGUF release via `llama serve`.

kyutai_labs released a technical blog on training PocketTTS, a 100M parameter on-device text-to-speech model using the drifting one-step generative objective from Deng et al., achieving under 1% WER with high quality and voice cloning — reportedly the first speech and autoregressive model trained this way.

AgBench introduces a benchmark suite and open artifacts for evaluating agentic AI on personal devices, comparing local, hybrid, and cloud execution across task success, latency, API cost, and data exposure using over 162 million data points. The study finds no single architecture dominates, with local execution reducing cloud cost and privacy risk but generally lowering task success.

webAI introduces TwIL-LM3-Pro, a 3.6B-parameter open-source reasoning model that runs locally on everyday computers with quantized builds, claiming top small-model scores on formal logic, SVAMP, MuSR, and BIG-Bench Hard. They also tease Meridian, an upcoming family of frontier-class on-device models.

The developer released LokalBot 0.9.2, a free, open-source Mac meeting notetaker that runs entirely on-device with a 6.8 GB stack of five local models (Qwen3-ASR for transcription, Nemotron 3 for speaker diarization, Qwen3.5 4B for notes, Harrier for embeddings, and LFM2.5 for autocomplete), along with benchmark improvements on an M4 Max.

Twin is a free, open-source, local-first AI buddy that floats on your Mac desktop — catching you up on texts, tracking your calendar, responding by voice, and nudging you before you forget, all stored in a local DuckDB file with no cloud or accounts. Users bring their own Claude, GPT, Gemini, or Grok API keys.

Meta launched Muse, an AI agent app with 3.4 million downloads that raises privacy concerns over cloud-based data handling, while the author is developing an on-device alternative to keep data local and secure.

PrismML showcases its tiny 1-bit Bonsai LLM at Qualcomm's Snapdragon Summit, which can run locally on smart glasses powered by the Snapdragon AR1 Gen 1 Platform, enabling real-time vision and language processing.

Wasmer introduces a new SDK for Swift that allows running Python, Node.js, FFmpeg, and Clang on iOS and macOS entirely on-device with near-native performance, without JIT compilation.

This research paper extends FunctionGemma 270M for practical on-device Android workflows by introducing a synthetic dataset and fine-tuning the model, achieving improved accuracy for function calling while balancing performance and coverage.

This paper reports on training a language model end-to-end in Rust, detailing failures in Rust ML frameworks like Candle and Burn, and proposing verification methods, with the conclusion that Rust is currently better suited for model serving than training.

Walkie is an on-device app that offers dictation, meeting transcription with speaker identification, and text-to-speech in 950+ voices, available for free on Mac, Windows, Linux, and iPhone.

Cactus Needle 3 is a small, sliceable foundation model for automation tasks that runs on-device, achieving performance comparable to larger models on function calling and structured extraction.

The post introduces the open-source Audio8 models, which enable on-device speech recognition and synthesis on phones and PCs, including an offline transcription version for iPhone.

Jexxa is a high-speed dictation tool for macOS that runs entirely on-device, ensuring privacy and fast performance with no data uploads.

This paper evaluates on-device language models for privacy-preserving stress prediction using multimodal data on mobile devices, finding that lightweight models achieve low latency and predictable resource usage while highlighting practical constraints for mobile mental health applications.

Oats is a free, open-source, on-device AI meeting note-taking tool available on macOS and Windows with features like transcription and speaker recognition.

Desert Ant Labs, a European AI lab, launches a suite of small, specialized on-device models for audio, vision, and text, offering fast inference and privacy benefits by running locally on devices.

Speechmark is a private macOS app that records, transcribes, and summarizes meetings entirely on-device, featuring speaker-attributed transcripts and integrations with Claude Code via MCP server.