on-device-ai

Stable Diffusion XL (SDXL) can now run locally in the browser using WebGPU, enabling high-quality AI image generation directly on-device with open-source code.

RepSquad is a workout app that uses on-device AI to count reps and score form in real time, enabling social challenges with friends and family for motivation.

Pixi launches an iOS messaging app that uses on-device AI to send interactive AR characters that respond to real-world surroundings and user emotions, aiming to make conversations more playful and present.

A developer created an Android app that runs AI models locally, supporting GGUF and LiteRT formats with multiple ways to add models.

This paper presents the first end-to-end RAG pipeline running entirely on a mobile NPU (Qualcomm Hexagon on Snapdragon X Elite), achieving up to 18x faster LLM prefilling and 4x lower energy vs. CPU, with no quality regression.

Proposes a decentralized information discovery system using on-device embedding models and peer-to-peer gossip, eliminating the need for central indexes like search engines.

Apple announced a major overhaul of its Apple Intelligence platform, revealing a new AI architecture built on foundation models co-developed with Google using Gemini technologies, enabling multimodal capabilities and privacy-preserving on-device and server processing via Private Cloud Compute.

Google announces the availability of Gemma 4 12B on laptops via Google AI Edge, enabling local, agentic, and multimodal workflows with tools like AI Edge Gallery and Eloquent.

A developer argues that the edge AI community overlooks small, specialized models that can run locally on devices like smartphones, using a self-built offline Morse code recognition feature as an example. The project uses a sub-5 MB AI model with TensorFlow/Keras and LiteRT, and the entire pipeline from data generation to mobile integration was custom-built.

A security researcher discovered that Meta's Stella companion app for smart glasses (v273.0.0.21) contains a fully assembled, functional facial recognition pipeline—including three on-device models, a biometric embedding database, and a notification system—that is dormant on stock accounts but operable when invoked directly. The pipeline can detect faces, generate 2048-dimension embeddings, and fire 'Person Recognized' notifications, raising significant privacy concerns even though Meta has not been observed activating it for regular users.

A tool or app that enables users to run large local AI models directly from their iPhone, bringing on-device LLM inference to iOS.

Unsloth, a popular LLM fine-tuning library, announces upcoming support for Apple Silicon devices, expanding its optimization capabilities beyond NVIDIA GPUs.

A commentary arguing that the AI competition is shifting from model quality to hardware placement and infrastructure, highlighting Microsoft's Project Solara, NVIDIA's RTX Spark, and ByteDance's custom CPU efforts as signs that agentic workloads are driving new silicon and deployment strategies.

Google released Gemma 4 12B, an Apache 2.0 open-source multimodal model supporting text, vision, and audio with a 256K context window. The article provides a guide for running it locally on Macs using Ollama, LM Studio, or llama.cpp.

Perplexity unveiled a hybrid local-cloud inference system at Computex 2026 that intelligently routes queries between on-device and cloud models, building on its earlier Personal Computer agent.

The article discusses the shift of AI from data centers to laptops and desktops, driven by Nvidia's new Arm-based processors and Microsoft's AI integration, leading to a major enterprise hardware upgrade cycle.

OpenBMB released UltraData-SFT-2605, a 15M-sample high-quality SFT dataset for fine-tuning AI models like MiniCPM5-1B to run on phones or laptops.

The Chromium team proposes a new Embedding API for the web platform that allows developers to generate vector embeddings on-device using Chrome's AI infrastructure, enabling privacy-preserving semantic search, retrieval-augmented generation, and content clustering while reducing latency and cost.

A developer created a Chrome extension called Dobby that runs Google's Gemma4 (Gemini Nano) locally on PC without needing a GPU, requiring only Chrome and 16GB RAM. The extension provides a simple interface to interact with the model for tasks like spell checking or summarizing.

A weekly roundup of the 10 fastest-growing GitHub repositories, highlighting AI infrastructure tools like code knowledge graphs, agent memory, on-device intelligence, and more.