edge-ai

A quiet revolution is making powerful AI models runnable on consumer hardware without expensive GPUs, thanks to breakthroughs in quantization and optimized implementations like llama.cpp's Gemma4 MTP support, democratizing access for hobbyists, small businesses, and edge computing.

The author announces a new blog post on clustering three Jetson Nano Orin Supers for distributed training and inference, continuing a series to help people build small compute clusters with accessible hardware.

A developer argues that the edge AI community overlooks small, specialized models that can run locally on devices like smartphones, using a self-built offline Morse code recognition feature as an example. The project uses a sub-5 MB AI model with TensorFlow/Keras and LiteRT, and the entire pipeline from data generation to mobile integration was custom-built.

Google released Gemma 4 12B, an open-source multimodal AI model under Apache 2.0 that runs locally on laptops with 16GB RAM, targeting enterprise edge deployment.

The author uses the Qwen3.6-35B-A3B model and oMLX tool on the new local machine for daily tasks, finding that both speed and quality far exceed expectations, even outperforming remote LLMs in PA and coding scenarios, demonstrating a significant improvement in on-device AI capabilities.

A deep benchmark of 8 tiny LLMs (135M to 1B parameters) on a $250 Jetson Orin Nano Super across four power modes finds 25W to be Pareto-optimal, with SmolLM2-135M achieving 165.1 tok/s and best efficiency.

NVIDIA announces JetPack 7.2 and NemoClaw support on Jetson, bringing agentic AI capabilities to edge devices like robotics and industrial automation, with performance boosts and new developer tools.

NVIDIA announced the Factory Operations Blueprint (FOX), a reference design for building autonomous factory manager AI agents that integrate real-time data, automate model training, and orchestrate specialized agents, with early adoption by major manufacturers.

Liquid AI releases LFM2.5-8B-A1B, an 8B MoE model with 1.5B active parameters and 128K context, optimized for edge devices.

A 7MB open-source L4 self-driving AI is trained to learn navigation, lane following, and drift recovery from visual and sensor input, designed to run on phones and embedded devices without heavy infrastructure.

NVIDIA has launched the $249 Jetson Orin Nano Super developer kit, an AI computer that runs large models like Llama 3 and Mistral locally, cutting monthly OpenAI costs from $200 to just $22 in electricity.

Demonstrates running a DCGAN with 12.6M int8 quantized parameters on a low-cost RISC-V microcontroller (CH32H417), generating 64x64 cat faces in 26 seconds using pure C inference and quantum entropy sampling.

BitCPM is a new open-source model from ModelBest, Tsinghua, and OpenBMB that uses ternary weights (-1,0,1) to run full-sized AI models on phones.

Developed a custom C++ inference engine for MiniCPM-V 4.6 on Orange Pi AIPro (Ascend 310B NPU), achieving 2x speedup over stock framework by writing optimized AscendC kernels for matmul and causal-conv1d, reaching 5.90 tokens/s.

BitCPM-CANN is the first open-source 1.58-bit ternary LLM trained entirely on Chinese-developed AI infrastructure (Huawei Ascend 910B), offering extreme memory reduction for edge deployment.

General Instinct launches a deployment layer that enables frontier AI models to run on constrained edge hardware like Jetsons and mobile NPUs, helping robotics and physical AI teams achieve low-latency offline inference.

A tweet argues the next AI boom will be compact intelligence on edge devices rather than larger data centers, with Liquid AI supporting the vision of running AI on phones, cars, and everyday devices.

A detailed examination of the real-world challenges faced when updating AI models on edge devices deployed in remote or disconnected environments, covering strategies like connectivity windows, technician visits, mesh propagation, and accepting staleness.

This paper proposes an Edge-AI-driven decentralized task allocation framework for circular smart manufacturing that uses learning-to-rank to align with the ordering-based nature of winner selection. Simulation results show improved delay, deadline adherence, and energy efficiency under high-load and tight-deadline scenarios.

This paper proposes a knowledge-adaptive edge expert agent architecture for ecological monitoring, separating visual perception from reasoning to reduce reliance on cloud resources and enable sustainable on-device AI in remote deployments.