edge-deployment

A tweet thread explains four AI agent deployment strategies: batch, stream, real-time, and edge, with detailed descriptions and use cases.

A benchmark study comparing traditional machine learning methods (Random Forest, XGBoost, SVM, Logistic Regression) against lightweight transformer variants (DistilBERT, TinyBERT, MobileBERT) for on-device fault detection across three public datasets. Traditional ML offers competitive accuracy at far smaller resource footprints, while TinyBERT-4L is the most deployment-friendly transformer.

Baidu's PaddlePaddle released PP-OCRv6, an OCR model supporting 48+ languages with tiny (1.5M), small (7.7M), and medium (34.5M) sizes, optimized for edge deployment and handwritten/printed/industrial/screen/card text.

Researchers from AMD propose Recover-LoRA, a method that uses low-rank adaptation with knowledge distillation on synthetic data to recover accuracy lost from aggressive 2-bit quantization of LLMs, achieving 80–95% accuracy recovery on 9 of 12 benchmarks for Qwen3-4B using only 10k synthetic samples.

Researchers from UiT and University of Oslo propose a differentiable NAS framework that jointly optimizes architectural configurations and mixed-precision quantization for LLM compression, achieving up to 1.4× faster inference or 6% higher accuracy across seven reasoning tasks compared to sequential NAS-then-quantization baselines.

RightNow-Arabic-0.5B-Turbo is an open 518M-parameter Arabic-specialized language model built on Qwen2.5-0.5B through vocabulary injection and continued pretraining, achieving competitive performance on Arabic benchmarks while being deployable on edge devices with quantization.

MiniCPM5-1B is fully open-sourced with weights, training data, and deployment code; it achieves top scores among sub-2B models and runs on edge devices.

Thinking Machines Lab and OpenBMB released MiniCPM-o 4.5, a 9B full-duplex omnimodal model with the Omni-Flow framework that enables continuous, time-aligned real-time video and voice interaction, surpassing previous models and available as open source.

OpenBMB releases MiniCPM-V 4.6, a 1.3B-parameter multimodal LLM with 262k context and significantly reduced visual encoding FLOPs, achieving strong benchmark performance and broad inference framework support.

A highly efficient AI model architecture using ternary weights (-1, 0, 1) that achieves competitive performance while requiring only 1.58 bits per parameter, enabling deployment on extremely constrained devices.

NXP and Hugging Face demonstrate techniques for deploying Vision-Language-Action (VLA) models on embedded robotic platforms, covering dataset recording best practices, VLA fine-tuning, and on-device optimizations including quantization and asynchronous inference scheduling for the i.MX 95 processor.

This paper introduces Tequila, a trapping-free quantization method for Large Language Models that improves ternary quantization accuracy and inference speed by repurposing deadzone-trapped weights as dynamic biases.