edge-deployment

RobustMAD introduces a benchmark to evaluate the real-world robustness of multimodal small language models for deployable industrial anomaly detection assistants. It reveals critical failure modes and provides guidance for next-generation systems.

This paper systematically evaluates component-wise quantization of small vision-language models on Jetson edge devices, finding that model architecture (MoE vs dense) significantly affects quantization sensitivity and that quantization errors are largely additive except along modality-alignment paths.

This paper investigates the cross-domain generalization failure of lightweight ML models for IIoT intrusion detection, finding they rely on coarse port features and that adversarial robustness does not correlate with cross-network performance.

This paper benchmarks 17 compact language models (1B-8B parameters) as generators in Russian-language RAG systems under CPU-only inference, finding that Qwen-family models offer strong quality-latency tradeoffs for private, GPU-free deployment.

A systematic study of compressing recursive reasoning models for edge hardware finds that aggressive quantization destroys global reasoning while preserving local prediction. The paper introduces per-channel calibrated INT4 to recover reasoning ability and provides deployment recipes fitting 8 MB SoC and 4 MB MCU targets.

A tweet thread explains four AI agent deployment strategies: batch, stream, real-time, and edge, with detailed descriptions and use cases.

A benchmark study comparing traditional machine learning methods (Random Forest, XGBoost, SVM, Logistic Regression) against lightweight transformer variants (DistilBERT, TinyBERT, MobileBERT) for on-device fault detection across three public datasets. Traditional ML offers competitive accuracy at far smaller resource footprints, while TinyBERT-4L is the most deployment-friendly transformer.

Baidu's PaddlePaddle released PP-OCRv6, an OCR model supporting 48+ languages with tiny (1.5M), small (7.7M), and medium (34.5M) sizes, optimized for edge deployment and handwritten/printed/industrial/screen/card text.

Researchers from AMD propose Recover-LoRA, a method that uses low-rank adaptation with knowledge distillation on synthetic data to recover accuracy lost from aggressive 2-bit quantization of LLMs, achieving 80–95% accuracy recovery on 9 of 12 benchmarks for Qwen3-4B using only 10k synthetic samples.

Researchers from UiT and University of Oslo propose a differentiable NAS framework that jointly optimizes architectural configurations and mixed-precision quantization for LLM compression, achieving up to 1.4× faster inference or 6% higher accuracy across seven reasoning tasks compared to sequential NAS-then-quantization baselines.

RightNow-Arabic-0.5B-Turbo is an open 518M-parameter Arabic-specialized language model built on Qwen2.5-0.5B through vocabulary injection and continued pretraining, achieving competitive performance on Arabic benchmarks while being deployable on edge devices with quantization.

MiniCPM5-1B is fully open-sourced with weights, training data, and deployment code; it achieves top scores among sub-2B models and runs on edge devices.

Thinking Machines Lab and OpenBMB released MiniCPM-o 4.5, a 9B full-duplex omnimodal model with the Omni-Flow framework that enables continuous, time-aligned real-time video and voice interaction, surpassing previous models and available as open source.

OpenBMB releases MiniCPM-V 4.6, a 1.3B-parameter multimodal LLM with 262k context and significantly reduced visual encoding FLOPs, achieving strong benchmark performance and broad inference framework support.

A highly efficient AI model architecture using ternary weights (-1, 0, 1) that achieves competitive performance while requiring only 1.58 bits per parameter, enabling deployment on extremely constrained devices.

NXP and Hugging Face demonstrate techniques for deploying Vision-Language-Action (VLA) models on embedded robotic platforms, covering dataset recording best practices, VLA fine-tuning, and on-device optimizations including quantization and asynchronous inference scheduling for the i.MX 95 processor.

This paper introduces Tequila, a trapping-free quantization method for Large Language Models that improves ternary quantization accuracy and inference speed by repurposing deadzone-trapped weights as dynamic biases.