Are We Underestimating Small Edge AI Models?[D]

Reddit r/MachineLearning News

Summary

A developer argues that the edge AI community overlooks small, specialized models that can run locally on devices like smartphones, using a self-built offline Morse code recognition feature as an example. The project uses a sub-5 MB AI model with TensorFlow/Keras and LiteRT, and the entire pipeline from data generation to mobile integration was custom-built.

A lot of recent discussion around Edge AI focuses on running increasingly larger local LLMs. Meanwhile modern smartphones already have enough compute for many practical computer vision tasks that don't require massive models at all. I recently built and released an Android feature that performs offline recognition of handwritten and printed Morse code from images and live camera frames. The final solution combines lightweight ML and computer vision techniques running entirely on-device. The AI module is under 5 MB, works fully offline, and runs on Android devices using LiteRT for inference. What made the project particularly interesting was that the entire ML pipeline was built from scratch: data collection, synthetic dataset generation, annotation, model training, evaluation, mobile optimization, and Android integration. Training was performed on a personal GPU workstation using TensorFlow/Keras, while annotation and dataset preparation relied on Label Studio and custom data-generation tools. While the problem itself is fairly niche, the project made me wonder whether we are overlooking a large class of small, highly specialized models that can solve practical tasks locally without requiring cloud infrastructure or large foundation models. What practical Edge AI applications do you think are currently underexplored? Demo video showing the feature running entirely on-device: • Downloading the optional AI module • Real-time camera recognition • Image recognition • Module removal [https://youtube.com/shorts/Y2qOK0N1Bvk](https://youtube.com/shorts/Y2qOK0N1Bvk)
Original Article

Similar Articles

The small open weight models are scarier in AI development

Reddit r/LocalLLaMA

The article argues that small open-weight models running locally on personal devices could disrupt major AI companies by offering private, fast, and free AI capabilities, potentially causing the AI bubble to burst.

How I see the future for local AI

Reddit r/LocalLLaMA

The author predicts that small specialized AI models in the 10B-40B parameter range will disrupt large AI companies by enabling efficient, on-device intelligence tailored to specific domains.