I implemented the YOLO26n model inference from scratch using ARM64 Assembly Language (No framework) [P]
Summary
The author implemented YOLO26n model inference from scratch using ARM64 assembly language without any external frameworks, demonstrating low-level AI inference techniques.
Similar Articles
QLLM, no transformer, no mamba and new noval architecture with O(1) inference is finally out as model
QLLM introduces a novel architecture without transformer or mamba, achieving O(1) inference and no KV cache. A 100M parameter model is released as a proof of concept, with open-source code.
CPU-only inference on a Celeron N5095 SBC: 6 models from 0.6B to 8B, benchmarked
This post benchmarks six AI models from 0.6B to 8B parameters running CPU-only inference on a Celeron N5095 single-board computer, providing performance comparisons.
Why we write our own C and C++ inference engines
LocalAI explains why it writes its own C/C++ inference backends, showing that its vllm.cpp port achieves comparable or better throughput and far smaller footprint than vLLM, with benchmarks across multiple models and hardware.
LiquidAI/LFM2.5-2.6B
Liquid AI released LFM2.5-2.6B, a 2.6B-parameter hybrid model optimized for on-device deployment with 128K context, agentic post-training, and fast inference (220 tok/s on Apple M5 Max) under 2.5GB memory.
Developing open source LLM from ground up from pretrain - rlhf(PPO/GRPO)
A developer shares progress on training a 7B parameter open source LLM from scratch using a DeepSeek architecture optimized for low VRAM, with the goal of democratizing AI development and eventually surpassing large proprietary models.