RF-DETR: Neural Architecture Search for Real-Time Detection Transformers

Papers with Code Trending Papers
detection-transformer neural-architecture-search real-time object-detection weight-sharing roboflow arxiv

Summary

RF-DETR introduces a lightweight detection transformer that uses weight-sharing neural architecture search to achieve state-of-the-art real-time object detection, outperforming prior methods on COCO and Roboflow100-VL while running up to 20x faster.

Open-vocabulary detectors achieve impressive performance on COCO, but often fail to generalize to real-world datasets with out-of-distribution classes not typically found in their pre-training. Rather than simply fine-tuning a heavy-weight vision-language model (VLM) for new domains, we introduce RF-DETR, a light-weight specialist detection transformer that discovers accuracy-latency Pareto curves for any target dataset with weight-sharing neural architecture search (NAS). Our approach fine-tunes a pre-trained base network on a target dataset and evaluates thousands of network configurations with different accuracy-latency tradeoffs without re-training. Further, we revisit the "tunable knobs" for NAS to improve the transferability of DETRs to diverse target domains. Notably, RF-DETR significantly improves on prior state-of-the-art real-time methods on COCO and Roboflow100-VL. RF-DETR (nano) achieves 48.0 AP on COCO, beating D-FINE (nano) by 5.3 AP at similar latency, and RF-DETR (2x-large) outperforms GroundingDINO (tiny) by 1.2 AP on Roboflow100-VL while running 20x as fast. To the best of our knowledge, RF-DETR (2x-large) is the first real-time detector to surpass 60 AP on COCO. Our code is at https://github.com/roboflow/rf-detr
Original Article
View Cached Full Text

Cached at: 05/23/26, 06:25 PM

Paper page - RF-DETR: Neural Architecture Search for Real-Time Detection Transformers

Source: https://huggingface.co/papers/2511.09554 Published on Nov 12, 2025

Abstract

RF-DETR, a light-weight detection transformer, uses weight-sharing NAS to optimize accuracy and latency for real-time detection across diverse datasets.

Open-vocabulary detectorsachieve impressive performance onCOCO, but often fail to generalize to real-world datasets with out-of-distribution classes not typically found in their pre-training. Rather than simply fine-tuning a heavy-weightvision-language model(VLM) for new domains, we introduce RF-DETR, a light-weight specialistdetection transformerthat discovers accuracy-latency Pareto curves for any target dataset withweight-sharing neural architecture search(NAS). Ourapproach fine-tunes a pre-trained base network on a target dataset and evaluates thousands of network configurations with different accuracy-latency tradeoffs without re-training. Further, we revisit the “tunable knobs” forNASto improve the transferability of DETRs to diverse target domains. Notably, RF-DETR significantly improves on prior state-of-the-artreal-time methodsonCOCOandRoboflow100-VL. RF-DETR (nano) achieves 48.0APonCOCO, beating D-FINE (nano) by 5.3APat similar latency, and RF-DETR (2x-large) outperforms GroundingDINO (tiny) by 1.2APonRoboflow100-VLwhile running 20x as fast. To the best of our knowledge, RF-DETR (2x-large) is the first real-time detector to surpass 60APonCOCO. Our code is at https://github.com/roboflow/rf-detr

View arXiv pageView PDFProject pageGitHub7.27kAdd to collection

Get this paper in your agent:

hf papers read 2511\.09554

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper36

#### stevenbucaille/rf-detr-small Object Detection• 32.1M• UpdatedApr 14 • 78 • 1 #### Roboflow/rf-detr-medium Object Detection• 33.7M• Updated3 days ago • 116 • 1 #### Roboflow/rf-detr-seg-xxlarge Image Segmentation• 38.6M• Updated3 days ago • 312 • 1 #### Roboflow/rf-detr-segmentation Image Segmentation• 34.2M• Updated3 days ago • 31 • 1 Browse 36 models citing this paper## Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2511.09554 in a dataset README.md to link it from this page.

Spaces citing this paper4

Collections including this paper4

Similar Articles

@Phoenixyin13: In the world of object detection, there have always been two schools: the YOLO school, the traditional powerhouse, following the principle that speed is the ultimate weapon. Extremely fast, it dominates industries, drones, and surveillance cameras. The Transformer school, the academic aristocrat, highly intelligent with superior accuracy, but due to massive computational consumption, it was like a delicate Lin Daiyu in the past, unable to run in scenarios requiring real-time response...

X AI KOLs Timeline

The RF-DETR model proposed at ICLR2026 combines Transformer's high accuracy with real-time performance, achieving high scores in 100 real-world scenarios and offering sizes from Nano to 2XL, potentially replacing YOLO in real-time detection.

Lightweight Transformer Models for On-Device Fault Detection: A Benchmark Study on Resource-Constrained Deployment

arXiv cs.LG

A benchmark study comparing traditional machine learning methods (Random Forest, XGBoost, SVM, Logistic Regression) against lightweight transformer variants (DistilBERT, TinyBERT, MobileBERT) for on-device fault detection across three public datasets. Traditional ML offers competitive accuracy at far smaller resource footprints, while TinyBERT-4L is the most deployment-friendly transformer.

On-Device Neural Architecture Search

arXiv cs.LG

Proposes a lightweight neural architecture search performed directly on the deployment device for near-sensor computing, validated on sEMG sign language and fault diagnosis datasets, achieving improved accuracy and reduced RAM occupancy.