Point density, not architecture, was the bottleneck for a 5-class radar-only object [P]

Reddit r/MachineLearning Papers

Summary

An engineer demonstrates that for a radar-only 5-class object classifier on the RadarScenes dataset, increasing point density from 1 to 5 points per instance roughly doubles macro F1, whereas architectural and feature engineering changes fell within the noise floor. The work highlights how extremely sparse radar points fail to convey size or velocity-spread signatures, causing confusion between classes like two-wheelers and pedestrians.

Hello all, TL;DR: point density, not model architecture, was the real bottleneck for a 5-class radar-only classifier on RadarScenes. Going from 1 to 5 points per instance roughly doubles macro F1 (0.381 → 0.764), while a whole set of architecture and feature changes all landed inside a measured noise floor. Real failure case attached: a stationary two-wheeler misread as a pedestrian. Setup I'm a perception / radar signal processing engineer getting into ML on radar data. Trained a 5-class classifier (car, large_vehicle, two_wheeler, pedestrian, pedestrian_group) on RadarScenes radar point clouds only, no camera or lidar. Per-instance histogram (16 bins) encoding into a 3-layer MLP. Main result: point density is the ceiling Macro F1 goes from 0.381 to 0.764 just by increasing points per instance from 1 to 5. Same trained model, nothing else changed. Why: a single point can't carry a size or velocity-spread signature. large_vehicle's F1 is 0.037 at n=1 vs 0.995 at n=11+. Some classes still work at n=1 (car separates cleanly on RCS/Doppler alone), others don't (two_wheeler and pedestrian collapse to the same near-zero-velocity signature when sparse). Ablation studies Wider/deeper networks, six alternative feature encodings, different bin edges, all landed inside the noise floor I measured with a 6-fold split sensitivity check (same train/val/test proportions, sequences reassigned per fold). Closest thing to an exception: swapping the histogram for explicit per-instance statistics (mean/median/std) actually made things slightly worse (0.658 vs baseline's 0.686), and pedestrian's own F1 fell outside its class-specific noise floor. Data caveats RadarScenes is naturalistically collected, not balanced: common classes get broad coverage, rare ones don't. two_wheeler merges two physically different speed regimes (bicycle vs a much rarer motorized variant). large_vehicle merges large_vehicle/truck/train/bus, RadarScenes' own recommended scheme, mostly forced by data scarcity in the rarer classes. pedestrian_group's own boundary, what counts as a "group", isn't sharply defined in the source labels either. Cross-validation also caught a data quirk: a two-wheeler tracked for hundreds of scans while stopped or moving sideways at a junction can dominate one sequence's vr_compensated distribution near zero. Land that sequence in one split and its two-wheelers read as stationary; land it in another and the rest look like normally moving two-wheelers with different RCS, which explains f1 score variance. [Image 1: validation set confusion matrix] [Image 2: real scene, camera + radar ground truth vs prediction] A real failure case A nearly stationary two-wheeler with a single radar point gets predicted as pedestrian, its compensated velocity is near zero, indistinguishable from someone standing still at that point count. A car in the same scene, also one point, is classified correctly, RCS and Doppler are enough for that class. Full writeup here: https://github.com/brunopinto900/radar-ml-autonomous-driving/blob/main/MLP_Report.md Curious what people would try next, especially for the very sparse cases, alternative representations that preserve more than the histogram encoding, and better ways to confirm an improvement is real rather than just split variance.
Original Article

Similar Articles

I built a mmWave material classification radar

Hacker News Top

A detailed account of building a mmWave radar prototype that uses FMCW, Capon beamforming, and a neural network to classify building materials, with a focus on detecting asbestos in walls.

RADAR: Relative Angular Divergence Across Representations

arXiv cs.LG

RADAR is a geometrically grounded metric that estimates cross-domain transferability in foundation models by analyzing layer-wise angular and distance changes in representations, using KL divergence between within-domain and cross-domain trajectory distributions.

Crowded in B-Space: Calibrating Shared Directions for LoRA Merging

Hugging Face Daily Papers

This paper introduces Pico, a data-free method that improves LoRA adapter merging by separately calibrating the output-side matrix B to reduce interference from shared directions while preserving task-specific information. Pico achieves 3.4–8.3 point accuracy improvements over existing merging methods across math, coding, finance, and medical benchmarks.

(Rant ;)) Make your benchmarks realistic

Reddit r/LocalLLaMA

A community rant urging realistic AI model benchmarks that account for context size, multimodal features, hardware specifics, and parallel processing, rather than just raw speed.