@mervenoyann: day 2 findings on this pipeline > it works, got map@50=0.8028 on road sign detection against human annotations, with on…

X AI KOLs Timeline News
multi-modal vlm road-sign-detection pipeline data-labeling model-training

Summary

Merve (@mervenoyann) shares day two findings of a pipeline using multiple small VLMs as judges for road sign detection, achieving map@50=0.8028 with only 1.3k examples. The thread compares model rejection rates and discusses dataset shrinking, super-specific prompts, and plans to generalize the library.

day 2 findings on this pipeline > it works, got map@50=0.8028 on road sign detection against human annotations, with only 1.3k examples see results below > Liquid rejects way more than Gemma-4 (530 vs 306 in hard document parsing, 1022 vs 116 in easy road sign detection, tbh it's smaller and more prone to hallucination when I vibe check) > in some cases (see document media parsing examples below) trained RF-DETR outperforms Qwen annotations it was trained on which is super cool, sometimes judges introduce bboxes (and I don't remove them) it's a win? > multiple VLMs as judges will shrink your dataset depending on the difficulty of the problem, sometimes taking only one "correct" from a judge is enough. since you are training small models it's better to kickoff training for consensus and single correct verdict separately > use super-specific prompts of what you want and don't want in labelling and judging especially if your labels as words could mean many things next up: make this library leaner to generalize better to be problem-agnostic, try again on segmentation, actually use Gemma for orchestration
Original Article
View Cached Full Text

Cached at: 06/18/26, 06:09 AM

day 2 findings on this pipeline

it works, got map@50=0.8028 on road sign detection against human annotations, with only 1.3k examples see results below

Liquid rejects way more than Gemma-4 (530 vs 306 in hard document parsing, 1022 vs 116 in easy road sign detection, tbh it’s smaller and more prone to hallucination when I vibe check) in some cases (see document media parsing examples below) trained RF-DETR outperforms Qwen annotations it was trained on which is super cool, sometimes judges introduce bboxes (and I don’t remove them) it’s a win? multiple VLMs as judges will shrink your dataset depending on the difficulty of the problem, sometimes taking only one “correct” from a judge is enough. since you are training small models it’s better to kickoff training for consensus and single correct verdict separately

use super-specific prompts of what you want and don’t want in labelling and judging especially if your labels as words could mean many things

next up: make this library leaner to generalize better to be problem-agnostic, try again on segmentation, actually use Gemma for orchestration

all my artifacts are here https://huggingface.co/collections/merve/vision-intern… labelled datasets, judged datasets, trained models, parts of pipelines and more

also shoutout to @huggingface infra I use Buckets, Jobs, Dataset Viewer and more heavily

@DataScienceHarp @skalskip92 @maximelabonne you might be interested in this ^

Similar Articles