SeededGrasp: Language-Guided Grasping in Complex Scenes with Multiple Embodiments

Hugging Face Daily Papers Papers
robotic-grasping language-guided multi-embodiment vision-language-model grasp-generation dataset

Summary

SeededGrasp proposes a data-efficient framework that uses a vision-language model to predict a seed point for a lightweight grasp generator, enabling language-guided grasping in complex scenes with multiple robot embodiments. The method outperforms baselines with 72% simulation and 78% real-world success, and includes a new large-scale multi-embodiment grasping dataset.

Practical robotic grasping in complex scenes requires both 3D spatial reasoning and alignment with task-specific requirements. Vision-language models (VLMs) offer a natural way to specify these requirements using language, but existing approaches either use a VLM to predict the grasp directly with limited spatial awareness, or train the VLM together with the grasping model, which requires significantly more data and compute. These limitations impede performance and have prevented scaling to multiple embodiments in complex scenes. We address this by proposing SeededGrasp, a novel data-efficient framework that enables a VLM to predict a seed point to be used as conditioning for a subsequent lightweight grasp-generation model. Our architecture decouples high-level semantic reasoning from low-level geometric execution, enabling multi-embodiment support while bypassing the need for expensive end-to-end training. To enable training such models, we release the first multi-embodiment tabletop grasping dataset comprising over 2.5M grasps in cluttered scenes. Experimental results demonstrate that our approach outperforms existing baselines, achieving 72% success in simulation and 78% in real-world grasping experiments. See our project site for data and code: https://uoft-isl.github.io/seeded-grasp/
Original Article
View Cached Full Text

Cached at: 07/24/26, 05:09 AM

Paper page - SeededGrasp: Language-Guided Grasping in Complex Scenes with Multiple Embodiments

Source: https://huggingface.co/papers/2607.20207

Abstract

Practicalroboticgraspingincomplexscenesrequiresboth3Dspatialreasoningandalignmentwithtask-specificrequirements.Vision-languagemodels(VLMs)offeranaturalwaytospecifytheserequirementsusinglanguage,butexistingapproacheseitheruseaVLMtopredictthegraspdirectlywithlimitedspatialawareness,ortraintheVLMtogetherwiththegraspingmodel,whichrequiressignificantlymoredataandcompute.Theselimitationsimpedeperformanceandhavepreventedscalingtomultipleembodimentsincomplexscenes.WeaddressthisbyproposingSeededGrasp,anoveldata-efficientframeworkthatenablesaVLMtopredictaseedpointtobeusedasconditioningforasubsequentlightweightgrasp-generationmodel.Ourarchitecturedecoupleshigh-levelsemanticreasoningfromlow-levelgeometricexecution,enablingmulti-embodimentsupportwhilebypassingtheneedforexpensiveend-to-endtraining.Toenabletrainingsuchmodels,wereleasethefirstmulti-embodimenttabletopgraspingdatasetcomprisingover2.5Mgraspsinclutteredscenes.Experimentalresultsdemonstratethatourapproachoutperformsexistingbaselines,achieving72%successinsimulationand78%inreal-worldgraspingexperiments.Seeourprojectsitefordataandcode:https://uoft-isl.github.io/seeded-grasp/

View arXiv pageView PDFProject pageGitHub2Add to collection

Get this paper in your agent:

hf papers read 2607\.20207

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.20207 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.20207 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.20207 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

Human Universal Grasping

Hugging Face Daily Papers

A flow-matching model generates diverse human grasps from RGB-D images, enabling zero-shot robotic grasping with improved performance over existing methods. The model, trained on a large egocentric dataset, significantly outperforms state-of-the-art baselines on a new benchmark.

Domain randomization and generative models for robotic grasping

OpenAI Blog

Researchers explore a data generation pipeline using domain randomization and procedurally generated objects to train a deep neural network for robotic grasp planning. The proposed autoregressive model achieves >90% success on unseen objects in simulation and 80% in the real world, despite being trained only on random simulated objects.