DeformSmith: Physics Harness-Guided Hierarchical Generation of Deformable Assets for Robot Manipulation

Hugging Face Daily Papers Papers
deformable-assets robot-manipulation physics-simulation 3d-generation ai-framework

Summary

DeformSmith is a framework for generating interactive, physically credible deformable assets for robot manipulation from text or images, using physics-guided hierarchical generation to improve quality and plausibility.

Creating deformable assets for robot manipulation requires jointly specifying their geometry, appearance, and physical properties. This is especially challenging for deformable objects, since text and images provide limited evidence about how they deform and respond to contact, yet these responses directly affect their suitability for interaction. Automated generation therefore needs to resolve coupled physical requirements and use interaction evidence to guide construction and refinement. We present DeformSmith, a framework that enables automated generation of interactive, physically credible deformable assets from text or a single image. Through hierarchical agentic construction and a shared physics-grounded harness, it progressively builds, tests, and refines geometry, physical models, material behavior, and robot interaction until the resulting asset is ready for simulation and manipulation. Robot interaction closes the generation loop through manipulation feedback and replayable interaction data. Results show that DeformSmith generates assets with better visual quality and physical plausibility than state-of-the-art baselines, including PhysGen3D, PhysGM, and PhysX-Omni, while supporting the synthesis of data for robotic manipulation of deformable objects. Project page: https://can-lee.github.io/deformsmith-web/
Original Article
View Cached Full Text

Cached at: 09/21/26, 03:18 AM

Paper page - DeformSmith: Physics Harness-Guided Hierarchical Generation of Deformable Assets for Robot Manipulation

Source: https://huggingface.co/papers/2609.18620 Published on Sep 17

·

Submitted byhttps://huggingface.co/Canlee

Can Lion Sep 21

Abstract

Creatingdeformableassetsforrobotmanipulationrequiresjointlyspecifyingtheirgeometry,appearance,andphysicalproperties.Thisisespeciallychallengingfordeformableobjects,sincetextandimagesprovidelimitedevidenceabouthowtheydeformandrespondtocontact,yettheseresponsesdirectlyaffecttheirsuitabilityforinteraction.Automatedgenerationthereforeneedstoresolvecoupledphysicalrequirementsanduseinteractionevidencetoguideconstructionandrefinement.WepresentDeformSmith,aframeworkthatenablesautomatedgenerationofinteractive,physicallycredibledeformableassetsfromtextorasingleimage.Throughhierarchicalagenticconstructionandasharedphysics-groundedharness,itprogressivelybuilds,tests,andrefinesgeometry,physicalmodels,materialbehavior,androbotinteractionuntiltheresultingassetisreadyforsimulationandmanipulation.Robotinteractionclosesthegenerationloopthroughmanipulationfeedbackandreplayableinteractiondata.ResultsshowthatDeformSmithgeneratesassetswithbettervisualqualityandphysicalplausibilitythanstate-of-the-artbaselines,includingPhysGen3D,PhysGM,andPhysX-Omni,whilesupportingthesynthesisofdataforroboticmanipulationofdeformableobjects.Projectpage:https://can-lee.github.io/deformsmith-web/

View arXiv pageView PDFProject pageGitHub2Add to collection

Get this paper in your agent:

hf papers read 2609\.18620

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.18620 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.18620 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.18620 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation

Hugging Face Daily Papers

PhysisForcing is a training framework that enhances embodied video generation for robotic manipulation by enforcing physical consistency through pixel-level trajectory alignment and semantic-level relational alignment losses in a DiT-based architecture, achieving notable improvements on benchmarks.

EgoPhys: Learning Generalizable Physics Models of Deformable Objects from Egocentric Video

Hugging Face Daily Papers

EgoPhys introduces a framework to construct deformable physical digital twins from egocentric RGB video using generalizable priors and a compact codebook, enabling zero-shot generalization to unseen objects without per-spring optimization. The system is demonstrated on a real robot, showing that egocentric human play video can serve as internal world representation for deformable-object planning.