LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures

Hugging Face Daily Papers Papers
llm-agents benchmark 3d-generation structural-design cad embodied-ai physical-reasoning

Summary

该论文提出 LMBuild，一个用于评估 LLM 智能体生成可建造、功能性结构能力的基准，包含交互式装配环境、基于 CAD 数据集构建的评测集以及涵盖结构可靠性、功能可供性和物理可实现性的评估框架，在 30 个系统上的评测揭示了功能推理仍是前沿模型的主要瓶颈。

LLM-based agents are increasingly capable of generating complex 3D structures, with the potential to reshape how objects are designed and realized in the physical world. Yet, producing elegant geometry is fundamentally different from producing objects that can be built and perform their intended functions. Existing evaluations largely focus on geometric quality while overlooking physical realizability. We introduce LMBuild, a benchmark for evaluating LLM agents on generating buildable and functional structures. LMBuild represents generated objects as assembled structures comprising part decompositions, joints, materials, and sequences. To support reproducible evaluation, we provide a unified framework consisting of: (1) an interactive environment in which agents can use tools to retrieve, create, and place components to construct objects; (2) a curated benchmark that repurposes established CAD datasets and augments them with knowledge from Wikipedia; and (3) a evaluation framework covering structural soundness, functional affordance, design quality, and physical realization. Evaluations across 30 systems reveal several intriguing findings: (a) Soundness and alignment are no longer the primary bottlenecks for frontier closed-source models, while functional affordance and physical operability remain substantially more challenging; (b) stronger models more effectively create new components, whereas weaker models tend to rely on retrieval; and (c) providing functional specifications substantially improves part completeness, kinematics, and physical operability. These results show that generating real-world structures requires deeper reasoning about functional affordances, mechanics, and designing and creating novel components. We expect LMBuild to provide a foundation for measuring progress and incentivizing research toward agents that generate buildable and functional structures.
Original Article
View Cached Full Text

Cached at: 10/06/26, 04:57 PM

Paper page - LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures

Source: https://huggingface.co/papers/2610.04292 Authors:

,

,

,

,

,

,

,

,

,

Abstract

LLM-basedagentsareincreasinglycapableofgeneratingcomplex3Dstructures,withthepotentialtoreshapehowobjectsaredesignedandrealizedinthephysicalworld.Yet,producingelegantgeometryisfundamentallydifferentfromproducingobjectsthatcanbebuiltandperformtheirintendedfunctions.Existingevaluationslargelyfocusongeometricqualitywhileoverlookingphysicalrealizability.WeintroduceLMBuild,abenchmarkforevaluatingLLMagentsongeneratingbuildableandfunctionalstructures.LMBuildrepresentsgeneratedobjectsasassembledstructurescomprisingpartdecompositions,joints,materials,andsequences.Tosupportreproducibleevaluation,weprovideaunifiedframeworkconsistingof:(1)aninteractiveenvironmentinwhichagentscanusetoolstoretrieve,create,andplacecomponentstoconstructobjects;(2)acuratedbenchmarkthatrepurposesestablishedCADdatasetsandaugmentsthemwithknowledgefromWikipedia;and(3)aevaluationframeworkcoveringstructuralsoundness,functionalaffordance,designquality,andphysicalrealization.Evaluationsacross30systemsrevealseveralintriguingfindings:(a)Soundnessandalignmentarenolongertheprimarybottlenecksforfrontierclosed-sourcemodels,whilefunctionalaffordanceandphysicaloperabilityremainsubstantiallymorechallenging;(b)strongermodelsmoreeffectivelycreatenewcomponents,whereasweakermodelstendtorelyonretrieval;and(c)providingfunctionalspecificationssubstantiallyimprovespartcompleteness,kinematics,andphysicaloperability.Theseresultsshowthatgeneratingreal-worldstructuresrequiresdeeperreasoningaboutfunctionalaffordances,mechanics,anddesigningandcreatingnovelcomponents.WeexpectLMBuildtoprovideafoundationformeasuringprogressandincentivizingresearchtowardagentsthatgeneratebuildableandfunctionalstructures.

View arXiv pageView PDFProject pageGitHub1Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.04292 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.04292 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.04292 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

Words to World: Benchmarking LLMs in Interactive Physical Environments

Hugging Face Daily Papers

该论文提出了一个用于评估大语言模型在闭环物理交互中表现的基准，模型需通过连接机械臂、摄像头和传感器的系统在真实环境中执行任务并根据物理反馈调整行动。实验揭示了LLM在物理行动中的常见失败模式，如幻觉状态变化、糟糕的异常恢复和不稳定的长程规划。

Competence-Aware Orchestration for LLM-Based Multi-Agent Systems

arXiv cs.AI

该论文提出了一种两阶段的 competence-aware 框架，用于 LLM 多智能体系统的自动编排，根据模型在当前查询上的能力动态调整角色激活与通信图，在六个基准上平均达到 93.81%，同时在高能力骨干上降低 36.26% 的 token 成本。

Foundation Models for Automatic CAD Generation

arXiv cs.AI

This paper presents a comprehensive empirical study on using foundation models (LLMs and VLMs) for automatic CAD generation from natural language, introducing the LLMForge framework with two critique regimes (IterTracer and IterVision) and evaluating seven models on a benchmark of 97 engineering design problems.