BrickBench: Evaluating Agentic Brick Design
Summary
We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design, where an agent must produce a physically buildable assembly from a discrete parts library while satisfying semantic and design criteria. The paper also introduces BrickAgent, an environment for coding agents, and finds leading agents meet physical requirements but still fall short of human designs.
View Cached Full Text
Cached at: 10/09/26, 06:19 AM
Paper page - BrickBench: Evaluating Agentic Brick Design
Source: https://huggingface.co/papers/2610.12452 Published on Oct 8
·
Submitted byhttps://huggingface.co/kulits
Kulitson Oct 9
Abstract
WeproposeBrickBench,abenchmarkforagentictext-conditionedLEGO-setdesign.Givenaprompt,anagentistaskedwithproducinganassemblythatnotonlysatisfiessemanticanddesigncriteria,butthatcanalsobephysicallybuilt.Todoso,itmustselectpartsfromadiscretelibraryandreasonjointlyaboutlocalandglobalconstraints.Wescorevalidity,alignment,anddesignacrossthreesettingsthatvaryinscaleandpartavailability.WeprovideBrickAgent,anenvironmentforcodingagentstoconstruct,inspect,andvalidatetheirdesigns.Wefindthatleadingagentslargelysatisfyverifiablephysicalandsemanticrequirements,butfallshortofhumandesigns.Wereleaseourbenchmarkandenvironmentathttp://www.brickben.ch
View arXiv pageView PDFProject pageGitHub1Add to collection
Get this paper in your agent:
hf papers read 2610\.12452
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.12452 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.12452 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.12452 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Can Agents Design Libraries for Agents?
The paper introduces LibraryDesignBench, a two-phase benchmark that has one AI agent write software libraries from underspecified requirements, then scores each library by how effectively other agent implementers use it to solve programming problems in minimal code. The evaluation metric combines squared pass rate with solution simplicity relative to a production-library reference.
AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation
AJ-Bench introduces a benchmark to evaluate Agent-as-a-Judge systems that interact with environments to verify agent behaviors across 155 tasks in search, data systems, and GUI domains.
AgBench: Agentic AI Benchmarks for Personal AI Devices
AgBench introduces a benchmark suite and open artifacts for evaluating agentic AI on personal devices, comparing local, hybrid, and cloud execution across task success, latency, API cost, and data exposure using over 162 million data points. The study finds no single architecture dominates, with local execution reducing cloud cost and privacy risk but generally lowering task success.
Embodied-BenchClaw: An Autonomous Multi-Agent System for Embodied Spatial Intelligence Benchmark Construction
This paper proposes Embodied-BenchClaw, an autonomous multi-agent system that automatically constructs embodied spatial intelligence benchmarks from user intent through a five-stage pipeline with process quality control and an extensible Skill Library.
AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
This paper introduces AgentCollabBench, a diagnostic benchmark for multi-agent systems that evaluates behavioral risks like instruction decay and context leakage across four major LLMs. It argues that communication topology is a critical factor in multi-agent reliability, often overshadowing raw model capability.