GraphDecisionBench: Benchmarking System One Models on Graph Tasks
Summary
GraphDecisionBench：一个模型无关的基准，用于评估 System One 模型在图结构任务上的决策能力，涵盖 14 种模型-接口配置，揭示了邻接识别准确并不等同于结构正确性等关键发现。
View Cached Full Text
Cached at: 10/12/26, 02:37 AM
Paper page - GraphDecisionBench: Benchmarking System One Models on Graph Tasks
Source: https://huggingface.co/papers/2610.06354
Abstract
Largelanguagemodels(LLMs)areincreasinglyexploredforgraphunderstandinganddecision-making,whileSystemOnemodelssuchasJevselectdirectlyfromsuppliedoptions.However,thecapabilitiesofSystemOnemodelsongraph-relatedtasksremainunclear.WeintroduceGraphDecide,amodel-independentbenchmarkthatcombinesstructuraltaskprofiles,matchedgraph-textinputcontrastsandheuristic-proposalcontrolstodiagnosegraphdecisionperformance.WeevaluateJevandrelatedchoice-basedmodelsalongsidelanguage-modelbaselines,coveringfourteenmodel-interfaceconfigurations.Jev’sresultsillustratethebenchmark’scentraldistinctions:accurateadjacencyrecognitiondoesnotguaranteebroaderstructuralcorrectness,jointgraph-textinputdoesnotconsistentlyimproveprediction,andfeasibleconstructiondoesnotestablishhighsolutionquality.Itstaskcontracts,candidateinterfacesandscoringrulessupportcomparisonacrossnativeselectorsandlanguage-modeladapters.Codeandaggregateresultsareavailableathttps://github.com/VictorYXL/JevGraphBench.
View arXiv pageView PDFGitHubAdd to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.06354 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.06354 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.06354 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Benchmarking System One decision models against trained classifiers and language models for automated decision gates
This paper benchmarks typed 'System One' decision models against supervised classifiers and generative language models for automated decision gates, finding that model rankings depend on label availability, calibration, and cost. It introduces condition-dependent design rules and shows cascaded models can match accuracy at a fraction of the cost.
TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models
TypedBench is a new benchmark for evaluating System One typed decision models on calibration, wording sensitivity, policy adherence, and decision cost under asymmetric cost matrices, comparing a hosted model, an open encoder, and open decoder families on identical items.
Microsoft-Decision-1, our model for fast decision-making
微软发布 Microsoft-Decision-1，一个专为快速决策打分设计的模型，可用于路由、分类、优先级排序与工作流控制，基于 Qwen3.5-9B 后训练，在 36 项基准测试中取得最高准确率，延迟比 GPT-6 Sol 快约 35 倍。
ClosureBench: A Constructive Benchmark for Compositional Graph Reasoning
ClosureBench is a constructive benchmark for evaluating language models on compositional graph reasoning tasks, revealing that frontier models degrade with complexity while a fine-tuned model maintains performance through program synthesis.
@yibie: https://x.com/yibie/status/2104076142500069608
This article demonstrates how to convert GLM-5.3-Flash into a Jev-style System 1 decision model, achieving typed decisions through a single forward pass. Benchmark tests show that it performs comparably to specialized models in terms of accuracy and speed.