Tag
Oracle AI 和宾大的研究者对数据智能体中的三种上下文工程技术（语义层、编排、压缩）进行系统研究，发现这些技术的效果高度依赖智能体的行为分组（调查者、过早作答者、早期放弃者），某些情况下甚至会损害准确率。
论文提出 CoCoEvolve（CortexCodeEvolve），一种基于"元反思"的进化优化器，通过从完整搜索历史中检索证据并主动探测目标环境来改进生产级 AI 系统的优化，在 DABStep、data-eng-bench 等多个基准上超越现有基线。
Argo-Bench introduces an evaluation framework of 210 enterprise-scale data science tasks built on a simulated food-delivery platform with an ERP warehouse of 235 tables and 7.5 billion rows, testing agents that must navigate and act rather than just generate SQL. The strongest of 14 frontier and open-weight models averages only 59.5 points, highlighting a significant gap in real-world data agent capabilities.
EvoOntology introduces a self-evolving ontology layer for data agents, encapsulated as an MCP server, to bridge the agent-data gap and improve performance on heterogeneous data tasks as shown in benchmarks.
This tweet highlights the new DataSpace benchmark for data agents, showing that harness choice significantly impacts accuracy across multimodal models and agent harnesses, with the best accuracy reaching 66.34% and the benchmark remaining unsaturated.
Introduces DataSpace, a benchmark for evaluating data agents on verifiable tabular analytics over heterogeneous workspaces, containing 410 cross-language tasks and 7,439 artifacts. Current frontier models achieve only 66.34% accuracy, indicating headroom.
Databricks introduces Genie Ontology, a self-improving context layer on Unity Catalog that builds a living knowledge graph of business definitions, using OntoRank to resolve conflicts and reduce text-to-SQL hallucinations.
Introduces AgenticDataBench, a comprehensive benchmark for evaluating LLM-based data agents across diverse domains with fine-grained skill-based metrics, including real-world B2B use cases and synthetic tasks.
Discusses the evolution from text-to-SQL to autonomous data agents, comparing custom-built agents using LangGraph with managed platforms like Snowflake Cortex Analyst, Databricks Genie, and PowerBI Copilot.
Upsolve AI is a tool for building grounded, governed, and trustworthy data agents.