Tag
AutoDataBench introduces a controlled, data-centric testbed to isolate and evaluate LLM agents' "data intelligence" — their ability to diagnose, organize, and construct training data — while holding non-data factors fixed, showing trajectories can also be reused for mid-training gains.
Databricks has opened up its Astra platform to approximately 3,500 engineers and drawn valuable conclusions from the initiative.
DI-Bench is a pipeline for systematically generating benchmarks for enterprise data intelligence tasks, combining knowledge retrieval and analytical computation. It evaluates models and finds they achieve only 32% accuracy on tasks where business rules modify computations.