Tag
The article describes a reinforcement learning method with verifiable rewards to fine-tune AI models for text-to-SQL, achieving human-level accuracy on the BIRD benchmark without scaffolding.
This paper introduces DataKernelBench, a benchmark for evaluating LLMs on optimizing GPU kernels for database queries, achieving speedups over baseline methods like torch.compile.
QO-Bench is a diagnostic benchmark for query-operator question answering over typed event tuples, covering 22,984 news articles and 614 corporate events across 18 query templates. It evaluates RAG, ReAct RAG, GraphRAG, and extraction-to-SQL systems, finding that operator execution—not just retrieval—is a core bottleneck that stronger models alone cannot resolve.