Specification-delta-driven data governance: an empirical study of the {\guillemotleft}spec-delta{\guillemotright} as the unit of change in lakehouse data platforms
Summary
This paper formalizes the concept of spec-delta for data governance in lakehouse platforms and presents an empirical study comparing spec-delta-driven workflows to traditional code-based approaches.
View Cached Full Text
Cached at: 08/21/26, 10:06 AM
# Specification-delta-driven data governance: an empirical study of the «spec-delta» as the unit of change in lakehouse data platforms Source: [https://arxiv.org/abs/2608.19838](https://arxiv.org/abs/2608.19838) [View PDF](https://arxiv.org/pdf/2608.19838) > Abstract:Spec Driven Development SDD has consolidated the idea that the specification rather than the code should be the primary artefact governing AI assisted work\. Tools such as GitHub Spec Kit, and proposals such as Constitutional SDD, have formalised this principle in the software domain, while the executable data\-contracts literature has extended it to schema and quality enforcement at run time\. Nevertheless, the treatment of the specification delta OpenSpec's core idea that every change should produce a reviewable increment of requirements as the unit of change in data platforms remains empirically unexplored, even though many data\-platform changes are contractual \(new datasets, service\-level agreements, metric semantics, access policies\) rather than purely code changes\. This work formalises the spec\-delta concept, proposes a taxonomy of data platform changes according to their suitability for incremental specification, and defines a controlled experiment comparing a spec\-delta\-driven workflow against a conventional code pull\-request workflow without a delta\. The response variables are discovery to deployment time, the density of defects reaching the Silver and Gold lakehouse layers, cross\-tool metric divergence, and reviewer cognitive load measured with NASA TLX\. The paper explicitly reserves a demonstration\-and\-laboratory section for instantiation on a real lakehouse environment\. The contribution is not a tool but reproducible evidence and an applicability guide that helps to avoid the up front over specification antipattern\. ## Submission history From: Pablo Ramírez Amador Mg \[[view email](https://arxiv.org/show-email/be91f171/2608.19838)\] **\[v1\]**Thu, 20 Aug 2026 09:38:48 UTC \(314 KB\)
Similar Articles
SDAD: Spec-Driven Agentic Development for the AI-Native SDLC
The paper formalizes Spec-Driven Agentic Development (SDAD) to restructure the software development lifecycle with AI, emphasizing precise specifications and multi-agent verification for disciplined agentic speed.
LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake
LakeQA is a new benchmark for exploratory question answering over a million-scale data lake, evaluating multi-hop reasoning and compositionality across text, tables, and knowledge graphs.
LakeQuest: A Three-Domain Benchmark for Grounded Question Answering across Data Lakes
Introduces LakeQuest, a human-validated benchmark of 9,846 QA pairs across three domains for evaluating end-to-end retrieve-and-synthesize pipelines over heterogeneous data lakes, revealing critical failure modes in modern QA systems.
@PawelHuryn: Spec-Driven Development was always BS. It never worked for humans. Waterfall lost to agile because you discover what yo…
This tweet argues that Spec-Driven Development is ineffective for AI agents, drawing parallels to the failure of waterfall methodology in software development. It advocates for 'intent engineering'—communicating context, strategy, and constraints to agents to handle unknown unknowns.
Towards Spec Learning: Inference-Time Alignment from Preference Pairs
This paper introduces spec learning, a framework that compiles a small set of preference judgments into natural-language specifications to condition LLMs at inference time, outperforming DPO on specialized domains without requiring model parameter updates.