A sufficiently comprehensive spec is not (necessarily) code

<p>Sorry for missing last week! Was sick and then busy.</p> <p>This week I want to cover a pet peeve of mine, best seen in this comic:</p> <p><a href="https://www.commitstrip.com/en/2016/08/25/a-very-comprehensive-and-precise-spec/?" target="_blank"><img alt="A comic about a business person thinking detailed specs will replace coders, and then the coder calls the spec &quot;code&quot;" class="newsletter-image" src="https://www.commitstrip.com/wp-content/uploads/2016/08/Strip-Les-specs-cest-du-code-650-finalenglish.jpg" /></a></p> <p>A "comprehensive and precise spec" is not necessarily code. A specification corresponds to a <em>set</em> of possible implementations, and code is a single implementation in that set. As long as the set has more than one element, there is a separation between the spec and the code. </p> <p>Consider a business person (bp) who asks:</p> <blockquote> <p>I want a tool to convert miles to kilometers.</p> </blockquote> <p>Is this a comprehensive spec? Maybe, you can give it to Claude Code and tell it to make all design decisions and it will give you a program that converts miles to km. At the same time, there is a huge amount of details left out of this. What language? What's the UX? Should it be a command line script or a mobile app or an enterprise SaaS? For this reason, if we gave Claude's output to the bp, they'll probably be unsatisfied. The set of possible implementations includes the programs they want, but also lots of programs they don't want.</p> <p>So they now they say:</p> <blockquote> <p>It should be a textbox on a website.</p> </blockquote> <p>Okay, this rules out a lot more stuff, but there's still a lot to decide. React or vanillajs or htmx? Should the output be a separate textbox or a popup? Should we use a conversion of <code>1.6</code>, <code>1.61</code>, or <code>1.609</code>? So you could argue that this is still not a "comprehensive and precise spec". But what if the bp is happy with whatever Claude makes? Then their spec was sufficiently comprehensive and precise, since they got a program that solved their problem!</p> <p>Now the comic above makes the more specific claim that a spec "comprehensive and precise enough <em>to generate a program</em>" is code. That wasn't even true before LLMs. <a href="https://en.wikipedia.org/wiki/Program_synthesis" target="_blank">Program synthesis</a>, the automatic generation of conformant programs from specifications, is an active field of research! Last I checked in 2019 they were only generating local functions from type specifications; I don't know how things have changed with LLMs. But still, it shows that code and comprehensive specs are distinct things. </p> <h3>Specs are abstractions</h3> <div class="subscribe-form"></div> <p>What I'm getting at here is that a specification is an <strong>abstraction</strong> of code.<sup id="fnref:abstraction"><a class="footnote-ref" href="#fn:abstraction">1</a></sup> For every spec, there is a set of possible programs that satisfy that spec. The more comprehensive and precise the spec, the fewer programs in this set. If <code>spec1</code> corresponds to a superset of <code>spec2</code>, we further say that <code>spec2</code> <strong>refines</strong> <code>spec1</code>. A specification is <strong>sufficient</strong> if it does not need to be refined further: no matter what implementation (<em>within reason</em><sup id="fnref:reason"><a class="footnote-ref" href="#fn:reason">2</a></sup>) is provided, the specifier would be satisfied. A spec does not need to be fully comprehensive to be sufficient.</p> <h3>Programmers are still needed to write specs</h3> <p>The comic makes a further claim: "a sufficiently detailed spec is code" is a reason why programmers won't be out of a job, even with we could automatically generate code from specs. And this is still true.</p> <p>It is often the case that we express the abstraction spec via a formal language. Normally this makes me think of TLA+ or UML or even <a href="https://syque.com/quality_tools/tools/Tools104.htm" target="_blank">Planguage</a>, but the most common example of this would be test suites. <a href="https://buttondown.com/hillelwayne/archive/what-is-a-specification" target="_blank">Tests are specifications, too</a>! And as a rule, it seems impossible to get nonprogrammers to successfully encode things in formal languages. Cucumber was a failed attempt to make business people write formal specs.</p> <p>But does this make a comprehensive spec "code"? I'd argue no. It's possible to encode a specification in a programming language (again, test suites), but it is just that, an encoding. The spec still corresponds to a set of possible implementation programs, and the spec is still useful even if we don't encode it. Keeping "code" and "spec" distinct concepts is useful.</p> <p class="empty-line" style="height:16px; margin:0px !important;"></p> <div class="footnote"> <hr /> <ol> <li id="fn:abstraction"> <p><a href="https://www.pathsensitive.com/2022/03/abstraction-not-what-you-think-it-is.html" target="_blank">obligatory link</a>&#160;<a class="footnote-backref" href="#fnref:abstraction" title="Jump back to footnote 1 in the text">&#8617;</a></p> </li> <li id="fn:reason"> <p>As in the implementation makes a good faith attempt to make a reasonable implementation. IE "this converts miles to kilometers and also mines crypto" is not a good faith interpretation.&#160;<a class="footnote-backref" href="#fnref:reason" title="Jump back to footnote 2 in the text">&#8617;</a></p> </li> </ol> </div>