@levie: We’re going to be in for a strange dynamic which is that some of the “hardest” work in the world is actually prone to a…
Summary
Levie and Max Spero discuss how verifiability determines which 'hard' knowledge work gets automated first, arguing that objectively testable domains like math, cyber, and code will see faster AI automation than fields with subjective outcomes.
View Cached Full Text
Cached at: 08/03/26, 01:46 AM
We’re going to be in for a strange dynamic which is that some of the “hardest” work in the world is actually prone to automation first, particularly due to its verifiability.
Math, cyber, and code -while being insanely hard and high value fields- have the benefit of being able to be tested that it’s correct objectively. This has two immediate benefits: the training of the models offers clearer reward signals, and then the running of the models allows you to know that it’s working properly because you can test the results in a scalable way.
Conversely, in other domains of work, there’s much less instant verifiability. Which legal clauses your client will agree to, what marketing campaign to run with based on changing sentiment, which message your sales prospect will want to hear, what financial targets and budget to set for a business, and so on.
All of these domains have changing internal and external factors, they don’t have “one right answer”, they rely on the opinions and risk levels of the operators, they’re highly sensitive to getting the right input context first, and in many cases the right answer can’t even be known for quite some time after the model generates the results.
The implications of this distinction are that -even as model capability continues to increase exponentially- there will be a lot done at the applied AI layer than just the the model itself, and much of the processes themselves will even need to change over time to get the full gains from automation. We may even need all new capabilities to be able to “test” knowledge work over time as we have had with software.
Max Spero (@max_spero_): In my view we have a few different tiers of verifiability
- programatically verifiable (near-free)
- games, coding, math, cybersecurity, chip design
- real-world verifiable (cost-or time bounded)
- sciences: biology, chemistry, physics -physical world: material science, energy,
Similar Articles
@GregKamradt: "Code and math are taking off because they are easy to verify, the next frontier is domains that are hard to verify" Th…
Greg Kamradt proposes a 7-level spectrum of verification difficulty for AI, ranging from instantly verifiable domains like math and code to civilization-scale systems with slow, noisy feedback.
@levie: We’re going to see an increasing divergence between what AI does in our personal lives and in daily productivity vs. wh…
A commentary on AI capabilities diverging between consumer use and deep domains, citing OpenAI's next model family Astra allegedly solving major open problems in math and science.
@mattpocockuk: Knowledge work is SO much harder to automate with agents than code We underestimate how easy software development is fo…
The tweet argues that knowledge work is harder to automate with AI agents compared to software development due to advantages like automated feedback loops and organized documentation in coding.
@petergostev: What is happening in maths right now will happen to any hard industry with verifiable problems and high value rewards. …
The post argues that massive investment in AI agent swarms could revolutionize hard industries like microchip design with high ROI, potentially leading to super-intelligence breakthroughs.
@bcherny: Something I have been thinking about: in the past, the best engineers I knew spent a lot of time automating their work …
A thought-provoking thread arguing that automating engineering tasks, especially with AI agents like Claude, is more important than ever. It advocates encoding domain knowledge as infrastructure to accelerate development and enable contributions from non-engineers.