@GregKamradt: "Code and math are taking off because they are easy to verify, the next frontier is domains that are hard to verify" Th…

X AI KOLs Timeline News
ai-verification difficulty-spectrum artificial-intelligence machine-learning reasoning code science

Summary

Greg Kamradt proposes a 7-level spectrum of verification difficulty for AI, ranging from instantly verifiable domains like math and code to civilization-scale systems with slow, noisy feedback.

"Code and math are taking off because they are easy to verify, the next frontier is domains that are hard to verify" This got me thinking - what does the spectrum of "easy to verify" look like? This is loosely aligned w/ @DarioAmodei's "intelligence bottlenecked" domains. My take of easy > hard: - Level 1: Instant, objective verification Math, code, formal proofs, chess tactics, parsing AI improvement is easiest here because the loop is tight - Level 2: Fast but incomplete verification Software engineering, UI implementation, data analysis, security bug finding You can test a lot, but not everything. “It passes tests” is not the same as “it is good” - Level 3: Human-evaluable creative work Copywriting, design, video thumbnails, sales emails, landing pages Verification is possible through humans or markets, but noisy. AI can improve by predicting human reaction, but taste shifts and metrics can be gamed There is no "right" answer, only feedback from humans - Level 4: Market-verifiable work Startups, investing, product strategy, hiring, pricing, distribution Reality gives feedback, but slowly and with tons of confounders - Level 5: Experimentally verifiable science Materials, biology, chemistry, medicine, robotics There is ground truth (physics), but experiments cost time and money. AI helps most when it can propose better candidates and reduce search space - Level 6: Institutionally verifiable systems Education systems (Alpha school), legal systems, city planning, corporate management systems You can measure outcomes, but the feedback cycle is long, and the counterfactual is hard - Level 7: Civilization-scale verification Democracy variants, alternative governance, monetary systems, cultural norms, geopolitical strategy Verification is slow, morally loaded, noisy, and often impossible to isolate. You may never get a clean answer, only accumulated historical evidence
Original Article
View Cached Full Text

Cached at: 05/21/26, 03:47 PM

“Code and math are taking off because they are easy to verify, the next frontier is domains that are hard to verify”

This got me thinking - what does the spectrum of “easy to verify” look like?

This is loosely aligned w/ @DarioAmodei’s “intelligence bottlenecked” domains.

My take of easy > hard:

  • Level 1: Instant, objective verification Math, code, formal proofs, chess tactics, parsing

AI improvement is easiest here because the loop is tight

  • Level 2: Fast but incomplete verification Software engineering, UI implementation, data analysis, security bug finding

You can test a lot, but not everything. “It passes tests” is not the same as “it is good”

  • Level 3: Human-evaluable creative work Copywriting, design, video thumbnails, sales emails, landing pages

Verification is possible through humans or markets, but noisy. AI can improve by predicting human reaction, but taste shifts and metrics can be gamed

There is no “right” answer, only feedback from humans

  • Level 4: Market-verifiable work Startups, investing, product strategy, hiring, pricing, distribution

Reality gives feedback, but slowly and with tons of confounders

  • Level 5: Experimentally verifiable science Materials, biology, chemistry, medicine, robotics

There is ground truth (physics), but experiments cost time and money. AI helps most when it can propose better candidates and reduce search space

  • Level 6: Institutionally verifiable systems Education systems (Alpha school), legal systems, city planning, corporate management systems

You can measure outcomes, but the feedback cycle is long, and the counterfactual is hard

  • Level 7: Civilization-scale verification Democracy variants, alternative governance, monetary systems, cultural norms, geopolitical strategy

Verification is slow, morally loaded, noisy, and often impossible to isolate. You may never get a clean answer, only accumulated historical evidence

How do you put a price on verifying a novel?

Are they though??

Math and code aren’t easy to verify? At least more so than writing a novel?

How do you standardize and formalize the process of writing a novel? Or creating a start up idea?

Similar Articles

The Verification Horizon: No Silver Bullet for Coding Agent Rewards

Hugging Face Daily Papers

This paper explores the challenges of verifying AI coding agents' outputs, arguing that verification is becoming harder than generation as models improve. It analyzes four reward constructions and shows that no fixed reward function remains effective as model capability grows.