Build the Right Thing: Product Engineering

my notes from day 1 at @aiDotEngineer conference about Product Engineering in the AI world. lecture by Kent Dodds

Core Thesis

• When AI agents level the implementation playing field, the differentiator becomes building the right thing

• Implementation is commoditized; deciding what to build is the scarce, durable skill

• The “last skill” a software engineer needs: product judgment

The Arrow Metaphor

• Engineers used to need precision to hit targets; agents now make hitting targets easy

• Far more targets are now reachable than ever before

• The real question: which target is worth aiming at?

• Not all targets are created equal; selection is the differentiator

Product Engineer vs. Product Manager

• Product engineer is in the code, thinking about system constraints, primitives, and infrastructure limits

• When a request comes in, they map it to existing architecture or identify where expansion is needed

• Key distinction: technical expertise married to user understanding

• Both roles are needed, but at early-stage startups one person often covers both

• “Engineering judgment comes from technical experience married to human issues”

Idea and Feature Validation (The Mom Test)

• Don’t ask users to evaluate your idea; they’ll say yes to be polite

• Ask about past behavior, not future intent:“Tell me about the last time this happened” “What did you do instead? What did it cost you?”

• Strong signal: users already spending time or money on a workaround

• Weak signal: “Yeah, that sounds like a good idea”

• Wayne’s cautionary tale: 1.2M AUD (~900K USD) spent on a high-scale system nobody usedCould have launched in two weeks with manual integration to test the market

Jobs-to-Be-Done Framework

• People hire products to make progress in a specific situation (Clayton Christensen)

• Feature requests are not jobs; always ask “what are you trying to solve?”

• Job statement format: “When [situation], help me [make progress] so I can [desired outcome] without [pain]”

• Example: “When there’s a workshop, help me get in quickly so I can learn, without missing the first 20 minutes”

• Break the job into three dimensions:Functional: does it actually do the thing? Social: who else is involved when the user uses it? Emotional: how does the user feel, including when others know they use it?

• Facial recognition example: understanding the job redirected the solution toward NFC or parallelized queues

The Kano Model: Prioritizing Features

• Three main categories:Basic needs: must-haves; absence causes dissatisfaction (e.g., order confirmation email, GPS tracking) Performance: more is better up to a point (e.g., ETA accuracy, page load speed) Delighters: unexpected features that create delight (e.g., surprise discount); absence causes no dissatisfaction

• Features only move one direction on the model: delighter → performance → basic needGPS tracking was a delighter in 2015; now a basic need

• Two additional categories:Indifference: users don’t care either way; remove these to reduce maintenance burden Reverse: features users actively dislike (e.g., facial recognition without consent)

• Implication for engineers:Basics: nail reliability, completeness, ownership Performance: measure and hit minimum thresholds Delighters: treat as experiments; don’t over-invest before basics are solid

User Feedback Loops

• Watch users use your software; it’s humbling and reveals obvious improvements

• Don Norman: “User error does not exist” — if users fail, the system is wrong

• Tactics for ongoing feedback:Slack channel aggregating Reddit, X, and support threads Dedicated channel with high-value customers

• Close the loop: early adopters will tolerate rough edges; mainstream users won’t

• “Big hire” vs. “little hire”: getting paid once vs. users returning repeatedlySuccess = repeated progress, not just initial purchase

Durable Skills and the AI Future

• Even if AGI does almost everything, human judgment on what to build remains valuable

• Product engineers are now team leads over however many agents they can run

• Your job: keep the system (primitives, architecture, testing) clean so agents can work effectively

• If AI eventually learns product judgment too: “I don’t know what we’re gonna do”

• Recommended resources:Competing Against Luck by Clayton Christensen (jobs theory) The Design of Everyday Things by Don Norman The Mom Test (book) Kent C. Dodds’ YouTube channel (youtube.com/@kentcdodds-vids)