Building AI where mistakes matter

Trustworthy AI does not replace care, it reduces the friction to provide it. Ever used Spotify's DJ function? It picks songs you already like, wraps them in a friendly voice, and creates this pleasant illusion that the algorithm *gets* you. Then you ask for something specific, like a niche genre or a particular mood, and the mask slips. It plays something completely off or straight up ignores you. Mildly annoying, sure. You skip the track, you move on with your day. Now imagine a different kind of wrong answer. A chatbot tells someone experiencing homelessness that a night shelter is open, but it actually closed two hours ago. Or it confidently recommends a food point that moved last month. The person walks there, finds nothing, and spends the night without food or a bed. That is not a skipped track. That is a real consequence landing on a real person. During my Applied Data Science & AI studies, I led a project exploring whether a small, locally hosted AI chatbot could support frontline volunteers at a church-based social organisation in Rotterdam. The visitors, many experiencing homelessness, came with practical questions about shelter, food, basic legal documents, referrals, and local services. The volunteers needed reliable answers quickly, often while managing emotional conversations, limited time, and situations that did not fit neatly into any FAQ. We were not trying to automate care. We were trying to reduce information friction so that volunteers could spend less time searching and more time actually helping. But the project taught me something I did not fully appreciate going in: building AI for contexts where mistakes matter is fundamentally different from building AI for contexts where they do not. This blog is about those differences, and the engineering and design decisions they demand. Full blog post on my Substack below.