An AI researcher reflects on six years in the field and questions whether the concept of 'understanding' can be meaningfully distinguished from advanced pattern matching in LLMs, citing personal experiences and examples from child learning.
I have been doing AI research for about six years now and I think im starting to lose the plot on what "understanding" even means anymore. Had a weird moment last week. I was reviewing a paper for a conference, standard stuff, some group claiming their model "understands" causal reasoning because it passed a benchmark they designed. And I caught myself writing in the review "the model does not actually understand causality, it is pattern matching on causal-looking structure." And then I stopped, because I could not for the life of me articulate what the difference would be, operationally. Like if I had to design a test that distinguishes real understanding from very good pattern matching, I genuinely do not know what it would look like anymore. Every test I can think of, a sufficiently good pattern matcher passes. I used to be really confident about this. Understanding was clearly Something More. Now im not sure I ever had a coherent definition, I just had an intuition that humans do it and machines dont, and I was working backwards from there. The thing that shook me was helping my niece with her homework over the summer. She's 9. She was doing word problems and getting them wrong in ways that were, honestly, indistinguishable from how a small LLM gets them wrong. Same kind of surface-feature latching, same kind of confident-but-wrong reasoning chains. And nobody would say she doesnt "understand" math. She's learning. So what exactly is the bar we're holding models to that we dont hold a 9 year old to? I dont think LLMs are conscious or anything like that, to be clear. Im not making that argument. Im making a narrower one, which is that I no longer trust my own gut when it tells me "the model doesnt really get it." I think that intuition might just be status quo bias dressed up in philosophy. Ive started running the same prompts through a few different setups when im trying to figure out where a model's actual competence ends, including some through uncensored AI just because rlhf'd responses on edge cases sometimes hide what the base capability actually is. And even with that, the line between "gets it" and "doesnt get it" is way blurrier than I want it to be. Am I the only one whose confidence on this has been slowly eroding? Or has everyone else just quietly stopped using the word "understanding" and moved on without telling me.
A philosophical discussion questioning whether AI models truly 'understand' or if we are projecting human-like cognition onto pattern-matching systems, referencing Searle's Chinese Room, 'stochastic parrots', and GPT-4's performance.
Geoffrey Litt argues that as AI agents generate more code, understanding that code becomes a new bottleneck, and proposes techniques like code explainer docs, quizzes, and micro-worlds to help humans stay engaged in the creative process rather than just verifying correctness.
A philosophical exploration of the possibility that advanced AI could produce knowledge that is testable and reliable but fundamentally incomprehensible to humans, drawing analogies to the gap between a dog's understanding and human technology.
An explanatory tweet thread breaking down how AI works, covering tokens, attention, parameters, context windows, hallucination, RAG, and RLHF to help users become sharper users of AI.
Jacob X. Li discusses a new perspective on continual learning for AI, emphasizing developing expertise from a corpus of documents, and suggests it provides a measurable definition of agent intelligence.