Is AI actually getting better at understanding context in long conversations, or does it still fall apart?

The recency bias problem is real and it's one of the more frustrating things about working with these models day to day. You spend the first part of a conversation establishing your situation carefully, and then ten exchanges later the model is giving you advice that directly contradicts something you told it at the start. The context window growth is genuinely useful, but you're right that raw length and actual comprehension are different things. A model that can technically "see" 200k tokens isn't necessarily treating all of them equally. In practice, earlier content gets deprioritized as the conversation accumulates. Whether that's an architectural issue baked into how transformers weight attention, or something that better training can fix, I don't think there's a clean answer yet. Probably both. RAG feels like a partial solution at best. It helps in specific setups where you're pulling from a structured knowledge base, but it doesn't really solve the problem of a model losing track of what you told it three minutes ago in the same conversation. What's worked for me: periodically restating the core constraints explicitly, especially before asking anything that depends on them. Not as a summary, just a quick "remember the goal here is X" before the relevant question. Annoying that it's necessary, but it does seem to help. Some people also keep a short running "context document" they paste at the start of each session if they're picking up a long project, which sidesteps the degradation issue entirely. Whether it matters depends on what you're using these for. For quick standalone tasks it's basically irrelevant. For anything that requires holding a complex mental model across a long working session, it's a real limitation and I don't think acknowledging that is overestimating the problem.