@HowToAI_: Microsoft Research + Salesforce has published a paper that should scare every single AI builder right now. It’s called …
Summary
A new paper by Microsoft Research and Salesforce reveals that LLM performance drops significantly in multi-turn conversations due to a 'Lost in Conversation' phenomenon, challenging the reliability of current single-turn benchmarks.
Similar Articles
@MSFTResearch: LLMs that get lost in conversation. Robots that learn zero-shot. Diffusion models built for imbalanced data. This is th…
Microsoft Research highlights ICLR papers on LLMs losing conversational coherence, zero-shot robot learning, and diffusion models for imbalanced data.
Did you see it when Salesforce's run their own AI Agents benchmark
Discussion of Salesforce's CRMArena-Pro benchmark showing agent success drops from 58% on single-turn to 35% on multi-turn tasks, plus practical advice for splitting agent workflows into narrow stages to reduce error compounding.
@dair_ai: Great paper from Salesforce AI Research on RL for multi-turn tool use. The finding is that you want to train the one ca…
A paper from Salesforce AI Research introduces Critical-State RL, a method to identify and train the most impactful model call in multi-turn tool use trajectories, improving accuracy on the BFCL v4 benchmark by up to 14 points.
@marfinxx: Microsoft Research created AutoGen to pioneer multi-agent LLM conversations essential for AI system engineers deploying…
Microsoft Research's AutoGen enables multi-agent LLM conversations, allowing AI system engineers to build conversable agents with hierarchical execution for complex tasks that single LLMs struggle with.
Is AI actually getting better at understanding context in long conversations, or does it still fall apart?
This article discusses the limitations of AI models in maintaining context over long conversations, highlighting recency bias and the distinction between context window size and actual comprehension. It suggests practical workarounds like restating constraints and using running context documents.