Tag
A commentary exploring how much of today's agent memory stack — retrieval, summarization, context compression — is merely a workaround for stateless model architectures, and how recurrent-memory designs like BDH-CQ could shift that responsibility into the model itself.
This paper systematically studies when recurrence helps in looped language models (LoopLMs), finding that extra recurrence can improve reasoning beyond the training horizon but degrade knowledge retention, and proposes channel-wise history-state injection with timestep conditioning as a more robust design for variable inference budgets.
A thread sharing field notes on fine-tuning, explaining how model architecture (e.g., heavily-quantized or mixture-of-experts) restricts which parts can be adjusted, urging practitioners to check model accessibility before planning work.