Chain of Thought is a scaling trap. the next wave is latent reasoning (Coconut / HRM / RecrusiveMAS)... but then we hit the black box wall. Where does BDH fit? [D]
This article argues that Chain of Thought reasoning is a temporary hack and that the future lies in latent reasoning methods like Coconut, HRM, and RecursiveMAS. It discusses the black box problem and proposes outer loop governance with DAGs and verification, while positioning BDH as a model that combines latent computation with stateful memory.
Read a long piece on the future of LLM reasoning that makes a provocative claim: Chain of Thought is a useful hack but we've started to confuse a readable trace with the actual computation. All in all, "generating text is not the same as thinking." There are two practical problems here: Faithfulness: CoT style traces can decouple from what the model actually "did." u can get plausible steps with a wrong answer, or messy steps with a right answer (so the trace isnt a reliable audit trail) Systems cost: Autoregressive reasoning serializes intermediate work into tokens. Longer traces inflate latency, cost and context usage The latent turn (stop making models "think in public"): A lot of recent work is shifting the inner loop into latent space and decoding language only at the end: Coconut (continuous / latent "thought" steps) HRM / HRM Text (separating slower planning from faster recursive execution) RecursiveMAS (agents passing latent embeddings instead of long text messages) A framing i propose: language as interface vs language as the compute substrate. I agree that language is essential for communication and abstraction but forcing search / constraint solving to be serialized into text is awkward and expensive. The black box wall If the "thinking" happens in dozens of latent loops, u lose the already imperfect window u had with CoT. In production, especially in high stakes domains, "no visibility" is a real blocker. One proposed solution is an outer loop governance layer (e.g., a symbolic / planning manager that builds an auditable DAG of subgoals + deterministic verification at each node: unit tests, constraints, rules, etc.). Auditability shifts from "read the model's inner monologue" to "audit the plan + checks + verified outputs." Where BDH fits BDH (Dragon Hatchling) is interesting in this landscape because it aims to keep language modeling capability while adding recurrent or stateful latent computation, rather than being "just" a supervised puzzle solver. Pathway reports 97.4% top 1 accuracy for a BDH based system on ~250k Sudoku Extreme puzzles, without CoT or solution backtracking. Sudoku is a useful diagnostic for constraint solving but not a complete measure of general reasoning. One point i found clarifying: many recursive latent reasoners excel at depth recurrence (iterating on a fixed snapshot of the problem) but real agentic / language settings are a stream: new tokens arrive continuously. That introduces time recurrence questions: when do u advance time, and what state / memory do u carry forward? BDH's stated research direction is basically trying to bring these together i.e. high bandwidth latent iteration and a principled state / memory story over time. It also provides a recoverable graph view and sparse, localized state, offering some native interpretability hooks but that is complementary to, not a replacement for, system level verification. I want views on: Is CoT increasingly a costly interface artifact rather than a scalable reasoning path? For high stakes use, do we inevitably need a DAG / verification outer loop, or can native model analysis hooks meaningfully reduce the governance burden (even if they cant replace it)? If latent recursion is the inner loop, what should the outer loop be in practice, DAGs, unit tests, formal specs, proof assistants, something else?
This paper applies dynamical systems analysis to interpret latent chain-of-thought reasoning in models like CODI and COCONUT, revealing structured dynamics with stable and unstable classes.
This research paper from MediaTek and National Taiwan University challenges the assumption that reasoning chains must be dense and sequential, showing that models can extract answers from sparse, shuffled, and noisy reasoning traces. The findings suggest that answer extraction is robust and order-independent, potentially enabling more efficient, parallelized reasoning generation.
This paper demonstrates that extended chain-of-thought reasoning degrades performance on deterministic state-tracking tasks due to information-theoretic limits of decoder-only attention, and proposes tool delegation when the reasoning horizon exceeds a threshold.
The article discusses a shift in LLM reasoning research from making reasoning explicit via chain-of-thought to exploring latent reasoning that doesn't require language traces, questioning whether visibility is necessary for effective reasoning.
OpenAI researchers study whether reasoning models can deliberately obscure their chain-of-thought to evade monitoring, finding that current models struggle to control their reasoning even when aware of monitoring. They introduce CoT-Control, an open-source evaluation suite with over 13,000 tasks to measure chain-of-thought controllability in reasoning models.