@jonasgeiping: We’re training models wrong and it’s due to chatGPT. Even the modern coding agents used daily still use message-based e…

We’re training models wrong and it’s due to chatGPT. Even the modern coding agents used daily still use message-based exchanges: They send messages to users, to themselves (CoT) and to tools, and receive messages in turn. This bottlenecks even very intelligent agents to a single stream. The models cannot read while writing, cannot act while thinking and cannot think while processing information. In our new paper, see below, we discuss LLMs with parallel streams. We show that multi-stream LLMs can … Be created by instruction-tuning for the stream format Simplify user and tool use UX removing many pain points with agents and chat models (such as having to interrupt the model to get a word in) Multi-Stream LLMs are fast, they can predict+read tokens in all streams in parallel in each forward pass, improving latency LLMs with multiple streams have an easier time encoding a separation of concerns, improving security LLMs with many internal streams provide a legible form of parallel/cont. reasoning. Even if the main CoT stream is accidentally pressured or too focused on a particular task to voice concerns, other internal streams can subvocalize concerns that would otherwise not be verbalized. Does this sound related to a recent thinky post :) - Yes, but I don’t feel so bad about being outshipped with such a cool report on their side by 23 hours. I’ll link a 2nd thread below with a more direct comparison. I actually think both are complementary in interesting ways.