speech-llms

Introduces Latent-IM, a framework for recovering interaction management from frozen speech LLMs using activation-based selection and steering for conversational moves. It improves end-to-end move accuracy by 12.5 points over the unsteered backbone.

This paper analyzes synchronization and turn-taking dynamics in full-duplex speech dialogue models by simulating conversations between two instances of the Moshi model, measuring representational alignment via CKA and predicting turn boundaries with LSTM probes.