qwen-4

Alibaba has announced Qwen 4, a new AI model, at the Apsara Conference.

The author speculates on the possible architecture of Qwen 4, discussing potential designs like embedding-offloaded linear attention or multi-head latent attention hybrids based on reverse engineering Deepseek.

Based on previous correct predictions, Qwen 4 is expected to be released in September, as sourced from Alibaba. The author praises the performance and cost-effectiveness of current Qwen 3.8 models.