Tag
The article discusses Sander Land's reproduction of Claude's tokenizer, revealing its surprisingly small vocabulary size compared to models like Qwen 3.8, and explores the theoretical implications of the softmax bottleneck in language models as described in a recent paper.