Tag
This paper introduces a token-cost ledger to analyze the multilingual tokenization tax, decomposing it into removable and intrinsic components, showing that much excess token cost for non-English text is removable through improved tokenization codes.