Tag
Liquid AI introduces LiquidCodec, a ~74M-parameter streaming audio codec built on the LFM3 architecture that turns speech into tokens for its LFM3-Audio-2B and LFM3-Omni-2B models, matching peers in reconstruction fidelity and compression while delivering much faster token-to-waveform decoding for on-device use.
EvenVLM introduces the first text and multimodal VQA dataset for the endangered Even language, with tokenizer expansion and LoRA fine-tuning of Qwen3-VL-4B improving translation, text benchmarks, and VQA accuracy from 0% to 65%.
CZ commented that more countries will realize issuing local stablecoins boosts demand for their currency and government bonds, citing HSBC's Hong Kong dollar stablecoin RedCoin reaching 50,000 users and HK$87.75M in circulation within its first week.
Matt Dancho explains why understanding tokenization is a must-have skill for Data Scientists moving into AI Engineering, highlighting how tokens affect context windows, cost, embeddings, and multilingual performance, and links a tokenization handbook plus a free workshop.
美国证券交易委员会(SEC)发布了2027财年检查优先事项,将先进AI(包括“超级智能”)以及代币化证券列为重点监管对象,涵盖市场基础设施、网络安全和投资者保护等风险领域。
SpecSlot 是一种像素空间 rectified-flow Transformer,通过将空间锚点扩展为分配不相交 DCT 系数带的多个槽位,并仅让最低频槽参与全局注意力,在不增加计算成本的情况下提升图像细节生成质量。
Cartesia 分享关于分层网络(H-Nets)的研究博客,展示无需分词器的字节级语言模型在受控实验中实现 3-5 倍计算效率提升,2 阶段 H-Net 以 10 倍更少计算与数据超越匹配的开源模型,并在代码等领域表现出色。
Cartesia 的 From Tokens to Bytes 系列第一篇探讨 Hierarchical Networks (H-Nets) 的扩展规律,展示学习式 tokenization 带来 3-5 倍计算效率提升,2 阶段 H-Net 以 10 倍更少算力和数据超越匹配的开源模型,并呈现 in-context tokenization 等新行为以及在生物与视觉等跨模态任务上的潜力。
ARMOR-DP is a modular framework for multilingual BabyLM that dynamically adjusts language sampling based on per-language validation loss, showing that the weakest language shifts during training and that dynamic protection outperforms fixed Chinese-protection baselines.
该论文提出了一个统一的六维分析框架,系统综述了2017–2026年大模型训练全流程(数据与分词、预训练、SFT与偏好对齐/强化学习)的395篇核心文献,并揭示了跨阶段效应与多个开放问题。
AdaptTok-Indic 提出一种面向印地语系语言的脚本感知自适应分词方法,结合 Unicode 归一化预处理、显式印度语连字处理和基于脚本路由的字节级边界预测,在孟加拉语和印地语上取得比 SentencePiece BPE 更低的 token 生长率和更好的下游表现。
The paper shows that letters-only pre-tokenizer regexes in Llama-3 and Qwen3 impose an irreducible ~2.2x token premium on Nepali, and proposes an English-preserving regex repair that, with 32k new merges, reduces the premium to 1.01x, testing it via continued pretraining experiments.
该论文挑战了离散视觉令牌必然存在语义损失的假设,提出因子化向量量化(Factorized Vector Quantization),在多模态理解上达到与连续视觉表示相当的水平,并在离散表示模型中取得最先进性能。
The paper introduces Common-Support Distillation (CSD), a unified logits-based framework for cross-tokenizer knowledge transfer that constructs a shared support between teacher and student and matches induced distributions with a single objective, outperforming existing cross-tokenizer distillation methods across multiple teacher-student settings.
The paper identifies that existing time series tokenization strategies (patch and phase tokenization) introduce spectral distortions such as spectral leakage and aliasing, and proposes TimeFT, a parameter-free, nearly distortion-free frequency-based tokenizer that serves as a drop-in replacement and improves performance across forecasting, classification, and anomaly detection tasks.
A Nature paper introduces 'byteification', a method converting subword language models into byte-level ones with under 1% of a typical pretraining budget, producing models like Bolmo 7B that outperform prior byte-level baselines while retaining subword-level performance.
Someone open-sourced the KRONOS model, which treats each candlestick as a token to predict the next candle (pre-trained on 12 billion candlesticks). It can be integrated 100% free into Claude Code and TradingView to build a full AI crypto trading and prediction system.
论文提出 Mediator,一种离散、本体无关的 EHR 分词器,将任意临床概念映射到共享向量量化码本,使封闭词汇模型可迁移至 eICU,零样本任务 AUPRC 相对提升 29%。
Thunder-DeID+ 提出一种边界感知的韩语去标识化方法,通过改进的 BPE 分词器与三层 PII 分类体系,将法院判决书上的 Strict Typed F1 从 0.76-0.77 提升至 0.87-0.88,并在微调后达到 0.93,实现向更广泛敏感文档的迁移。
这篇论文通过在Tagalog上的受控实验(MorphAustro-Tok与BPE及打乱形态素的对照组比较),发现形态感知分词确实能提升机器翻译效果,但形态正确性本身并不贡献收益——增益仅来自分词粒度与一致性,而非语言学上正确的切分。