embeddings

An educational walkthrough of what happens between a user's message and a chatbot's reply, covering tokenization, context windows, conversation memory, chunking/embeddings, RAG retrieval, system prompts, function calling, and safety evaluation.

开发者将 Karpathy 的 stories260k 模型编译成一个 NPU，并直接嵌入 Qwen 3.5-4B 的权重中，实现了一个 LLM 在另一个 LLM 内部逐位精确地运行（包含矩阵乘、softmax 和注意力），无需外部工具。

This paper investigates how different pretrained visual representations organize Indian foods into similarity neighborhoods, finding that food similarity depends on representation, modality, and abstraction level rather than being an intrinsic property, using an expanded Khana dataset and seven visual models.

A Voyage AI practitioner argues that embedding inputs with a cheap embedding model and training a simple sklearn classifier on labeled data beats hype-driven zero-shot decision models like Jev, reporting a 37% error reduction at lower cost (with the caveat that this advantage only applies when training labels are available).

A technical report compares taxonomy-label search with a two-stage embedding retrieval plus LLM filtering approach on a corpus of 7,354 randomized trials, finding the new method recovers 92% of systematic-review evidence versus 45% for label search, at a cost that scales with search depth.

论文研究通过聚类数学题目的语义嵌入来自动构建教育分类体系，并在 ASSISTments 数据集上评估其在深度知识追踪任务中的表现，结果显示预测性能提升约 3-5 个 AUC 百分点。

This paper introduces an embedding-based subset selection method using Whisper's pretrained encoder and a greedy k-center algorithm to pick diverse training subsets for ASR fine-tuning. Evaluated on Hebrew and Danish, it shows k-center selection beats random sampling, especially at small subset sizes, due to greater lexical diversity.

Google 推出 EmbeddingGemma 2，一个统一文本、图像、音频与视频表示的开源多模态嵌入模型，采用简化逐层嵌入与多阶段训练课程，在各模态基准上建立新的帕累托前沿，性能超越大 2 至 12 倍的模型。

A developer shares a tutorial for giving AI agents persistent memory across sessions using Pydantic AI and a vector database: extracting facts from conversations, embedding them, and retrieving relevant memories to inject into a new session's context.

The paper introduces Signals and Queries (S&Q), a lightweight first-stage relevance filter for Parlant 4.0 that uses offline-trained logistic classifiers on LLM-generated examples to cut at least 30% of candidate behavioral policies per turn while preserving high recall, outperforming top-K and cosine-threshold retrieval.

Exa 3.0 launches as the biggest search quality update to Exa's search API for AI agents, featuring a larger embedding model, billions of new documents, Dynamic Highlights that use 95% fewer tokens, and new pricing tiers starting at $4 per 1,000 requests.

The post explains why RAG needs both embedding-based retrieval and reranking: dual-encoder models are fast and allow offline precomputation of document vectors, making them suitable for large-scale retrieval but weaker at capturing fine details; cross-encoders score each pair more accurately but are computationally expensive, so recall-then-rerank is a necessary trade-off.

An in-depth article on a high-frequency Agent interview question — "Walk me through your RAG pipeline" — systematically covering the principles, trade-offs, and common follow-up questions across the full RAG workflow: offline indexing (parsing, chunking, vectorization, index building) and online QA (retrieval, reranking, context construction).

This preprint argues that learned embedding tables dominate small language models' size and shows a frozen 16 MB vocabulary of 512-bit codes can match or beat trained tables on benchmarks, enabling a 46 MB BitNet ternary model with 5-6x faster CPU inference at a seventh of the memory.

介绍一种基于768维嵌入空间上的吸收马尔可夫链（Absorbing Markov Chain）的语义分流引擎，通过在60节点领域流形上做随机游走来替代朴素余弦相似度路由，实现<2ms的零token意图分类，从而在唤醒LLM之前完成路由。文中给出JevBench基准测试结果：四层级联系统（Gevva0）在240个条目上达到100%召回、89.17%精度，平均p50延迟46ms，并配备否定/语法、流形邻近OOD和熵与边界三重防护网。

研究人员发布了一个包含约16万篇巴西葡萄牙语金融新闻文章的大规模语料库，并基于语义相似度（余弦相似度、KNN、FAISS）建立了多分类弱监督基准，结果显示任务对标注质量和类别分布高度敏感。

互动式指南讲解向量嵌入的历史、原理与实践应用，通过可视化展示语义空间、向量运算与语义相似性，并演示语义搜索、聚类、零样本分类等真实场景。文中使用 Kanon 2 Embedder 的真实嵌入构建了一个交互式词语地图游戏。

TopicMerge 提出了一种多阶段「分而治之」主题建模流水线：先用高效嵌入方法将语料库过度分割为细粒度微簇，再由 LLM 引导迭代合并，实现 SOTA 聚类性能的同时将 LLM 调用量减少 92% 以上。

该论文展示预训练语言模型无需任务特定训练即可零样本完成记录链接与姓名族群推断两项社会科学常规任务；作者提出 EnsembleLink 算法，在记录链接上匹敌或超越既有方法，并证明 LLM 在美国选民文件上的族群推断准确率高达 83.8%，优于 name-supplemented BISG，并可推广到宗教、种姓等其他群体类别。

该论文利用 Pythia 和 OLMo 等开源模型的中间检查点，系统研究了大语言模型在训练过程中形态结构、嵌入漂移、重复主导与共现效应的演化规律，发现模型在语义嵌入稳定之前就已习得形态结构，且训练数据规模有助于发散识别，而共现和重复则有负面影响。