Tag
A paper reports a systematic gap in modern text embedders: sentiment is highly decodable via linear probes but nearly absent from their cosine-similarity metric, and proposes SISE, a tiny supervised-contrastive head that fixes this on frozen encoders.
This paper studies learned post-training quantization calibration for 4-bit text embedders (e5-base-v2, BGE-M3, EmbeddingGemma-300m), finding that calibration outcome ranges from gains to catastrophic collapse depending on the 'dose' of update, and proposes a validation-gated abstention rule that never selects harmful checkpoints while retaining most beneficial ones.
The paper shows that scaling text embeddings (e.g., swapping T5-small for T5Gemma-2-270M) greatly improves continuous diffusion language models, and that distilling the scaled embeddings into a more connected latent space makes them easier to generate — achieving Gen. PPL 17.8 that outperforms GPT-2-M.
This paper presents a black-box, model-agnostic method to analyze temporal and geographic signals in language model embeddings using simple projections, finding that embeddings encode meaningful chronological and spatial structure for interpretability and retrieval tasks.
This paper proposes a denoising-aware embedding inversion pipeline (DAEI) that effectively attacks noise-protected text embeddings, revealing significant privacy risks and challenging the efficacy of Gaussian noise as a defense.
This arXiv paper investigates whether simple linear transformations can translate representations across nine heterogeneous text embedding models, finding that shared structure and transferability depend jointly on architecture, training objective, pooling, and data distribution, challenging the notion of universal latent compatibility.
This paper proposes an evaluation framework for predicting item parameters from text embeddings using regularized regression and reliability/design ceilings. Results show that difficulty is substantially predictable from text, while discrimination and pseudo-guessing are limited by reliability, not text signal.
This paper evaluates twelve recent text encoders on their ability to encode affective cues from three psychological emotion theories, finding that instruction-aware open-weight encoders match or exceed proprietary ones at word level, while task-tuned embeddings are superior at sentence level.
This paper introduces the MELD dataset for evaluating whether text embedding models capture mathematical equivalence across different terminologies, and finds that current models fail. It proposes a contrastive learning approach to align informal and formal mathematical statements, improving retrieval on both informal-formal and natural language tasks.
This paper presents a human-in-the-loop pipeline for measuring a computer science program's coverage of curricular guidelines, applied longitudinally to CS2013 and CS2023. The framework reveals near-constant topical coverage but a gap in cognitive depth due to the newer guideline's raised expectations.
The paper identifies that LLM text embeddings overly express high-frequency uninformative tokens and proposes EmbedFilter, a linear transformation that filters out this subspace to improve semantic representations and enable dimensionality reduction.
JFinTEB introduces the first comprehensive benchmark for evaluating Japanese financial text embeddings, addressing a gap in domain-specific and language-specific evaluation resources. The benchmark includes retrieval and classification tasks evaluated across Japanese-specific, multilingual, and commercial embedding models, with datasets and evaluation framework publicly released.
OpenAI released text-embedding-ada-002, a unified embedding model that consolidates five previous models into one with superior performance, 4x longer context (8192 tokens), smaller dimensionality (1536), and 99.8% lower pricing than previous Davinci embeddings.
Saattrupdan 提供的 multilingual-e5-large-instruct 文本嵌入模型,基于 xlm-roberta-large 初始化并通过弱监督对比预训练与指令微调,支持 100 种语言的检索与语义相似度任务。
A CLIP-based embedding model hosted on Replicate that generates 768-dimensional embeddings for both images and text using the clip-vit-large-patch14 architecture, costing ~$0.00022 per run.