Tag
This paper presents Dual-Embedding Watermarking (DEW), a semantic watermarking scheme for LLMs that improves robustness against paraphrasing and translation by leveraging contextual and token-level embeddings. Experimental results show improved detection after paraphrasing and translation compared to prior methods.
This paper investigates whether Word2Vec can generate meaningful semantic embeddings for Toki Pona, a constructed language with only ~130 words, using a corpus of 1.4 million sentences, and examines the effect of non-Toki Pona tokens on embedding quality.
The author shares their experience switching from semantic embeddings to BM25 for tool selection in agents, finding that BM25 achieves 81% top-1 accuracy vs. 64% for embeddings on a corpus of 200 query-tool pairs, because tool descriptions are short and keyword-driven rather than semantically rich like documents.
This paper proposes Evidence-Calibrated Query Clustering (ECC), an algorithm that aligns semantic embeddings with latent LLM capability demands using posterior model comparisons and Bradley-Terry modeling, significantly improving capability ranking quality for LLM evaluation.