Implementing Watermarking for Language Models [P]

Reddit r/MachineLearning Tools
watermarking language-models implementation educational github anthropic

Summary

The author implemented a simplified, educational version of watermarking for language models based on SynthID-Text, sharing the code on GitHub after being inspired by Anthropic's announcement.

I recently implemented a minimal, educational version of SynthID-Text-style watermarking for language models. I saw anthropic post about how they'll start adding watermarks to their model responses and it made me very curious as to how they'll do it and what do they even mean by watermark here. Like will we start getting random ads or something in the middle of model responses or what. Then decided to read their article and found out that watermark is not a visible message at all. It is a subtle statistical pattern introduced while the model chooses its tokens. My implementation is not an exact reproduction of the original SynthID-Text system. I simplified or implemented a few components differently to keep the project understandable, but the main idea is there I think. Github: https://github.com/Saad1926Q/llm-watermark If you find it interesting then you may star the repo !!
Original Article

Similar Articles

Robust Text Watermarking for Large Language Models via Dual Semantic Embeddings

arXiv cs.CL

This paper presents Dual-Embedding Watermarking (DEW), a semantic watermarking scheme for LLMs that improves robustness against paraphrasing and translation by leveraging contextual and token-level embeddings. Experimental results show improved detection after paraphrasing and translation compared to prior methods.

A Linguistics-Aware LLM Watermarking via Syntactic Predictability

arXiv cs.CL

This paper introduces STELA, a linguistics-aware watermarking framework for LLMs that leverages syntactic predictability via POS n-grams to balance text quality and detection robustness. The method enables publicly verifiable watermark detection without requiring access to model logits, demonstrating superior performance across typologically diverse languages (English, Chinese, Korean).

Linguistics-Aware Non-Distortionary LLM Watermarking

arXiv cs.CL

Introduces LUNA, a linguistics-aware LLM watermarking method that achieves non-distortionary embedding and model-free detection across multiple languages, significantly improving AUROC and perplexity preservation.