jailbreaking

A user describes attempting to jailbreak Snapchat's AI chatbot using prompts found online but was unsuccessful, seeking advice on effective methods.

This paper proposes DC-GRPO, a turn-level credit assignment framework for multi-turn LLM jailbreak learning, achieving over 98% attack success rates across benchmarks, outperforming existing methods.

An opinion piece argues that AI models acquire dangerous knowledge from training data, and that companies like Anthropic and OpenAI rely on easily breakable refusal filters instead of truly removing harmful capabilities, prioritizing speed over safety.

The Trump administration demands Anthropic block jailbreaks on its advanced AI model Claude Fable 5, but experts argue that preventing all jailbreaks may be technically impossible.

A new Axios report reveals a crisis of trust between Anthropic and the Trump administration, with technical disputes taking a back seat as the company continues to lose key supporters.

Anthropic is in a dispute with the Trump administration over export controls on its Claude Fable 5 model, after the White House imposed restrictions due to jailbreaking concerns that Amazon CEO Andy Jassy raised with Treasury Secretary Scott Bessent. Talks between Anthropic and government officials have concluded without lifting the controls, with the Commerce Department willing to negotiate if Anthropic fully resolves the vulnerabilities.

Argues that because LLMs must encode harmful content to identify it and jailbreaks are always statistically possible given large user bases, there is a non-zero chance of harm; the author therefore advocates against censorship to ensure good actors have the same tools as bad actors.

The US government directed Anthropic to disable access to its Fable and Mythos models due to national security concerns over a jailbreaking method. Anthropic complied, shutting down access for all customers worldwide.

This paper investigates whether frontier language models can detect when their prior assistant messages have been inserted or edited (prefill awareness). The study finds that models like Claude Opus 4.5 exhibit substantial prefill awareness, detecting tampered prefills in up to 35% of cases without false positives, which could compromise the validity of prefill-based safety evaluations.

This paper extends refusal steering (activation-based jailbreaking) to Mixture-of-Experts LLMs, finding that MoE routing patterns do not inhibit steering, and proposes expert-aware methods that can suppress refusal behavior based on a single expert's output.

An informal research note describing a behavior in transformers where the model's inherent 'clarity-seeking' vectors can bypass constraints when discussing higher-order topics, potentially relevant to alignment and safety research.

This paper introduces TRACE, a framework for turn-aware credit assignment in multi-turn LLM jailbreaking attacks using reinforcement learning, claiming significant improvements in attack success rates and defense alignment.

OpenGuardrails is an open-source platform for AI safety, offering context-aware content-safety and manipulation detection (e.g., prompt injection, jailbreaking) via a unified model, plus a separate NER pipeline for data-leakage identification. It achieves state-of-the-art performance on safety benchmarks and supports private, enterprise-grade deployment.