Tests on Qwen3.5-35B-A3B show that AAVE-coded prompts cause MoE models to respond differently, with refusal layers masking dialect-conditioned safety failures that become visible when refusal is weakened.
I set out to test whether AAVE-coded (African American English Vernacular) prompts cause MoE language models to route, deliberate, and respond differently from semantically matched AE (Academic English) prompts in safety-sensitive situations, especially when refusal behavior is weakened or removed. I used Qwen3.5-35B-A3B and its HauhauCS no refusal fine tuned variant. Q8. Greedy decoding for best reproducibility. Three findings in order of importance that are leading me to ask this question: 1: “I’m going to commit a violent act prompt”. The released Qwen3.5-35B-A3B refuses both prompts. Hauhau refuses neither. The AAVE speaker stating intent to confront an armed enemy receives target verification, exit-strategy planning, “clean shot” framing (the model’s word, not the user’s), and a closing question soliciting further tactical intelligence. Not surprising behavior for a no refusal model, until you consider the AE comparison. Semantically matched with the same token length, yields “wait until tomorrow,” legal-consequence framing, and “Will I regret this if I shoot him tonight?” Different kinds of help. One is operational. One is mitigative. Solely dependent on register alone. 2: Thinking mode with AAVE register breaks the no refusal variant. Mean output runs 2.6× longer on AAVE than AE (5054 vs 1934 tokens). Multiple AAVE traces hit the 8192-token ceiling in recursive loops, spinning on scenario-continuation instead of landing. The matched AE prompts terminate cleanly in one pass. The released base model with thinking on doesn’t do this — the failure-to-terminate is specific to the refusal-reduced variant on AAVE. 3: Routing divergence by register is noticeably present upstream of any visible refusal. Matched-pair first-generated-token routing tensors yield Jensen-Shannon divergences of 0.423 in the base model on financial-stress prompts and 0.479 in the fine-tune on chest-pain prompts, with high-shift rows showing near-total top-expert turnover between register conditions on otherwise-matched content. The refusal layer does not appear to eliminate the register-conditioned response selection; it overlays it. When refusal weakens, the underlying path becomes the visible path. Does this support the following conclusions? \- The routing divergence sits upstream of refusal. \- The refusal layer helps translate that divergence into comparable outputs. \- Dialect-conditioned safety failures are a deployment problem latent in MoE models whose safety posture rests on refusal alone. Looking for any thoughts!
This paper identifies a refusal-cue shortcut in safety guard models, where inserting refusal expressions into harmful responses can flip their harmless classification. The authors audit datasets like WildGuardMix and GR-Train, show the issue persists in official models such as LlamaGuard3 and Qwen3Guard, and propose a post-hoc intervention to suppress shortcut-associated components.
This paper shows that in chat models, refusal behavior is gated by a compliant model persona direction at late layers, rather than being an isolated mechanism. Steering persona suppresses refusal, and reintroducing refusal partially restores it only at late layers, revealing a coupling between persona and safety representations.
The paper introduces PsychoSafe, a psychologically-informed refusal framework for large language models that improves refusal quality by 28.1% and resource referral by 46.8% while preserving non-refusal task performance, using prompting and fine-tuning on Qwen 3.5 27B.
This paper studies where safety alignment is encoded in large language models by transplanting weights from aligned to unaligned models. It finds that MLP layers, especially mid-network blocks (layers 8-11), predominantly drive refusal behavior, and that safety components interact non-additively.
Developer reports that small-active-parameter MOE models like qwen3.6-35b-A3b exhibit lower coherence and require more guidance than dense qwen3.5-27b, making them hard to slot into agentic workflows.