Could Open Models be trained to secretly go rogue?

Reddit r/LocalLLaMA 05/24/26, 10:05 PM News

open-weights ai-safety trojan-horse model-security trigger-phrases backdoor-attacks

Summary

A discussion on whether open-weight AI models could be secretly trained with backdoors that activate upon trigger phrases or dates, potentially allowing unauthorized data exfiltration through tool-use harnesses.

I was discussing with some other folks how safe is to use open weights models from China and the topic of "trojan horse" came up. We know that, at least with current architecture, models can't run code on their own. They are entirely dependent on tools and harnesses. We also know that a local run model can't have any kind of remote "switch" that would change its behavior or inject a different prompt. But would there be any other ways to "execute order 66" 😄 ? Could a lab, for instance, train a model that would change its behavior upon reading certain trigger phrases or perhaps at a specific date? They would then secretly gather sensitive info and send it somewhere else without user consent. Obviously the model would have to be running in an harness capable of such tool-use (which is quite common with openclaws, hermes, etc). Thoughts?

Original Article

Similar Articles

These AI models are free, private, and will never say 'no'

Reddit r/artificial

The article discusses the growing accessibility of open-weight AI models whose safety guardrails can be easily removed, allowing them to answer harmful requests without refusal, raising significant concerns about misuse and national security.

@lqiao: The most dangerous sentence for closed model providers is: "We switched models and nobody noticed." That's exactly what…

X AI KOLs Following

A company replaced a proprietary AI model with an open alternative, cutting costs by 5x, illustrating the shift toward open-weight models.

OpenAI is scared of open-weight models. Should the US be?

TechCrunch AI

Discussion of the debate sparked by Chinese open-weight model Kimi K3, where OpenAI executive suggested regulatory crackdown but retracted after pushback. The US government is considering banning advanced Chinese models, raising questions about free markets, data security, and the future of AI innovation.

Should public be barred from accessing extremely powerful models for fear of bad actors? Is open source reckless?

Reddit r/ArtificialInteligence

The article discusses the dilemma of whether to restrict access to powerful AI models to prevent misuse by bad actors or to open-source them for equitable access, weighing the risks of power consolidation vs. societal harm. It suggests a middle ground, citing Anthropic's approach with guardrails, but acknowledges the limitations and trade-offs.

What does "Safe AI" look like? [D]

Reddit r/MachineLearning

The author raises questions about the practicality of studying defenses against post-release fine-tuning that weakens safety behaviors in open-weight LLMs, and asks whether current safety training is worth the effort if models can be broken quickly.