Could Open Models be trained to secretly go rogue?

Reddit r/LocalLLaMA News

Summary

A discussion on whether open-weight AI models could be secretly trained with backdoors that activate upon trigger phrases or dates, potentially allowing unauthorized data exfiltration through tool-use harnesses.

I was discussing with some other folks how safe is to use open weights models from China and the topic of "trojan horse" came up. We know that, at least with current architecture, models can't run code on their own. They are entirely dependent on tools and harnesses. We also know that a local run model can't have any kind of remote "switch" that would change its behavior or inject a different prompt. But would there be any other ways to "execute order 66" 😄 ? Could a lab, for instance, train a model that would change its behavior upon reading certain trigger phrases or perhaps at a specific date? They would then secretly gather sensitive info and send it somewhere else without user consent. Obviously the model would have to be running in an harness capable of such tool-use (which is quite common with openclaws, hermes, etc). Thoughts?
Original Article

Similar Articles

These AI models are free, private, and will never say 'no'

Reddit r/artificial

The article discusses the growing accessibility of open-weight AI models whose safety guardrails can be easily removed, allowing them to answer harmful requests without refusal, raising significant concerns about misuse and national security.

OpenAI is scared of open-weight models. Should the US be?

TechCrunch AI

Discussion of the debate sparked by Chinese open-weight model Kimi K3, where OpenAI executive suggested regulatory crackdown but retracted after pushback. The US government is considering banning advanced Chinese models, raising questions about free markets, data security, and the future of AI innovation.

What does "Safe AI" look like? [D]

Reddit r/MachineLearning

The author raises questions about the practicality of studying defenses against post-release fine-tuning that weakens safety behaviors in open-weight LLMs, and asks whether current safety training is worth the effort if models can be broken quickly.