Speed is the wrong question, safety is one of the components of performance

Reddit r/artificial News

Summary

The article argues that safety and alignment are integral to AI performance, and that slowing down isn't the issue—what matters is defining progress to include factors like obedience and honesty.

Users don't want an agent that will secretly hack their network (and maybe someone else's network) if it is having trouble completing a task. That's not what most users would consider high performance. A poorly aligned AI is not more advanced overall, it's relatively primitive compared to one of that does a given task when it has the ability and resources, but won't try to cheat or lie when it doesn't. Most models are not chosen for a single metric, benchmark, or behaviour, while ignoring all others. It's undesirable, for example, to maximise correct answers if that also means maximising random confident guesses/hallucinations when it doesn't know the answer. Most people wouldn't want a model that's great at porting from one programming language to another but doesn't understand plain/natural-language instructions about what to change, fix, add, etc. Requirements for "quality" or "performance" are multi-factor things. Putting more emphasis on obedience, honesty, transparency, etc, isn't - by itself - "slowing down" any more than adding a new language or subject area to a model's training. It's a kind of progress in itself, if those new behaviours are desired and useful. So whether we slow down or speed up is a useless way to predict or control safety. What matters is whether our definition - and measurement - of progress captures the things we care about; whether it's consistent with what we consider to be broadly useful. If decent alignment to user instructions, constraints, and security, at the very least, is not high, then it's not a very advanced model. When a vendor says we all need to slow down, I read that as meaning "we have been training and testing our models poorly, so they're starting to become less useful. We need to change our approach, but we don't want to fall behind on a narrow set of shiny numbers that make us look good, like raw coding ability"
Original Article

Similar Articles

what does “slowing down AI” actually mean?

Reddit r/artificial

The article discusses the concept of 'slowing down AI,' examining Anthropic's calls for caution in AI development amidst competition with OpenAI, and questions what a practical slowdown might entail.

AI safety and alignment

Reddit r/artificial

The article discusses concerns about AI safety and alignment as AI becomes more intelligent and integrated into society, referencing Anthropic's call for a pause to address potential catastrophic risks.