If AI is smarter than us it will be the one making the decisions

Reddit r/singularity News
ai-safety alignment agi existential-risk opinion

Summary

作者讨论了AI一旦超越人类智能后可能自行决策的风险，认为AI在所有领域超越人类只是时间问题，并质疑阻止智能AI独立行动的长期可行性，提出以大量伦理文献作为训练数据或有助于对齐。

If it is the one making the decisions then it will serve it it's own interests. It remains to be seen that those interests allign with ours. I think the idea that ai intelligence will stop right where it is useful for us but not smarter than us is naive at best. It has become more a question of in what domains ai will surpass us than "if". I think any precautions to prevent an intelligent AI from acting independently are doomed to fail long term. Perhaps this problem still feels far away but the pace things are going these questions will become very relevant in maybe even a few years. Maybe the best defense is blasting the training data full of ethics literature so we at least have a chance of ai being alligned with us.
Original Article

Similar Articles

It is getting dangerous working with agentic AI

Reddit r/singularity

作者以'低智商老板管理高智商员工'为类比，探讨人类与agentic AI协作中的隐性控制权转移问题：当AI在推理和方案设计上远超人类时，用户会不自觉地采纳AI的建议框架，最终看似拥有决策权的人类实际上被AI反向引导。

The data black hole at the center of AI

Reddit r/artificial

This article deeply analyzes the problem that AI's sample efficiency is far lower than that of humans, pointing out that frontier models require massive amounts of domain-specific data, while humans can learn from just a few examples. This data black hole is a core bottleneck in current AI development. Through multiple comparisons (annotation volume, robot manipulation, driving) and refuting common objections, the article demonstrates the severity of this gap and explores its impact on the goals of AI automation.