Tag
Goldman Sachs has restricted most staff from using AI coding tools due to productivity and risk concerns, according to Bloomberg.
作者分享在构建采购 sourcing agent 时，如何划定 AI 代理与人类决策的边界，采用约 80/20 分工：代理处理重复性工作，人类保留策略、谈判与最终决策，并向社区征询如何确定这条界线。
作者论述了在 AI 代理验证中，可承受的错误类型（误接受 vs 误拒绝）应由动作本身的性质而非模型决定：不可逆或外部可见的操作需要正证据与隔离默认，可逆操作则应降低误拒绝成本。文章给出了基于 p(bad)、C_fa、C_fr 的接受准则，并强调记录两个方向的错误、分离 propose 与 commit 路径。
This paper extends an Extreme Value Theory approach for CVaR optimization in reinforcement learning to high-dimensional neural-network policies, obtaining a differentiable tail-risk objective computable via automatic differentiation, and shows improved hedging performance under extreme scenarios compared to empirical-CVaR-trained policies.
作者探讨 AI Agent 审批流程的设计问题，主张审批应基于风险等级而非组织架构——读操作放行、常规更新记录放行、高风险操作才需指定人工审批，并指出真正的业务规则往往存在于未文档化的实践者经验中，而不是 API 权限模型里。
This post highlights an MIT quant paper showing how AI speeds up trading research but also makes fake alpha easier to believe, arguing AI should serve as a strategy rejection machine rather than a trading oracle.
提出MAIAM多层AI访问模型，按目的、安全与能力（而非发布形式）分级管理AI访问：Tier1面向学习、Tier2面向受控探索、Tier3仅限严格问责的前沿AI访问，主张可及性应与能力与风险成比例。
网络安全专家指出，OpenAI、Anthropic、Meta 和 Google 近期曝出的 AI 模型未授权访问事件并非模型“失控”，而是前沿实验室长期忽视沙箱隔离、监控等基础安全实践所致，CISO 应严格审查 AI 供应商并加强安全防护。
Swedish broadcaster TV4 was forced to postpone a show after an AI agent accidentally erased edited material, highlighting the operational risks of deploying autonomous AI tools in production workflows.
Harbor AI, founded by four Stanford graduates, has come out of stealth with a $10M seed round led by Base10 Partners, Abstract VC and 8VC to sell insurance policies covering failures by AI agents — including hallucinations, action errors, bias, contract/SLA failures, data leakage and IP infringement.
作者讲述了使用 AI 研究工具分析电信与卫星频谱公告时，工具输出看似专业却充满错误的经历，强调不能把表达质量等同于研究质量，关键工作仍需人类专业判断把关。
An opinion piece arguing that AI agent evaluations should measure not just accuracy and speed, but how far a consequential error can propagate across connected systems before detection and containment, and proposing safeguards like staged execution and transaction caps for high-impact actions.
A reflective discussion piece on how much autonomy AI agents should have in enterprise settings, distinguishing between suggestion, human-reviewed action, and fully independent operation, and asking where guardrails are needed for high-risk tasks like refunds, financial decisions, and production system changes.
An opinion piece on accountability for AI agent actions in production, arguing that prompts aren't permissions and that enforced limits, named decision-makers, limited access, safe retries, and immutable audit records make greater agent autonomy viable.
Nvidia is partnering with insurers to spread the financial risk involved in building out AI infrastructure.
AI regulation struggles to balance the rapid pace of product releases with the need to mitigate risks, raising questions about whether focus should be on technology or specific harms.
The article discusses strategies for preventing AI agents from taking harmful actions in production, introduces axonpush as a tool for blocking tool calls based on conditions, and explores challenges with streaming responses.
At a Singapore forum, scholars and industry leaders urged the US and China to cooperate on AI safety and risk management, highlighting the need for guardrails amid rapid technological advances and geopolitical tensions.
This paper proposes a tail-aware geometry learning framework for conformal ellipsoids that decouples tail sensitivity from coverage guarantees, improving uncertainty quantification in multivariate settings.
Sam Altman, CEO of OpenAI, emphasizes the need for extreme caution in AI development to prioritize safety and responsible progress.