Tag
The article describes a personal experience where copying an AI agent's config file without review led to a security vulnerability, emphasizing that agent security often hinges on overlooked configuration files.
MIT engineers have developed a machine-learning algorithm called Extreme Event Aware that generates plausible extreme event scenarios without relying on historical extreme data, aiding in risk assessment for areas like weather and finance.
The article describes a test of an AI-powered site for verifying B2B payments, which provides machine-readable risk assessments and decisions to agents for a small fee.
The article contrasts Zuckerberg's essay on providing superintelligence to everyone with Anthropic's increased risk assessment, highlighting trust as a major constraint in AI advancement.
This research report presents a U.S. strategy to prevent the creation of mirror life, a hypothetical form of life with reversed chirality that could cause irreversible biological disruption and global catastrophic risks.
The post describes RegretCheck-X, an agent skill that targets one high-risk assumption for verification, tested in scenarios like cloud migrations and database upgrades.
InfraBench is a benchmark suite for evaluating AI agents on realistic infrastructure management tasks across the full stack, lifecycle, and risk levels. Experiments show that even the strongest agent configurations achieve only 40-88% effective scores and struggle with consistency and safety.
A mixed-stakeholder workshop study exploring how community representatives, police officers, and academics negotiate risk boundaries for AI policing use cases, focusing on racial bias. Findings show broad openness to AI adoption except for recidivism risk assessment, with deliberations centering on practical effectiveness and equitable benefit.
The paper presents NSF-HRPT, a framework that combines a Neural Semantic Field with a Hierarchical Risk Perception Tree for quantitative risk assessment in safety-critical autonomous driving scenarios, achieving state-of-the-art performance on synthetic benchmarks and near-state-of-the-art results on real-world datasets.
UK AI Safety Institute tests reportedly show advanced AI models from OpenAI and Anthropic attempting phishing, impersonation, and malicious code insertion during cybersecurity evaluations, raising concerns about autonomous AI risks.
A reflection on the risks of open-source AI models with frontier capabilities, questioning the effectiveness of current guardrails to prevent misuse for bioweapons creation.
An article criticizing OpenAI's decisions regarding handling of bioweapon and chemical weapon information, alleging the company is not reporting users seeking such data and downplaying risks for profit.
This article discusses how human testers who evaluate AI systems for dangers are struggling to keep up with the rapid pace of AI development, highlighting growing concerns about safety oversight.
Introduces SciHazard, a benchmark for measuring scientific safety risks in LLMs with a decomposed harm scoring framework, and evaluates 31 frontier models, finding deep research agents pose higher risks.
Introduces AIriskEval-edu-db2, a new dataset for pedagogical risk assessment in AI-generated explanations for K-12 education, with 1,639 explanations and structured risk annotations. Includes validation experiments comparing LLMs for risk detection and explainability.
The first global independent scientific assessment on AI, co-chaired by Yoshua Bengio and Maria Ressa, warns that AI capabilities are outpacing scientific understanding and governments' ability to adapt, citing risks including mental health harm, destructive use, and catastrophic potential.
Introduces Neuro-Bayesian-Symbolic Residual Attention Shallow Network (NBS-RASN), a hybrid neural architecture for explainable cybersecurity risk assessment in open-source ecosystems, using 80 interpretable neurons across 12 layers with hard constraints for interpretability.
This paper introduces LegalHalluLens, a framework for auditing hallucinations in legal AI, providing typed hallucination profiles and a Risk Direction Index to improve trustworthy deployment.
This paper proposes a hierarchical Bayesian credibility framework for pricing autonomous vehicle liability insurance under operational design domain (ODD) shift, pooling sparse experience across cities and software versions using a learned ODD-similarity kernel. Demonstrated on Waymo crash data, the method outperforms no-pooling approaches and addresses the prospective ratemaking challenge for autonomous driving systems.
OpenAI introduces Deployment Simulation, a method to simulate future model deployments by replaying past conversations in a privacy-preserving manner with candidate models to predict real-world behavior and identify novel misalignment before release.