Why responsible AI development needs cooperation on safety

# Why responsible AI development needs cooperation on safety Source: [https://openai.com/index/cooperation-on-safety/](https://openai.com/index/cooperation-on-safety/) We’ve written a policy research paper identifying four strategies that can be used today to improve the likelihood of long\-term industry cooperation on safety norms in AI: communicating risks and benefits, technical collaboration, increased transparency, and incentivizing standards\. Our analysis shows that industry cooperation on safety will be instrumental in ensuring that AI systems are safe and beneficial, but competitive pressures could lead to a collective action problem, potentially causing AI companies to under\-invest in safety\. We hope these strategies will encourage greater cooperation on the safe development of AI and lead to better global outcomes of AI\. Companies have a stronger incentive to cooperate on safety if the mutual benefits from safe development are higher\. The prospect of cooperation can be improved by highlighting the benefits of establishing good safety norms early, such as preventing incidents of AI failure and misuse, and establishing safety standards that are based on a shared understanding of emerging AI systems\. Collaborative efforts like[Risk Salon⁠\(opens in a new window\)](https://risksalon.org/), which hosts events for people working in fraud, risk, and compliance, are a good example of this\. These events facilitate open discussions between participants from different companies, and seem to be primarily motivated by the shared gain of improved risk mitigation strategies\. Reducing any advantages companies can expect to get by not cooperating on safety should increase overall compliance with safety standards\. For example, companies producing USB connectors don’t expect to gain much from deviating from USB connector standards, because doing so will render their product incompatible with most devices\. When standards have already been established and deviating from them is more costly than any benefits, advantage is low\. In the context of AI, reducing the cost and difficulty of implementing safety precautions would help minimize the temptation to ignore them\. Additionally, governments can foster a regulatory environment in which violating high\-stakes safety standards is prohibited\. We’ve found four strategies that can be used today to improve the likelihood of cooperation on safety norms and standards in AI\. These are: **1\. Promote accurate beliefs about the opportunities for cooperation** Communicate the safety and security risks associated with AI, show that concrete steps can be taken to promote cooperation on safety, and make shared concerns about safety common knowledge\. **2\. Collaborate on shared research and engineering challenges** Engage in joint interdisciplinary research that promotes safety and is otherwise conducive to fostering strong collaboration \(e\.g\. work that involves combining complementary areas of expertise\)\. **3\. Open up more aspects of AI development to appropriate oversight and feedback** Publicize codes of conduct, increase transparency about publication\-related decision\-making, and, provided that security and IP concerns are addressed, open up individual AI systems to greater scrutiny\. **4\. Incentivize adherence to high standards of safety** Commend those that adhere to safety standards, reproach failures to ensure that systems are developed safely, and support economic, legal, or industry\-wide incentives to adhere to safety standards\. We think collective action problems may be a principal source of policy challenges as AI systems become increasingly powerful\. This analysis focuses on the roles that industry can play in preventing such problems, but we anticipate that legal and political mechanisms will also play an important role in preventing and mitigating these issues\. We also anticipate that identifying similar mechanisms to improve cooperation on AI safety between states and with other non\-industry actors will be of increasing importance in the years to come\. There is a great deal of uncertainty about the challenges that future AI systems may pose, but we believe that encouraging greater cooperation on the safe development of AI is likely to have a positive impact on the outcomes of AI development\. While we acknowledge that such challenges exist, we advocate for a more thorough mapping of possible collaborations across organizational and national borders, with particular attention to research and engineering challenges whose solutions might be of wide utility\. Areas to consider might include joint research into the formal verification of AI systems’ capabilities and other aspects of AI safety and security with wide applications; various applied “AI for good” projects whose results might have wide\-ranging and largely positive applications \(e\.g\. in domains like sustainability and health\); and joint development of countermeasures against global AI\-related threats such as the misuse of synthetic media generation online\. To achieve greater cooperation on safety, we need to make it common knowledge that such cooperation is in everyone’s interest, and that methods for achieving it can be identified, researched, and implemented today\.