Urging an International AI Treaty: An Open Letter

We call on governments worldwide to actively respond to the potentially catastrophic risks posed by advanced artificial intelligence (AI) systems to humanity, encompassing threats from misuse, systemic risks, and loss of control. We advocate for the development and ratification of an international AI treaty to reduce these risks, and ensure the benefits of AI for all.

Leading experts, including Geoffrey Hinton, Yoshua Bengio, and the CEOs of OpenAI and Google DeepMind, have publicly voiced their concerns about the catastrophic risks posed by AI, and called for the reduction of this risk to be treated as a global priority. Similarly, AI experts have called for a pause on the advancement of AI capabilities. In recent months, world leaders have increasingly drawn attention to the need to ensure that AI is developed safely.¹ Half of AI researchers estimate more than a 10% chance that AI could lead to human extinction or a similarly catastrophic curtailment of humanity's potential. We consider that the gravity of these risks warrants immediate and serious attention.

We believe the central aim of an international AI treaty should be to prevent the unchecked escalation of the capabilities of AI systems while preserving their benefits. For such a treaty, we suggest the following core components:

• Global Compute Thresholds: Internationally upheld thresholds on the amount of compute used to train any given AI model, with a procedure to lower these over time to account for algorithmic improvements.
• CERN for AI Safety: A collaborative AI safety laboratory akin to CERN for pooling resources, expertise, and knowledge in the service of AI safety, and acting as a cooperative platform for safe AI development and safety research.
• Safe APIs: Enable access to the APIs of safe AI models, with their capabilities held within estimated safe limits, in order to reduce incentives towards a dangerous race in AI development.
• Compliance Commission: An international commission responsible for monitoring treaty compliance.

Crucially, the effectiveness and impact of such a treaty hinges on widespread agreement across the international community. These measures, combined, promote a safer exploration of AI's immense potential by fostering a cooperative, safety-first approach to AI research and development.

Such an AI treaty has the potential to play a role similar to the International Atomic Energy Agency (IAEA), which was formed to manage nuclear risks and promote the peaceful uses of nuclear energy. Similarly, an AI treaty could not just reduce risks from AI, but also ensure that the benefits of AI are accessible to all. The potential of AI extends far beyond our current understanding and should be viewed as a common good. It is essential that this extraordinary resource benefits all of humanity.

Humanity has shown remarkable unity when faced with global threats, as demonstrated by international cooperation on nuclear non-proliferation. We believe the risks posed by AI systems warrant at least as much caution and coordination.

We urge members of the international community to actively engage in discussions around an AI treaty, and strive towards implementing a robust set of international regulations. We advocate for the formation of a working group, with broad international support, to develop a blueprint for such a treaty. This responsibility does not rest solely on a few shoulders, but on the collective strength of the global community. Our future hangs in the balance. We must act now to ensure AI is developed safely, responsibly, and for the betterment of all humanity.