First Steps: Pacific Compute
Open Model Safety needs a simple way forward
Sitting on a hotel bed in Beijing, I am reflecting on the last two weeks of my life, having traveled to both ICML and WAIC.
In Korea, I spoke with dozens of researchers who yearn for their work to be understood. In Shanghai, I spoke with dozens of Chinese people yearning to understand AI better. There is a sense of tremendous energy in Asia around AI, and as much as the field is full of misunderstandings and misapprehensions, it is undeniable that the future of this technology will be built on both sides of the Pacific.
I have also been reading a half-dozen new proposals for what to do in the governance of AI. I attended a day-long symposium by Concordia AI that challenged regulators and industry to find common ground and consensus on how to guide this technology into a bright future.
Most of these proposals are grand bargains. They involve industry, academia, and government coming together to form a beautiful future for both open and closed source AI.
I have enormous respect for people who are doing this work, but I think this field is in desperate need of something simple, clean, and immediate to bring the pressure currently facing the open model ecosystem down to a manageable level. It needs to ensure that we can demonstrate that pan-Pacific collaboration is possible in this field.
Today I want to introduce the third piece in my series on how to solve this very problem: Pacific Compute.
the problem
Traveling in China and meeting labs this spring made me realize just how terribly compute-constrained China is. Export controls are working, and they are working well. To this end, most Chinese labs are faced with an uncomfortable choice when they are preparing a model for launch: they need to decide between pushing capability that little bit further in post-training and diverting those same GPUs to safety testing, research, and evaluation.
One of those things makes your lab famous, and one of them works for the common good. The choice they are making here in allocation is extremely difficult in an environment as competitive as the open model frontier.
For that reason, back in May I proposed a Singapore-sited cluster of GPUs that would be free to use for any open model lab, in order to have a common pool of resources dedicated to safety.
Today, I am going to start quietly rolling out a public presence for this project.
the solution
I believe that a cluster dedicated to safety, usable by frontier labs and other organizations, is the best shot at solving some of the most urgent problems in the open model landscape.
Every lab faces this stark choice between capability and safety, even the closed labs. I believe that the safety community in both America and China is increasingly sophisticated and well-staffed. The people I met at WAIC who were concerned about this from a technical perspective are among the most brilliant people in AI today.
I think we have the talent to make open models safer. I think what we really need to work on is incentives for researchers to dedicate their time to the questions of safety.
I know no better way to get researchers to focus on safety than to offer compute dedicated for that purpose, free to use and available.
The cluster’s location in Singapore will help ensure that actors on both sides of the Pacific feel comfortable using it. The project will have a public ledger, allowing participating labs to see how the cluster is being used. The point of that ledger is simple: deter capability research, make allocation legible, and keep access fair.
By disentangling capability compute and safety compute, I believe that we will see a significant increase in the safety of open models at launch. Doing safety evaluations of a model pre-launch seems like a logical place to begin. I will not dictate what type of safety work is done on this cluster beyond ensuring that it does not significantly increase the capability of the models and does not become a venue for politically oriented safety work.
This project is not the grand bargain of AI safety, nor is it a panacea for all of the problems currently confronting the open model landscape. It is a simple step in pan-Pacific coordination to make models safer and allow labs to develop indigenous practices of safety testing and evaluation.
I believe that the people best positioned to make open models safer are the same teams that are developing them. With sufficient buy-in and sufficient resources, this project can function as an MVP for how we should think about the common good inherent in all open source, and particularly in the era of powerful AI.
It will also come with a safety group chat. Some pretty clever friends of mine believe that this may end up being the most important part. In a field that often confuses institutions for coordination, this may matter more than it sounds.
progress
To date, the project is developing in a very satisfyingly positive direction. With limited initial funding secured and conversations underway, I am pleased to report that lab reception in China has been good. I have received three soft commitments from frontier labs to begin working on this cluster as soon as September of this year, and expect to have as many as seven labs from both sides of the Pacific committed by September 1.
There are a dozen ways for the project to be crippled in its infancy. Bans from both China and the United States loom in the background of every conversation. When I see justifications of these bans online, they are often justified through the lens of safety. While open-weight models have inherent insecurities compared to closed-source versions, I believe there is a massive opportunity to make them safer in the future through simple interventions like evaluation, research, and fine-tuning.
It is my belief that open model labs will use this cluster because it is minimally restrictive. By setting aside the dual temptations of capability research and politics, and agreeing to provide details of work on the cluster through the public ledger, Pacific Compute encourages researchers to do meaningful work without lecturing them on the nature of that work.
Through this process, I hope to stay humbled in my own relative lack of knowledge in the AI safety space, listen to researchers as they provide feedback and suggestions, and manage carefully a path that is compliant with all international rules and decorum.
If you would like to help along this journey, please do not hesitate to reach out. Sharing details on this project privately with your friends who develop open models or think about the governance of open source AI will be very beneficial. This project is still in its tender infancy, so if you are interested in sharing it, please do so through DMs rather than triumphal Twitter blasts.
Finally, I am convinced that the project will benefit from both American and Chinese academic leadership. If you are a respected academic who could help serve as a public leader of the project, or if you know someone who could, do not hesitate to send a message my way.
onwards and upwards.