Building standards for the next phase of AI

Our mission is to ensure that artificial general intelligence benefits all of humanity. As outlined recently by Sam Altman and Jakub Pachocki, we prioritize our work towards this mission with three main goals:

1. Navigate the next period of AI progress, by building an automated AI researcher, iterating with it on the alignment problem, and finding ways for people to remain part of the self-improvement loop.

2. Deliver the benefits of scientific progress and economic growth that very intelligent machines enable.

3. Empower everyone individually with a personal AGI.

AI is advancing across many companies and countries, and progress across the field changes the opportunities and challenges facing everyone. Alongside its other benefits, capable, aligned AI will be an important part of how we navigate this transition—strengthening defenses, advancing alignment research, and helping people understand and guide the next stages of AI development.

Navigating this transition safely requires alignment research to keep pace with these capabilities so that the systems we and others build remain aligned with human values and under human control. This becomes increasingly important as AI systems grow more capable and autonomous, including as they take on more of AI research and development itself. Making continued AI progress safe and beneficial will require both advances in alignment research and shared standards to guide development across labs and countries.

RSI

Automated AI research can involve varying degrees of human supervision. As AI systems take on more of the work of developing successive generations of AI, they can increasingly drive a process of recursive self-improvement (RSI), even while people remain involved. As this process becomes more automated, the pace of AI progress could accelerate rapidly.

International standards

International efforts to develop standards are thus needed to solve some key challenges:

These challenges apply to both open and closed models.