OpenAI has announced that it’s building a new team at the company that will consist of the top machine learning researchers and engineers. The objective of the team will be to develop scalable training methods that ensure superintelligent artificial intelligence doesn’t go rogue.
In the announcement, OpenAI said that superintelligence will be the “most impactful technology” ever created by man and that it will help to solve many of the world’s problems. Unfortunately, OpenAI thinks that superintelligent AI could be very dangerous and could lead to human extinction.
The company said that it has no way to steer or control superintelligent AI at present. For current AI models, OpenAI uses reinforcement learning and human feedback, which both rely on humans supervising AI.
Eventually, OpenAI doesn’t think humans will be able to reliably supervise AI systems that are much smarter than us, which will make current alignment techniques redundant. To address this challenge, it’s assembling a new Superalignment team with the following objectives:
- Develop a scalable training method.
- Validate the resulting model.
- Stress test our entire alignment pipeline.
Being such an important issue for OpenAI, the new team will have access to 20% of the computing power that has been secured so far. They will have the next four years to solve the problem of superintelligence alignment.
The new team is the company’s “chief bet” to solve the superintelligence alignment issue but the company still expects many other teams to contribute to solving the issue.
It’s very important to note that a superintelligence isn’t necessarily going to be malicious, but it could be, and that’s what OpenAI is concerned about. It believes that superintelligence could be reached this decade and says that it could fail in its goal of steering it, though it remains optimistic.
As it builds the Superalignment team, OpenAI is looking to bring on board people who have been successful in machine learning. It’s looking for people to fill roles such as research engineer, research scientist, and research manager.