Jan Leike, a former leader in AI safety at OpenAI, has joined rival company Anthropic, which makes the Claude AI chatbot. He left OpenAI earlier in the month because the company was giving up on superalignment where they develop ways to keep control over hypothetical super-intelligent AIs.
Leike said that he will continue working on superalignment at Anthropic. His new team will work on scalable oversight, weak-to-strong generalisation (where weaker AI systems guide stronger AIs to achieve better results), and automated alignment research. He said in his post that if anyone with the necessary skills is interested in working in his team, then he can be contacted by direct message.
I"m excited to join @AnthropicAI to continue the superalignment mission!
— Jan Leike (@janleike) May 28, 2024
My new team will work on scalable oversight, weak-to-strong generalization, and automated alignment research.
If you"re interested in joining, my dms are open.
According to TechCrunch, a source familiar with what’s going on said Leike will report to Jared Kaplan, Anthropic’s chief science officer. In turn, Anthropic employees working on scalable oversight will report to Jan Leike.
In May, Jan Leike and OpenAI co-founder Ilya Sutskever both left OpenAI. Around this time, it was revealed by insiders that OpenAI had dissolved the superalignment team and transferred those busy with that to other projects in the company. On his departure, Leike criticised the direction of the company, spurring Sam Altman and Greg Brockman to write a response explaining that OpenAI does care about safety.
Since Leike left, OpenAI has announced the creation of the Safety and Security committee, which will keep an eye on the company’s safety procedures as it develops new products to ensure they’re safe.
OpenAI is never too far from controversy, just today, Neowin reported that when ChatGPT was launched back in 2022, none of the board members, who would go on to overthrow Sam Altman temporarily, were given any notice of the product’s launch. Former board member Helen Toner claimed that she first heard about it from Twitter.
Given Anthropic’s strong emphasis on safety, hopefully, Leike will be able to get undisturbed with his work to ensure super-intelligent AIs don’t get too powerful and overthrow humanity.
Source: Jan Leike via TechCrunch