Back in May, OpenAI announced GPT-4o, its most advanced multimodal model. It is faster and cheaper than the GPT-4 Turbo, with stronger vision capabilities. Today, OpenAI shared the System Card for the GPT-4o model. Along with the model System Card, OpenAI also released the Preparedness Framework scorecard to provide an end-to-end safety assessment of GPT-4o and shared details on the model"s capabilities, limitations, and safety evaluations across multiple categories.
In addition to the safety evaluations and mitigations conducted for GPT-4 and GPT-4V, OpenAI conducted more safety work focusing on GPT-4o"s audio capabilities. The risks evaluated include speaker identification, unauthorized voice generation, the potential generation of copyrighted content, ungrounded inference, and disallowed content. Based on the results of these evaluations, OpenAI has implemented safeguards at both the model and system levels.
GPT-4o scored low in three of the four Preparedness Framework categories, with a medium score in persuasion alone. According to OpenAI"s Preparedness Framework, only models with a score of "medium" or below can be deployed, and only models with a score of "high" or below can be developed further.
OpenAI also collaborated with more than 100 external red teamers to evaluate the model before releasing it to the general public. Red teamers will make exploratory capability discovery, assess novel potential risks posed by the model, and stress-test mitigations.
The OpenAI team wrote the following regarding GPT-4o"s System Card:
"OpenAI has implemented various safety measurements and mitigations throughout the GPT-4o development and deployment process. As part of our iterative deployment process, we will continue to monitor and update mitigations in accordance with the evolving landscape."
OpenAI"s release of the System Card for its models demonstrates its commitment to responsible AI development, ensuring the safe and accessible use of AI for everyone. You can download the PDF version of the GPT-4o System Card here for further reference.