OpenAI's new chatgpt-4o-latest model re-claims the No.1 position in LMSYS Chatbot Arena

Last week, OpenAI announced its newest frontier model, officially named gpt-4o-2024-08-06. This new model came with Structured Outputs support in the API. Yesterday, OpenAI surprised everyone by announcing another new frontier model named chatgpt-4o-latest. While OpenAI still recommends developers use gpt-4o-2024-08-06 for most API usage, the chatgpt-4o-latest model will allow developers to test OpenAI's latest improvements for chat use cases.

According to OpenAI's official documentation page, chatgpt-4o-latest will be a dynamic model that will be continuously updated to the current version of GPT-4o in ChatGPT. The new chatgpt-4o-latest model supports a context of 128,000 tokens and 16,384 output tokens. This model is intended for research and evaluation only.

Last week, Google introduced a new experimental Gemini 1.5 Pro model in the LMSYS Chatbot Arena to claim the No. 1 position with a score of 1297. It was the first time Google topped the Chatbot Arena since its release. With the new chatgpt-4o-latest model, OpenAI has reclaimed the No. 1 position in the Arena with a record score of 1314. According to the scores, the new ChatGPT-4o demonstrates notable improvements in Coding, Instruction-following, and Hard Prompts.

New ChatGPT-4o Category Rankings in LMSYS Chatbot Arena:

Overall: #1
Math: #1-2
Coding: #1
Hard Prompts: #1
Instruction-Following: #1
Longer Query: #1
Multi-Turn: #1

This new chatgpt-4o-latest model has already been powering OpenAI's ChatGPT since last week. If you noticed improvements in ChatGPT's responses over the last week, it is safe to assume that the improvement is because of the new model.

OpenAI recently announced its DevDay 2024 developer conference, which will be held in three cities around the world. During the announcement, OpenAI explicitly mentioned that they will not reveal their next big frontier model, which began training earlier this year, during these events. Instead, these developer events will focus on improvements to the API and the dev tools.