At DevDay 2024, OpenAI announced several new developer tools, enabling them to create advanced AI applications more easily. As expected, the company did not announce any new models during the event.
OpenAI made four major announcements for developers. First, it announced a public beta of the Realtime API, which will allow developers to build low-latency, multimodal experiences in their apps. This will enable developers to build applications similar to ChatGPT’s Advanced Voice Mode with natural speech-to-speech conversations using six preset voices. For developers who don"t need the low-latency benefits of the Realtime API, OpenAI also announced audio input and output in the Chat Completions API.
The Realtime API text input tokens will cost $5 per 1 million and $20 per 1 million output tokens. Audio input will cost $100 per 1 million tokens, and output is $200 per 1 million tokens. So, it will cost around $0.06 per minute of audio input and $0.24 per minute of audio output.
Second, OpenAI announced vision fine-tuning on GPT-4o, allowing developers to fine-tune the model using images along with text. This will be useful in scenarios like improved visual search functionality, improved object detection for autonomous vehicles, accurate medical image analysis, and more. Vision fine-tuning is now available for all developers using the latest GPT-4o model snapshot, ‘gpt-4o-2024-08-06’ on paid usage tiers. OpenAI is also offering 1 million training tokens per day for free through October 31, 2024, to fine-tune GPT-4o with images. After that, fine-tuning training will cost $25 per 1 million tokens, and inference will cost $3.75 per 1 million input tokens and $15 per 1 million output tokens.
Third, OpenAI is finally catching up with Google and Anthropic with the new Prompt Caching support that allows developers to reduce costs and latency. Using Prompt Caching, developers can get a 50% discount and faster processing times. Also, developers need not do anything to enable Prompt Caching, as it is automatically applied on the latest versions of GPT-4o, GPT-4o mini, o1-preview, and o1-mini, as well as fine-tuned versions of those models.
Fourth, OpenAI announced a new Model Distillation suite that will allow developers to fine-tune smaller models using outputs from bigger frontier models. This will allow them to match the performance of advanced models on specific tasks at a much lower cost. Previously, model distillation involved a huge process and several disconnected tools. With the new Model Distillation suite, OpenAI now offers the entire distillation pipeline directly within its platform.
Model Distillation is now available to all developers. OpenAI is offering 2 million free training tokens per day on GPT-4o mini and 1 million free training tokens per day on GPT-4o until October 31st. After that, the cost of training and running a distilled model is the same as OpenAI"s current standard fine-tuning prices.
By streamlining workflows and reducing costs, OpenAI is making it easier for developers to innovate and push the boundaries of what"s possible with AI.