Google today released two updated production-ready Gemini 1.5 models: Gemini-1.5-Pro-002 and Gemini-1.5-Flash-002. These new models deliver minor improvements over the original Gemini 1.5 models released in May.
The updated Gemini 1.5 series models deliver a ~7% increase in MMLU-Pro, a ~20% improvement in MATH and HiddenMath benchmarks, and ~2-7% improvements in vision and code use cases. Additionally, Google has improved the overall helpfulness of model responses. Both models now respond in a more concise style. According to Google, the default output length of the updated models is ~5-20% shorter than previous models.
In addition to the model improvements, Google is also making a significant change: They are reducing the pricing of Gemini 1.5 series model APIs.
- A 64% price reduction on input tokens.
- A 52% price reduction on output tokens.
- A 64% price reduction on incremental cached tokens for Gemini 1.5 Pro, from October 1st, 2024, on prompts less than 128K tokens.
Google is increasing the rate limits, allowing developers to build complex AI applications. The paid tier rate limits for 1.5 Flash are now 2,000 RPM, and 1.5 Pro is now 1,000 RPM, up from 1,000 and 360, respectively. Google has also reduced latency with the new models. Developers can expect 2x faster output and 3x less latency.
With the updated Gemini 1.5 (-002 models), Google has improved the model"s ability to follow user instructions while balancing safety. Also, by default, Google will not apply AI content safety filters to these new models. Instead, developers can use the filters based on their requirements.
Finally, Google released an improved version of the Gemini 1.5 model called “Gemini-1.5-Flash-8B-Exp-0924.” This improved experimental version includes significant performance increases across both text and multimodal use cases. All these updated Gemini 1.5 models are now available for developers via Google AI Studio and the Gemini API. For larger enterprises and Google Cloud customers, these new updated Gemini 1.5 models are available on Vertex AI.