Intel has unveiled its latest AI chip, the Gaudi 3, in a bid to challenge NVIDIA"s leadership in the AI industry. The new chip promises significant performance and efficiency improvements over NVIDIA"s H100 GPU, the latter has a faster HGX H200 too.
According to Intel, the Gaudi 3 is more than twice as energy efficient and can perform AI tasks one and a half times faster than the H100. This improved efficiency could help reduce the exorbitant costs associated with training large AI models. The chip also comes in different modules, including an 8-chip configuration on a motherboard and a card compatible with existing Intel server designs.
Intel says it has tested the chip on popular AI models from Meta"s Llama 2 and Falcon project, showing that Gaudi 3 is capable of training transformers and running complex applications.
According to Intel benchmarks, Gaudi 3 is on average 50% faster at training large language models such as Llama with 7 billion and 13 billion parameters and GPT-3 with 175 billion parameters. In addition, Gaudi 3 is expected to achieve 40% better power efficiency than the H100 for inference.
In comparison to Nvidia H100, Intel Gaudi 3 is projected to deliver 50% faster time-to-train on average across Llama2 models with 7B and 13B parameters, and GPT-3 175B parameter model. Additionally, Intel Gaudi 3 accelerator inference throughput is projected to outperform the H100 by 50% on average1 and 40% for inference power-efficiency averaged2 across Llama 7B and 70B parameters, and Falcon 180B parameter models.
The full spec details of the Gaudi 3 are given in the image below:
Gaudi 3 connects to its host CPU via a high-speed PCIe Gen 5 x16 link. Intel is also developing liquid-cooled variants for even higher performance.
The launch marks Intel"s most serious challenge yet to NVIDIA"s dominance, which is estimated at around 80% of the AI chip market. With customers like Dell, HPE, and Supermicro on board to build servers with the new chips, Intel is aiming to steal a significant share.
"Gaudi 3 will be available to OEMs – including Dell Technologies, HPE, Lenovo and Supermicro – in the second quarter of 2024," Intel wrote in the blog post.