Baidu, ByteDance, Tencent, and Alibaba have recently placed multiple bulk orders with Nvidia to fuel their generative Artificial Intelligence (AI) programs.
According to a Financial Times report (Paywall), Chinese companies have placed orders for 100,000 units of Nvidia"s A800 processor, valued at $1 billion. The companies plan to receive this shipment from the US chip maker by the end of this year. Furthermore, the companies have also placed an additional cumulative order of $4 Billion for processors to be delivered in 2024.
It is worth noting that A800 processor is the weakened version of the A100 processor that is used data centers, which in turn is the weaker version of H100 processor. Due to the export restrictions announced by the US government, Nvidia can only sell A800 processors to Chinese companies.
According to the export restrictions, Nvidia is allowed to sell processors with chip-to-chip transfer speeds lower than 600GBps, which rules out A100 and A800 processors.
With the threat of further restrictions as well as the overwhelming demand for GPUs, Chinese companies are stock-piling GPUs to ensure they have hardware available to fulfil their AI needs.
Two employees with direct knowledge of the matter said ByteDance had already stockpiled at least 10,000 Nvidia GPUs to support its ambitions. They added that the company had also ordered almost 70,000 A800 chips to be delivered next year, worth about $700 million.
While the Chinese companies did not comment on the matter, Nvidia did issue a statement in regard to the report:
Consumer internet companies and cloud providers invest billions of dollars on data centre components every year, often placing orders many months in advance.
With the rise in the demand for GPUs, fueled by the advancements in generative AI, experts fear that a 2020-like GPU shortage can occur again.
Currently, the sky is the limit when it comes to GPU demand, fulfilment of which is constrained by how many GPUs can Nvidia produce in a day or in a month. An Nvidia distributor noted that the price of A800 processor "in the hands of distributors has risen by more than 50 per cent".
As more and more companies jump into the AI space, we may see more bulk orders being placed to Nvidia as companies scramble to secure the hardware required to run their large language models (LLM).