NVIDIA's CEO, Jensen Huang, proclaimed during a Tuesday press event that the company has introduced their next-gen GH200 Grace Hopper superchip platform, specifically designed for the accelerating computing needs of the generative AI era.
Huang noted the growing demands of generative AI necessitate data centers to have specialized acceleration computing platforms. The new GH200 chip platform showcases superior memory technology and bandwidth, elevating the capability to maintain lossless connections in GPU aggregation. Moreover, the server design is easily deployable throughout an entire data center.
Highlighting the trend, the surge in large models has given rise to various AI-native applications, stimulating a sharp increase in computational requirements. Consequently, data centers tailored for data-intensive AI applications are swiftly emerging.
New Transformations for Data Centers
The Wall Street Journal reported that analysts have observed traditional cloud computing providers rapidly updating their data centers with advanced chips and other enhancements to meet the demands of AI software. This shift has opened opportunities for builders to create new facilities from the ground up.
A data center, resembling a massive warehouse, houses several servers, networking, and storage devices for storing and processing data. In contrast to traditional centers, AI-focused ones deploy more servers using high-performance chips. As a result, the power consumption for these AI data center servers can reach or exceed 50 kilowatts per rack, compared to the 7 kilowatts typical in conventional data centers.
This implies that AI data centers must expand their infrastructure to supply higher power. Given the added power usage generates more heat, AI centers also require alternate cooling solutions, such as liquid cooling systems, to prevent equipment from overheating.
Manju Naglapur, Senior Vice President of the services and consultancy firm Unisys, mentioned that specially constructed AI data centers can accommodate servers utilizing AI chips, like NVIDIA's GPUs. These centers are designed to handle extensive data storage simultaneously for AI applications and are equipped with fiber-optic networks and more efficient storage devices to support large-scale AI models.
Constructing AI data centers involves significant investments in both funds and time. Research from Data Bridge Market Research indicates that by 2029, global spending on AI infrastructure is projected to reach $422.5 billion, with an expected compound annual growth rate of 44% over the next six years. Raul Martynek, CEO of DataBank, asserted that the rapid deployment of AI could lead to a shortage of data center capacity within the next 12 to 24 months.
AI Powerhouse CoreWeave Secures $2.3 Billion in Financing
Presently, industry giants are placing their bets on AI data centers. Blackstone, the real estate benchmark, is divesting from real estate to invest in AI data centers, while Meta has expressed intentions to build a new AI data center.
As previously mentioned, AI powerhouse CoreWeave secured debt financing of $2.3 billion (approximately 165 billion yuan) by using NVIDIA's H100 as collateral. CoreWeave announced this funding will accelerate their AI data center constructions. This financing comes after two previous rounds this year - $221 million in April and $200 million in May. Established six years ago, CoreWeave currently operates seven AI data centers, with plans to double this number by year's end.
In partnership with NVIDIA and Inflection AI, CoreWeave is in the process of constructing a mega-sized AI server cluster with the aim to operate 22,000 NVIDIA H100 units. Once completed, this will be the world's largest AI server cluster.
It's worth noting that, according to CoreWeave's official site, their services are 80% cheaper than traditional cloud computing providers. NVIDIA's latest HGX H100 server, with eight 80G memory H100s and 1T memory types, starts at a mere $2.23 per hour (16 yuan).
Compared to its predecessor, the new GH200 Grace Hopper platform, with its dual-chip configuration, offers 3.5 times the memory capacity, three times the bandwidth, and houses 144 Arm Neoverse high-performance cores, 8 petaflops of AI performance, and 282GB of the latest HBM3e memory technology.
It's no wonder, in this era of explosive growth in large language models, that Jensen Huang confidently asserts, "The more you buy, the more you save!"
