Surging Demand Sparks a Computing Power Revolution, Liquid Cooling Will Strongly Drive Industry Growth!

 

Since 2023, OpenAI's AI chatbot model, ChatGPT, has received widespread acclaim. It uses large amounts of training data to simulate human language behavior, generating text through semantic analysis, allowing it to interact with users in a natural and realistic manner, often indistinguishable from human interaction.

 

In many areas, such as poetry creation and scriptwriting, it even surpasses human capabilities. However, the development of artificial intelligence requires extremely powerful computing power, leading to increased server power consumption, higher power density per unit in data centers, and rising energy consumption. Traditional air cooling technology can no longer meet cooling needs, making liquid cooling a necessity that will undoubtedly drive industry growth!

 

▲ OpenAI's AI chatbot model, ChatGPT

 

 

Ⅰ The Explosive Success of ChatGPT: What Kind of Computing Power Does It Require?

 

Data, algorithms, and computing power are the three core elements of artificial intelligence, with computing power being called the "engine" or "motor" of AI. Both data and algorithms rely on computing power. According to OpenAI estimates, since 2012, the computing power required to train the world's top AI models has doubled approximately every 3-4 months, with an annual increase in computing power required for top training models reaching as much as tenfold. During the training phase, ChatGPT consumed a total of about 3640 PF-days (which means running at 1 PetaFLOP/s efficiency for 3640 days). This highlights that strong computing power is the key to supporting the further application and popularization of AI-generated content (AIGC) in the future.

 

Take ChatGPT as an example: it is supported by the GPT-3.5 model, which has gone through three iterations (GPT-1, GPT-2, GPT-3), with the number of parameters increasing from 117 million to 175 billion, and the amount of pre-training data increasing from 5GB to 45TB (equivalent to 160 times the entire English version of Wikipedia). This massive amount of data and computation requires higher computing power to support its operation. According to data from Similarweb, in January 2023, ChatGPT had about 13 million daily active users, each asking questions averaging around 1,000 words.

 

This generated a total of approximately 13 billion words (17.33 billion tokens). Assuming the tasks are evenly distributed over 24 hours, the number of A100 GPUs required would be 17.33 billion * 2 * 300 billion / (20% * 24 hours * 3600 seconds) = 601.75 PetaFLOP/S. Given that traffic peaks exist, assuming peak traffic is five times the daily average, a total of 602 DGX A100 servers would be needed to meet the current traffic demand, making the computing power requirements extremely stringent.

 

This illustrates that to train even better AI models and to truly apply AI models across various industries, more efficient computing power and more abundant computing resources are needed. Computing power has become a critical factor limiting the industrialization of AI, and it is bound to trigger a new computing power revolution! With the rapid increase in AI computing power demand, the power of related CPUs/GPUs is accelerating, necessitating stronger and more effective cooling solutions to keep the equipment running smoothly.

 

 

Ⅱ Extensive Applications of Computing Power

 

In the digital economy era, computing power is not only a key productivity factor but also set to become a universally applied one. As the construction of computing power infrastructure progresses, new models and business forms of computing power applications are rapidly emerging, deeply integrating computing power into more industries, such as manufacturing, transportation, education, aerospace, finance, and media. At the same time, computing power scheduling, management, and trading are becoming focal points of industry exploration. Computing power is evolving from a "resource" into a "factor," becoming an essential part of people's production and lives.

 

The development of various industries has triggered an explosion in computing power demand. To maximize processor performance, higher cooling efficiency is required, and traditional air cooling technology can no longer meet the surging computing power demands. Liquid cooling servers, with their advantages, are set to gain a significant market share and strongly drive industry development!

 

 

Ⅲ Significant Increase in Liquid Cooling Demand

 

In 2022, Intel's fourth-generation server processors saw single CPU power consumption surpass 350 watts, and NVIDIA's single GPU chip power consumption exceeded 700 watts. The computing power density of AI clusters generally reached 50kW per cabinet, and traditional air cooling technology can no longer cope! Higher cooling efficiency is needed to maximize processor performance.

 

Meanwhile, from a policy perspective, the initiation of China's "Eastern Data, Western Computing" project and the dual carbon goals have made the industry pay more attention to energy conservation, emission reduction, and high-quality development. Stricter technical requirements have been set for the Power Usage Effectiveness (PUE) of data centers, with the average PUE of newly built large and super-large data centers across the country to be reduced to below 1.3, and further reduced to below 1.25 at national hub nodes.

 

Liquid cooling meets market cost-effectiveness and policy guidance needs in many aspects, including economic efficiency and cooling performance, making it both highly valuable and economically beneficial. According to CCID Consulting, China's liquid-cooled IDC market is expected to exceed 120 billion yuan by 2025, with a growth rate of over 30% and a penetration rate currently around 20%. The demand for liquid cooling is increasing significantly, and the market prospects are promising.

 

 

Ⅳ The Liquid Cooling Server Industry Chain

 

1. The Panorama of the Liquid Cooling Industry

The liquid cooling server industry encompasses much more than just liquid cooling servers; the entire industry chain is long, with wide subfields and many participating players, making it a capital-intensive industry. Liquid cooling technology is currently in a rapid development stage, and if the relevant players have advantages in financing or customer acquisition, they can build certain barriers to entry.

 

Enterprises in various subfields are gradually infiltrating related fields based on their core advantages, constructing their own ecosystems. According to the division from infrastructure to applications, the liquid-cooled data center industry can be divided into three major subfields: liquid cooling infrastructure, midstream industry, and industry applications.

 

The Liquid Cooling Infrastructure Sector provides the essential equipment and facilities for constructing liquid-cooled data centers. It is divided into subfields such as liquid cooling cabinets, coolant, cooling systems, power systems, network systems, and data center infrastructure management systems. These subfields mainly supply equipment, software systems, and refrigerants for server operation. In addition, the infrastructure sector includes construction contractors, network operators, and power suppliers. Construction contractors are responsible for the building and installation of the aforementioned equipment and facilities, network operators provide basic network access, and power suppliers provide the electricity needed for equipment operation. These areas provide a relatively basic peripheral environment for data centers.

 

The midstream industry includes server manufacturers, data center hosting providers, and some new entrants. Server manufacturers research, design, and produce servers suitable for liquid cooling environments or directly produce cold plate liquid-cooled servers. Data center hosting providers build and operate data centers, leveraging their own advantages to integrate resources from the infrastructure sector and offer server hosting services to application users. As data centers rapidly develop, new companies are entering the liquid-cooled data center field.

 

Data center application users are mainly divided into three subfields: cloud computing and cloud computing power, the internet and blockchain, and other users. They are the primary users of data centers. Cloud computing and cloud computing power providers generate revenue by offering customers flexible computing power allocation and scheduling through virtualization. Some major internet and blockchain companies often build data centers directly in collaboration with data center operators, while other users generally either directly host or lease server or computing power resources from data center operators.

 

2. Overview of the Liquid Cooling Industry

Compared to air cooling, liquid cooling technology offers advantages of high stability and high cooling efficiency. From a comprehensive application perspective, liquid cooling gradually reveals its economic advantages when power densities exceed 15kW per cabinet. According to the UPS Application journal, the CPU core temperature of liquid-cooled servers can be 20-30°C lower than the maximum temperature in air-cooled systems, significantly improving reliability.

 

Intel's white paper, Green Data Center Innovation Practices-Cold Plate Liquid Cooling System Design Reference, indicates that air-cooled data centers can typically manage cooling for cabinets with power levels up to 12kW. For cabinets exceeding 15kW, the existing air-cooled data centers have reached the ceiling of their air convection cooling capabilities. Liquid cooling technology, which provides stronger cooling capabilities, can support higher power densities.

 

Compared to air cooling solutions, the trend toward liquid cooling driven by AIGC and the "Eastern Data, Western Computing" initiatives will inevitably lead to a rise in both the volume and price of the data center temperature control market. The liquid-cooled server industry follows three technological routes: cold plate, immersion, and spray cooling. Cold Plate Liquid Cooling involves a cooling method where the coolant does not directly contact the server's heat-generating components.

 

▲ Cold Plate Cooling

 

▲ Spray Liquid Cooling System

 

▲ Single-Phase Liquid lmmersion Cooling

 

▲ Two-Phase Cooling

 

Instead, the main heat-generating components are attached to a cold plate, with the coolant circulating within the cold plate's internal liquid cooling system to remove heat. To enhance heat exchange capabilities, the heat exchange area must be maximized, requiring the cold plate to be as closely attached as possible to the surface of the server's heat-generating components. Irregularly shaped heat-generating components are difficult to cool using this method. Only relatively regular-shaped chips are suitable for cold plate cooling.

 

This presents a significant issue for cold plate liquid cooling solutions: approximately 30% of the server's heat, generated by other components like storage, memory, and power supplies, still requires traditional air cooling. This results in the need for two cooling systems to be used simultaneously in cold plate liquid cooling solutions, increasing both costs and the actual system PUE. The current cold plate solution is relatively mature, with a solid commercial foundation, and has already seen numerous commercial applications.

 

▲ Cold plate liquid cooling solutions

 

Immersion Liquid Cooling refers to the complete immersion of server equipment in coolant, where cooling is achieved through direct contact between the coolant and the heat-generating components. In this method, the server's heat-generating components are fully submerged in the coolant, allowing for sufficient contact with the coolant, which continuously carries away the heat generated, thereby cooling the server. Immersion cooling offers higher cooling efficiency, with the potential to increase power density per cabinet by over three times. It is expected to account for more than 40% of the market by 2025, with full immersion liquid-cooled servers likely to become a future technological trend.

 

▲ Immersion Liquid Cooling

 

Spray Cooling Liquid Cooling System is a direct-contact liquid cooling technology designed to precisely spray coolant onto the heat-generating components of servers. The coolant is applied directly to the heat-generating devices or solid thermal conductive materials connected to them, either by gravity or system pressure. This process allows for heat exchange to cool the server. During heat exchange, the coolant level inside the server is kept below the heat-generating components or the solid thermal conductive materials connected to them. The system uses external heat exchange units to cool and recycle the coolant.

 

 

3. Liquid Cooling Market Landscape

As an advanced technology, liquid cooling spans multiple disciplines and fields, with high technical barriers.

Currently, the adoption of liquid cooling technology among domestic and international data center infrastructure manufacturers remains relatively low, with few companies mastering this technology.

 

However, several companies both domestically and internationally have started offering customized liquid cooling solutions to their clients, and some tech companies are gradually building and deploying their own liquid-cooled data centers. Despite this, the adoption rate of liquid cooling technology among data center infrastructure manufacturers remains low, and the market has yet to see the emergence of dominant leaders, leaving the competitive landscape still uncertain.

 

In Chinese market, some companies have, through years of technological accumulation and experience, demonstrated significant technical and market advantages, and it is expected that in the future, leading companies will dominate the market, forming a competitive landscape where major players hold significant market share.

 

4. Chip-Level Liquid Cooling

Reducing the distance between liquid cooling equipment and core heat sources like chips is an important trend. In the future, cooling is expected to evolve from room-level, cabinet-level, and server-level to chip-level, achieving better chip cooling through direct contact between cooling components and the chip surface. Liquid coolers dissipate heat by moving large amounts of heat away through the friction between the liquid flow and the internal surface of the radiator. Compared to conventional air cooling systems, liquid cooling can significantly improve cooling efficiency. Currently, liquid cooling technology is mainly divided into indirect and direct types.

 

According to the latest China Semiannual Liquid Cooling Server Market (Second Half of 2022) Tracking report published by the International Data Corporation (IDC), the Chinese liquid cooling server market reached $1.01 billion in 2022, representing a year-on-year growth of 189.9%. IDC predicts that from 2022 to 2027, the compound annual growth rate of China's liquid cooling server market will reach 56.6%, with the market size expected to reach $9.5 billion by 2027, indicating a bright future for the liquid cooling industry!

 

 

▲ The forecast for the liquid-cooled server market in China from 2023 to 2027