Special Report on Liquid Cooling in the Data Center Industry

 

--The Singularity Has Arrived Amid the AI Wave, Domestic and International Leaders Are Set to Erupt

 

I Rising Chip Power Consumption: Liquid Cooling Becomes Essential

 

1. Liquid Cooling Ensures High-Power Components Operate Normally

More than half of electronic component failures are due to excessive temperatures, with temperature accounting for 55% of failures. According to the "10-degree rule," every 10°C increase in temperature from room temperature doubles the failure rate and reduces lifespan. GPU failure rates increase, affecting server lifespan. The power consumption of individual chips is rapidly increasing, leading to greater heat generation. Mainstream processor chips like CPUs have a power rating of around 200W, with the latest CPUs exceeding 350W, and GPUs surpassing 1000W. Air cooling is no longer sufficient to cool high-heat-density electronic components, while liquid cooling offers much higher heat exchange efficiency. The volumetric heat capacity of liquids is 1000-3500 times greater than air, and their thermal conductivity is 20-30 times higher, giving liquid cooling a far superior cooling capability in the same space.

 

Liquid cooling efficiently absorbs the heat generated by components, providing precise cooling for heat-generating elements and keeping CPU core temperatures below 65°C (about 25°C lower than air cooling). It also prevents significant temperature fluctuations during sudden high-frequency operations, ensuring system safety and reliability. Liquid cooling even allows chips to overclock, improving chip performance by around 10%-30%.

 

▲ Electronic Component Failures

 

2. Domestic and International Chip Cooling Technologies Are Gradually Shifting to Liquid Cooling

Liquid cooling can maximize chip potential. For instance, in servers of equal performance, the liquid-cooled version outperforms the air-cooled version by about 10%. The GB200 adopts a new, efficient liquid-cooled cabinet architecture, and NVIDIA's future products may fully transition from air cooling to liquid cooling. Besides NVIDIA, domestic chips like Huawei Ascend have also seen a substantial increase in liquid cooling usage.

 

NVIDIA GB200 NVL72 is a multi-node liquid-cooled rack-level expansion system that combines 36 Grace Blackwell superchips, including 72 Blackwell GPUs and 36 Grace CPUs connected via fifth-generation NVLink. The liquid cooling technology used in the GB200 NVL72 not only improves computing density and reduces footprint, but also significantly cuts carbon emissions and energy consumption through high-bandwidth, low-latency GPU communication. Compared to traditional air-cooled NVIDIA H100 infrastructures, the GB200 delivers 25 times the performance at the same power consumption while reducing water use. According to TrendForce, the initial GB200 NVL36 architecture may feature both air and liquid cooling solutions, but due to higher cooling requirements, the NVL72 will prioritize liquid cooling. The GB200 cabinet system's liquid cooling supply chain is divided into five main components: cold plates, coolant distribution units (CDU), manifolds, quick disconnects (QD), and rear door heat exchangers (RDHx). The CDU, a key system, regulates coolant flow throughout the system to ensure the cabinet temperature is controlled within a preset TDP range. For NVIDIA AI solutions, Vertiv is the main CDU supplier, while Chicony, Auras, Delta, and CoolIT continue their testing and validation efforts.

 

▲ Liquid Cooling

 

3. Manufacturers Compete in Liquid-Cooled Servers; Shipment Volumes Grow Rapidly

According to IDC, China's liquid cooling industry is still in its early stages, but the country has already closed the technological gap with foreign counterparts. In fact, China's relevant industrial chain holds a leading edge in large-scale commercial applications. In 2023, China's liquid-cooled server market reached $1.42 billion, growing 48% year-on-year, with shipments reaching 161,000 units, a 57.3% increase. IDC forecasts a compound annual growth rate (CAGR) of 45.8% for China's liquid-cooled server market from 2023 to 2028, with the market expected to reach $10.2 billion by 2028.

Mainstream server manufacturers are aggressively expanding in the liquid cooling market. In terms of market share, Inspur, HyperFusion, and NingChang ranked as the top three vendors in 2023, collectively holding over 70% of the market.

 

Although the market is relatively concentrated, rapid growth and diversified industry demand are gradually narrowing the market share gaps among mainstream vendors. Inspur continues to pursue its "All in Liquid Cooling" strategy, aiming to achieve price parity between air and liquid cooling. Its liquid-cooled servers cater to clients in industries such as the internet, finance, services, manufacturing, and public utilities. The company has successfully deployed liquid-cooled servers at scale in leading internet companies and in the finance, education, and research sectors, establishing a new global benchmark with the first fanless, full cold-plate server.

 

▲ Market Share of Liquid-Cooled Server Manufacturers

 

According to IDC, in 2023, the internet industry remained the largest buyer of liquid-cooled servers in China, accounting for 46.3% of the market, with expectations of continued strong demand. Additionally, telecom operators and government-related users are also experiencing rapid growth in demand for liquid-cooled data centers. Other industries like finance, services, manufacturing, and public utilities are actively exploring appropriate liquid cooling solutions.

 

 

II Liquid Cooling Boosts the Value of Temperature Control, Industry Turning Point in Sight

 

1. Liquid Cooling Penetration Increases with Rising Server Power and Carbon Neutrality Goals

Data centers are major electricity consumers, accounting for about 2%-3% of national electricity use. Policies are increasingly stringent on energy consumption in data centers. In 2021, China's data center electricity consumption reached 216.6 billion kWh, accounting for 2.6% of the total national electricity consumption, with carbon emissions of 135 million tons, or 1.14% of total national emissions. With the dual carbon goals, data centers face unprecedented energy consumption and cooling challenges. After construction, electricity costs account for 60%-70% of the total operation and maintenance costs of traditional air-cooled data centers.

 

PUE (Power Usage Effectiveness) is the indicator used to evaluate energy efficiency in data centers, which is the ratio of total energy consumed by a data center to the energy consumed by its IT equipment. Before 2013, the average PUE for large data centers in China exceeded 1.7, but by the end of 2019, this had dropped to 1.46. The "Three-Year Action Plan for New Data Center Development (2021-2023)" aims to reduce the PUE of new large-scale data centers to below 1.3 by the end of 2023, with targets of below 1.25 for cold and severe cold regions. The "Eastern Data, Western Computing" policy mandates that nodes in Inner Mongolia, Gansu, Ningxia, and Guizhou reduce PUE to below 1.2.

 

The energy consumption of air conditioning systems in data centers is key to reducing PUE to reasonable levels. According to estimates, when the energy consumption of the air conditioning system accounts for 38%, 26%, and 17.5% of the total, the corresponding PUE values are 1.92, 1.5, and 1.3, respectively.

 

The power density of intelligent computing servers has increased significantly, reaching levels that air cooling systems cannot support. Previously, air cooling systems adapted to higher heat densities by bringing the cooling source closer to the heat source or by sealing off cold and hot aisles, which worked for cabinets requiring cooling below 12kW. However, as rack density exceeds 20kW, these methods become less effective. Vertiv believes that deploying facilities with ultra-high-density racks (30kW or more) leaves little choice but to use liquid cooling. No matter how the system is configured or optimized, air cooling cannot provide the cooling capacity needed to maintain the reliability of IT systems.

 

Therefore, to reduce data center PUE and meet the cooling needs of high-power cabinets, cooling technologies have advanced as follows:

 

• Phase 1 (1998-2004): Mainly used direct expansion air-cooled systems, including compressors, evaporators, expansion valves, and condensers, with refrigerants usually being Freon.
• Phase 2 (2005-2009): Primarily used water-cooled systems, including chillers, cooling towers, water pumps, and terminal units for chilled water.
• Phase 3 (2010-2023): Adopted evaporative cooling technology, which produces cool air or water using dry air. This technology can provide cooling air or water depending on needs. Because it doesn't require traditional compressors, it consumes less energy and is used in data centers requiring year-round cooling. Indirect evaporative cooling, the most effective solution for utilizing natural cooling sources, can reduce cooling energy consumption by 30% compared to traditional chilled water systems.
• Phase 4 (2024-present): Liquid cooling technology fundamentally improves the way the main equipment dissipates heat. It better meets the precise cooling needs of high-density racks and chips and offers advantages such as lower energy consumption, higher heat dissipation, lower noise, and lower total cost of ownership (TCO). Liquid cooling refers to using liquid as the cooling medium to exchange heat with heat-generating components in the server, carrying the heat away to ensure the server operates within a safe temperature range. It is suitable for scenarios requiring increased computing power, energy efficiency, and deployment density.

 

2. Cold Plate Liquid Cooling Is Highly Mature and Widely Used

Based on how the cooling medium contacts the server, liquid cooling can be categorized into indirect cooling and direct cooling. Indirect cooling primarily involves cold plate liquid cooling, which is further divided into single-phase and two-phase cold plate cooling based on whether the cooling medium undergoes phase change. Direct cooling includes immersion and spray cooling, where liquid comes into direct contact with the chip or rack, and is divided into single-phase and two-phase immersion cooling based on whether the cooling medium undergoes phase change.

 

Cold plate liquid cooling is currently the most widely used in the data center industry, and its primary application in the liquid cooling industry is in high-density computing scenarios. Cold plate liquid cooling refers to installing a cold plate directly on a chip for heat exchange, with a pump-driven liquid circulating to remove heat through a heat exchanger. The working principle of cold plate liquid cooling involves embedding microchannel heat sinks on top of the heat source, dissipating heat through liquid flowing through these microchannels, and then transferring it to a heat exchanger, which further dissipates the heat into the environment.

 

Cold plate liquid cooling, with its mature technology, advantages in cost and operational stability, is currently the mainstream solution for high-performance computing and AI computing in data centers. Cold plate liquid cooling typically results in a PUE of 1.2 or below, even reaching 1.05 in some cases. Direct liquid cooling technologies like immersion cooling generally deliver better performance but are less widely used due to higher technical risks and costs. Cold plate liquid cooling has several advantages over other liquid cooling technologies, including simple installation, low cost, scalability, easy maintenance, and short construction times. For instance, Meixin proposes that cold plate liquid cooling could reduce computing power energy consumption by up to 30%.

 

In practice, 75% of global data center liquid cooling systems use cold plate liquid cooling technology, and cold plate liquid cooling has also become the mainstream solution for China's AI computing centers. Cold plate liquid cooling is expected to continue its high growth. The compound growth rate of China's liquid-cooled servers will exceed 50% in the next five years, with cold plate liquid cooling systems dominating the market.

 

3. Liquid Cooling Is More Effective and Is Being Widely Promoted By Operators

Currently, the rapid increase in cabinet heat density due to the construction of data centers for AI computing has heightened demand for liquid-cooled solutions. The market share of liquid-cooled servers continues to rise, while the proportion of traditional air-cooled servers is rapidly decreasing.

 

▲ Annual Revenue Ofliquid Cooling Compared To Air Cooling 

 

In 2022, five liquid cooling industry standards led by the China Academy of Information and Communications Technology were officially implemented. The three major telecommunications operators plan to pilot liquid cooling technology in 10% of new data center projects in 2024, with over 50% of data center projects applying liquid cooling technology by 2025. China Telecom has already planned and constructed an intelligent computing center cluster in Shanghai, capable of supporting large-scale model training with trillions of parameters. The cluster, powered by domestic computing capabilities, will have 10,000 cards in a single pool, making it the first domestically produced large-scale liquid-cooled computing cluster to support a single pool of 10,000 cards.

 

The operators' pace of promoting liquid cooling is exceeding market expectations, which will drive cost optimization and standard improvement. To enhance the competitiveness of data center cabinets and reduce power consumption in temperature control, third-party data center vendors may accelerate the application of liquid cooling, leading to a wider regional distribution of liquid-cooled data centers in China. As liquid cooling data center technology matures and deployment costs decrease, more IDC (Internet Data Center) service providers are exploring the adoption of liquid cooling. In the east, users who have already applied liquid cooling data centers are seeing increased demand for liquid cooling solutions to meet stricter energy consumption requirements, further expanding the development of liquid-cooled data centers in central and western regions.

 

4. High-Speed Growth in the Liquid Cooling Market

The liquid cooling industry chain includes upstream suppliers of product components, midstream liquid cooling server and infrastructure providers, and downstream computing power users. The upstream mainly includes suppliers of product components and liquid cooling equipment, such as cold plates, CDUs (Coolant Distribution Units), quick disconnects, solenoid valves, immersion liquid cooling tanks, manifolds, and coolant products. The midstream includes liquid cooling server manufacturers, chipmakers, as well as liquid cooling integration facilities, modules, and cabinets. Downstream includes telecommunications operators, internet companies, and customers in industries such as telecommunications, the internet, government, finance, transportation, and energy.

 

In the upstream part of the industry chain, the heat exchange in cold plate liquid cooling systems is divided into two parts: the primary heat exchange system and the secondary heat exchange system. The secondary system acquires heat from heat sources like servers through direct/indirect heat exchange and transfers it to the primary system. The primary heat exchange system uses outdoor cooling equipment to dissipate the heat outside, completing the overall cooling process. The two systems exchange heat through the Coolant Distribution Unit (CDU).

 

Specific component breakdowns: A quick disconnect is a plug-and-socket component with fluid cut-off functions on both the plug and socket, allowing quick plug-and-play maintenance of liquid cooling systems. When the plug and socket are connected, fluid flows through to supply liquid to the cold plate; when disconnected, the fluid stops, preventing leakage outside the system. A manifold is a device that connects the CDU to the cold plate in liquid cooling servers, typically installed within a cabinet. Its function is to evenly distribute coolant to each cold plate and collect the heated coolant, sending it back to the CDU through connected pipelines.

 

A Coolant Distribution Unit (CDU) is a module used to exchange heat between the high-temperature coolant in the secondary system and the primary cooling source. It provides coolant distribution and manages temperature, pressure, and flow monitoring for liquid-cooled IT equipment. The CDU mainly consists of heat exchangers/condensers, circulation pumps, filters, coolant storage tanks, and accessories (valves, pipelines, connectors, sensors, etc.), and has functions such as heat exchange, circulation, coolant purification, and storage.

 

Between 2020 and 2022, liquid cooling technology matured rapidly, and validation of application models achieved significant success, leading to a reduction in per-unit power cooling costs. According to CCID Consulting data, the cold plate approach maintained over 90% market share, while immersion and spray cooling together accounted for around 10%. After a weighted average of the three methods, the cooling cost of 1kW in liquid-cooled data centers was approximately 6,500 RMB in 2022, with an expected drop to 5,000 RMB per 1kW by 2023.

 

Recently, Supermicro's CEO Charles Liang predicted that within the next 12 months, 15% of new global data center deployments will use its direct liquid cooling technology, and this proportion will reach 30% in the following year, a substantial increase from just 1% over the past 30 years, creating another major hockey stick growth curve. Compared to traditional air cooling, direct liquid cooling is more economical and environmentally friendly. Assuming that the penetration rate of liquid cooling in new data centers in China reaches 22% by 2026, the market space for liquid cooling will be 11.1 billion RMB.