Key Points of Liquid Cooling in the Evolution and Practical Applications of AI Computing Power
Sep 25, 2024
Leave a message
In recent years, with the rapid development of technologies such as artificial intelligence, big data, and large models, the demand for efficient cooling has been steadily increasing. Liquid cooling technology has gained widespread attention and application. Many well-known companies have invested in the research and application of liquid cooling technology, driving its continuous innovation and development. The application of liquid cooling technology is also gradually expanding in fields such as 5G communication and edge computing, providing strong support for their growth.
According to market research agencies, the global liquid cooling market is expected to maintain rapid growth in the coming years, reaching billions of dollars by 2025. In the Chinese market, the application of liquid cooling technology is also gradually increasing, and the market size is expected to double in the next few years.
I Evolution of Computing Power and Cooling
In the evolution of computing power, cooling plays a critical role. Every major breakthrough in computing power has often been accompanied by improvements in cooling technology. Early on, air cooling was the primary method, using fans to move air and dissipate heat. This is a more traditional and common cooling method.
As computing power increased and heat generation grew, more efficient heat pipe cooling technology emerged. Heat pipes transfer heat through the evaporation and condensation of a working fluid, offering good thermal conductivity. Liquid cooling technology gradually rose to prominence, effectively absorbing and transferring heat through circulating liquid, providing higher efficiency compared to air cooling. As computing demands in heterogeneous, HPC, and AI systems continue to grow, the importance of liquid cooling is becoming more apparent.
▲ Behind the Evolution of Cooling Technology is the Continuous lteration of Chip Technology
Advanced packaging has become a crucial path for extending Moore's Law as semiconductor processes approach physical limits. In addition to shrinking device size through process technologies, developing new materials, and improving circuit structures to enhance transistor density, changing packaging methods to increase integrated circuit capacity is also an important direction. In scenarios such as multi-chip 2.5D and 3D packaging, which enhance system performance, liquid cooling becomes indispensable in high-efficiency cooling solutions as system power and heat density increase in computing network frameworks.
As AI training and inference reconstruct computing network architectures, the growth rate of large model parameters is significantly faster than that of GPU memory. High integration, large memory, and multi-GPU systems are better suited for large model training and inference. With the significantly increased chip density in AIDC cabinets, the evolution from traditional cooling to efficient liquid cooling is inevitable.
▲ AIDC cabinets
II Application Scenarios and Technology of Liquid Cooling
On the chip level, when the typical power consumption of a chip exceeds 300W, liquid cooling is required to ensure the release of computing power. On the system level, the power of AI servers has increased from the 10kW level to tens of kW per cabinet, creating an urgent need for liquid cooling to penetrate. On the data center level, the only way to reduce IDC PUE from above 1.5 to 1.2 is by adopting liquid cooling.
Currently, the mainstream liquid cooling solutions in China include cold plate, immersion, and spray types, with cold plate being the most widely used.
▲ Cold Plate, Immersion, and Spray Types
▲ Cold Plate, Phase Changelmmersion, Single Phase lmmersion and Spray Cooling
As demands for computing performance increase, liquid cooling technology is playing a crucial role in the following key application scenarios:
1. Data Centers: Cooling servers and other IT equipment to improve energy efficiency and reduce operating costs.
2. Supercomputers: Handling large-scale computing tasks to ensure high performance and stability.
3. Artificial Intelligence: Training and running deep learning models to accelerate computation.
4. Medical Devices: Keeping equipment like MRI machines at operating temperatures.
5. Industrial Manufacturing: Cooling processing equipment to improve production efficiency and product quality.
6. Electric Vehicle: Cooling battery packs to extend battery life and improve safety.
7. Aerospace: Cooling electronics and engine components.
8. Research: Cooling various experimental equipment.
9. Gaming Computers: Providing high-performance cooling solutions.
10. Cryptocurrency Mining: Maintaining efficient operation of mining equipment.
III Development Trends in Liquid Cooling Technology
In the context of energy conservation and emissions reduction, the advantages of liquid cooling technology are gradually becoming apparent, and several new trends are emerging:
1. Higher Efficiency: Continuous improvement of cooling efficiency to meet growing computing demands.
2. Lower Energy Consumption: Reducing energy consumption through optimized design and materials.
3. Broader Applications: Expanding to more fields such as 5G communication and edge computing.
4. Smart Management: Achieving intelligent monitoring and management of liquid cooling systems.
5. Environmental Sustainability: Using environmentally friendly coolants and materials.
6. Integrated Design: Integrating with other technologies to improve overall system performance.
7. Cost Reduction: Lowering costs as technology matures and scales.
8. Improved Reliability: Enhancing the overall reliability and stability of liquid cooling systems.
9. Customized Solutions: Providing customized liquid cooling solutions for different application scenarios.
10. Heat Recovery: Exploring the reuse of heat generated by liquid cooling systems.
IV Characteristics and Application Scenarios of Common Liquid Cooling Technologies
▲ Characteristics and Application Scenarios of Common Liquid Cooling Technologies
V The Limit of Computing Power is Electricity
Discussing "East Data West Computation," as IDC/AIDC are high-energy-consuming industries, matching computing power with electricity is a practical need. According to Omdia 2020, the global electricity consumption of data centers accounts for 2% of society's total electricity consumption.
PUE is an important standard for evaluating the economic feasibility and energy consumption of IDC projects. "East Data West Computation" requires data center PUE levels higher than current standards (generally requiring PUE around 1.2 for national projects). The core to achieving energy-saving targets lies in temperature control energy-saving equipment. "East Data West Computation" signifies a significant increase in China's overall computing power level, and the demand for supporting temperature control cooling and energy-saving equipment will increase in tandem.
PUE = IDC total energy consumption / IT equipment energy consumption
IT equipment energy consumption = rated power per cabinet × number of powered cabinets × 24 hours × number of days per year × load factor
VI Improving the Economics of Liquid Cooling
Breaking down the cost structure of AIDC, liquid cooling penetration has already demonstrated economic viability, due to power density rather than just the cost of liquid cooling.
From the Capex perspective: construction (space cost), power distribution (power capacity), and thermal management equipment costs (air cooling or liquid cooling) make up the majority of the initial investment (not considering ICT equipment, cost share > 50%).
From the Opex perspective: electricity and depreciation are the primary ongoing operational costs (cost share can exceed 80%).
The core factor in measuring the economics of liquid cooling lies in the electricity savings achieved through PUE optimization, increased density, and whether these can offset the additional initial investment in equipment.
▲ Method of Improving the Economics of Liquid Cooling
VII The Logical Framework of Liquid Cooling Economics
In traditional IDC construction costs, construction, power distribution, and air conditioning are the core factors affecting the project's economic feasibility. As the power density of cabinets increases, the influence of power equipment Capex and annual electricity Opex in the IDC business model has grown significantly.
▲ The Logical Framework of Liquid Cooling Economics
VIII Liquid Cooling for Computing Power
Similar to the AI computing power system, the efficient and stable operation of energy storage systems also requires strict temperature and humidity conditions. Temperature directly affects battery capacity and efficiency degradation, and it is directly related to thermal runaway incidents. Currently, the mainstream cooling technologies for energy storage include air cooling, liquid cooling, heat pipe cooling, and phase-change cooling. Air and liquid cooling are the industry mainstream.
According to China Energy Storage Network, the battery cost in energy storage systems accounts for about 55%, PCS accounts for about 20%, BMS and EMS combined account for about 11%, and the cost of thermal management varies between 2-4% depending on the cooling technology chosen.
With the construction of large-capacity, high-density energy storage stations, such as new energy power stations and off-grid storage, driven by large energy groups and large system integrators, the penetration of liquid cooling in energy storage is increasing. The expansion of energy storage temperature control from IDC precision temperature control, industrial temperature control, and new energy vehicle temperature control suggests possible future developments.
▲ Energy Storage Systems
The downstream concentration of the energy storage temperature control industry is high, with strong bargaining power and negotiation leverage. Once a supply qualification is established, the relationship is sticky, making first-mover advantages important. Certification of core AI chips and terminal manufacturers, along with service capabilities, are the key barriers to entry for liquid cooling in computing power.
The large-scale application of air and liquid cooling systems in energy storage, along with the rapid growth in demand, makes product delivery capabilities and cost control critical. Reducing ICT and IDC investment costs through manufacturing and cost control is essential.
As storage capacity increases, product customization demands grow stronger. Liquid cooling systems require high customization in terms of the number of flow paths, flow rates, and flow speeds, prompting customers to choose manufacturers with co-design capabilities. The component segment focuses on standardized products with significant performance variations across products, while the system segment focuses on non-standard products, requiring an understanding of thermal management technology and knowledge of ICT and IDC systems.
Ⅸ How to Choose the Right Liquid Cooling Technology
Choosing the appropriate liquid cooling technology requires considering the following factors:
- Cooling Needs: Determine the cooling requirements of your equipment or system. Different applications and devices have varying cooling demands, such as high-performance computing, data centers, or gaming PCs, which may need stronger cooling capabilities.
- Technology Type: Understand the different types of liquid cooling technologies, such as cold plate, immersion, and spray types. Each technology has its characteristics and applicable areas, requiring careful selection based on specific needs.
- Cost: Liquid cooling technology generally involves higher costs, including equipment, installation, and maintenance costs. Choose a technology that fits your budget.
- Space Requirements: Liquid cooling systems often require some space for installation and operation. Consider the size and space constraints of your equipment.
- Reliability and Maintenance: Choose reliable liquid cooling technology and suppliers to ensure system stability and reliability. Understand the system's maintenance requirements to facilitate daily upkeep and troubleshooting.
- Compatibility: Ensure that the liquid cooling technology is compatible with your equipment and components without causing damage to other parts.
- Performance and Efficiency: Compare the cooling performance and efficiency of different liquid cooling technologies and choose the one that meets your needs.
- Environmental and Safety Considerations: Consider the environmental impact and safety of the liquid cooling technology, opting for environmentally friendly, non-toxic, and non-flammable coolants.
- Technical Support and Services: Choose a supplier that offers good technical support and after-sales service to address issues promptly.
Taking all these factors into consideration, you can choose the most suitable liquid cooling technology for your needs. Before making a decision, it is advisable to consult professional liquid cooling technology suppliers for more detailed information and recommendations.
As technology advances, liquid cooling technology will become more mature and widespread, with applications expanding continuously. In the future, liquid cooling may integrate with AI and IoT, leading to more intelligent heat management. The development of liquid cooling technology will bring more opportunities and challenges to various industries, requiring continuous innovation and exploration.