Development and Discussion of Liquid-Cooling Data Center Engineering Technology

 

Since the specific heat capacity of liquid per unit volume is about 1,000 times that of air, liquid cooling (heat dissipation) has far greater cooling capacity than conventional air cooling. It is an effective solution for dissipating heat generated by ultra-high heat flux densities. In 1964, to address the overheating and downtime of large mainframes, IBM developed the world's first refrigerated water-cooled computer, the System360, pioneering liquid-cooled computers. Recently, driven by national and local energy efficiency management policies, liquid cooling technology has once again gained widespread attention in the data center industry, leading to the development of a series of liquid-cooled data center engineering technologies.

 

 

I Liquid Cooling Implementation Methods

 

In a broad sense, liquid cooling refers to all technologies related to liquid cooling. The industry is familiar with methods such as installing cold water coils on the back panels of racks, which are considered liquid cooling, falling under the broad concept of liquid cooling. The China Institute of Electronics (CIE), during the development of a series of liquid-cooled data center design standards, clarified the concept of liquid cooling through industry discussions. According to their definition, liquid cooling specifically refers to the direct removal of heat generated by electronic chips using liquids, so the narrow definition of liquid cooling only refers to the cooling process inside the server.

 

From the perspective of the narrow definition of liquid cooling, it can be classified into two main categories: direct contact of the cooling liquid with the electronic chip or indirect contact via a high thermal conductivity solid material. These can then be further subdivided based on the form of contact, type of cooling liquid, and whether the cooling liquid undergoes a phase change. Liquid cooling methods are generally considered to fall into eight categories (see Table 1).

 

▲ Table 1: Liquid Cooling Implementation Classification

 

 

II Demand for Liquid Cooling in Data Center Scenarios

 

Like conventional air-cooled systems commonly used in data centers, the role of liquid cooling is to remove the heat generated by IT equipment such as servers and other facility equipment (e.g., UPS batteries) to maintain a relatively stable environment with appropriate temperature and humidity in the data center.

 

Firstly, heat dissipation needs are the core driving force behind the development of liquid cooling technology. As the demand for data processing capabilities grows rapidly, the integration of electronic chips is likely to continue to grow exponentially in the future. This will lead to increasing power density and heat flux density. The need for faster data transmission speeds and enhanced user convenience will drive higher integration of equipment, which in turn will lead to greater heat dissipation requirements for IT equipment such as servers. This will also raise the environmental demands for operation. As a result, air cooling will require higher-speed, larger-diameter fans and larger heat dissipation channels, resulting in significant noise, greater environmental heat impact, and increased construction and operating costs. Liquid cooling offers a better cost-performance ratio.

 

Secondly, energy efficiency management policies are driving the adoption of liquid cooling technology. Another important reason the data center industry is focusing on liquid cooling is due to national and local energy efficiency management policies. The national and local requirements for data center Power Usage Effectiveness (PUE) are becoming stricter, bringing liquid cooling into the spotlight of the data center industry. The latest national standard, Data Center Energy Efficiency Limit Values and Energy Efficiency Grades (GB 40879-2021), requires energy consumption to be below 1.3, which is difficult to achieve using air cooling alone in most parts of the country, necessitating the use of liquid cooling technology.

 

Thirdly, the convenience of waste heat recovery may drive the adoption of liquid cooling in data centers. Data centers built using liquid cooling solutions have relatively higher-quality waste heat and are easier to recover. Waste heat recovery projects in liquid-cooled data centers are an effective way to achieve comprehensive energy utilization and improve energy efficiency. Some scholars have already proposed the idea of constructing large-scale data centers as heat sources for cities or industrial parks.

 

 

III Development of Liquid-Cooled Data Center Engineering Technology

 

The process of removing the heat generated by electronic chips through liquid cooling is just the beginning of the cooling process in a data center. Continuous heat generation from electronic chips requires sustained, stable, and reliable operation of liquid cooling data center engineering technology to maintain chip cooling.

 

The principles and practice of liquid cooling differ from air cooling. Specifically, the concepts of heat dissipation and cooling show subtle differences: starting from above room temperature or below room temperature. To better organize the liquid-cooled data center engineering technology framework, electronic chips are viewed as the source, with the goal of transferring the heat generated by the chips to outside the data center, ensuring the stable operation of IT equipment. Thus, liquid-cooled data center engineering technology is divided into primary and secondary cooling processes. This concept differs from the primary and secondary sides of conventional data center air cooling.

 

The primary cooling process of liquid-cooled data center engineering technology cools high heat flux density components of electronic IT equipment and transfers the generated heat outside the rack. It is also referred to as primary cooling (heat dissipation), initial cooling, internal cooling, or internal circulation cooling. The primary cooling process is strictly a liquid cooling process, and typically involves a closed loop of liquid cooling equipment or components at the chip end, a cooling distribution unit (CDU), coolant dispensers, and pipelines. The cooling distribution unit (CDU) contains pumps and heat exchangers, providing the circulation force for the cooling liquid. A typical primary cooling process involves circulating a certain temperature and flow of coolant from the CDU to the liquid cooling equipment or components at the chip end. The coolant exchanges heat with the chips through direct contact or indirect contact via materials with high thermal conductivity such as metals. The heated high-temperature coolant or coolant vapor then flows back to the CDU via the pipelines, where it exchanges heat with the secondary coolant. After cooling, the low-temperature coolant is driven back to the liquid cooling equipment or components at the chip end by the CDU, completing a full cycle. Common coolants used in the primary cooling process include ethylene glycol solutions, propylene glycol solutions, deionized water, etc., and some solutions use fluorinated liquids, though the physical requirements for coolants vary significantly across different liquid cooling solutions.

 

A cooling distribution unit (CDU) is a commonly used piece of equipment. A typical CDU architecture is shown in Figure 1. In addition to providing circulation force and heat exchange for the coolant in the primary cooling process, this equipment also serves the role of distributing cooling power (not just coolant flow). Therefore, it typically has the following functions:

 

1) Temperature and flow control: Dynamic monitoring of coolant temperature and flow in the primary cooling process via temperature and flow sensors. Based on built-in models, the CDU dynamically adjusts the coolant's temperature, flow, or supply pressure to provide adequate cooling capacity while avoiding condensation in the primary cooling loop.

2) Ensuring physical separation between the primary coolant and secondary coolant.

3) Online or bypass filtration of the coolant.

4) Supporting network management.

 

▲ Figure 1: A Typical Cooling Distribution Unit (CDU) Architecture

 

The secondary cooling process of liquid-cooled data center engineering technology transfers the heat removed by the primary cooling process outside the data center, also known as secondary cooling, external cooling, external circulation cooling, or heat dissipation. The secondary coolant in this process can be air, cooling water, water-based solutions (e.g., ethylene glycol solutions, glycerin solutions), or refrigerants, collectively referred to as secondary coolants. If air is used as the secondary coolant, the secondary cooling process is akin to the conventional cooling process inside a data center. If a water-based solution is used, the secondary coolant circulates in what is called a secondary cooling loop.

 

The heat exchange between the primary and secondary cooling loops occurs in the CDU. After heat exchange in the CDU, the high-temperature secondary coolant enters a cooling source or waste heat recovery equipment, transferring the heat to the environment or enabling its reuse. The secondary coolant then flows back into the heat exchanger after cooling, completing a full cycle. The cooling source could be a cooling tower, dry cooler, chiller, etc. Since the primary cooling process allows for the secondary coolant's inlet temperature to reach 30°C or higher, the cooling source can operate solely on natural cooling, which explains why liquid cooling technology can achieve relatively ideal Power Usage Effectiveness (PUE). A typical diagram of liquid cooling engineering technology is shown in Figure 2.

 

▲ Figure 2: Liquid Cooling Engineering Technology Diagram

 

Some liquid cooling technologies use a design based solely on the primary cooling process, where the coolant directly transfers heat to the environment. However, in most cases, this requires high physical purity standards for the coolant, increasing costs. If the pipelines are too long, the overall economics of the system are affected. Therefore, primary cooling designs are best suited for compact configurations.

 

 

IV Discussion on the Future Development Trends of Liquid-Cooled Data Centers

 

Overall, the future development of liquid-cooled data centers still carries uncertainty.

 

Firstly, the driving force for the promotion of liquid cooling technology will still primarily be based on the cooling (heat dissipation) demands of IT equipment. However, this is built on predictions of future trends in electronic chip development, as well as the assumption that the cost-effectiveness of liquid cooling technology will gradually establish an advantage over air cooling technology. Nonetheless, there remains a possibility that the high costs of liquid cooling could lead to alternative chip designs that reduce cooling and heat dissipation requirements.

 

Secondly, the construction and usage model of liquid-cooled data centers differ significantly from conventional air-cooled data centers. The characteristic of liquid-cooled data centers is the high degree of coupling between IT equipment and infrastructure. In terms of technical principles and reliability assurance, they essentially cannot achieve the same level of decoupling as conventional air-cooled data centers. As a result, various technical solutions and equipment vendors operate largely independently of each other. Data center owners are likely to become deeply bound to a specific technology solution provider. The application of liquid cooling technology in data centers can be seen as developing based on the disruption of the original commercial model, construction model, and industrial model of conventional air-cooled data centers. Traditional cabinet leasing services are largely inapplicable to liquid-cooled data centers. This presents a significant obstacle to the further promotion of liquid cooling technology.

 

Thirdly, the reliability of liquid-cooled data centers still requires validation. Although current liquid-cooled data center solution providers have proposed solutions such as enhanced reliability testing and the addition of leak detection and alarm devices for the coolant, concerns about potential leaks in the liquid cooling loop within the cabinet, leading to damage to IT equipment, remain a topic of widespread concern in the industry.

 

The resolution of these uncertainties is deeply dependent on the scale of application. To solve this seemingly unsolvable problem, monopolizing the market and controlling the upstream and downstream industries, which has been proven countless times to violate the laws of economic development, is not the answer. History has repeatedly shown that those who can first establish an industrial chain and ecosystem based on division of labor and cooperation are more likely to succeed in competition. Before finding a technical path that decouples IT equipment from infrastructure, standardization will not be a magic solution to the decoupling problem, nor will it be an effective means of limiting or attacking competitors. However, using standardization as a tool to steadily advance decoupling based on finding common ground while preserving differences, effectively sharing experiences, and promoting the separation of manufacturing and service processes to achieve specialization and refinement, can help reduce the overall application costs. Continuous improvement through application testing may lead to the formation of a positive feedback loop that expands the scale of liquid cooling, potentially becoming the necessary path for the development of liquid cooling technology.

 

 

V Conclusion

 

Depending on factors such as server compatibility, the type of coolant, and the operating temperature of liquid cooling, the engineering technologies for liquid-cooled data centers vary accordingly. Currently, the engineering technologies under each technical route are still in the process of development and finalization, and there is not yet an industry-recognized perfect technical route. Nor is there consensus on the specific applicable subfields of each technical route. Seeking the ultimate certainty will depend on factors such as the economics, reliability, and maintainability of liquid-cooled data center engineering technology, as well as finding optimal solutions through continuous practical application.