Data Centers Enter the Liquid Cooling Era

 

Currently, 46% of the electricity used by typical data centers is allocated for air cooling.

 

Dell and Supermicro have stated that by utilizing liquid cooling directly on GPUs within the rack, they can reduce data center power consumption by one-third. Rack suppliers have told EE Times that by replacing the predominant air cooling solutions of today's data centers, liquid cooling will enable operators to focus more on computing rather than air conditioning.

 

 

Artificial intelligence is driving this demand. As newly constructed hyperscale data centers around the world begin to stress the power grid, the prospect of higher energy efficiency follows suit.

 

Dell and Supermicro are key suppliers of high-end liquid-cooled racks for data centers and have been ramping up production to meet expected surges in demand. Supermicro reports that it can deliver over 1,000 racks globally each month.

 

Supermicro's Vice President of Technology and AI, Vik Malyala, told EE Times, "We can anticipate the technologies our partners will launch, whether it's Intel, AMD, or Nvidia, as well as networking companies like Broadcom and Intel. We collaborate with them and then review their roadmaps."

 

GPU manufacturers like Nvidia have created reference designs for cold plates and heat sinks, which Dell and Supermicro are closely adopting.

 

Dell engineering expert Tim Shedd told EE Times, "We want to ensure that any products we design are supported and backed by chip manufacturers. We've added proprietary features accumulated over the years in the industry, such as proprietary leak detection technology to protect customers and servers."

 

Supermicro claims its cooling solutions include proprietary information on cold plate designs. Malyala added that data center operators have been reluctant to abandon their reliance on air cooling, but he expects this situation to change.

 

"As data center deployments scale up, we know efficiency will become a critical factor," Malyala said. "Power consumption is a nightmare."

 

Dell estimates that 46% of the power used by a typical data center is for air cooling. Shedd stated, "By deploying liquid cooling, we can reduce energy consumption to 10% or lower without needing to install cold plates on everything, but rather using smart mixed technologies: just air and liquid cooling." He added that the key to reducing energy consumption is minimizing the number of air compressors in the data center.

 

Today, most data centers use air blown from the facility perimeter toward the servers, with fans on the servers providing additional cooling.

 

"I need relatively cold water, typically around 45°F, to adequately cool the air," Shedd said. "This means I need compressors and refrigeration to cool the water for most of the year. With liquid cooling, for sections of the data center using cold plates and liquid cooling, I can typically cool with water at 90°F or even 100°F. Now, under most climate conditions, I almost don't need compressors, resulting in significant energy savings."

 

Market research firm Dell'Oro reported in July that liquid cooling is transitioning from a niche market to mainstream business, with its value expected to reach $15 billion by 2030 due to increased demand for thermal management.

 

 

I Liquid Cooling System Shipments Expected to Grow by 20%

 

Malyala indicated that Supermicro expects a shipment growth rate of at least 20% this year for systems supporting AI infrastructure, including liquid cooling. He noted, "We're examining many data centers and realize that many of them are operating at extremely low efficiencies because those building the infrastructure aren't the ones paying for cooling."

Shedd pointed out that demand is driven by advancements in 1,000-watt GPUs and AI needs that may not be obvious. "We need to bring AI to the data rather than bringing data to the processing power. I don't necessarily want to send data to the cloud due to security concerns. Governments don't want to send their data-especially sensitive, processed, and valuable data-out. That's not a data center with 10,000 GPUs; that's a rack."

 

Dell has identified use cases for liquid cooling in new areas. "We previously focused primarily on high-performance computing and specific commercial verticals like banking and other fields requiring intense computing," Shedd explained. "Now we see use cases refining to a rack in a factory where they want to collect data from the production line in real time, process the data, and make decisions. This may require the use of liquid cooling in that rack."

 

Malyala noted that edge security is an increasingly important business for liquid-cooled rack suppliers. "We're talking about every phone we have," he said. "How do we effectively secure them at former central locations? Real-time threat detection and protection are happening-it's moving toward the edge."

 

 

II Turning Off the Air Conditioning

 

By converting data centers and even racks to liquid cooling, operators should be able to "turn off air conditioning or at least run it more efficiently," Malyala stated. "You're talking about 40% energy savings. That essentially means you can place 40% more infrastructure in the same space."

 

Dell also views the energy savings from liquid cooling as a way to sell more racks to customers and enhance data center processing density.

 

The effects of energy savings could expand even further. Singapore supplier Sustainable Metal Cloud stated in August that its immersion cooling technology has installation costs 28% lower than liquid solutions, with potential energy savings of up to 50%.

 

Installing liquid cooling in existing data centers requires significant modifications to the facilities, which is something clients find daunting. "For customers, we need to meet their needs based on their real situations," Shedd said. "Some customers cannot currently deploy liquid cooling. Installing all the piping and plumbing is just too cumbersome. So we work with them to continually improve server fan efficiency. We strive to enable customers to operate at higher temperatures so they can reduce reliance on compressors."

 

Dell anticipates further innovations in liquid cooling. Future processor thermal densities will exceed that of a gas torch. Chief Operating Officer Jeff Clarke mentioned in a LinkedIn blog in September that Dell is re-engineering cold plates to manage the most challenging processor functions without compromising data center energy efficiency. He stated, "We're building new integrated racks for the future, so each rack can handle over 400 kW of power and cooling."