Key takeaways
- AI rack density moves cooling from a room-level airflow problem to a rack-level heat-transfer problem.
- Direct-to-chip liquid cooling is the near-term bridge for dense GPU clusters because it removes heat at GPUs, CPUs and memory packages.
- Immersion cooling can be effective, but it changes hardware handling, tank layout, fluid management and maintenance workflows.
- PUE and WUE must be evaluated together: dry coolers and closed loops can reduce water use, but site climate and grid load still matter.
For two decades, most data centre cooling was an exercise in disciplined airflow. Cold air came up through raised floors or contained aisles, passed through servers, and returned to computer room air-conditioning or air-handler units to shed heat. The model worked because the load was predictable: many enterprise racks lived in the 5 kW to 15 kW range, and even dense compute zones could often be managed with containment, in-row cooling, or rear-door heat exchangers.
AI infrastructure broke that arithmetic. GPU training and inference clusters compress more power into fewer cabinets, and nearly all of that power becomes heat. A rack that would once have looked extreme now looks routine. The practical question for operators is no longer whether liquid cooling will appear inside data halls. It is where the pipes go first.
The rack is now the thermal unit
The data centre used to be planned around rooms, rows and megawatts. AI has pulled the unit of design down to the rack. Power distribution, cooling loops, floor loading, leak detection and service access now converge around cabinets built for dense GPU trays rather than general-purpose servers.
That shift shows up in the language used by chip vendors. NVIDIA describes Blackwell-class AI systems as rack-scale computers, not just boards that happen to sit in a rack. Its later AI factory cooling plans move the discussion further upstream: coolant entering fully liquid-cooled systems at up to 45 degrees Celsius, dry coolers and liquid loops designed as part of the compute platform.
Air can still cool enormous amounts of IT load when the watts are spread out. It struggles when the heat is concentrated at the silicon. Fans move volume; liquid moves heat. Water has far higher volumetric heat capacity than air, so a smaller stream can remove more energy with less physical turbulence and less fan penalty.
Why air cooling hit the wall
Cold aisles were built for a different density
Traditional air-cooled halls rely on separation: cold aisle, hot aisle, contained return path. The physics is clean enough when temperature deltas are moderate and air volumes remain manageable. Raise rack power far enough, and the facility begins paying twice: more fan energy to move air, then more mechanical cooling or water consumption to remove the heat from the building.
The silicon has less tolerance than the room. GPUs, CPUs, high-bandwidth memory, power shelves and optical interconnects all create local hot spots. Air passing across the front of a server may be cold enough in aggregate and still fail to remove heat evenly from the components doing the work.
Rear-door heat exchangers buy time
Rear-door heat exchangers remain useful because they intercept exhaust heat at the rack boundary. They can support dense air-cooled deployments without sending all heat back into the room. But they do not change the heat path inside the server. The hottest devices still rely on air to carry energy away from the board before the rear-door coil can touch it.
That is why direct-to-chip liquid cooling is becoming the preferred bridge for high-density AI clusters. It removes heat at the source while allowing most server form factors, rack service practices and facility layouts to remain recognisable.
Liquid cooling is not one system
"Liquid cooling" is often used as a single term. In procurement, it describes several different architectures with very different consequences for operations, maintenance and retrofit cost.
| Cooling model | Where it removes heat | Operational tradeoff |
|---|---|---|
| Direct-to-chip cold plates | At GPUs, CPUs and other high-power packages | Works well with rack-scale AI hardware, but requires CDUs, manifolds, leak controls and trained service teams. |
| Rear-door heat exchangers | At the rear of the rack, after heat leaves servers | Useful for retrofits and mixed halls, though it does not solve silicon-level hot spots by itself. |
| Single-phase immersion | Across the full server board submerged in dielectric fluid | Can cool components evenly, but changes hardware handling, tank layout, fluid management and maintenance workflows. |
| Two-phase immersion | Through boiling and condensation inside a sealed tank | Offers strong heat transfer, but introduces fluid containment, vapor management and a narrower supplier ecosystem. |
Direct-to-chip has won the first major adoption wave because it is less disruptive. Cold plates sit on the hottest packages, coolant moves through sealed loops, and the rest of the chassis can look relatively familiar. Technicians still need procedures for hoses, fittings, pressure checks and fluid quality. They do not need to drain a tank for every routine intervention.
The retrofit market will be hybrid
Hyperscalers can design new AI campuses around liquid from the start. Colocation providers have a harder job. They must serve ordinary enterprise workloads, cloud nodes, network gear and high-density AI clusters inside facilities that were often built around air.
The result is a mixed thermal estate. Standard racks remain on air or rear-door exchange. AI islands get liquid loops, CDUs, reinforced pathways, extra monitoring and revised service rules. The room may look the same from the corridor. Under the floor and above the rack, it is becoming two facilities at once.
Recent deployments point in that direction. Digital Realty's direct liquid-cooled AI deployment for Imperial College London is a useful marker: the colocation market is no longer treating liquid cooling as lab infrastructure. It is being packaged as a sellable high-density capacity tier.
This is where capital planning gets awkward. Retrofitting for liquid cooling is not just plumbing. Operators must validate slab load, pipe routes, containment, leak detection, CDU redundancy, maintenance clearances, water chemistry and heat-rejection capacity. One underbuilt section can strand the rest.
Water is becoming the second metric
The power number is already large. A U.S. Department of Energy report found that data centres accounted for 4.4 percent of U.S. electricity use in 2023 and projected a range of 6.7 percent to 12 percent by 2028. The International Energy Agency says data centres used about 415 TWh globally in 2024 and projects that electricity consumption will more than double to about 945 TWh by 2030.
Cooling now sits inside that demand story. Power usage effectiveness still matters, but water usage effectiveness is becoming harder to ignore. Evaporative cooling can reduce electrical load in hot climates, but it consumes water. Closed-loop liquid systems and dry coolers can reduce operational water use, but they can move more burden back to the electrical side when ambient temperatures rise.
The largest platform companies are already adjusting their messaging. NVIDIA says its newer liquid-cooling designs can use warmer water and dry coolers, reducing dependence on chilled water and evaporative systems. Microsoft has said its newest AI-focused data centre designs recirculate coolant directly to chips and avoid water-consuming cooling towers during normal operation.
Those claims do not erase the older installed base. Most data centres were not built for this thermal model. Many still consume water during seasonal peaks, and many are tied to electricity grids where extra load has its own water and emissions profile. Cooling efficiency, water policy, grid capacity and permitting are now the same conversation.
What operators will buy next
The buying centre is widening. Mechanical engineers still specify cooling equipment, but cloud architects, finance teams, sustainability leads and site-selection teams now have to understand the thermal stack. A cheap megawatt can become an unusable megawatt if it cannot be cooled at the rack.
Demand will concentrate around four areas. Operators need coolant distribution units sized for mixed densities. They need manifold designs that let technicians service dense AI racks without turning routine maintenance into a shutdown event. They need telemetry that links chip temperature, flow rate, pump health, humidity, leakage, rack power and workload scheduling. They need heat-rejection strategies that survive hotter summers without silently trading water savings for grid stress.
Immersion will still have a place, especially where operators control the hardware stack and can standardise tanks, fluids and maintenance. Direct-to-chip will take more of the broad enterprise and colocation market because it fits into the least disruptive path from today's halls to AI-ready capacity.
The old data centre bargain was simple: bring in power, move air, keep servers inside a safe temperature envelope. AI has narrowed the margin. If the DOE's upper case materialises, U.S. data centres could consume 12 percent of national electricity by 2028. Every watt still leaves as heat.
Planning AI-ready data centre capacity?
Browse Singapore data centre and AI computing providers that can help evaluate high-density racks, power, cooling, colocation and infrastructure design.
Frequently asked questions
Why do AI data centres need liquid cooling?
AI data centres need liquid cooling when GPU racks concentrate too much heat for practical airflow. Direct-to-chip cold plates remove heat at GPUs, CPUs and high-bandwidth memory before that heat spreads into the room.
What is the difference between direct-to-chip cooling and immersion cooling?
Direct-to-chip cooling uses cold plates and coolant loops on high-power packages while the server remains rack-mounted. Immersion cooling places hardware in dielectric fluid, which changes tank layout, service handling, fluid management and supplier selection.
Does liquid cooling reduce water use?
Liquid cooling can reduce operational water use when it is paired with closed loops and dry coolers. The result still depends on local climate, heat-rejection design, electrical load and the older cooling systems still operating on site.
What should operators check before approving AI rack capacity?
Operators should check rack power, floor loading, coolant supply temperature, flow rate, pressure drop, CDU redundancy, manifold access, leak detection, water chemistry, dry-cooler or chiller capacity, maintenance clearances and whether the wired and power infrastructure can support the rack without stranding usable megawatts.
Sources and further reading
- NVIDIA - Blackwell rack-scale AI systems
- NVIDIA - Liquid cooling AI factories
- Digital Realty - Direct liquid-cooled AI deployment for Imperial College London
- U.S. Department of Energy - Data centre electricity demand report
- International Energy Agency - Energy and AI executive summary
- Axios - Microsoft lower-water-use AI data centre designs