Key Takeaways
- Modern AI GPU clusters generate 50–100+ kW per rack — far beyond what air cooling can reliably handle.
- Immersion cooling typically achieves PUE of 1.02–1.10, compared with 1.3–1.6 for air-cooled facilities.
- Australia's high energy costs make the OPEX case for immersion cooling particularly compelling.
- Data sovereignty requirements for defence and government make on-premise high-density compute essential.
- Pilot deployments (4–8 nodes) can be operational in 6–8 weeks to validate performance before scaling.
The AI Cooling Challenge
Australia's growing appetite for AI workloads — from large language model training to GPU-accelerated inference — is driving demand for high-density computing infrastructure. Data centres built around modern GPUs such as NVIDIA H100, B200, and emerging next-generation accelerators generate heat densities that traditional cooling approaches struggle to support. A single rack of high-end GPUs can easily reach 50–80 kW, with some configurations exceeding 100 kW. At these densities, air cooling hits practical limits: airflow requirements become prohibitive, hot spots emerge, and reliability suffers.
Why Air Cooling Falls Short
Air-cooled data centres typically top out at around 15–20 kW per rack in production environments. Beyond that, the physics are unforgiving. Air has low thermal mass and poor heat capacity compared with liquids. To move heat away from dense GPU clusters, facilities must push enormous volumes of conditioned air through narrow channels — requiring large CRAC/CRAH units, raised floors, and complex airflow management.
Even then, thermal gradients within the rack can cause GPU throttling and reduced training performance. Facilities often resort to aggressive overcooling, which drives up energy consumption and undermines the business case for running AI workloads in-house.
How Immersion Cooling Works for AI Workloads
Immersion cooling submerges servers or GPU nodes directly in a dielectric fluid — a liquid that conducts heat but not electricity. Heat transfers from components to the fluid by direct contact, eliminating fans and most airflow requirements. The fluid is circulated through a heat exchanger, where it rejects heat to a secondary loop (often dry coolers or adiabatic systems). This approach is particularly well-suited to AI workloads because:
- Thermal capacity: Liquids transfer heat 10–50x more effectively than air, enabling sustained GPU operation at peak power without throttling.
- Density: Rack power density can reach 100+ kW with immersion, versus 15–25 kW with advanced air cooling.
- Simplicity: No raised floors, no per-node plumbing (unlike direct-to-chip liquid cooling), and typically lower cooling OPEX.
- Topology tolerance: GPU-to-GPU NVLink and InfiniBand topologies work well in immersion tanks without per-node coolant connections.
For a deeper technical comparison of immersion approaches, see our guide on single-phase vs two-phase immersion cooling.
Australian Market Context
Energy Costs
Electricity prices in Australia are among the highest in the OECD. Cooling can account for 30–50% of data centre energy consumption in air-cooled facilities. Immersion cooling typically reduces cooling-related energy use by 30–50%, with PUE often falling into the 1.02–1.10 range. For AI facilities operating at scale, this translates to substantial annual OPEX savings. Our article on the business case for immersion cooling explores PUE, power, and payback in more detail.
Climate Advantages
Many Australian regions offer favourable conditions for heat rejection. Dry coolers can achieve approach temperatures of 3–8°C above ambient; in temperate zones, free cooling may be available for significant portions of the year. Hot, humid regions like northern Queensland require careful design — adiabatic or hybrid systems — but remain viable.
Data Sovereignty
For defence, government, and regulated industries, on-premise or locally hosted AI infrastructure is often a requirement. Immersion cooling for defence and government supports these requirements by enabling high-density compute within controlled, sovereign facilities — without depending on hyperscale cloud providers.
Deployment Considerations
Before committing to immersion cooling for AI infrastructure, teams should evaluate:
- Floor loading: Immersion tanks with fluid typically weigh 500–1500 kg depending on size. Existing slabs may need assessment or reinforcement.
- Fluid supply chain: Dielectric fluids must be sourced and, at end of life, responsibly disposed. Australian availability varies by fluid type; synthetic esters and engineered two-phase fluids may require longer lead times.
- Hardware compatibility: Most modern GPUs and servers can be immersed, but OEM warranty positions differ. Immersion-specific SKUs are increasingly available from major vendors.
- Operational readiness: Maintenance procedures — fluid monitoring, node replacement, leak response — differ from air-cooled operations. Training and runbooks should be part of the deployment plan.
A structured engagement — from consulting and audit through system design and deployment — helps de-risk these considerations.
Getting Started
Organisations evaluating immersion cooling for AI workloads typically begin with a pilot: a single tank with 4–8 GPU nodes to validate thermal performance, operations, and integration with existing infrastructure. Pilots can often be operational in 6–8 weeks. Full-scale deployments — whether retrofits or greenfield — typically take 3–9 months depending on scale and site readiness.
If you're planning AI infrastructure in Australia and want to explore whether immersion cooling fits your facility and workload, contact us for a consultation. We provide site assessment, thermal analysis, and TCO modelling to support informed decisions.
Frequently Asked Questions
Is immersion cooling suitable for all GPU types used in AI?
Most modern GPUs used for AI — including NVIDIA A100, H100, and comparable AMD and Intel accelerators — are compatible with immersion cooling. OEM guidance varies; some offer immersion-specific SKUs or certification. High-power GPUs (300 W+ per device) benefit the most from immersion's thermal headroom.
What PUE can Australian AI data centres achieve with immersion cooling?
Typical ranges are 1.02–1.10 for immersion-cooled facilities, versus 1.3–1.6 for air-cooled. Actual results depend on climate, heat rejection design, and IT load profile. Pilot deployments allow you to measure performance in your specific context before committing to scale.
How does immersion cooling affect AI training reliability?
Immersion cooling generally improves thermal stability by removing hot spots and reducing thermal cycling. More consistent temperatures can reduce GPU throttling during sustained training runs and extend component life. Operators should follow fluid monitoring and maintenance schedules to maintain reliability over time.