An AI data centre is a computing facility purpose-built for artificial intelligence workloads: training large models, fine-tuning them and serving them to users at high volume. NVIDIA calls the concept an "AI factory", and the metaphor is more useful than most marketing language. Raw data goes in, electricity and capital are consumed at industrial rates, and trained models and inference responses come out.
The phrase gets attached to everything from a single GPU hall to a gigawatt campus, so this article takes it apart layer by layer: what makes an AI data centre different from the cloud facilities that came before it, how the compute, networking, storage, power and cooling layers fit together, and why the constraint on the whole system has shifted from software to physical infrastructure. It closes with the part that matters locally — how Singapore is managing its own AI data centre buildout.
- An AI data centre is built so thousands of accelerators can behave like one machine — that single requirement drives every other design decision.
- Rack power has jumped from 5–25 kW in conventional facilities to roughly 130 kW for current GPU racks, with announced designs reaching 600 kW.
- Five interdependent layers — compute, networking, storage, power and cooling — must all perform; a weakness in any one stalls the entire cluster.
- The largest AI campuses now draw close to a gigawatt of IT power, which is why grid access, not chip supply, increasingly sets project timelines.
- Singapore is expanding capacity deliberately: at least 300 MW under the Green Data Centre Roadmap, plus a further 200 MW call for applications that closed in March 2026.
What is an AI data centre?
A traditional cloud data centre hosts thousands of loosely related workloads — websites, databases, email, storage, virtual machines — spread across general-purpose servers. If one server fails, the workload moves. The building is designed around resilience, floor space and uptime.
An AI data centre inverts that model. Training a large model requires thousands of accelerators to execute trillions of operations in lockstep, exchanging gradients and parameters continuously with near-lossless, microsecond-level communication. The cluster behaves less like a room full of servers and more like one enormous computer — and the building becomes part of the machine. Compute, networking, storage, power delivery and cooling collapse into a single tightly engineered envelope.
The scale is what separates this cycle from previous infrastructure booms. According to Epoch AI's Frontier Data Centers tracker, the largest known AI facility as of mid-2026 is xAI's Colossus 2 in Memphis at roughly 946 MW of IT power — more than 1.1 million H100-equivalents of compute. Close behind are the Anthropic–Amazon campus in New Carlisle, Indiana (about 910 MW), Microsoft's Fairwater site in Atlanta (about 636 MW), Meta's Prometheus (about 631 MW) and OpenAI's Stargate campus in Abilene, Texas (about 421 MW). For comparison, a large conventional cloud data centre typically draws tens of megawatts.
Training, fine-tuning and inference shape the facility differently
Training runs are sustained, synchronised bursts that can occupy tens of thousands of accelerators for weeks — they need maximum interconnect bandwidth and tolerate almost no stragglers. Inference — serving large language models and other systems to users — is spikier, latency-sensitive and spread across the day. Fine-tuning sits in between. Many operators now build separate zones, or entire separate sites, for each: training clusters chase cheap power and land, while inference capacity stays close to users. That split matters later when we get to Singapore.
How an AI data centre differs from a traditional data centre
The differences run through every layer of the stack. The table below summarises the practical gaps rather than the marketing ones.
| Requirement | Traditional data centre | AI data centre |
|---|---|---|
| Compute | CPU-centric general-purpose servers. | GPU and accelerator-centric systems built for massive parallelism. |
| Networking | Standard Ethernet; mostly north–south traffic between users and servers. | 400G/800G fabrics with RDMA, InfiniBand or Ultra Ethernet; dominated by east–west GPU-to-GPU traffic, plus NVLink-class links inside the rack. |
| Storage | Balanced capacity and latency for mixed workloads. | Parallel file systems delivering terabytes per second for petabyte-scale datasets and frequent checkpoints. |
| Rack power | Roughly 5–25 kW per rack. | 80–130+ kW today, with announced designs at 600 kW per rack. |
| Cooling | Air cooling with hot/cold aisle containment. | Direct-to-chip liquid cooling as the default; immersion for extreme densities. |
| Scale | Halls of a few megawatts to tens of megawatts. | Campuses measured in hundreds of megawatts, approaching a gigawatt. |
One consequence deserves emphasis: the layers are interdependent in a way traditional facilities never were. A congested network fabric idles millions of dollars of GPUs. A storage system that cannot absorb a checkpoint stalls a training run. A cooling excursion of a few degrees throttles an entire pod. In an AI data centre, the weakest layer sets the output of the whole factory.
The compute layer: GPU clusters and rack-scale systems
Accelerators are the reason the facility exists. NVIDIA still dominates, and its rack-scale GB200 NVL72 is the clearest illustration of where the hardware has gone: 36 Grace CPUs and 72 Blackwell GPUs in a single liquid-cooled rack, joined by an NVLink domain that gives every GPU 1.8 TB/s of bandwidth to its neighbours — around 130 TB/s aggregate. The rack draws roughly 120–140 kW and is sold, cooled and managed as one unit. The successor Vera Rubin NVL144 platform arrives in late 2026 at similar rack power, and the Rubin Ultra NVL576 "Kyber" generation announced for 2027 is specified at about 600 kW per rack with 800-volt DC power distribution.
The alternatives are credible and growing. AMD's Instinct MI300 and MI350 series lead into the MI400 generation and its 72-GPU Helios rack in the second half of 2026, scaling out over Ultra Ethernet. Google's seventh-generation TPU, Ironwood, deploys in pods of up to 9,216 chips. AWS shipped its 3-nanometre Trainium3 in December 2025, and several large model developers run on custom silicon. For a deeper look at why GPUs suit these workloads, see our GPU architecture explainer.
At facility level, the unit of planning is no longer the server but the pod: a block of racks, switches, storage and cooling commissioned together. Frontier sites chain pods into clusters of hundreds of thousands of accelerators. That is why operators talk about "time to first token" for a new site — the months between breaking ground and the first useful model output — as a competitive metric.
Networking and storage: moving data at cluster scale
Distributed training synchronises constantly. Every training step, GPUs exchange gradients across data-parallel groups and shuffle activations across model-parallel groups. The resulting east–west traffic dwarfs anything a conventional enterprise network carries, and it is unforgiving: collective operations wait for the slowest participant, so one congested link can idle thousands of accelerators at a synchronisation barrier. Fabric design therefore optimises for deterministic tail latency, not just headline bandwidth.
Three tiers do the work. Inside the rack, NVLink-class links join GPUs into one memory domain. Between racks, clusters run lossless RDMA fabrics — InfiniBand or increasingly 400G/800G Ethernet with RoCE, now standardised by the Ultra Ethernet Consortium — arranged in non-blocking fat-tree or rail-optimised topologies. Optical circuit switching, which Google uses in its TPU pods, is emerging as a way to cut switching power and cost at the top tiers. All of it rides on very high-density fibre plant, which makes structured cabling discipline a first-order design input rather than an afterthought.
Storage: feeding the accelerators and surviving failures
Training datasets run from petabytes to exabytes, and the storage layer has two jobs that pull in different directions. It must stream data fast enough that GPUs never wait — parallel file systems such as Lustre, BeeGFS, WEKA and VAST deliver aggregate throughput measured in terabytes per second across NVMe tiers. And it must absorb checkpoints: periodic snapshots of model state, often multiple terabytes each, written frequently because at cluster scale hardware failures are routine, and losing a week of training to a single node fault is not acceptable. Metadata performance on billions of small files matters almost as much as raw bandwidth.
Power and cooling: the physical limits of an AI data centre
An AI rack draws ten to twenty times the power of a conventional one, and a single campus can demand hundreds of megawatts to more than a gigawatt. That pushes the electrical system to the front of the design. The chain runs from high-voltage substations and transformers through UPS systems, batteries, busways and rack-level distribution — and each stage now has an AI-specific complication: GPU workloads swing sharply within milliseconds, producing harmonic-rich, fast-transient loads that can disturb the upstream grid. Batteries and power electronics are moving closer to the compute stack partly to smooth those swings. Announced next-generation racks shift to 800-volt DC distribution to cut conversion losses, and some operators are exploring on-site generation — including agreements around small modular reactors, which are designed to deliver up to around 300 MWe per factory-built unit, though none yet powers a data centre. Our data centre power explainer covers the conventional chain in detail.
Cooling follows the same trajectory. Air cooling becomes impractical somewhere around 40–60 kW per rack, which today's GPU racks exceed comfortably. Direct-to-chip liquid cooling — coolant circulated through cold plates on the hottest silicon — is now the mainstream answer for 80 kW-plus racks, supplemented by rear-door heat exchangers and, for extreme densities, single- and two-phase immersion. Warmer coolant loops, with inlet temperatures up to about 45°C, let facilities reject heat through dry coolers instead of energy-hungry chillers. The industry-wide numbers show how far there is to go: the Uptime Institute's 2025 global survey found average PUE stuck at 1.54 for a sixth consecutive year, with only 22% of operators using direct liquid cooling — while purpose-built hyperscale AI facilities operate in the 1.1 range. Our guides to AI data centre cooling and liquid-cooled racks go deeper on the engineering.
Facility design and the software that runs the factory
The building itself changes around these systems. Liquid-cooled racks are heavy, so floors are rated higher. Halls need ceiling height and riser space for coolant distribution units, manifolds and leak detection. Fire protection shifts toward clean-agent and water-mist systems that can coexist with electronics and dielectric fluids. Physical security remains layered — perimeter, mantrap, biometric, per-rack — because the hardware inside is now among the most valuable industrial equipment on earth.
Redundancy thinking is also being rebalanced. Classic Tier III/IV design assumes every load justifies concurrent maintainability or 2N paths. Some AI operators now accept leaner redundancy on training pods — a checkpointed training job can restart, and the capital saved buys more accelerators — while keeping storage, network spines and control planes fully protected. It is a deliberate trade, not corner-cutting, and it only works because the software layer expects failure.
That software layer is the least visible but binds everything: schedulers such as Slurm and Kubernetes allocate pods of GPUs to jobs; NCCL and MPI libraries run the collective communication; frameworks such as PyTorch orchestrate the training loop; MLOps tooling tracks experiments, serves models and watches for drift. Facility-side, DCIM platforms increasingly fuse IT telemetry with power and cooling data — often applying machine learning to the data centre's own operations. The operational metric that matters is goodput: the fraction of theoretical compute that actually produces model progress after failures, stragglers and restarts are accounted for.
AI data centres in Singapore: a managed buildout
Singapore is a special case in this story. It is one of Asia's densest connectivity and cloud hubs, with roughly 1.4 GW of operational data centre capacity — but it is also land-, power- and water-constrained, which led to a pause on new builds between 2019 and 2022 and a pilot call for application in 2023 that awarded about 80 MW across four operators.
The current framework is deliberately managed rather than open-ended. The Green Data Centre Roadmap, published in May 2024, plans at least 300 MW of additional capacity in the near term, with more tied to green energy deployment, and pushes operators toward liquid cooling, higher hall temperatures and a PUE of 1.3 or better at full load — demanding targets in a tropical climate. In December 2025, EDB and IMDA followed with a second Data Centre Call for Application (DC-CFA2) to allocate at least 200 MW to operators that can demonstrate best-in-class efficiency and sustainability; applications closed in March 2026.
The practical pattern for the region: the largest training clusters are landing where power and land are abundant — including Johor and Batam just across the border — while Singapore concentrates on inference capacity, regional connectivity and high-value, efficiency-led facilities. For enterprises here, the near-term questions are concrete: whether your colocation provider can support high-density racks and direct-to-chip cooling, how AI capacity is priced when megawatts are allocated rather than abundant, and which providers have credible liquid-cooling operations experience.
Challenges and the road ahead
The constraints on the AI data centre buildout are physical and financial rather than architectural. Grid interconnection queues are congested in most major markets. Transformers, switchgear, high-bandwidth memory and advanced packaging remain supply-bottlenecked. Water use draws local scrutiny. And the capital involved has reached energy-industry scale: the IEA estimates cumulative data centre investment of US$3.9 trillion between 2026 and 2030, with leading operators lifting 2026 capital expenditure by 75% to around US$715 billion.
The direction of travel for the technology itself is fairly clear: rack densities continuing past 130 kW toward 600 kW; liquid cooling as the default rather than the exception; 800 VDC distribution and behind-the-meter generation; optical interconnects and co-packaged optics in the network; prefabricated, modular construction to compress build times; and AI increasingly used to optimise the data centres that run it. Whether every announced gigawatt gets built is a separate question — connection queues contain speculation as well as demand — but the facilities that do get built will look less like the server rooms of the cloud era and more like precision industrial plants. That is what an AI factory actually is.
Planning AI-ready capacity in Singapore?
Browse Singapore data centre operators, colocation providers and infrastructure specialists with experience in high-density racks, liquid cooling and AI-ready facilities.
Frequently asked questions
What is an AI data centre?
An AI data centre is a facility purpose-built to train and run artificial intelligence models at scale. Unlike a traditional data centre, which hosts many independent workloads on general-purpose servers, it is designed so thousands of GPUs or other accelerators can operate as one tightly coupled machine — which drives extreme rack power density, liquid cooling, specialised low-latency networking and very high-throughput storage.
Why are AI data centres called AI factories?
The term, popularised by NVIDIA, describes what the facility does: it consumes raw data, electricity and capital, and produces trained models and inference responses as output. Like a factory, throughput depends on every layer of the production line — compute, networking, storage, power and cooling — working together, and a bottleneck in any one layer reduces the output of the whole plant.
How much power does an AI data centre use?
Current GPU racks such as NVIDIA's GB200 NVL72 draw roughly 120–140 kW each, versus 5–25 kW for conventional racks, and announced 2027-generation racks reach about 600 kW. At campus level, Epoch AI's tracking puts the largest sites near a gigawatt of IT power — xAI's Colossus 2 at roughly 946 MW and the Anthropic–Amazon New Carlisle campus at about 910 MW as of mid-2026.
Can a traditional data centre be converted to run AI workloads?
Only partially. Moderate GPU deployments can fit existing halls with rear-door heat exchangers or spot liquid cooling, but frontier-class clusters need power delivery, floor loading, coolant distribution and network fabrics that most legacy buildings cannot retrofit economically. Many operators instead run hybrid estates: conventional workloads on air-cooled halls, AI pods in purpose-built high-density zones.
Is Singapore building new AI data centres?
Yes, under a managed framework. After pausing new builds from 2019 to 2022, Singapore released capacity through a pilot call for application in 2023, then published the Green Data Centre Roadmap in May 2024 targeting at least 300 MW of additional capacity plus more tied to green energy. A second call for application (DC-CFA2), announced in December 2025, allocates at least 200 MW to operators with best-in-class efficiency; applications closed in March 2026.
What is the difference between training and inference infrastructure?
Training uses large synchronised clusters that run for weeks and need maximum interconnect bandwidth, so training sites chase cheap power and land. Inference serves user requests in real time, is latency-sensitive and scales with demand across the day, so inference capacity sits closer to users. Many operators now build separate facilities for each, which is why regions like Singapore focus on inference and connectivity while giant training campuses rise where power is abundant.
Sources and further reading
- Epoch AI — Frontier Data Centers hub
- International Energy Agency — Key Questions on Energy and AI (April 2026)
- NVIDIA — GB200 NVL72 product page
- Data Center Dynamics — Rubin Ultra NVL576 rack expected to be 600 kW (2027)
- Uptime Institute — 15th Annual Global Data Center Survey (2025)
- IMDA — Green Data Centre Roadmap (May 2024)
- Morgan Lewis — Singapore Announces Data Center Capacity Allocation Call (DC-CFA2)
- AI Data Centre Infrastructure: Power, Cooling and GPU Cluster Demand
- AI Data Centre Cooling: Liquid Cooling for High-Density Racks
- GPU Architecture Explained: How Graphics Cards Work
- data centre power design explained
- Inside Large Language Models: How the Engines Behind ChatGPT, Claude, and Gemini Actually Work