NVIDIA Technologies Explained: CUDA, GPUs and the AI Infrastructure Stack

NVIDIA started as a graphics-chip company. In 2026 it is the company whose technology underpins most of the world's AI infrastructure — the GPUs, interconnects, networking and software that train and serve modern AI models. Understanding NVIDIA AI infrastructure as a stack, rather than a list of chips, is the fastest way to make sense of both the products and the market around them. The scale of that shift shows up in the accounts: in its most recent quarter (Q1 FY2027, ended April 2026), NVIDIA reported US$81.6 billion in revenue, of which US$75.2 billion — more than 90 per cent — came from the data centre segment, up 92 per cent year on year.

Most explanations of NVIDIA stop at "they make AI chips". That undersells what the company actually sells, and it makes buying decisions harder to reason about. This article maps the full stack — CUDA and the software foundation, Tensor Cores and the GPU roadmap, NVLink and Spectrum-X networking, rack-scale systems like the GB200 NVL72, and the enterprise software layer — then covers the honest limitations, the competitive landscape as of mid-2026, and what is actually deployed in Singapore.

What is the NVIDIA AI infrastructure stack?

NVIDIA describes large AI deployments as "AI factories": facilities that consume data and electricity and produce trained models and inference responses. Whatever you think of the branding, it captures how the company positions its products — not as components, but as the whole production line. A useful way to read the portfolio is as four layers that are designed, and increasingly sold, together.

The layering explains NVIDIA's pricing power. A rival can match one layer — a fast chip, a good switch — but a buyer standardising on the full stack gets components that are co-designed and pre-integrated, from the cooling manifold to the inference server. It is also why the unit of purchase has shifted from a chip, to a server, to an entire rack, and why the facility housing it has become specialised enough to deserve its own explainer: see our guide to AI data centres.

CUDA: the software foundation of NVIDIA AI infrastructure

CUDA (Compute Unified Device Architecture), released in 2006, let developers use GPUs for general-purpose computation rather than just graphics. For years it was a niche tool for scientific computing. The inflection point came in 2012, when AlexNet — trained on two consumer GeForce GTX 580 cards — won the ImageNet competition by a wide margin and demonstrated that GPUs were the natural substrate for deep learning. Every wave of AI since, including today's large language models, has been built on that foundation.

CUDA itself is a programming model, but its gravity comes from what sits on top: cuDNN, the deep-learning primitives library that frameworks such as PyTorch and TensorFlow call behind the scenes; TensorRT and TensorRT-LLM, which compile trained models into their fastest deployable form; and hundreds of domain libraries NVIDIA groups under CUDA-X. Millions of developers have written against these interfaces over nearly two decades.

This is the moat in practical terms. When a new model architecture appears, it is implemented and optimised for CUDA first; competing stacks such as AMD's ROCm port it later. For a buyer, the hardware price gap between vendors can be real, but so is the engineering cost of leaving the ecosystem — a trade-off we return to below.

The silicon: Tensor Cores, precision and GPU generations

A modern NVIDIA data centre GPU contains two kinds of engines. CUDA cores are general-purpose parallel processors. Tensor Cores — introduced with the Volta generation in 2017 — are specialised units that do one thing extremely fast: multiply matrices at reduced numerical precision, which is the dominant operation in neural networks. (For the fuller hardware picture, see our GPU architecture explainer.)

Each generation has pushed precision lower because neural networks tolerate it and throughput roughly doubles with every halving: FP16 arrived with Volta, TF32 and BF16 with Ampere, FP8 with Hopper, and 4-bit NVFP4 with Blackwell. Feeding those engines requires enormous memory bandwidth, which is why data centre GPUs use stacked high-bandwidth memory rather than conventional DRAM — the Rubin generation carries 288 GB of HBM4 at roughly 13 TB/s per package (see memory hierarchy explained).

The roadmap now runs on a roughly one-year cadence:

• Hopper (2022–2024): H100 and H200 — the GPUs behind the first LLM buildout wave, still widely deployed.
• Blackwell (2024–2026): GB200 and the Blackwell Ultra GB300, designed from the start as rack-scale products paired with the Arm-based Grace CPU.
• Rubin (2026–): in full production since 1 June 2026. Each Rubin package carries 336 billion transistors and delivers about 50 petaflops of FP4 inference compute.
• Rubin Ultra (announced for 2027): the NVL576 "Kyber" rack generation, specified at roughly 600 kW per rack with 800-volt DC power distribution.

NVLink, InfiniBand and Spectrum-X: moving data between GPUs

Scale-up: NVLink and NVSwitch

Training a large model requires GPUs to exchange data constantly, and ordinary PCIe is far too slow. NVLink is NVIDIA's proprietary GPU-to-GPU interconnect: the fifth generation gives each Blackwell GPU 1.8 TB/s of bandwidth — an order of magnitude beyond PCIe Gen5 — and NVSwitch chips extend that into a rack-wide domain in which 72 GPUs address each other's memory almost as if they were one device. Rubin systems move to NVLink 6.

In 2025 NVIDIA opened this previously closed interconnect through NVLink Fusion, which lets partners build semi-custom silicon that plugs into NVIDIA's rack architecture. The ecosystem now spans MediaTek, Alchip, Astera Labs, Synopsys and Cadence, with Fujitsu and Qualcomm CPUs able to pair with NVIDIA GPUs — and in March 2026 NVIDIA invested US$2 billion in Marvell to bring custom XPUs into the fold. The strategic read: even where the accelerator is not NVIDIA's, the rack increasingly is.

Scale-out: Quantum InfiniBand and Spectrum-X Ethernet

Between racks, NVIDIA sells two fabrics: Quantum InfiniBand, the traditional low-latency choice for training clusters, and Spectrum-X, an Ethernet platform tuned for AI traffic patterns. Networking produced about US$31 billion of revenue in FY2026, and IDC's tracker shows NVIDIA becoming the largest data centre Ethernet switch vendor by revenue in early 2026, in a market that grew 39.8 per cent to US$15.4 billion in a single quarter. ConnectX network cards and BlueField DPUs complete the picture, offloading storage, security and network processing from the GPUs.

Rack-scale systems and the enterprise software layer

The clearest expression of the full-stack strategy is the GB200 NVL72: 36 Grace CPUs and 72 Blackwell GPUs in one liquid-cooled rack, joined by an NVLink domain with around 130 TB/s of aggregate bandwidth, drawing roughly 120–140 kW. It is sold, deployed and supported as a single unit — which is also why liquid cooling has moved from exotic to default. The successor Vera Rubin NVL144 systems ship to AWS, Microsoft Azure, Google Cloud, Oracle, CoreWeave, Lambda, Nebius and Nscale from late 2026.

Not every workload needs a whole rack, and two technologies matter for sharing: Multi-Instance GPU (MIG) partitions one physical GPU into up to seven isolated instances, and vGPU shares GPUs across virtual machines — the standard route for VDI and mixed estates (see server virtualization explained).

On top sits the commercial software layer, which is how NVIDIA reaches enterprises that will never rack their own DGX: NVIDIA AI Enterprise, the supported and security-patched distribution of the stack; NIM microservices, which package models as containerised inference endpoints; NeMo for building and customising models; Omniverse for industrial digital twins; and DGX Cloud, NVIDIA's own rented-supercomputer offering hosted inside the major clouds.

Beyond the data centre: RTX, DLSS and the consumer stack

The consumer portfolio runs on the same silicon philosophy, and it is worth a brief tour because the two halves feed each other. The RTX platform pairs RT Cores (dedicated ray-tracing hardware) with the same Tensor Cores found in data centre parts. DLSS is the neatest illustration of the loop: the upscaling and frame-generation models are trained on NVIDIA's supercomputers, then execute in real time on the Tensor Cores of a GeForce card. G-SYNC synchronises display refresh with GPU output, Reflex cuts input latency, and NVENC/NVDEC provide hardware video encode and decode — the same blocks that accelerate video pipelines in the data centre. Studio drivers and NVIDIA Broadcast serve creators; PhysX, once a separate physics accelerator, lives on inside the simulation stack.

For enterprise readers the relevance is practical: RTX workstations run CAD, rendering and increasingly local AI workloads; the consumer business amortises the R&D behind every generation; and CUDA code written on a laptop GPU deploys unchanged on a cluster — which is exactly how NVIDIA likes it.

Benefits, limitations and the lock-in question

The case for standardising on NVIDIA AI infrastructure is straightforward:

• Performance density and time-to-result: rack-scale systems deliver the highest widely available training and inference throughput per square metre.
• Ecosystem maturity: drivers, kernels, monitoring tools, reference architectures and — not least — a deep hiring pool of engineers who already know the stack.
• Integration risk transferred to the vendor: compute, interconnect, networking and software are validated together before they reach you.
• A predictable annual roadmap, which simplifies multi-year capacity planning.

The limitations deserve equal weight:

• Cost: premium pricing across every layer, and demand still outruns allocation for the newest parts.
• Facility burden: 120–140 kW liquid-cooled racks exceed what most existing enterprise server rooms can host, pushing buyers toward colocation or cloud.
• CUDA lock-in: the deeper your custom kernels and tooling, the higher the cost of ever moving to ROCm, TPUs or custom silicon.
• Concentration risk: a single vendor for compute, networking and software is a supply chain and commercial exposure.
• Geopolitics: export controls have already redrawn the addressable market once, and could again.

Mitigations exist. Writing to framework level (PyTorch rather than raw CUDA), exchanging models through ONNX, serving through portable runtimes such as Triton, and specifying open scale-out networking (Ultra Ethernet rather than proprietary fabrics) all preserve optionality without giving up much performance today.

The NVIDIA AI infrastructure market in 2026: competition, China and sentiment

NVIDIA holds roughly 80–85 per cent of data centre accelerator revenue in 2026 — down from about 92 per cent in 2023, which tells you the alternatives are real without threatening the lead. The serious challengers each attack a different weakness:

We looked at the AMD challenge in depth in our Helios market analysis. China deserves its own paragraph: Washington approved conditional H200 licences for about ten Chinese firms from December 2025, but Beijing ordered domestic companies to prioritise homegrown silicon and customs has blocked shipments — by mid-July 2026 a US Commerce official described actual deliveries as "trivial" against some US$10 billion in approved licences. Practically, analysts now treat the China market as closed to US accelerators, and NVIDIA's growth story no longer assumes it.

Market sentiment in mid-2026 is genuinely split, and both cases rest on real numbers. Bulls point to hyperscaler capital expenditure exceeding US$600 billion in 2026, buyers that are profitable and cash-generative, and inference — now roughly two-thirds of AI compute — becoming a recurring, revenue-linked workload; NVIDIA itself projects US$3–4 trillion of AI infrastructure spending by 2030. Bears point to customer concentration (four customers contributed 61 per cent of NVIDIA's revenue), to depreciation schedules that book five-to-six-year lives on hardware whose economic edge may last two to three, and to circular financing structures — vendor equity stakes and take-or-pay compute deals — that can make end-demand look larger than it is. Neither side has been proven wrong yet; a buyer's job is to size commitments so that either outcome is survivable.

NVIDIA AI infrastructure in Singapore

Singapore's deployments skew toward inference and sovereign-AI capacity rather than frontier training, which fits a power-constrained market. Singtel's RE:AI cloud runs H200 clusters and brought the region's first GB200 NVL72 systems online, with Nanyang Technological University among the anchor deployments. Sustainable Metal Cloud operates immersion-cooled H100 clusters of 2,048 GPUs at a PUE of about 1.02 — among the most efficient GPU infrastructure anywhere. The National Supercomputing Centre's ASPIRE 2A+ system adds H100 capacity for research, with a next-generation S$270 million system planned. Capacity growth sits inside the managed framework covered in our AI data centre explainer.

What buyers here should check

• Consumption model first: GPU cloud (local or regional), colocation with your own systems, or managed private AI — each shifts the power and cooling burden differently.
• NVLink domain size, not just GPU count: 72 GPUs in one NVLink domain behave very differently from 72 GPUs across nine servers.
• Software entitlement: whether NVIDIA AI Enterprise licensing and support are included, and who holds the relationship.
• Data residency and egress: where checkpoints, fine-tuned weights and inference logs physically live.

Future outlook

Three threads are worth watching. First, the cadence itself: Rubin ramps through 2027, and the 600 kW Rubin Ultra NVL576 generation will test what buildings, grids and supply chains can absorb — reports of manufacturing slippage on that rack, denied by NVIDIA, are a reminder that the roadmap's hardest constraints are now physical rather than architectural. Second, NVLink Fusion signals a company converting its interconnect into an industry standard: capturing value from racks even when the accelerator inside is someone else's. Third, the economics are tilting from training to inference, where cost per token and performance per watt — not peak petaflops — decide deals, and where competitors are strongest.

For most organisations the practical question is no longer whether NVIDIA's technology is capable — it demonstrably is — but which slice of the stack to consume, through which channel, and with how much optionality preserved. Understanding the layers is the first step in negotiating any of them.

Sourcing AI infrastructure in Singapore?

Browse AI computing specialists, GPU cloud operators and data centre providers who design, host and manage NVIDIA-based infrastructure for enterprise deployments.

Browse AI computing providers

Frequently asked questions

Sources and further reading

• NVIDIA — GB200 NVL72 product page
• NVIDIA — NVLink Fusion
• NVIDIA Newsroom — NVLink Fusion semi-custom AI infrastructure ecosystem
• NVIDIA Newsroom — Marvell joins the NVLink Fusion ecosystem (March 2026)
• IDC — NVIDIA becomes #1 in datacenter Ethernet switching (Q1 2026)
• TechPowerUp — Vera Rubin NVL144 servers set for 2026 volume production
• CNBC — Rubin Ultra rack delay report and NVIDIA response (July 2026)
• Brookings — The US is out of the AI chip market in China