How Much Power Does a GPU Use? – TDP, Servers and Real Draw

A spec sheet says 700 watts. The facility engineer says the rack needs 40 kilowatts. The utility says the campus wants 200 megawatts. All three can be talking about the same chips, and none of them is wrong. The gap between them is where most confusion about GPU power consumption comes from. How much power does a GPU really use once you count the server, the rack and the building around it?

A top-end data center GPU draws roughly 700 to 1,000 watts or more at full load, but once you count the server and the building around it, each one costs about 1.5 to 2 kilowatts at the utility meter. NVIDIA rates its H100 SXM at up to 700 W. Add the processors, memory, networking and fans or pumps in the server, and each GPU accounts for roughly 1.2 to 1.5 kW. Cooling and electrical losses add the rest. These are estimates that move with chip generation, workload and cooling design.

What Does a GPU's TDP Actually Mean?

Thermal design power (TDP) is the maximum sustained power the chip is designed to dissipate as heat, and so the number the cooling system has to handle. It is a design ceiling, not a constant draw. A GPU running a light job, waiting on data or sitting idle uses much less.

NVIDIA's H100 product page lists a maximum TDP of up to 700 W for the SXM version and 350 to 400 W for the PCIe-based H100 NVL. Same chip family, very different power envelopes. The SXM module sits in a dense multi-GPU board built for high-power operation, while the PCIe card has to fit a standard server slot.

Operators can also cap power in software. Setting a lower power limit trades some speed for lower draw and less heat, which can be useful when a site is short on power or cooling. Many facilities plan for TDP anyway, because a cluster running a big job will push every chip toward its limit at the same moment.

How Does a GPU Compare With a Server CPU?

A data center GPU typically draws two to several times as much power as a server CPU. Mainstream server processors usually carry TDP ratings in the low hundreds of watts, while top accelerators sit at 700 W and up. The bigger difference is how many of them share a box: a general-purpose server might hold two CPUs, while an AI training server holds two CPUs plus eight GPUs.

That is why AI servers land around 10 kW while ordinary servers often draw well under 1 kW. The GPUs are not just hungrier chips; they arrive in packs.

How Does GPU Power Consumption Change With Each Generation?

Each generation tends to draw more power per chip while doing much more work per watt. NVIDIA's Blackwell generation is widely reported at around 1,000 W or more per GPU in its top configurations, up from 700 W for the H100 SXM. More power per chip is the price of more computing per chip.

That trend pushes design limits in two directions at once. Per GPU, efficiency improves, so a given job can finish with less energy. Per rack, though, power keeps climbing, because builders pack the newest chips as tightly as possible. NVIDIA's GB200 NVL72 puts 72 Blackwell GPUs and 36 Grace CPUs in one liquid-cooled rack, a density that air cooling could not support.

From Chip to Server: Where the Extra Watts Come From

The GPU is the biggest load in an AI server, but it is not alone. A typical eight-GPU training server also carries two host CPUs, large amounts of memory, high-speed network cards, local storage, power supplies that lose a few percent as heat, and either fans or a liquid cooling loop.

Here is a simple model for an eight-GPU server built around 700 W chips. These are illustrative inputs:

1. GPUs: 8 × 700 W = 5,600 W.
2. CPUs, memory, network cards and storage: about 2,500 W.
3. Fans, power supply losses and board overhead: about 1,900 W.
4. Server total: about 10,000 W, or 10 kW.
5. Per GPU: 10,000 W ÷ 8 = 1,250 W.

In other words, the "700 W chip" really costs about 1.25 kW at the server level in this example. Anyone sizing a room from chip TDP alone will come up short by nearly half.

How Much Power Does a GPU Rack Use?

It depends on how many servers fit and how they are cooled. Air-cooled racks with four eight-GPU servers land around 40 kW using the model above (4 × 10 kW). That is already several times what a traditional enterprise rack was built for.

Liquid-cooled rack-scale systems go much further. If a rack holds 72 GPUs at about 1,000 W each, the accelerators alone draw 72 kW, and the CPUs, switches and power conversion push the total past 100 kW. At that density, liquid cooling is no longer optional; there is simply too much heat to blow out with air.

For the facility, rack power drives everything else: the size of the busway, the number of coolant distribution units per row, the floor loading and the spacing of the hall. All of that power ends up as heat, and at 100 kW per rack someone has to plan for it on day one.

How Much Power Does a GPU Use at the Meter?

At the utility meter, each GPU carries its share of the networking, storage, cooling and electrical losses for the whole building. Continuing the model:

• Server level: 1,250 W per GPU.
• Add cluster networking and storage, about 12 percent: roughly 1,400 W per GPU.
• Multiply by a power usage effectiveness (PUE) of 1.2 for cooling and electrical overhead: 1,400 W × 1.2 = 1,680 W per GPU at the meter.

The chart below lays out those steps. Only the first bar is a vendor figure; the rest are the illustrative assumptions in this article.

Scale that to a cluster. A site with 100,000 GPUs at about 1.68 kW each needs roughly 168 MW at the meter. Running near full output all year, that is 168 MW × 8,760 hours ≈ 1.47 million megawatt-hours. Newer, hotter chips push the per-GPU number higher.

Why Does GPU Power Draw Swing So Much?

Because the work comes in bursts. A GPU computing flat out sits near its power limit, then drops sharply while it waits for data from memory or from other GPUs across the network. In a big training job, thousands of chips do this in step, so the whole cluster's load rises and falls together.

This is not just an operator's hunch. A 2025 Microsoft Research paper, written with engineers from OpenAI and NVIDIA, documented exactly these synchronized swings in large training clusters. On a small cluster nobody notices. On a cluster drawing tens or hundreds of megawatts, those swings show up at the substation and can stress on-site generators.

For operators, the fixes sit at several layers: software that smooths the timing of jobs, GPU firmware that limits how fast power can change, and batteries or UPS systems that absorb the fast swings before they reach the grid or the generators. If you run on-site power, ask your equipment vendor how their units handle repeated load steps, not just a single start.

How Much Electricity Does One GPU Use in a Year?

A single 700 W GPU running at full power around the clock uses 0.7 kW × 8,760 hours = 6,132 kilowatt-hours (kWh) a year at the chip. With the server and building overhead from the model, that rises to about 1.68 kW × 8,760 = 14,717 kWh.

For perspective, that all-in figure is more than a typical American home uses in a year, which EIA averages put at roughly 10,000 to 11,000 kWh. One GPU, fully loaded, outweighs a house. A hundred thousand of them outweigh a small city.

Real clusters rarely run at exactly 100 percent all year. Jobs pause, chips wait on data and maintenance takes nodes offline. But large training clusters tend to run hard, so planners usually assume high utilization rather than hoping for slack. Those totals add up nationally too, which is why data centers' share of US electricity has become a headline number.

What Can Operators Do to Lower GPU Power Use?

They can lower the power per job, even if the total keeps rising. The main tools are practical ones:

• Power capping, which trims peak draw with a modest hit to speed.
• Liquid cooling, which lowers fan power and lets the cooling plant run warmer and more efficiently.
• Higher utilization, so fewer idle chips burn standby power.
• Newer hardware, which finishes the same work with less energy per operation.

None of these shrinks the utility request much on a growing campus. They do make each megawatt go further, which is the most an operator can promise.

GPU Power Consumption Adds Up Layer by Layer

The single most common sizing error is starting from the chip. A 700 W spec sheet number turns into roughly 1.7 kW per GPU at the meter in a good facility, and more with hotter chips or a worse PUE. That gap is the difference between a power request that holds up and one that comes back short.

When you size a room, a lease or a power request, start from the all-in number, not the chip TDP. Multiply by GPU count, check the PUE your cooling can hold on the hottest day, and you will have a figure that the facility engineer and the utility will both recognize.

← All guides