How expensive are AI servers to run per month
Discover the true monthly costs of running an AI server, from hardware depreciation and kilowatt colocation rates to power and cooling overhead expenses.
In this article8
- The components of AI server monthly run costs
- Server power draw and the impact of memory
- Data center colocation rates per kilowatt
- Cooling overhead and energy efficiency
- Total cost of ownership versus continuous utilisation
- Shifting CapEx to OpEx: When to rent instead of own
- Frequently asked questions
- Sources

The components of AI server monthly run costs
Calculating the monthly operating cost of an artificial intelligence compute node requires separating fixed capital amortization from variable facilities charges. In standard enterprise IT, hardware depreciation over four to five years represents the primary financial expense, with power and hosting treated as minor overheads. In accelerated AI systems, however, elevated power densities and rapid hardware release cycles compress capital write-downs to two to three years while shifting monthly facility expenses from flat rack rents to usage-based power charges.
Monthly running costs for modern AI infrastructure comprise four primary components: accelerated server depreciation, direct electricity consumption, high-density colocation space, and cooling overhead. According to the IEA's breakdown of data centre electricity consumption by equipment type, servers account for around 60% of electricity demand in modern data centers, with storage at roughly 5%, networking equipment up to 5%, and the remainder going to cooling and environmental control[1]. On the hardware side, accelerating server bills of materials has turned high-bandwidth memory and thermal packaging into leading cost drivers.
| Cost Component | Financial Classification | Primary Cost Determinant | Monthly Budget Behavior |
|---|---|---|---|
| Server Hardware | Capital Expenditure (amortized) | Accelerator architecture, HBM capacity, purchase price | Fixed monthly write-down across 24 to 36 months |
| Direct Power | Operating Expenditure | Thermal design power (TDP), workload utilization rate | Variable based on active compute draw (kW) |
| Colocation Base Rent | Operating Expenditure | Contracted power capacity, rack footprint, tier level | Fixed per-kW or per-rack monthly commitment |
| Cooling and Facility Overhead | Operating Expenditure | Facility Power Usage Effectiveness (PUE), liquid loop | Variable or bundled via contracted PUE multiplier |
Because an 8-GPU AI server can demand between 6 kW and 10.2 kW of continuous power under full inference or training loads, facilities fees scale directly with power density. Understanding how hardware depreciation interacts with data center billing models is necessary for budgeting enterprise deployments accurately.
Server power draw and the impact of memory
The electrical demands of modern compute nodes are determined by accelerator processor logic, system host processors, and high-bandwidth memory (HBM) subsystems. Current-generation accelerators illustrate the scale: an NVIDIA H100 is rated at 700 W per GPU, while a Blackwell B200 runs at 1,000 W air-cooled or 1,200 W liquid-cooled, so an air-cooled 8-GPU HGX B200 system lands at roughly 60 kW across four systems per rack[2]. At this scale, memory architecture exerts a direct influence on energy consumption and thermal dissipation.
As model parameter sizes grow, moving weights between memory stacks and compute cores consumes a rising share of the node's power budget. Standards development organizations have focused specifically on mitigating these memory energy profiles. For over 50 years, JEDEC has been the global leader in developing open standards for the microelectronics industry, including main memory specifications such as DDR SDRAM, Low Power DDR (LPDDR), and High Bandwidth Memory (HBM), which govern interconnect efficiency and bandwidth density.
- Accelerator Logic: Compute engines executing tensor math dominate raw node power draw at full utilization, and their rated TDP sets the floor for every other facility calculation.
- High-Bandwidth Memory (HBM): Memory stacks and silicon interposers add a material share of total accelerator wattage to sustain multi-terabyte-per-second throughput, and that share grows as HBM capacity per package rises.
- Host CPU and System Memory: Dual host CPUs and system DDR5 memory add several hundred watts per node on top of the accelerators.
- Auxiliary Chassis Systems: High-velocity chassis fans, power supplies, and PCIe or NVLink network interfaces consume further overhead before any facility cooling is counted.
When evaluating AI accelerator cost trends, the energy profile of the memory subsystem directly dictates both power bills and chassis thermal design requirements.
Data center colocation rates per kilowatt
Deploying enterprise AI hardware requires specialized high-density colocation facilities. Traditional data centers are designed for roughly 5 kW to 8 kW per rack, whereas AI infrastructure routinely needs 15 kW to 25 kW for standard deployments, 30 kW to 50 kW for high-density racks, and 60 kW to over 100 kW for liquid-cooled systems[3]. Rack-scale platforms sit at the top of that range: an NVIDIA GB200 NVL72 cabinet draws roughly 120 kW nominal, against an industry-average rack of about 7.6 kW[2].
This power intensity has fundamentally altered colocation commercial agreements. Facility operators no longer price space primarily by the physical cabinet unit or floor square footage; instead, wholesale deployments are priced per kilowatt of contracted capacity. CBRE puts the average asking rate for a 250-to-500-kW requirement in primary North American wholesale colocation markets at a record $196.25 per kW per month in H2 2025, a 6.6% year-over-year increase, with primary-market vacancy at a record low of 1.4% at year-end.[4]
| Deployment Type | Typical Power Density | Monthly Space and Power Structure | Key Infrastructure Requirement |
|---|---|---|---|
| Standard Enterprise Colocation | 5 kW to 8 kW per rack | Flat per-cabinet rent with metered or bundled power | Standard perimeter air cooling (CRAC) |
| Mid-Density Inference Rack | 15 kW to 25 kW per rack | Per-cabinet rent plus a rising power component | Hot/cold aisle containment, reinforced busbars |
| High-Density AI Cluster | 30 kW to 100 kW+ per rack | Priced per kW of contracted capacity (record primary-market average of $196.25/kW/month in H2 2025) | Direct-to-chip liquid cooling, dedicated CDU loops |
The arithmetic follows directly from the contracted rate. At the record primary-market average of $196.25 per kW per month, an enterprise holding a dedicated high-density rack at a continuous 30 kW commits to several thousand dollars a month in colocation and power before dedicated fiber cross-connects, managed network ports, or initial setup charges.[4]
Cooling overhead and energy efficiency
Cooling high-density server infrastructure constitutes the largest component of non-compute power consumption in a data center. The efficiency of this auxiliary infrastructure is measured by Power Usage Effectiveness (PUE), defined as total facility energy divided by IT equipment energy. A PUE of 1.25 indicates that for every 1.00 kW consumed by the server, an additional 0.25 kW is required for facility cooling, power transformation, and distribution.
Cooling systems and environmental control account for about 7% of data centre electricity consumption in efficient hyperscale facilities and over 30% in less efficient enterprise data centers, according to the IEA equipment-type breakdown[1]. Traditional forced-air cooling systems encounter physical limitations at heat densities above 15 kW to 20 kW per cabinet, creating severe thermal bottlenecks for dense GPU deployments.
- Air-Cooled Facilities: The average annual PUE reported across the industry was 1.55 before optimisation, and air cooling consumes roughly 0.5 kW to 1.2 kW for every kW of IT load. Heat rejection relies on computer room air handlers (CRAH) and chilled water loops.
- Direct-to-Chip Liquid Cooling: PUE falls to about 1.05 to 1.15 once cold plates replace air as the primary heat path. Cold plates circulate coolant at 15 to 30 degrees Celsius directly over GPU and CPU packages, removing 60% to 80% of server heat while chassis fans handle the remainder.[5]
- Immersion Cooling: Two-phase immersion pushes PUE towards 1.02 by eliminating pumps and chassis fans, with nodes submerged in dielectric fluid so facility auxiliary power draw drops further.
Under evolving regulatory frameworks such as European datacenter energy reporting mandates under the Corporate Sustainability Reporting Directive (CSRD), facility PUE and cooling emissions must be disclosed as material environmental metrics alongside baseline power costs.
Total cost of ownership versus continuous utilisation
The economic viability of purchasing dedicated AI server hardware depends heavily on sustained utilization. When an organization purchases an 8-GPU compute node, the primary monthly expenses are fixed: hardware depreciation amortizes at a constant monthly rate, while data center colocation contracts require a fixed monthly commitment for allocated kilowatts regardless of whether the processors are running workloads or sitting idle.
When a dedicated AI server runs near-continuously, fixed capital and colocation expenses are distributed across millions of processed tokens or completed training steps, resulting in a predictable and defensible unit cost. However, when utilization falls to a small fraction of available capacity, the effective cost per useful compute hour rises sharply because the fixed monthly bills for amortized capital, rack reservations, and idle power draw do not decrease.
- Fixed Amortization Lock-in: An accelerated write-down schedule of two to three years turns the purchase price of a GPU node into a constant monthly capital charge, booked whether or not the accelerators run a single job.
- Reserved Power Minimums: High-density colocation contracts bill reserved capacity, so a 10 kW reservation at the record primary-market average of $196.25 per kW per month is charged in full even when the hardware is powered down.
- Idle Power Penalty: Enterprise AI servers continue to draw a meaningful fraction of their peak wattage in low-power idle states, consuming grid electricity while waiting for batch jobs.
- Maintenance and Support Overheads: Annual enterprise vendor hardware support contracts add a recurring percentage of hardware purchase value each year, divided across monthly operating budgets.
For organizations with intermittent workloads, internal development cycles, or unpredictable customer traffic, the total cost of ownership of dedicated hardware creates substantial financial drag.
Shifting CapEx to OpEx: When to rent instead of own
The decision between capital equipment ownership and cloud-based operational expenditure turns on workload predictability, burst requirements, and hardware lifecycle risk. Purchasing dedicated servers requires upfront capital, engineering staff to manage thermal distribution, and long-term facility commitments. If enterprise application demand fails to materialize, the organization remains burdened with depreciating silicon and fixed monthly colocation fees.
For pilot programs, fine-tuning tasks, and fluctuating inference traffic, on-demand compute models remove the fixed financial overhead of server ownership. Rather than committing to multi-year hardware leases and monthly kilowatt minimums, organizations can consume accelerator capacity strictly as needed. In production inference setups, teams can evaluate LLM cost per token benchmarks to determine whether per-token routing or raw infrastructure provides the optimal cost baseline.
For organizations seeking on-demand GPU capacity without fixed long-term commitments, On-demand GPU VM provides on-demand compute infrastructure billed strictly per second with no ongoing subscription or base fee. Capacity can be provisioned or released with two to three weeks notice and zero scale-down penalties, allowing engineering teams to align monthly infrastructure expenditure directly with active enterprise utilization.
Frequently asked questions
How much power does a single AI server use?
A high-density AI server rack can draw 30 to 100 kilowatts of power, far exceeding traditional enterprise servers. The high electrical demand is driven by multiple GPUs running concurrently alongside supporting components like high-bandwidth memory.
What is the average monthly colocation cost for AI infrastructure?
In primary North American wholesale markets, asking rates for a 250-to-500-kW requirement averaged a record $196.25 per kilowatt per month in H2 2025. Because AI deployments are priced by power density rather than physical space, a fully loaded rack incurs significantly higher monthly fees than standard hardware.
Why is cooling such a large factor in AI server expenses?
Dense AI compute generates intense heat. Depending on facility efficiency, cooling systems can consume from about 7 percent of total data center electricity in efficient hyperscale sites to over 30 percent in less efficient enterprise facilities, making thermal management a major portion of the operating bill.
Does memory impact the operating cost of an AI server?
Yes, main memory configurations are a major driver of both the initial bill of materials and ongoing power draw. Organizations like JEDEC, which has defined microelectronics standards for over 50 years, continuously iterate on memory architectures to balance high bandwidth with power efficiency.
When should an enterprise rent rather than run its own AI servers?
For pilot programs, burst capacity needs, or intermittent workloads, the fixed monthly costs of colocation and hardware depreciation make ownership inefficient. Renting on-demand GPU capacity avoids these baseload expenses and shifts the financial model from fixed capital to scalable operating costs.
Sources
- AI Energy Consumption Statistics aimultiple.com
- Nvidia Blackwell Explained: Data Center Impact moduledge.com
- Colocation for AI Workloads: Power, Cooling & GPU Density Requirements Explained databank.com
- North America Data Center Trends H2 2025 cbre.com
- 5 Misunderstood Facts About Direct-to-Chip Liquid Cooling chatsworth.com
Rankings and scores on Optinest cannot be bought — no commercial relationship changes how a provider is assessed. How we score
Related reading
All Memory →How much does inference actually cost per month
Understand the true drivers behind monthly AI inference costs. We break down GPU hosting, token multipliers, and how serverless open models reduce bills.

Renting 2- and 4-GPU servers instead of a full node
Compare European providers that rent 1, 2 or 4 GPUs instead of a full 8-GPU node, with per-GPU rates from each pricing page and the NVLink trade-off explained.

GPU failure rate: data center statistics and node reliability
Discover the real-world failure rate of data center GPUs based on published cluster studies, from H100 hardware interruptions to the impact on training MTTF.
