What the H200 GPU price hike signals about the future of AI compute
H200 GPU rental prices are rising despite newer GPUs. Explore the reasons behind the hike, market trends, and how AI teams can control compute costs.
A GPU is supposed to get cheaper as it ages. The H200 did not get the memo.
It is now a generation behind Blackwell, and it is still getting more expensive to rent. That is worth more attention than the headline number, because it tells you something about how AI compute is changing, and what to do about your own budget while it does.
How much has the H200 GPU price actually gone up?
It depends who you ask. Anyone quoting a single number for the h200 gpu price is picking a measure, and the measures disagree. Here is what has been reported in October 2026:
|
Measure |
Reading |
When |
|
Nebius on-demand list rate |
$4.50 to $5.40 per GPU-hour (+20%) |
From 1 October |
|
Crusoe on-demand list rate |
$4.29 to $5.40 (+26%) |
16 October |
|
AWS EC2 Capacity Blocks |
About +15%, as reported |
7 October |
|
Transaction-based index (OCPI-H200) |
Up 20% in a month, to $6.20 |
24 October |
|
Median listed on-demand price, 34 providers |
$4.77, up about 8% in 90 days and 28% in 12 months |
28 October |
|
a16z-based tally |
Up about 160% since January |
Mid-October |
Why do the numbers disagree so much?
Because they measure different things. A posted list price lags what customers actually pay. A transaction index reflects completed deals, which can move faster. Some trackers exclude reserved contracts or spot rates, and the mix of providers changes the median.
The safe reading is that the direction is clear and the size depends on the measure. Whichever you pick, H200 rental has gone up this year, not down.
Why is the H200 getting more expensive when a newer GPU exists?
Because memory, not the age of the chip, is the scarce part. The H200 carries 141GB of HBM3e against 80GB on the H100, and that extra memory is exactly what 70B-class models and long-context serving want. At the same time, memory is the part of the supply chain under the most strain. Micron said earlier this month that DRAM would stay tight through 2027 and into 2028.
Demand adds to it. Inference and agent workloads use far more tokens per task than a chatbot, and that load lands on the same limited pool of memory-heavy cards.
The result runs against what many people assumed. Rental rates for older GPUs were expected to fall with age. Instead, a16z's data shows older A100 and H100 cards still priced above their scheduled obsolescence.
What does the hike signal about the future of AI compute?
Four things stand out.
Compute is behaving like a scarce commodity, not a depreciating gadget
Finance teams assume hardware loses value on a schedule. When demand outruns supply, rental prices can rise instead. AWS raised its H200 instance prices by 15 percent in January, which was reported as its first GPU price increase in roughly two decades, and the pressure has continued since.
Memory is setting the price of each tier
The cards under the most pressure are the memory-heavy ones. If your model does not need 141GB, paying for it is the most avoidable cost on the page.
The market is splitting in two
Hyperscalers and specialist GPU clouds now sit far apart. One index puts the median hyperscaler H200 price around $10.30 per GPU-hour and the median specialist cloud around $4.78, about 2.2 times lower. The same chip, a very different bill.
Prices will be volatile, not one-way
Over the summer, floor prices were falling, with monthly cuts across several GPUs. In October the floors stopped falling and the H200 firmed. Plan for swings in both directions, and do not assume the current direction will last.
What does this mean for AI teams in India?
The moves reach Indian teams with a lag, either through rupee rates from local providers or through dollar-billed hyperscalers, where currency movement adds on top. CloudPe has forecast that Indian cloud GPU rental prices could rise 15 to 20 percent over the next 12 to 18 months, which is a conditional forecast, not a promise.
Buying is not an easy escape. A single H200 card is listed at roughly $32,000 to $47,500 before servers and power. For most teams, renting remains the practical route, and subsidised compute under the IndiaAI Mission, priced through the programme and not the open market, is the one option I would expect to move least. Access there is approval-gated.
What does a price rise like this cost on a real project?
Here is an illustrative case, using three rates from the table above: $4.50 before the Nebius change, $5.40 after it, and $6.20 as the transaction index reading.
|
Project |
At $4.50 |
At $5.40 |
At $6.20 |
|
Fine-tuning run, 8 GPUs for 100 hours (800 GPU-hours) |
$3,600 |
$4,320 |
$4,960 |
|
Inference service, 2 GPUs running all month (1,460 GPU-hours) |
$6,570 |
$7,884 |
$9,052 |
The training run costs about $720 more at the new list rate. The always-on service costs about $1,300 more every month, which is the kind of change that quietly turns a profitable feature into a break-even one. Steady workloads feel a price rise the most, since the extra cost repeats every month.
How should you plan your AI budget around this?
-
Budget per task, not per hour. Cost per request or per token is what you actually control
-
Check whether you need the memory. If your model fits in 80GB or less, an H200 may be paying for capacity you do not use
-
Commit to your steady baseline, and keep the bursty part on flexible pricing
-
Cut idle time first. It is the one saving no market move can take away
-
Ask about notice terms. Find out how much warning a provider gives before changing rates
-
Re-compare quotes every month. Prices have been moving faster than annual budgets
Conclusion
The H200 price hike is not just a bigger invoice. It signals that AI compute is scarce, memory-constrained and volatile, and that the old assumption of steadily falling prices no longer holds. Budget with a buffer, use only the memory you need, remove idle hours, and compare quotes often. The teams that treat compute as a managed cost, not a fixed one, will feel the next move least.
Frequently asked questions
Will the H200 price come back down?
Nobody can say for certain. Memory supply is expected to stay tight into 2027 and 2028, which argues for firmness. But floors fell for months over the summer, so prices can move both ways. Plan for firm prices, and treat a drop as a bonus.
Should I switch to an H100 or a B200 instead?
It depends on your model. An H100 is cheaper per hour if your workload fits in 80GB. A B200 can look cheap at its lowest in-stock price near $3.75 but typically rents closer to $8. Compare cost per token for your own workload, not the headline hourly rate.
Is this only affecting small providers?
No. Hyperscalers have raised prices too, and their H200 rates are the highest in the market.
Does a fixed-term contract protect me from all this?
For the length of the term, yes. The rise shows up at renewal, and contract rates have been climbing too: one index showed one-year H100 contracts up close to 40 percent between October 2025 and March 2026. If your renewal is near, ask for quotes early.


Prateek
