Back to Devoured - November 12, 2026
DEVOURED
What an H100 actually costs per hour (GitHub Repo)

What an H100 actually costs per hour (GitHub Repo)

Data GitHub
The cost of renting an H100 SXM 80GB GPU varies by 457% between community providers and major hyperscalers like Azure.
What: Research into on-demand GPU pricing shows hourly costs ranging from $2.69 on RunPod Community to $12.29 on Azure's ND96isr_H100_v5 instances. This price spread accounts for differences in enterprise-grade infrastructure, specifically the availability of 400 Gbps InfiniBand and GPU Direct RDMA, which are necessary for large-scale multi-node training.
Why it matters: This disparity underscores that cloud GPU pricing reflects more than just the raw silicon; it includes premium costs for high-speed interconnects, SLAs, and enterprise-grade networking stacks required for distributed model training.
Deep dive
  • Comparing on-demand prices for the H100 SXM 80GB reveals massive price gaps: $2.69/hr (RunPod Community), $3.99/hr (RunPod Secure), $6.88/hr (AWS p5.48xlarge), and $12.29/hr (Azure).
  • Lower-cost providers lack the high-performance networking (400 Gbps InfiniBand) standard in hyperscaler enterprise instances.
  • The research emphasizes grouping by exact model and memory capacity (SXM, PCIe, NVL) to avoid misleading price comparisons.
  • Pricing lists from hyperscalers like AWS and Azure ignore reserved instance or spot pricing, which would likely narrow the spread.
  • Availability is a hidden factor; low advertised rates for unavailable hardware are not functional prices for production workflows.
Decoder
  • InfiniBand: A high-performance, low-latency network interconnect used in data centers for high-performance computing and GPU clustering.
  • GPU Direct RDMA (Remote Direct Memory Access): A technology that allows GPUs to access memory from other GPUs over the network without involving the CPU, essential for high-throughput distributed training.
  • SXM: A form factor for GPUs developed by NVIDIA designed for high-performance server boards, offering higher power limits and bandwidth compared to the standard PCIe cards.
Original article

109 — What an H100 actually costs per hour

On-demand GPU rental prices across AWS, Azure and the neoclouds, normalised to dollars per GPU per hour and grouped so only identical GPUs are compared.

Result

H100 SXM 80GB $/GPU-hr A month (730 h)
RunPod Community $2.69 $1,964
RunPod Secure $3.99 $2,913
AWS p5.48xlarge $6.88 $5,022
Azure ND96isr_H100_v5 $12.29 $8,972

4.57× between cheapest and dearest, for the same chip. One GPU running continuously for a month is $1,964 or $8,972 depending only on where you rent it.

The comparison is grouped, deliberately

"An H100" is three different products. The SXM 80GB, the PCIe 80GB and the NVL 94GB all get marketed as H100s and they do not cost the same. Putting them in one chart manufactures a spread that isn't there, so every comparison here is within a single (model, memory) group.

The same applies within a vendor: Azure lists the same A100 PCIe 80GB at one, two and four GPUs per VM for $3.67, $4.78 and $4.78 per GPU-hour. The spread uses each vendor's cheapest route to a given GPU, so an Azure-vs-Azure gap never shows up as a market gap. RunPod Secure and Community are counted as one company.

What the 4.57× is not

It is not pure margin. Going from Azure's ND series to RunPod Community gives up:

  • InfiniBand. Azure's ND SKU has 400 Gbps per GPU of Quantum-2 InfiniBand with GPU Direct RDMA, which is what makes multi-node training work. That is the reason the ND series is priced above Azure's own single-H100 NC SKU ($6.98).
  • A datacentre SLA. RunPod Community is third-party hosts renting out their own hardware.
  • Enterprise compliance, committed capacity, and the rest of the hyperscaler surface.

If you are fine-tuning on one node, you are paying for an interconnect you will never use. If you are training across 64 nodes, the cheap row is not an option.

Files

File What it is
collect.py Pulls AWS Price List + Azure Retail Prices. No auth required.
hyperscalers.json What those APIs returned.
vendors.json Neocloud prices, transcribed, with source URL and date read.
compute.py Grouping, per-vendor normalisation, spreads.
compare.json Output.
python3 collect.py   # ~1 GB AWS index, slow
python3 compute.py

Limitations

  • On-demand list prices only. Reserved, committed-use and spot pricing all cut the hyperscaler numbers, and spot in particular would narrow the gap a lot.
  • Single region each (AWS us-east-1, Azure eastus).
  • GPU counts come from vendor documentation, not from the price label — AWS reports "640 GB HBM3" for a p5 and Azure reports an accelerator-memory total, and neither states a per-GPU model. Doc URLs are in compare.json.
  • Lambda is excluded from the headline spread. Its published page states the listed rates take effect 2026-11-16; current rates are behind a login. Its rows are shown separately and labelled.
  • Availability is not priced. A cheap H100 you cannot get is not a price.
Explore all 61 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi