GPUDatacenterReleased Nov 18, 2024

NVIDIA H200 NVL

141 GB HBM3e · 6144-bit · 16,896 cores

The PCI Express add-in card form of the HBM3e-equipped Hopper accelerator, designed for standard air-cooled enterprise racks where a full HGX system is impractical. Two or four cards can be joined with an NVLink bridge, and the card supports Multi-Instance GPU partitioning into up to seven instances, confidential computing and a hardware root of trust, with seven NVDEC and seven NVJPEG decoders for data processing pipelines. An NVIDIA AI Enterprise subscription is included. Announced at Supercomputing 2024 in November 2024, with partner systems from that December.

General

Manufacturer: NVIDIA
Model: H200 NVL
Segment: Datacenter
GPU chip: GH100
Architecture: Hopper
Release date: Nov 18, 2024
Price: Search on Amazon

Memory

Memory size: 141 GB
Memory type: HBM3e
Bus width: 6144-bit
Bandwidth: 4,800 GB/s
Memory clock: —

Clocks & silicon

GPU clock: 1,365 MHz
Boost clock: 1,785 MHz
Shader units: 16,896 CUDA Cores
Tensor / AI cores: 528
RT cores: —
Transistors: 80 billion
Technology (nm): 5 nm
TSMC 4N (5 nm-class)
TDP / TBP: 600 W

Interface & APIs

Bus interface: PCIe 5.0 x16 (dual-slot)
SLI / multi-GPU: NVLink bridge (900 GB/s, 2 or 4 GPUs)
DirectX: N/A (compute-only)
OpenGL: N/A (compute-only)
Vulkan: N/A (compute-only)
Other APIs: CUDA (compute capability 9.0)

AI support

Precisions: FP8FP16BF16INT8INT4TF32FP64
AI accelerator: Fourth-generation Tensor Cores (528) + Transformer Engine
Peak AI TOPS: 3,341 TOPS
Notes: Peak Tensor Core throughput published by NVIDIA for the H200 NVL: FP64 Tensor Core 60 TFLOPS, TF32 835 TFLOPS, BFLOAT16 and FP16 1,671 TFLOPS each, FP8 and INT8 3,341 TFLOPS each. The TF32, BFLOAT16, FP16, FP8 and INT8 figures are quoted with sparsity; dense rates are half. Standard FP64 is 30 TFLOPS and FP32 60 TFLOPS. NVIDIA does not claim INT4 Tensor Core support on Hopper.

All NVIDIA GPUs →

Datacenter gpus from NVIDIA announced around the same time as the NVIDIA H200 NVL. Add any of them to your compare list to see the specs side by side.

gpu