GPUDatacenterReleased Nov 13, 2023

NVIDIA H200 SXM 141GB

141 GB HBM3e · 6144-bit · 16,896 cores

An upgraded Hopper SXM module that keeps the compute engine of the H100 but replaces its memory with HBM3e, adding capacity and bandwidth for large language model inference and memory-bound HPC work. It is pin-compatible with HGX H100 systems and retains the Transformer Engine, Multi-Instance GPU partitioning and confidential computing. As a compute-only product it has no display outputs, RT Cores or video encoder. Announced at SC23 in November 2023, with systems shipping from 2024.

General

Manufacturer: NVIDIA
Model: H200 SXM 141GB
Segment: Datacenter
GPU chip: GH100
Architecture: Hopper
Release date: Nov 13, 2023
Price: Search on Amazon

Memory

Memory size: 141 GB
Memory type: HBM3e
Bus width: 6144-bit
Bandwidth: 4,800 GB/s
Memory clock: —

Clocks & silicon

GPU clock: —
Boost clock: 1,980 MHz
Shader units: 16,896 CUDA Cores
Tensor / AI cores: 528
RT cores: —
Transistors: 80 billion
Technology (nm): 5 nm
TSMC 4N (5 nm-class)
TDP / TBP: 700 W

Interface & APIs

Bus interface: SXM5 (PCIe 5.0 x16 host)
SLI / multi-GPU: NVLink 4.0 (900 GB/s)
DirectX: N/A (compute-only)
OpenGL: N/A (compute-only)
Vulkan: N/A (compute-only)
Other APIs: CUDA (compute capability 9.0)

AI support

Precisions: FP8FP16BF16INT8INT4TF32FP64
AI accelerator: Fourth-generation Tensor Cores (528) + Transformer Engine
Peak AI TOPS: 3,958 TOPS
Notes: Fourth-generation Tensor Cores with a Transformer Engine. Peak throughput with structured sparsity: FP8 and INT8 3,958, FP16 and BFLOAT16 1,979 TFLOPS, TF32 989 TFLOPS; dense rates are half. FP64 is 34 TFLOPS and FP64 Tensor Core 67 TFLOPS. NVIDIA does not claim INT4 Tensor Core support on Hopper. Multi-Instance GPU partitions the card into up to seven instances of 18 GB, and there are seven NVDEC and seven NVJPEG decoders.

All NVIDIA GPUs →

Datacenter gpus from NVIDIA announced around the same time as the NVIDIA H200 SXM 141GB. Add any of them to your compare list to see the specs side by side.

gpu