GPUDatacenterReleased Mar 21, 2023

NVIDIA H100 NVL 94GB

6016-bit · 16,896 cores

A PCIe version of NVIDIA's Hopper H100 built for large language model inference, announced at GTC in March 2023. It offers more memory than the standard H100 PCIe, with 94 GB of HBM3, and was introduced as a pair of cards linked by NVLink bridges to provide 188 GB of combined memory. The dual-slot, passively cooled card is offered in partner and NVIDIA-Certified systems with one to eight GPUs.

General

Manufacturer: NVIDIA
Model: H100 NVL 94GB
Segment: Datacenter
GPU chip: GH100
Architecture: Hopper
Release date: Mar 21, 2023
Price: Search on Amazon

Memory

Memory size: 94 GB
Memory type: HBM3
Bus width: 6016-bit
Bandwidth: 3,938 GB/s
Memory clock: 2,619 MHz

Clocks & silicon

GPU clock: 1,080 MHz
Boost clock: 1,785 MHz
Shader units: 16,896 CUDA Cores
Tensor / AI cores: 528
RT cores: —
Transistors: 80 billion
Technology (nm): 4 nm
TSMC 4N
TDP / TBP: 400 W

Interface & APIs

Bus interface: PCIe 5.0 x16
SLI / multi-GPU: NVLink bridge (600 GB/s)
DirectX: N/A (compute-only)
OpenGL: N/A (compute-only)
Vulkan: N/A (compute-only)
Other APIs: CUDA (compute capability 9.0)

AI support

Precisions: FP8FP16BF16INT8INT4TF32FP64
AI accelerator: 4th Gen Tensor Cores (528) + Transformer Engine
Peak AI TOPS: 3,341 TOPS
Notes: Peak Tensor Core throughput with sparsity: FP8 and INT8 3,341, FP16 and BF16 1,671, TF32 835 TFLOPS; dense rates are half. FP32 60 TFLOPS, FP64 30 TFLOPS (60 TFLOPS via FP64 Tensor Cores). Transformer Engine with FP8 support, CUDA compute capability 9.0, up to 7 MIG instances of 12 GB each. Board power is configurable from 350 to 400 W.

All NVIDIA GPUs →

Datacenter gpus from NVIDIA announced around the same time as the NVIDIA H100 NVL 94GB. Add any of them to your compare list to see the specs side by side.

gpu