GPUDatacenterReleased Aug 8, 2023

NVIDIA L40S

48 GB GDDR6 ECC · 384-bit · 18,176 cores

A universal data center GPU that pairs the graphics and media engines of Ada Lovelace with fourth-generation Tensor Cores, for customers who want a single card covering generative AI training and inference, 3D rendering, simulation and virtual workstations. Unlike NVIDIA's HBM-based accelerators it keeps display outputs and full RTX graphics support, uses ECC-protected GDDR6 memory, and is certified for NVIDIA vGPU and OVX systems. Announced at SIGGRAPH on August 8, 2023, with availability through system partners that fall.

General

Manufacturer: NVIDIA
Model: L40S
Segment: Datacenter
GPU chip: AD102
Architecture: Ada Lovelace
Release date: Aug 8, 2023
Price: Search on Amazon

Memory

Memory size: 48 GB
Memory type: GDDR6 ECC
Bus width: 384-bit
Bandwidth: 864 GB/s
Memory clock: 2,250 MHz

Clocks & silicon

GPU clock: —
Boost clock: 2,520 MHz
Shader units: 18,176 CUDA Cores
Tensor / AI cores: 568
RT cores: 142
Transistors: 76.3 billion
Technology (nm): 5 nm
TSMC 4N (5 nm-class)
TDP / TBP: 350 W

Interface & APIs

Bus interface: PCIe 4.0 x16
SLI / multi-GPU: None (no NVLink)
DirectX: 12 Ultimate (12_2)
OpenGL: 4.6
Vulkan: 1.4
Other APIs: CUDA (Compute Capability 8.9), OpenCL 3.0

AI support

Precisions: FP8FP16BF16INT8INT4TF32FP64
AI accelerator: 4th Gen Tensor Cores (568) + 3rd Gen RT Cores (142)
Peak AI TOPS: 1,466 TOPS
Notes: Fourth-generation Tensor Cores with FP8 support. Peak throughput with structured sparsity: FP8, INT8 and INT4 1,466 TFLOPS/TOPS (733 dense), FP16 and BF16 733 TFLOPS (362 dense), TF32 366 TFLOPS (183 dense). FP32 is 91.6 TFLOPS and RT Core performance is 212 TFLOPS. Three NVENC and three NVDEC engines with AV1 encode and decode. MIG is not supported.