GPUDatacenterReleased Dec 6, 2023

AMD Instinct MI300X

192 GB HBM3 · 8192-bit · 19,456 cores

The Instinct MI300X is AMD's CDNA 3 data center accelerator for large-scale generative AI training and inference, launched on December 6, 2023. Its chiplet design stacks eight compute dies on top of I/O dies with 256 MB of Infinity Cache, and its 192 GB of HBM3 lets a single accelerator hold large language models that would otherwise need to be split across several GPUs. It is typically deployed in eight-GPU platforms linked by Infinity Fabric.

General

Manufacturer: AMD
Model: Instinct MI300X
Segment: Datacenter
GPU chip: Aqua Vanjaram
Architecture: CDNA 3
Release date: Dec 6, 2023
Price: Search on Amazon

Memory

Memory size: 192 GB
Memory type: HBM3
Bus width: 8192-bit
Bandwidth: 5,300 GB/s
Memory clock: 2,600 MHz

Clocks & silicon

GPU clock: —
Boost clock: 2,100 MHz
Shader units: 19,456 Stream Processors (304 CU)
Tensor / AI cores: 1,216
RT cores: —
Transistors: 153 billion
Technology (nm): 5 nm
TSMC N5 + N6
TDP / TBP: 750 W

Interface & APIs

Bus interface: OAM (PCIe 5.0 x16 host)
SLI / multi-GPU: —
DirectX: —
OpenGL: —
Vulkan: —
Other APIs: ROCm (HIP), OpenCL

AI support

Precisions: FP8FP16BF16INT8INT4TF32FP64
AI accelerator: 1,216 Matrix Cores (4 per Compute Unit x 304 CUs) for FP8/FP16/BF16/INT8/TF32/FP64 matrix math
Peak AI TOPS: 2,614.9 TOPS
Notes: AMD-published peak compute: FP8/INT8 2614.9 TFLOPS/TOPS dense (5229.8 with 2:4 structured sparsity); FP16/BF16 1307.4 TFLOPS dense (2614.9 sparse); TF32 653.7 TFLOPS dense (1307.4 sparse); FP64 81.7 TFLOPS vector (163.4 TFLOPS matrix). AMD does not list INT4 among the supported data types.

All AMD GPUs →

Datacenter gpus from AMD announced around the same time as the AMD Instinct MI300X. Add any of them to your compare list to see the specs side by side.

gpu