AMD boosts data center AI solutions with Instinct MI300 Series

Chipmaker AMD launched its highly anticipated AMD Instinct MI300 Series, a suite of powerful data center AI (artificial intelligence) solutions. This release includes the AMD Instinct MI300X accelerators and the AMD Instinct MI300A accelerated processing unit (APU), both engineered to deliver exceptional performance for various AI and HPC (High-Performance Computing) workloads.

 “AMD Instinct MI300 Series accelerators are designed with our most advanced technologies, delivering leadership performance, and will be in large-scale cloud and enterprise deployments,” said Victor Peng, president of AMD. “By leveraging our leadership hardware, software, and open ecosystem approach, cloud providers, OEMs, and ODMs are bringing to market technologies that empower enterprises to adopt and deploy AI-powered solutions.”

The MI300X accelerators, operating on the new AMD CDNA 3 architecture, outshine their predecessors with almost 40% more compute units, 1.5x more memory capacity, and 1.7x more peak theoretical memory bandwidth, tailored for AI and HPC workloads. With 192 GB of HBM3 memory capacity and 5.3 TB/s peak memory bandwidth, these accelerators cater to the escalating demands of AI workloads, setting a benchmark in the industry.

AMD opens pervasive AI developer contest
AMD expands ‘Zen 4’ lineup into handheld game consoles

In comparison to the NVIDIA H100 HGX, the AMD Instinct Platform presents up to a 1.6x increase in throughput while running inference on LLMs like BLOOM 176B. It stands as the sole option capable of single MI300X accelerator inference for a 70B parameter model, streamlining enterprise-class LLM deployments and offering exceptional Total Cost of Ownership (TCO).

Open-source AI software

The MI300A APUs, the world’s first data center APU for HPC and AI, leverage advanced packaging and the 4th Gen AMD Infinity Architecture. These APUs merge high-performance AMD CDNA 3 GPU cores, the latest AMD “Zen 4” x86-based CPU cores, and 128GB of next-gen HBM3 memory, delivering approximately 1.9x better performance-per-watt on FP32 HPC and AI workloads compared to their predecessors.

AMD’s commitment to open-source AI software development is evident through the latest AMD ROCm 6 open software platform. This platform amplifies AI acceleration performance by around 8x when running on MI300 Series accelerators in Llama 2 text generation compared to prior hardware and software versions. ROCm 6 introduces support for various new features vital for generative AI, reinforcing AMD’s position to leverage widely used open-source AI software models, algorithms, and frameworks.