Scalability to define future of HBM for AI chips

The road map for next-generation high-bandwidth memory. Courtesy of KAIST TERALAB

Professor Kim Joung-ho of the Korea Advanced Institute of Science & Technology (KAIST), a leading expert in high-bandwidth memory (HBM) design, presented a road map for the future specifications of AI-specific memory, emphasizing that scalability will define its future.

During an online presentation Wednesday, KAIST Terabyte Interconnection and Package Laboratory (TERA lab), led by Kim, unveiled the standard technical specifications of HBM4 to HBM8 that will span from this year to 2040.

Currently, HBM3e is the most advanced commercially manufactured HBM chip. Memory chip makers such as Samsung Electronics and SK hynix are now preparing for the mass production of HBM4, highlighted by its wider bandwidth and customizable base dies that run functions demanded by AI chipmakers such as Nvidia and AMD.

TERA lab estimated that HBM5 will likely debut in 2029. One of the key advancements expected in HBM5 is 3D near-memory computing, which uses advanced packaging technologies to embed compute or cache dies directly within or on top of the memory stack.

To address the chip cooling issue, HBM5 will also adopt immersion cooling, a method that submerges the entire HBM package in coolant for more effective thermal management.

Professor Kim Joung-ho at Korea Advanced Institute of Science & Technology / Korea Times file

HBM6, which will likely be introduced in 2032, is expected to adopt a multitower architecture to overcome memory capacity limits. Instead of stacking memories higher, it expands capacity by placing additional HBM stacks next to each other.

The lab expects HBM7 to become available in 2035, featuring a hybrid architecture that combines HBM with other memory types such as low-power DDR and flash. This combination is aimed at boosting both performance and capacity.

HBM8 is estimated to emerge in 2038, and this chip will incorporate a full 3D architecture, the lab noted. In this structure, HBM can be mounted directly on top of the graphics processing unit (GPU) or positioned on both sides of the interposer alongside the GPU.

“The presentation is purposed at enhancing the industry’s understanding of the future of HBM technology, and to contribute to the sustainable growth of Korea’s semiconductor industry,” Kim said.

“The lab plans to share the presentation with major domestic chipmakers such as Samsung Electronics and SK hynix, and aims to hold similar presentations overseas, particularly in Silicon Valley.”