Cooling design emerges as battleground for next-gen AI memory

Song Jai-hyuk, president and chief technology officer at Samsung Electronics Device Solutions division, introduces a mock-up of the firm's high-bandwidth memory 5 chip during the COMPUTEX 2026 exhibition in Taiwan, Tuesday. Courtesy of Samsung Electronics

Chip designs aimed at keeping artificial intelligence (AI) accelerators cool are emerging as a new battleground for memory makers, with Samsung Electronics and SK hynix each revealing heat dissipation structures for their upcoming high-bandwidth memory (HBM) products.

The memory makers are increasing stacks and data transfer speed for HBMs to meet growing demands for AI computing performance, but facing intensified thermal management challenges. Starting with the eighth-generation HBM5, Samsung Electronics and SK hynix plan to introduce new heat-dissipation technologies to reduce heat at the chip level, improving system reliability and helping data center operators lower total cost of ownership.

During COMPUTEX 2026 in Taiwan earlier this week, Samsung Electronics unveiled a mock-up of its HBM5 chip.

“Thermal management is becoming a critical factor for next-generation memory as AI systems grow increasingly powerful and densely integrated, requiring advances in not only memory performance but also heat dissipation,” said Song Jai-hyuk, president and chief technology officer of Samsung's Device Solutions division.

At the center of Samsung Electronics' HBM5 is its Heat Path Block (HPB) technology. Designed to more efficiently disperse and dissipate heat generated within the chip, the technology acts as a thermal chimney, creating a dedicated pathway for heat to escape while reducing thermal resistance and improving operational stability.

HPB has already been adopted in application processors such as Samsung's Exynos application processors (APs). In January, the company said it adopted HPB for its Exynos 2600 processor, where a copper-based HPB structure mounting on top of AP dies and creates a more efficient heat-transfer path. The company said the technology reduced thermal resistance by up to 16 percent.

For HBM applications, however, Song said Samsung is considering a silicon-based HPB structure. He added that the key is to optimize the placement of the base die and core dies, suggesting that the HPB is being integrated into the overall memory stack design rather than simply placing HPB on top of the chip.

Song said the core die is being developed in line with the HPB structure to optimize the overall HBM architecture. He added that this gives Samsung an advantage because the company can manufacture the base die through its own foundry operations.

Samsung has already completed the implementation and validation of HPB technology for its seventh-generation HBM4E product, samples of which were shipped to customers for the first time on May 29. The company is planning a full-scale adoption of HPB for HBM5.

SK Group Chairman Chey Tae-won, left, shakes hands with TSMC Chairman Wei Che-chia in Taipei, Taiwan, Wednesday. Courtesy of SK hynix

SK hynix also recently revealed iHBM, a new technology that incorporates an Integrated Cooling Engine (ICE) into HBM to create a dedicated heat-transfer path. The silicon-based structure allows heat to escape through die-to-die physical layers, effectively serving as a thermal chimney within the memory stack.

The company said the technology reduces thermal resistance by more than 30 percent compared with conventional designs, enabling stable operation even under high-temperature, high-workload conditions.

SK hynix plans to apply iHBM to its HBM5 and subsequent products targeting high-performance computing and AI data center applications. The company added that the technology is based on packaging processes already proven in mass production, allowing customers to adopt it without significant design changes.

In line with this, SK hynix is strengthening its partnership with TSMC, which manufactures the base die used in its HBM4 products.

SK hynix said Thursday that SK Group Chairman Chey Tae-won met with TSMC Chairman Wei Che-chia in Taiwan a day earlier, where they agreed to deepen cooperation across a wide range of areas, including next-generation HBM development and advanced packaging technologies.

SK hynix said the two companies plan to step up efforts to take an early lead in the emerging market for custom AI memory designed to meet the diverse requirements of global tech companies.

“As memory makers adopt more aggressive thermal solutions such as HPB and ICE, closer integration across the entire development process will become increasingly important,” an industry official said. “How effectively foundries and memory makers work together will likely become a key competitive factor.”