Request an invitation ↗
Journal

/

AI infrastructure

HBM memory and advanced packaging: the supply chain behind AI chips

What HBM and CoWoS packaging are, why AI chip supply now hinges on memory, and what SK hynix, Micron and Samsung report about capacity into 2027.

November 20, 2026

8 min read

The Journal

In short

  • HBM, high-bandwidth memory, is DRAM stacked in towers and placed next to the processor in the same package. US export regulators describe it as found in almost all advanced chips bound for AI data centres, and NVIDIA's Rubin GPU carries up to 288 GB of HBM4.
  • The stacks only work once they are packaged with the processor, a step called advanced packaging. TSMC, whose version is called CoWoS, says demand for it has grown even greater since generative AI arrived in late 2022.
  • In their latest reports, all three memory makers say demand is running ahead of supply. SK hynix expects that to last through 2030, and Micron says it has no line of sight to when supply and demand will balance.
  • Since December 2024, US export rules have controlled HBM as a product in its own right. BIS noted then that all HBM stacks in production exceeded the new threshold.

What HBM is

A processor can only work as fast as data reaches it. The US Commerce Department's Bureau of Industry and Security (BIS) describes HBM as "found in almost all advanced computing ICs destined for advanced AI data centers", and put the reason plainly in its December 2024 rule: in advanced AI, "advanced logic chips must be paired with advanced memory to avoid this memory bottleneck."

BIS also explains why ordinary memory will not do: "HBM units are optimized for very high memory bandwidth, unlike general consumer-grade dynamic random access memory (DRAM) chips". High-bandwidth memory takes ordinary DRAM dies, stacks them on top of one another on a base die, and connects the stack to the processor through a very wide interface. JEDEC, the body that sets memory standards, published HBM4 in April 2025 as JESD270-4. Its figures show the approach: a 2,048-bit interface at up to 8 Gb/s, for up to 2 TB/s from a single stack, 32 independent channels where HBM3 had 16, and stacks from 4 to 16 dies high, holding up to 64 GB.

Announcing in September 2025 that it had readied HBM4 for mass production, SK hynix reported speeds above 10 Gbps against the standard's 8 and power efficiency more than 40% better than the previous generation.

NVIDIA's Blackwell Ultra GPU, described in August 2025, holds up to 288 GB of HBM3E at 8 TB/s. Its Rubin GPU, described in July 2026, holds up to 288 GB of HBM4 in 12-high stacks at up to 22 TB/s, 2.8 times the bandwidth of Blackwell. In NVIDIA's words, "High-capacity, high-bandwidth HBM4 is critical for hosting multitrillion-parameter models, extending context length without KV cache offload". For a comparison of the processors themselves, see the Journal's piece on GPU vs TPU.

Why packaging matters

An HBM stack cannot be fitted to a board like a memory module. It has to sit beside the processor on a shared base, wired to it through thousands of fine connections. That step is advanced packaging. TSMC's version is called CoWoS, short for Chip on Wafer on Substrate.

In TSMC's description, CoWoS integrates "multiple SoCs and high-bandwidth memory stacks" on an interposer that carries the wiring between them. In the CoWoS-S version it is a slab of silicon, which can be up to 3.3 times the size of a lithography reticle, about 2,700 square millimetres. A newer variant, CoWoS-L, has been in volume production at 3.5 reticles since 2024, for chips that need more room for processors and memory.

TSMC has run CoWoS in volume since 2012. "With the advent of generative AI in late 2022, market demand for" the platform "has become even greater," TSMC writes, and it "continues to expand its advanced packaging capacity, helping customers accelerate innovation to meet rapidly changing market challenges."

Adding supply therefore means building DRAM fabs, HBM stacking lines and packaging plants, and Micron is building packaging of its own. Micron's Singapore HBM advanced packaging facility is due to start output in early calendar 2027, and its DRAM fab expansion in Japan in late calendar 2028.

What the memory makers report

SK hynix, Micron and Samsung published their latest quarterly reports between late September and late October 2026.

SK hynix reported its third quarter of 2026 on 28 October. Quarterly revenue passed 100 trillion won for the first time. HBM4 shipments "began in earnest" during the quarter, and the company expects HBM4 to overtake HBM3E in both shipment volume and revenue in the fourth quarter. It has "signed LTAs with 14 major customers, including leading global cloud service providers (CSPs)". Those long-term agreements now cover a volume that "corresponds to more than 50% of total production capacity for 2027 and 2028." The company points to the "technical difficulty of leading-edge processes and the long lead time required to build new production facilities", and "expects the structure in which demand exceeds supply to persist through 2030."

Micron published its prepared remarks for fiscal Q4 2026 on 30 September. It has completed agreements for "the vast majority of our calendar 2027 HBM bit supply with significant price increases year over year". It expects industry DRAM bit shipments to grow approximately in the low-20s percentage range in both 2027 and 2028, with the industry supply constrained in both years. "We expect industry HBM bit shipments to grow faster than conventional DRAM through calendar 2028," it adds. On timing, it is blunt: "we do not have line of sight to when supply and demand will return to balance." A year earlier, in September 2025, it still expected "to conclude agreements to sell out the remainder of our total HBM calendar 2026 supply in the coming months."

Samsung Electronics reported its third quarter on 29 October. Its memory business set a record "amid capacity constraints" and "significantly scaled up sales of HBM4". For 2027, "the Business expects AI infrastructure capital expenditure to reach a historic level as the AI ecosystem expands. Additionally, increasing AI-driven demand will far exceed the industry supply in comparison with 2026." Samsung's foundry business also named HBM base-die sales as a driver of its improved results. The base die is the logic die on which each stack is built.

None of the three reports gives a date when supply will catch up, and SK hynix and Micron describe contracts that commit output years ahead.

Why ordinary memory feels it

HBM is made from DRAM. BIS's rule defines it to include DRAM integrated circuits above its bandwidth-density threshold, whether or not they follow the JEDEC standard, and a single HBM4 stack can hold up to 16 dies. The same companies make HBM and conventional DRAM.

Micron's remarks connect the two directly. It describes a "structural gap between DRAM supply and demand growth rates", says "cleanroom additions are required to augment node-transition supply growth and help narrow the gap", and expects supply-demand conditions across memory and storage to be "much tighter in calendar 2027 and 2028 than they were in 2026." Samsung reports higher market prices "across all applications".

Export controls treat HBM as strategic

On 2 December 2024, BIS added HBM to the Commerce Control List under a new paragraph, ECCN 3A090.c, saying that "HBM is critical to both AI training and inference at scale". The rule controls HBM with a memory bandwidth density above 2 GB per second per square millimetre and notes that all HBM stacks then in production exceeded that threshold. A narrow licence exception covers only HBM below 3.3 GB/s per square millimetre, under strict conditions.

A January 2026 rule set case-by-case licence review for exports of some accelerators to China or Macau, partly defined by a total DRAM bandwidth below 6,500 GB/s, giving the NVIDIA H200 and AMD MI325X as examples. A rule on remote access to advanced semiconductors, published on 19 November 2026, restates the same 2 GB per second per square millimetre definition for HBM.

Questions for a board or a buyer

  • What does our AI roadmap assume about memory? A plan built around specific accelerators assumes their HBM and packaging arrive on time.
  • Who holds our allocation? If the cloud provider or server maker holds the long-term contract, ask how capacity is shared when supply is short.
  • How exposed are ordinary IT budgets? Samsung reports that strong server demand raised market prices across all applications, so servers, PCs and storage are affected too.
  • Which rules apply where we build? For multinationals, HBM controls shape where accelerators can be shipped and used, a theme close to sovereign AI.

FAQ

What is HBM memory?

HBM, or high-bandwidth memory, is DRAM built as a stack of dies on a base die and placed beside a processor in the same package, joined to it through a very wide interface. The JEDEC HBM4 standard specifies a 2,048-bit interface and up to 2 TB/s per stack. It is used in AI accelerators and high-performance computing, where ordinary memory modules cannot feed the processor fast enough.

Who makes HBM memory?

SK hynix, Samsung Electronics and Micron all report HBM sales in their quarterly results, and all three reported growing HBM or HBM4 sales in their latest quarter. TSMC does not make the memory, but its CoWoS packaging joins HBM stacks to processors, and it is the partner Micron has named for manufacturing its HBM4E base logic die.

What is HBM memory used for?

Mainly AI accelerators and supercomputers. A recent data centre GPU design such as NVIDIA's Rubin carries up to 288 GB of HBM4, which holds model weights and working data close to the processor. JEDEC's HBM4 announcement also lists high-performance computing, high-end graphics cards and servers.

What is CoWoS?

CoWoS, short for Chip on Wafer on Substrate, is TSMC's 2.5D packaging technology. It places one or more processor dies and several HBM stacks on a shared interposer that carries the wiring between them. TSMC has produced it since 2012 and offers three variants: CoWoS-S, CoWoS-R and CoWoS-L.

Are there export controls on HBM?

Yes. Since December 2024, US rules have controlled HBM with a memory bandwidth density above 2 GB per second per square millimetre as ECCN 3A090.c. BIS noted that all HBM stacks then in production exceeded that level, and its November 2026 rule on remote access restates the same definition.

AI infrastructure on the Davos programme

Panel 3 of the Davos Super Intelligence Summit 2027, held on 19 January 2027 at the Grandhotel Belvedere in Davos, covers AI infrastructure: AI chips and compute, data centres, energy and scaling, and sovereign AI infrastructure. The programme sets out all four panels.

Sources

An evening in Davos, by invitation

  • Date19 January 2027
  • Time16:00 – 20:00 CET
  • VenueGrandhotel Belvedere