Samsung Put the Math Inside the Memory. It Claims an Eightfold Bandwidth Jump.

A conventional LPDDR5X-9600 package moves data to a processor at 76.8 GB/s. On stage at Hot Chips 2026 at Stanford, Samsung showed a package that looks identical from the outside — same 561-ball JEDEC footprint, same 16 GB, same low-power DRAM standard — and claimed 614 GB/s. The trick is that most of that data never leaves the chip. Samsung put sixteen small multiply-accumulate engines inside the memory, one per DRAM bank, so a chunk of the arithmetic in AI inference happens where the numbers already live.

LPDDR5X-PIM, detailed publicly for the first time in architectural depth on August 25, is Samsung's attempt to make processing-in-memory a product rather than a conference demo. The company ran Meta's Llama 3.1 8B on its own edge AI accelerator SoC twice, once with ordinary LPDDR5X and once with the PIM part. The conventional run took 12.3 seconds and produced 27.0 tokens per second. The PIM run took 5.4 seconds and produced 81.3 tokens per second — a 2.28x cut in model run time and a 3.01x gain in throughput. The test used a 320-token context, SINT8 activations, SINT4 weights and SINT32 output.

How the logic gets inside the DRAM

Each of the chip's sixteen banks gets its own PIM block: a scale register file and source register file feeding parallel MAC trees, with an ALU handling both integer and floating-point paths and results landing in a 1-kbit vector register file. That is a meaningful advance on Samsung's HBM-PIM generation, where fitting the logic meant sacrificing banks. Multi-precision support is new too — fifteen selectable combinations via the MAC precision fields in the configuration register, reaching 2.4 TOPS with SINT4 weights or roughly 1.2 TFLOPS per package in FP8.

The harder problem was never the math. It was the memory controller. PIM parts have historically demanded bespoke host silicon, which is why they stayed in labs. Samsung's answer is Address Align Mode, which maps DRAM addresses directly onto MAC instructions, deriving vector- and source-register addresses from row and column addresses rather than an instruction register file. The chip flips between single-bank mode, where it behaves like plain DRAM, and multi-bank PIM mode, with the host issuing PIMX_RD and PIMX_WR commands in place of standard reads and writes. Samsung says the mode switch is faster and more reliable than the HBM-PIM approach it replaces. In practice it means a drop-in part on a conventional controller.

Power is the obvious objection to putting compute inside a chip named for being low-power. Samsung's presenter conceded peak draw will be "much higher" because PIM current is bursty, but argued that eliminating round trips to the host nets out favorably. "We're not having significant power increase," Samsung principal engineer Karam Hwang said. One attendee also pressed on a detail visible in the benchmark slide: the PIM and non-PIM runs produced different output text. Samsung said accuracy optimization is ongoing and expects the speedup to hold — an admission that this is validated silicon, not a finished product.

Why It Matters

The economics here are more interesting than the architecture. Samsung opened its talk with a cost chart: memory's share of AI accelerator component spending climbed from 52 percent in Q1 2024 to 63 percent in Q4 2025, and HBM is nearly all of it. Nvidia has warned major customers of roughly 15 percent price increases on AI servers, attributed largely to memory. Micron told Hot Chips a day earlier that the HBM wafer penalty widens with every generation. Korean coverage has taken to calling it chipflation.

LPDDR5X-PIM does not touch HBM's bandwidth — 614 GB/s is a rounding error next to an HBM4 stack. But that comparison misreads the target. Decode-phase LLM inference is dominated by GEMV, matrix-vector work with almost no arithmetic reuse, where a GPU sits idle waiting on memory. That is exactly the operation PIM accelerates, and exactly the workload spreading to phones, AI PCs and edge boxes where an HBM stack is unaffordable and thermally impossible. Nvidia's Vera CPU already uses LPDDR5X via SOCAMM2; Intel's Crescent Island accelerator uses it too. PIM rides the form factor that is already winning below the data center.

The skepticism is earned. Samsung has been showing PIM for five years — Aquabolt-XL HBM2-PIM landed at Hot Chips 33 in 2021 with claims of 8.9x on GEMV microkernels and over 60 percent energy savings, then a 2.5x system gain on a Xilinx Alveo card — and none of it shipped in volume. Nothing about this design guarantees a different ending. "This one feels like one where a large customer is going to have to come in and say this is needed, implement it, and then it will get widely adopted," wrote ServeTheHome's Patrick Kennedy, who added the structural worry: handing a memory vendor control of the compute side gives it leverage buyers may not want, and "you need standardization at least so a vendor can second source."

That is the thing to watch. Samsung says it hopes for an initial JEDEC LP6-PIM specification this year, and a standardized LPDDR6-PIM would turn a proprietary part into a second-sourceable one — which matters because SK hynix and Micron are both reported to be targeting LPDDR6 for their own PIM entries. SK hynix already has GDDR6-AiM and the AiMX accelerator card, shown again at CES 2026, and its published AiMX-xPU work paired three AiMX boards with an H100 to hit 900 tokens per second on OPT-30B at batch 32. Samsung ships a simulator and SDK today, with a datasheet available on request. The chip won Best of Show at FMS 2026 in July. What it has not won is a named customer.