Quiet Breakout: The AI Chip Race Just Moved Inside the Memory, and Nvidia Isn’t In It

At the AI Infra Summit in Santa Clara last September, SK Hynix wheeled out a Supermicro server. Two Nvidia H100 GPUs sat where you’d expect them. Beside the GPUs, in the same chassis, were four PCIe cards Nvidia did not make: the Korean memory company’s AiMX accelerators, running Llama models off DRAM dies that have multiply-and-accumulate circuits soldered into the same silicon as the cells holding the weights. The H100s were there for the heavy linear algebra. The AiMX cards were there for everything else — the long token generations, the attention heads, the parts of inference that have been quietly turning into memory problems instead of compute problems.

That demo is the surface. The patent record is what’s underneath. And the patent record says that the next phase of AI hardware has been routed around Nvidia entirely.

Where the new AI silicon is being filed

We pulled every US grant whose title contains “in-memory computing,” “compute-in-memory,” “processing-in-memory,” or close variants, from 2018 through April 2026. There are 367 of them. The annual count went from 5 in 2018 to 86 in 2025, with 26 already issued in the first sixteen weeks of 2026. This is a category that, in the patent record, did not exist a decade ago.

The roster of who has been filing them is the part you do not hear in earnings calls.

Counts pool variant assignee spellings.

Now look at who is missing. Nvidia: zero compute-in-memory grants in that window. Google: zero. Amazon, Microsoft, Meta, Tesla: zero. Cerebras, Groq, SambaNova, Mythic: zero. The companies that the financial press treats as the AI hardware story have, as a group, zero issued US patents on doing arithmetic inside memory cells. Four of the top six filers are memory makers. The fifth and sixth (AMD, Qualcomm) sell processors that depend on that memory.

This is not a vocabulary trick. Read the actual claims and the inventions converge. SK Hynix’s PIM patents (US 11,237,799 in 2022 through US 12,417,076 in September 2025) describe MAC operators sitting inside DRAM banks, with the multiplications happening on data that never crosses the memory interface. Micron’s US 12,131,793 covers triple-modular-redundancy programming for MAC values stored in 3D self-selecting memory. d-Matrix’s US 12,353,985 wraps the same idea in a chiplet — a digital in-memory compute device on TSMC 6nm, slotted into a PCIe card, optimized for the attention computations inside transformer models. AMD’s recent grants are in a different layer entirely: scheduling MAC threads, cache-coherence directories, sparse-matrix predicates for instructions that execute in memory rather than in a core. AMD is writing the operating-system glue for chips it does not yet manufacture. The mechanism is the same across all of them: keep the weights where they live, and bring the multiplier to the cell.

Why memory makers, of all people

The reason this is happening in DRAM and SRAM, not in GPUs, is that the bottleneck has moved. Over the last decade, AI compute has scaled roughly 80x. Memory bandwidth has scaled roughly 17x. TrendForce calls the gap the “memory wall,” reviving a term Bill Wulf and Sally McKee used in 1995. For training, you can paper over it by adding more HBM stacks; for inference, especially the long-context, low-batch generation that running LLMs in production actually looks like, you cannot. The GPU sits idle waiting for weights. Every byte of attention key-value cache that has to walk across the bus is energy spent on transit instead of arithmetic.

That is the structural reason HBM has become the most contested supply chain in semiconductors. SK Hynix sold out HBM4 capacity through 2026, Samsung and Micron are right behind, and Nvidia’s Rubin Ultra roadmap calls for 1 TB of HBM4E per accelerator — a thirteenfold increase over the A100 in five years. But the more bandwidth you pile next to the compute, the more obvious it becomes that the next compression is to put compute into the bandwidth. SK Hynix’s December 2025 vision deck for SK AI Summit phrased it as moving from “AI memory provider” to “AI memory creator.” In patent-filing terms, that transition has been underway since early 2022.

The academic spine

The trick is older than the gold rush. The most-cited paper in our literature index, IBM’s 2020 Nature Communications report on phase-change memory inference (420 citations), proved you could hit ResNet-level accuracy on memristor crossbars whose conductance encoded the weights. The citations branch out from there: XNOR-SRAM (387 cites), CONV-SRAM (382 cites), C3SRAM, the 64-tile charge-domain compute macro, all from 2018–2020, almost all SRAM-based, a quiet majority of them coming out of one Princeton lab.

That lab is Naveen Verma’s. Verma has been a Princeton ECE professor since 2009; the Princeton Alumni Weekly profile last year called him the engineer who has spent fifteen years asking what would happen if you stopped moving data around and started moving the multipliers instead. In 2022 he co-founded EnCharge AI with former IBM Fellow Kailash Gopalakrishnan. The company has raised $144 million in venture money and an additional $18.6 million from DARPA’s OPTIMA program — the Pentagon’s name for “Optimum Processing Technology Inside Memory Arrays,” which is roughly as specific as DARPA program names get. EnCharge’s analog SRAM chip is an academic line drawn straight to a deployable product.

The other startup line is digital. d-Matrix was founded in 2019 by Sid Sheth and Sudeep Bhoja, two networking veterans from Inphi and Broadcom who decided the next bandwidth bottleneck would not be in the cable but in the memory die. In November 2025 the company closed a $275 million Series C at a $2 billion valuation. Its Corsair card claims 30,000 tokens per second on a 70-billion-parameter Llama model at 2 milliseconds per token, which, if it holds in production, is a 10x throughput improvement over a comparable GPU rack at a fraction of the power.

What this means for the next two years

Three things follow.

First, the AI hardware capex map is wrong if it assumes Nvidia keeps capturing the inference layer at the margins it captures training. The companies most likely to ship the silicon for cheap, ubiquitous LLM inference — phones, automobiles, edge boxes, agentic web traffic running 24/7 — are the ones who already control the memory die. SK Hynix is one of them. Samsung is another. Micron is a third. AMD has positioned itself to be the fourth by writing the cache-coherence and scheduling layer for chips it can co-design.

Second, the M&A logic gets weird. d-Matrix at a $2 billion valuation is a digital-in-memory chiplet startup with no fab. Acquiring it is more useful to a company that owns memory wafers than to one that owns logic wafers. EnCharge AI, with DARPA money and a Princeton lineage, is a similar shape. Both are likelier to be bought by a Hynix or a Samsung than by a Nvidia or an AMD.

Third, the moat that everyone is paying $4 trillion for — Nvidia’s CUDA software ecosystem — is being engineered around. The chips coming out of the memory-side patents do not run CUDA kernels. They run dataflow graphs in the memory array itself, with whatever software stack the memory vendor ships. AMD’s PIM scheduler patents are the bridge layer. If that bridge gets built before Nvidia files its own first compute-in-memory patent, the next AI infrastructure cycle will look unfamiliar — running on hardware that, in our patent index at least, the company that defined this one has not yet started inventing.

Method note. Patent counts come from a search of US utility grants in our local mirror of USPTO weekly grant XML, restricted to titles matching “in-memory computing,” “compute-in-memory,” “processing-in-memory,” “processing in memory,” or “in-memory processing,” with publication dates between January 2018 and April 21, 2026 (the most recent grant week in the index at time of writing). Assignee counts pool variant spellings (e.g., SK Hynix Inc. and SK hynix Inc.) and subsidiary filings under the parent name. The company list above includes all assignees with five or more grants; “zero” claims for Nvidia, Google, Amazon, Microsoft, Meta, Tesla, Cerebras, Groq, SambaNova, and Mythic were checked individually. Citation counts and publication years for academic prior art come from OpenAlex. Funding figures and product specs are from company announcements (SK Hynix newsroom, d-Matrix press releases, Princeton Engineering, Princeton Alumni Weekly) and from TrendForce’s HBM market estimates. None of this is investment advice.