2nd Order Thinkers
2nd Order Thinkers.
12 mins why memory is going to make or break AI
0:00
-17:47

12 mins why memory is going to make or break AI

I've read the Micro's earnings report so you don't have to. In this, you'll learn why AI companies are suddenly fighting for memory.

Micron

As of this writing, the stock has nearly tripled since January.

Everyone is talking about Micron: YouTube, news, your neighbor, colleague… in particular, how you’ve missed out big time if you haven’t bought the stock!

But that’s not why I’m here and not why you’re here either.

I’ve read their and others' earnings calls in the last few days, and in this, I’m going to explain to you…

  • Why does it suddenly get so much attention?

  • Why does the stock price boom?

  • The RAM shortage and why does AI make this worse?

  • How! How this memory party is not a good sign for companies adopting AI, and even worse news for AI labs?

  • And finally, what to look out for?

Let’s start with who Micron is in 30 seconds


Who’s Micron (in 30 seconds)

Think of it as one of the companies building the plumbing behind AI. NVIDIA’s GPUs are where the action happens, but Micron’s memory is what moves the data and feeds it to the GPU. And one doesn’t work w/o the other.

And as AI shifts from training models to actually serving answers all day, memory becomes even more important.

So the market is realizing that if everyone needs more AI hardware, everyone also needs more DRAM, NAND, and HBM (all of which are memories).

They’ve just released their latest earnings report.

It turns out it’s not just their stock that’s popular, but their products as well. To an extent, 1/3 of their products over the next 5 years are booked, totaling $100 billion committed.

So now we know what Micron sells.

Next to ask iswhy does the industry suddenly treat memory like a scarce asset?


The Bottleneck

Historically, memory has become cheaper per bit. That was basically the whole industry model. DRAM is in anything with a processor running software, think your smartphones, laptops, gaming consoles, smart TVs, and cars.

For the last few years, the focus was on GPUs, so the story was to get more Nvidia GPUs, build more data centers, and with more demands, the token costs will keep falling.

But that version leaves out the boring part that suddenly matters a lot in 2026: a GPU is only useful if you can keep feeding it data.

So the more AI shifts from training models to serving answers all day, the more memory becomes the choke point. Every prompt, long context window, agent, or answer being generated token by token has to be held, moved, and pulled back fast enough for the GPU to use it, exactly via everything that companies like Micron, SK Hynix, and Samsung sell(the one that also sells phones).

NVIDIA’s product still practically does the math. But Micron is part of what keeps the data pipeline from drying up.

And this is where the story gets interesting, the dynamic.

Memory demand is exploding, but memory supply takes years to build up.

You can’t just “make more memory chips” with a snap of a finger, because there’s a long list to tick before production can start, such as fabs. Cleanrooms. Equipment. Power. Permits. Skilled workers.

Then you need years before those factories are producing usable chips at scale.

And it gets worse. Because each new generation is more complex than the last, manufacturing only gets harder.

Instead of following the pattern we have been used to for the last 60+ years, Micron is telling investors that the blended DRAM cost per bit is increasing by 60-80%.

This isn’t unique to Micron.

Its memory peers are pointing in the same direction. For instance, Samsung has warned that memory supply still falls short of customer demand, and that the gap could widen in 2027. SK Hynix has sent a similar message to investors, saying it expects demand to outstrip supply for at least the next few years.

Same as Micron, they said there’s no clear line of sight for when supply actually catches demand.

Since the demand is loud and clear and the supply only arrives years later, market forces do what they always do: they push the price up.

And AI makes this worse because it does not just need normal memory. It needs the expensive stuff: HBM. It’s high-bandwidth memory, a memory that’s complex and costly but is best suited to relieve the memory-GPU bandwidth bottleneck for AI chips. But making HBM eats more factory capacity than regular DRAM.

So the more the industry shifts capacity toward AI memory, the more pressure it puts on everything else.

Natually, it hits servers, phones, PCs, cars, industrial systems, basically anything that has a processor.

And you're in these industries, you really don't want to get caught without enough memory, just look at Apple, they've been paying up just to lock up DRAM supply, and Asus has basically been stockpiling chips like it's a bank run.

That is why this earnings report matters: it shows how keen Micron’s clients are to sign supply contracts with them.

Because no customers sign multi-year agreements out of generosity, only they are more scared of a shortage than of overpaying.

And the contract details I’m about to reveal are even stronger proof of how eager they are to reserve supply.

So exactly how unusual are the commitments?


The Unfair Terms

The stock price boom is not just due to Micron’s last quarter’s earnings, but also to the future cash flows implied by customers locking in scarce supply years ahead.

Because every major player now knows, memory is the bottleneck.

In the report, Micron said it signed 16 strategic customer agreements (SCAs) with customers across data center, consumer, and automotive. And 14 of those have a total of $100 billion of revenue at the minimum contract price.

Minimum.

I emphasized it because this $100 billion is the worst-case scenario, based on the minimum volumes and prices in the contracts. It can only be improved from there.

There are four details in the paperwork; once I point them out, you’ll see how obvious this is and how desperate their customers are in the memory supply:

Detail #1 The Deposit

Micron expects around $22 billion in deposits and related financial commitments. So, rather than a handshake and a ceremony, the customers are putting money behind it.

Detail #2 Take-or-pay

And the structure matters: as I’ve explained in the SpaceX and Cursor article, the take-or-pay contract is the exact same type of contract signed by Micron’s customers.

In short, their customers have committed to buying a certain amount, and if they didn’t end up taking it, they still have to pay.

Why would a customer agree to that? It’s all for peace of mind; they are buying the right to know they will actually get memory when they need it.

Detail #3 The floor pricing

There’s a ceiling price that protects the buyer. It says that if memory becomes even more expensive, my price cannot go above this level. Equally, there’s a floor price that protects Micron. It says: if memory prices collapse later, you still pay at least this much.

So the buyer gives up the upside if prices fall, because they are more scared of what happens if prices rise, or if supply simply is not there. While Micron gets guaranteed cash flows earlier, which they will need to fund some of the production and expansion.

Detail #4 Locked in at a tight-market price

You’d thought, ok, maybe the customers are just buying in at a low point?

In reality, Micron just reported DRAM prices up in the 60% range quarter over quarter and NAND prices up 80%.

So this does not look like customers locking in a bargain deal. The contracts are built around today’s tight market.

That tells you how they see the market. They are thinking, “Memory is expensive, and it might get worse, and if we don’t reserve supply now, someone else will.

So far, it sounds great for Micron.

However, is it also unicorns and rainbows to everyone downstream?


The risk is downstream, the return is questionable

Micron is great if you own Micron. It’s probably fine if you own Nvidia. The demand for hardware is still there.

Most people assume inference only gets cheaper over time, and have built businesses or adoptions around this theory. But if memory stays scarce or doesn't get cheaper fast enough, that assumption falls apart.

This is the pass-through problem. So companies building products on top of that memory have to choose among absorbing the cost, raising prices, or shipping less.

Three downstream categories have seen or will soon run into issues:

First stop: The warning from the chip manufacturer

AMD, which specializes in GPUs, has warned that higher memory and component costs are affecting its clients; Apple has also raised the prices of its devices, citing memory costs as the reason.

AI labs face the same issue, just at a much larger and more strategic level.

Second stop: The margin problem for the AI labs

The frontier labs are still trying to turn usage into durable revenue while carrying enormous infrastructure costs. If memory keeps the cost base elevated, the labs have less room to subsidize customers, while putting more pressure to prove that each token sold has a decent margin.

So again, the “memory stocks are going up” is too simple, and I’m glad you’re here for substance.

Share this with someone who’d also appreciate this analysis.

Share

Once the labs have less room to absorb costs, the pressure shifts to customers.

Third stop: The ROI problem for the enterprise AI users

A lot of AI budgets have one hidden assumption: inference gets cheaper over time.

Maybe not quarterly, but eventually.

The model gets better, the hardware gets better, a task gets more efficient, and the same workflow costs less next year than it costs today, as many AI leaders (like Nadella) have been citing Jevons paradox.

However, this assumption isn’t true today in 2026 and will not be in 2028, especially since many AI use cases are still borderline.

These use cases work in a demo, a pilot, and when the vendor is subsidizing the price, because everyone wants adoption. But at full scale, with thousands of requests, real latency requirements, and real infrastructure costs, the ROI goes negative.

I don’t mean to say that token prices can never fall.

Software efficiency can still help. Smaller models. Caching. Better routing between models. And of course, once the memory supply pipeline is resolved, it will certainly help as well.

But it does mean the token cost decline is not automatic.

If you built your AI roadmap assuming inference costs would keep falling, you should worry about warnings from memory manufacturers that hardware may not deliver that discount you’ve been hoping for on schedule.

The first things to break will not be the obvious use cases. Customer support, coding tools, search, ads, basically anything with some sort of labor savings or revenue impact can still survive higher costs.

I see the following at immediate risk:

  • The weaker use cases

  • The agent that saves ten minutes but runs all day.

  • The internal chatbot is just for revising your emails and drafting your reports.

Basically, the workflow has no clear payback period, even under optimistic token pricing.

If inference costs remain higher for longer, those projects will be delayed or narrowed, and chief product officers will have to be much more careful about what to commit to.

Again, I am not saying this will cause AI demand to vanish. Demand can be real even if it doesn't clear the ROI hurdle.


What to look out for next

So we’ve answered the first three questions: why Micron matters, why the stock moved, and why some AI leaders may already understand the trap they’re in.

The last question is: where does the AI industry go next?

I’ll watch these four things.

One: memory supply.

If Micron, SK Hynix, and Samsung can add enough capacity by 2028, the pressure can ease. If they cannot, expensive memory becomes a longer-running tax on AI infrastructure.

And that matters again because of the ‘inference will get cheaper over time’ assumption and the few paths for AI companies to become profitable.

Two: token pricing and lab margins.

If OpenAI, Anthropic, Google, and the rest are trying to keep prices in an affordable range while infrastructure costs rise, then someone is absorbing the pain.

Maybe software efficiency offsets it. Maybe smaller models, caching, routing, and MoE keep squeezing more out of the same hardware.

But if none of it keeps pace with the price hike, margins get squeezed.

That is the trap for frontier labs. They need better models to stay ahead, but better models are expensive to train and serve. If customers also become more selective about ROI, the labs get pressure from both sides.

Three: substitution pressure across the stack.

When one input gets expensive, the market doesn’t just pay up forever.

It starts substituting.

That said, the ROI here is messy — every substitution comes with its own trade-offs. A few examples I’ve seen:

  • Labs change how inference is run. Batch more requests. Shorten context where possible. Keep repeated answers in cache (for example, the AI video summary feature on YouTube). Push some workloads closer to the user.

  • Open weights reduce dependence on OpenAI or Anthropic, but they still need hardware.

  • Local AI avoids some cloud inference costs, but it requires more memory on the device.

  • Edge AI helps with privacy and latency, but it also pushes memory demand into phones, PCs, cars, and industrial equipment.

  • Custom chips may reduce dependence on Nvidia over time, but they do not remove the need for HBM, DRAM, power, networking, and data center capacity.

Subscribe, so you don’t have to keep track of these evolutions, but I do this for you :-)

So the workaround market tells you how painful the bottleneck is and where they’re about to push it.

If you believe there is any company that isn’t worried about cost, they would just use the best model for everything; you’re a fool searching for the perpetual motion machine.

It’s against the natural law of economics.

And since the pressure is across the whole stack: models, memory, chips, cloud capacity, edge devices, so who gets access to scarce supply first will be in a much better position than their peers.

Which brings us to the next nature point to watch.

Four: who controls the bottleneck.

This may be the bigger strategic shift.

As I said, the trade-offs.

Open weights are useful, but they still need somewhere to run. Local AI is useful, but it does not solve enterprise scale w/o a huge cost and overhead. Edge AI helps with privacy and latency, but it also increases memory demand pressure on phones, PCs, cars, and other devices.

So the workload keeps finding its way back to infrastructure.

That gives more power to hyperscalers: Microsoft, Amazon, Google, Oracle, and the neoclouds underneath them. Increasingly, they’re looking for their own vertical solution.

Google has TPUs. Amazon has Trainium. Microsoft has Maia. OpenAI is now doing the same thing, working with Broadcom on custom inference silicon. Because renting the frontier stack forever is too expensive and too strategically important to leave entirely outside the company.

And that is why frontier labs are in the hardest seat: they don’t have a cash-cow revenue stream, while the ecosystem demands they spend on building their own stack.


So my final takeaway is simple.

Always look for where the bottleneck sits, who controls it, and where it’s shifting to next.

Here’s one open question to you: Do you think there’s a scenario in which models stop growing because the bottleneck gets too expensive to expand?

Discussion about this episode

User's avatar

Ready for more?