Memory as a Service: Smart Money Is Already Quietly Betting. After Two Weeks of Deep Research, Here’s a Breakdown of CXL and the Key Players Breaking the Memory Wall

I’m based in Silicon Valley, leading an AI inference team at a major tech company. Our goal is to spot the next investment opportunity by following AI bottlenecks from a front-row Silicon Valley perspective. This article draws on conversations with core engineers from NVIDIA, Marvell, and Astera Labs Inc., to break down CXL, which has been hot in the Valley recently.

Where the bottleneck is, that’s where the money is. We’re in a critical period for witnessing the direction of storage. This article follows the previous one, continuing to break down an important path to breaking the memory wall—and the most controversial one: CXL. After reading this, you’ll have a clear picture.

Munger in US@Kay2289123 · Jul 2 This article took a lot of legwork, used up my connections, visited over a dozen companies in Silicon Valley working on optical modules, interconnects, and storage, had in-depth conversations with front-line managers and R&D engineers, and took about 4 weeks to write. I believe this is something you won’t see elsewhere.

I didn’t just talk—I also put real money down. Words are cheap, so here’s a screenshot.

Please note: don’t blindly follow my trades. Anyone impersonating me in the comments is a scammer. I’m on WhatsApp…Show moreQuoteMunger in US@Kay2289123 · Jun 30 Article: 50 hours spent, deep dive into HBM memory dominance — Silicon Valley’s 6 technical routes to solve the memory shortage. Don’t miss this one. Writing took two weeks, 50 hours, to bring you the most cutting-edge, real value investment opportunities. We visited NVIDIA, Micron, Intel, SNDK…25195867526K

In December 1994, William Wulf and Sally McKee of the University of Virginia wrote a paper only a few pages long, titled “Hitting the Memory Wall.” It was formally published in March 1995.

Back then, there was no ChatGPT, nor the kind of clusters today composed of many servers, often using tens of thousands of AI compute chips (GPUs). The two authors were worried about something more fundamental: processors were getting faster, but memory wasn’t keeping up at the same pace. In the long run, chips would spend a lot of time “waiting for data.” No matter how powerful the compute, the whole machine would run slowly.

They called this invisible wall the Memory Wall.

Thirty years later, the classic memory wall still exists: chips still stall waiting for data. AI has added a new wall next to it: the amount of data that needs to be stored keeps growing, and the fastest memory is both expensive and too small to hold it all.

Meta published “Vistara: Making CXL Real” at ISCA 2026 (top computer architecture conference). Meta’s server lifespan is 3–5 years, but memory lasts 7–10 years; about 40% of its fleet can’t add more memory—so they “salvage” retired DDR4, using CXL.

In my last article, I described several routes the industry is taking to bypass the memory wall. This one focuses on the most controversial of them: CXL.

In 2024, it was once judged “not suitable for AI.” Today, Meta, Microsoft, and a group of chip companies are putting real programs, products, and acquisition money on this path.

1. Don’t Memorize Abbreviations First: Put These Nouns in a Server

If these terms are new to you, first see where they fit in a machine. Think of a server as a high-performance computer in a data center. It has chips for compute and memory for temporary data storage.

CPU is a general-purpose processor that runs the OS, schedules tasks, and handles routine computing.

GPU was originally for graphics processing, but now excels at many parallel similar calculations, making it the main chip for training and running AI models.

DRAM is a memory technology. We often talk about DDR4, DDR5—these are different generations, like previous and next-gen versions of the same product.

Host memory doesn’t refer to a different material but to location and purpose: it usually denotes DDR/DRAM installed next to the CPU, used by the whole server. So, DRAM answers “what it is,” host memory answers “where it’s installed and who mainly uses it.”

HBM stands for High Bandwidth Memory. It’s also essentially DRAM, but packaged in a more compact, stacked way, placed right next to the GPU. It can transfer more data per second, so it’s fast and expensive. “Bandwidth” here can be simply understood as how much data can be moved per second.

PCIe is a common high-speed connection in servers. GPUs, network cards (that connect the server to the network), and SSDs (solid-state drives for long-term storage) can all exchange data with the CPU via PCIe.

CXL is not a new type of memory, but a set of communication rules built on top of PCIe. The CXL controller in the middle translates instructions on both sides, allowing the CPU to read/write external DRAM almost like local memory.

Fabric originally means “woven material.” Here, it refers to a network woven from many CXL connections and switches. Switches act like intersections in the data path; multiple servers connect to a pool of shared memory through them, so each machine doesn’t have to be pre-loaded with tons of memory.

Later, you’ll see link, interconnect, and dedicated channel. These all refer to paths for data transfer between devices. “Dedicated” means it mainly serves a certain type of device or task, e.g., NVLink mainly allows GPUs to exchange data at high speed; CXL solves how external memory connects to the server. They are not the same path.

Think of PCIe as an already laid road. CXL is the set of rules that allows “memory data” to travel on that road. CXL Fabric is the network formed by roads, intersections, and traffic management systems.

This diagram also has an important boundary: most publicly deployed CXL products currently expand host memory on the CPU side first; they don’t directly expand HBM next to the GPU.

From a speed and capacity standpoint, data in an AI server is often tiered: the most frequently read, latency-sensitive data goes to HBM; then local DRAM next to the CPU; data that doesn’t need to be read immediately can go to external DRAM accessed via CXL; rarely used data lands on SSD.

If you remember only one sentence, think of it this way: HBM mainly pursues speed, CXL mainly pursues capacity and cost savings. Not all data needs to occupy the most expensive, fastest position all the time.

2. The Memory Wall Is Turning into an Increasingly Expensive Bill

AI has two main phases. Training: feeding lots of data to the model so it learns patterns. Inference: after the model is trained, it starts answering user queries.

During training, GPUs need to exchange data continuously at high speed—speed is paramount. During inference, capacity becomes equally important.

To remember what was discussed earlier, the model saves a temporary note in memory called the KV cache. The longer the conversation, the more users served simultaneously, the larger this note. Its size depends on the model, context length, and number of concurrent requests; with many long conversations running, it’s not uncommon to take tens of GB. GB is a unit of capacity; 1 TB ≈ 1,000 GB.

The most frequently accessed content still needs to be in HBM. But content that has been seen, not frequently used in the short term, but might be recalled later, needs a larger, lower-cost location.

So, today’s AI faces two memory pressures: data not delivered fast enough, and insufficient capacity at high cost.

SemiAnalysis estimates that hyperscalers—cloud providers building massive data centers—spent about 8% of CapEx on memory in 2023–2024, which could approach 30% by 2026. CapEx here is the money they spend building data centers, buying servers, and chips.

These aren’t exact budgets disclosed on a comparable basis by each company, just directional references. But from 8% to 30%, it’s clear that memory has gone from a server component to a major expense item that management must track separately. You can see this from $MU $SNDK.

Another set of numbers comes from Microsoft Azure.

CPUs have many compute units that can run tasks simultaneously, called CPU cores. If a cloud server’s cores are first rented out by customers, the remaining memory sits idle and can’t be automatically moved to another server. Microsoft tracked 100 general-purpose production clusters over 75 days and found that in some high-utilization clusters, the proportion of DRAM trapped in individual servers exceeded 30%. The paper’s cost model also assumes DRAM accounts for about half of a server’s total cost if memory pooling is not used.

These two numbers can’t be directly added. The >30% is an idle ratio observed at a few outlier high-utilization points; 50% is a model assumption in the paper for cost calculations. Together, they illustrate a real problem: an expensive component can have a significant portion that can’t be used by other machines.

When memory was cheap, this waste was tolerable. When memory is both expensive and scarce, having a few servers share a pool of memory, and recycling old memory, makes economic sense.

3. Why CXL Was Once “Sentenced to Death”

In 2024, bearish views on CXL weren’t unreasonable. By 2026, you should take it seriously again.

At the time, the central task of AI infrastructure was training large models. Tens of thousands of GPUs had to continuously exchange data; any slowdown in any link could stall the whole group.

Buyers cared most about peak speed between GPUs. NVIDIA’s NVLink, a dedicated GPU interconnect, and data center networks like InfiniBand and high-speed Ethernet, were designed for that. CXL, on the other hand, brings external memory into the server—it’s not designed for peak GPU-to-GPU bandwidth.

In March 2024, SemiAnalysis wrote directly: “CXL Is Dead in the AI Era.” It argued at the time that chips have limited numbers of high-speed lanes and limited power budgets. If these resources are prioritized for faster GPU links like NVLink, less room remains for CXL.

More practically, the whole ecosystem wasn’t ready either. It’s not enough for the CPU to recognize CXL; the motherboard, boot firmware, OS, and CXL memory cards must all be compatible. Sellers were enthusiastic, but buyers actually building AI clusters were in no rush.

So, the real question at that time was: can CXL become the main data path between GPUs, between servers? The answer is still no. CXL’s job is to expand memory, not replace GPU high-speed links.

4. It’s Not a Sudden Comeback—It Finally Found Its Right Problem

CXL’s return to the spotlight is driven by three changes.

First Change: AI Moves from Centralized Training to Ongoing Inference

Training values speed more. Inference cares about speed, but also about how much memory each response uses, how many users can be served concurrently, and the cost per response.

CXL external memory is slower than HBM, but usually much faster than SSD, and the CPU can read/write it like memory. As long as software can distinguish which data is urgent and which can wait, later data can be offloaded to CXL memory.

Current public deployments mainly expand CPU host memory. Whether KV cache can be moved from HBM to CXL depends on how the GPU accesses this memory and whether software can automatically manage data placement. Plugging in a CXL card doesn’t automate this.

Second Change: Old Memory Becomes Worth Reusing

In publicly available data, Meta’s Vistara provides the most complete set of production deployment data.

Meta didn’t invent new memory. It salvaged DDR4 from retired servers and connected it via custom CXL chips to servers using newer DDR5. The OS then decides: frequently accessed data stays in local DDR5, infrequently accessed data goes to external DDR4.

According to Meta’s Vistara paper, about 40% of the company’s servers hit memory capacity limits first; in the full lifecycle model used, DRAM DIMMs account for 69% of server component carbon emissions; DIMMs typically last 7–10 years, while servers are refreshed about every 3–5 years.

This means when a server is retired, the memory inside often still has useful life.

Vistara has entered Meta’s production systems, used for AI inference, data processing, databases, and software compilation/testing. In some AI services that separate compute and memory deployment, it reduces the number of servers by up to 25%. In distributed caching scenarios, because more data can stay local instead of reading from other servers over the network, average access latency actually decreases by 29%.

To clarify: the paper says Vistara entered an infrastructure with millions of servers, not that Meta deployed millions of Vistara units.

The trade-off is also in the paper. In Meta’s configuration, CXL expanded memory bandwidth is about one-tenth of local memory, and access latency is about 60% higher (about 150 nanoseconds more). (Nanosecond: one billionth of a second; in chip world, that’s a noticeable gap.) This is not a fixed characteristic of all CXL products, but it means CXL memory can’t be used for just any data. Meta only puts non-time-sensitive data there, keeping hot data in local memory.

Vistara’s difficulty is that chip and software must be developed together. Software must continuously judge which data is hot and which is not, and place them appropriately. For unsuitable programs, the system simply turns off external memory.

Fortunately, if you follow recent software trends, you’ll notice many vendors are pushing toward KV cache offloading. I work in AI inference myself, and this is clear to me.

Third Change: Underlying Links Get Faster, Buyers Start Testing

In November 2025, Microsoft put M-series cloud servers using CXL memory into limited preview, open to select customers for testing. Astera Labs’ Leo CXL 2.0 controller can connect up to 2 TB of memory per chip, claiming to boost server memory capacity by over 1.5x.

This is not full commercial availability, but it’s moved from “listening to product briefings” to “letting customers run real programs on it.” The Azure & Astera limited preview shows CXL has moved from lab testing to customer validation, still some way from general availability.

In the same month, CXL 4.0 specification was released, doubling the maximum signal transfer rate per lane from 64 GT/s to 128 GT/s. The new spec also allows combining multiple ports and enhances memory error correction and fault handling. Standard upgrades don’t mean immediate revenue, but they signal that CPU vendors, cloud providers, and chip companies are still investing in compatibility.

5. What Kind of Business Will CXL First Become?

CXL’s vision is big, but commercialization will not happen overnight.

Direct Attach Expansion: Add Memory to a Single Server

Plug a CXL memory card into a server to bypass limited DIMM slots and increase capacity. Most publicly visible deployments and products are at this level.

Memory Pooling: Let Multiple Servers Share a Memory Pool

Multiple servers connect to external memory via a CXL switch. The switch is like a data path intersection; software decides which machine gets how much capacity and when.

Microsoft’s research suggests small-scale pooling can reduce a fair amount of DRAM waste; but the larger the pool, the more layers, latency, and hardware costs increase. Sharing is not a free lunch.

CXL Fabric: Extend Sharing Across More Racks

A rack is a cabinet holding multiple servers in a data center. When servers, switches, and memory pools across multiple racks form a larger CXL network, you enter the Fabric stage. Memory is no longer fixed to any single machine but can be allocated across a larger scope.

The safest judgment currently: direct attach expansion already has production deployments—Meta Vistara is a clear example; pooling is being validated; cross-rack Fabric remains a longer-term direction.

6. CXL Won’t Replace HBM, Nor NVLink

When people hear CXL is getting attention again, they immediately ask: Will HBM be replaced?

My answer: no, at least not in CXL’s current mission.

HBM sits next to the GPU, handling the most frequently read, latency-sensitive data. CXL currently is better suited for expanding CPU-side capacity, storing data less sensitive to latency. They are not the same product, nor in the same location.

If future platforms can reliably move some KV cache data that doesn’t need immediate access from HBM to slower, cheaper memory, then new inference workloads might not require HBM to grow proportionally with inference volume. But this is still speculation; no existing business data proves CXL will significantly dampen HBM demand. As total inference volume grows, they could easily grow together.

NVLink’s boundary is also clear. It handles high-speed data exchange between GPUs; CXL handles how external memory connects, expands, and is shared.

So I won’t ask “Can CXL beat HBM?” The more accurate question is: Which data must stay in HBM, and which data can be moved to cheaper memory layers?

7. In What Order Will Money Appear Along This Chain?

The closest to commercialization is the CXL controller, but vendors generally don’t disclose this revenue separately.

Astera Labs’ Leo, Marvell’s Structera X, Montage Technology’s MXC—all make this “translation chip.” Every expansion card needs one. Speed, stability, compression capabilities, and platform compatibility will differentiate products. Therefore, controllers have a better chance to retain profit than simply assembling memory modules.

Next layer is the CXL memory module. Samsung’s CMM-D, Micron’s CZ120, SK hynix’s CXL modules all combine controller and DRAM into a pluggable product. For memory manufacturers, this is a new DRAM shipping channel, but not automatically a high-margin new market.

When direct attach moves to pooling, CXL switches become important. They connect multiple servers to multiple memory pools, handling access ordering, fault tolerance, and isolation between tenants. This market is still early; validation cycles will be longer than single-server expansion.

Once hardware is in place, software is needed to decide where data should go. Put hot data in the wrong place, and memory cost savings may be quickly eaten by speed loss. Meta can design its own chips and modify Linux; most enterprises cannot, so they need tools for data tiering, memory allocation, and monitoring.

Further out is optical interconnect.

As CXL speed increases, electrical signals degrade over copper wires. Inside a server and within a rack, copper is still simple and cheap; if memory pools need to span multiple racks, distance, power, and cabling space will push systems toward optical fiber.

Public demonstrations have pushed such optical links from ~20m to 100m, but demo specs differed. They prove technology works, not that customers will buy at scale.

So, optical links for PCIe/CXL today remain at demonstration and engineering preparation stage, not mature orders.

8. Why I Single Out Marvell

Marvell hasn’t separately disclosed CXL revenue, but it has positioned itself in direct attach expansion, pooled switching, and long-term optical interconnect.

Marvell’s core product at the single-server expansion layer is the Structera X 2404.

This CXL 2.0 memory expansion controller supports up to 12 DDR4 DIMMs and includes hardware compression, shrinking some data on-chip to increase effective usable capacity from the same physical memory. It targets exactly the reuse of old DDR4: when servers retire, memory doesn’t have to exit the data center at the same time. Its value proposition: help customers buy less new DRAM.

For pooling, Marvell filled the switch gap via acquisition of XConn.

Marvell completed acquisition on February 10, 2026, with a deal value announced at ~$540 million. XConn already had a product in production; next-gen was in sampling (providing samples to customers for testing).

Marvell expects XConn to start contributing revenue in the third quarter of FY2027, reaching ~50M annualized revenue (extrapolated from quarterly run rate) in the fourth quarter, and ~100M in FY2028. “FY” follows Marvell’s fiscal calendar, not the same as calendar year. These are management expectations, not achieved revenue.

Marvell subsequently introduced the Structera S 30260, offering 260 high-speed lanes to connect more servers and memory devices, aiming at larger memory pools. As of this writing, the product is still planned for sampling in Q3 2026. Sampling, mass production, and recognizable revenue are three different things.

Farthest out is Celestial AI.

When the deal was announced in December 2025, Marvell gave an upfront consideration valued at ~$3.25 billion; up to another $2.25 billion could be paid if Celestial AI meets agreed revenue targets. Celestial AI uses light to transfer data between chips, currently first serving high-speed connections inside AI clusters; applying this technology to remote memory pools is still a future concept.

So, Celestial AI is not yet a mature “optical CXL product.” By acquiring it now, Marvell is buying the technical capability for cross-rack optical connectivity, waiting for future demand.

Marvell’s targets are equally aggressive: Celestial-related business reaching $500M annualized revenue in Q4 FY2028, $1B annualized in Q4 FY2029. These are company targets, not locked-in orders.

The three layers in the diagram are not at the same maturity: Structera X is a current product; XConn already has an in-production switch, Structera S still planned for sampling; Celestial AI is a farther-out optical capability.

Marvell is not the purest CXL play. Astera Labs is more focused on controllers; Rambus designs CXL interface IP and licenses it to other chip companies; memory makers sell more DRAM through modules. Marvell’s differentiator is having both controllers and switches, plus the optical capability from Celestial AI for the longer term.

Marvell’s Q1 FY2027 revenue was $2.418B, up 28% YoY; data center revenue was $1.833B, 76% of total. But “data center revenue” includes Ethernet, storage, general-purpose servers, and inter-datacenter connectivity—not all AI, and certainly not all CXL.

I’ll mainly watch whether Structera X can start generating revenue from single-server expansion, and whether XConn completes customer validation according to the company’s guidance.

9. The Next Bottleneck: Whether the Entire System Works Together

The easiest way to pitch CXL is “plug in a card, memory expands infinitely.” Reality isn’t that simple.

The CPU recognizing CXL only means hardware has an entry point. The motherboard, boot firmware, OS, and management tools must all cooperate. The applications running on top must be able to tolerate the extra latency.

When pooling, the system must decide how much memory each server gets, and ensure a machine failure doesn’t degrade others. When spanning racks, optical links must prove they can run reliably long-term.

So, from now on, I’m watching three things: Will Azure move CXL cloud servers from limited preview to general availability? Will switches go from sampling to customer procurement and revenue? Will optical links leave demo tables and run reliably in cross-rack environments?

Until those signals appear, I’ll split CXL into two parts. Single-server direct attach expansion has real deployments; memory pooling and cross-rack Fabric still await customer validation.

Its current use is specific: HBM continues to handle the hottest data next to the GPU; CXL brings larger, cheaper DRAM into the server. Controllers are already in market first; switches, software, and optical interconnects will need to prove themselves step by step as the sharing scope expands. Marvell’s positioning also bets on this order.

Two years ago, the market said CXL had no place in the AI era. Now the more appropriate assessment is: It won’t take over GPU high-speed interconnects, but it has found a role in expanding host memory. Next, orders and long-term usage data will tell us how big this role can ultimately become.

Undoubtedly, the next move in storage directly determines our investment opportunities. I’ll continue to break down this main thread, along with other ways to break the memory wall.

References and Further Reading

• Wulf & McKee: Hitting the Memory Wall

• Meta: Vistara, from chip, OS to hyperscale deployment

• Microsoft Azure: Design trade-offs for CXL memory pool

• Astera Labs: Azure M-series CXL memory limited preview

• CXL Consortium: CXL 4.0 specification

• Marvell: Structera X memory expansion controller

• Marvell: Completed acquisition of XConn

• Marvell: Structera S 30260 CXL switch

• Marvell: Acquisition of Celestial AI

Disclaimer: This article is personal industry research and tech popularization, not investment advice. Market estimates, company guidance, and product roadmaps come from publicly available information at the time of writing; all are subject to change.