Intel's 256-core Xeon 7 CPUs are a Diamond in the rough

DEEP DIVE Intel's Xeon 7 platform, better known by its codename Diamond Rapids, was supposed to launch this year with up to 192 cores backed by a whopping 16 memory channels.

But that's not going to happen. This being Intel, the launch was, to no one's great surprise, delayed until 2027 to give what remains of the company's engineering team time to cut and polish the chip — and polish they certainly have.

At the Hot Chips show this week, Intel revealed the chip won't ship with 192 cores after all — but hold your disappointment because Chipzilla's flagship SKU will actually cram 256 performance cores into a single socket.

In fact, spec-for-spec, the platform is surprisingly well positioned to compete with AMD's sixth-gen Venice Epycs, which will also top out at 256 cores. Well, at least on the high end it will.

An eye on HPC

As we reported earlier this year, Diamond Rapids won't be a mainstream part. It lacks hyperthreading — what the rest of the industry calls simultaneous multithreading — entirely and it can only be offered in high-core-count SKUs aimed at HPC applications.

There will be no eight channel Diamond Rapids-SP parts. Those got canned earlier this year, which leaves us with just the high-end AP variants. These always are the cooler of the two anyway, but they're not what enterprises have historically bought.

What this means is Diamond Rapids — which for the sake of brevity we're going to refer to as DMR from here on out — won't compete with AMD in every regime, just the very top end.

Intel's next chip, known as Coral Rapids, will see the return of hyperthreading and presumably the next round of SP processors, but we wouldn't write off DMR just yet.

The chip is without a doubt Intel's most sophisticated Xeon ever built. In many ways it feels a bit like Intel's Rome moment. As you may recall, AMD's Epyc Rome saw the House of Zen embrace a truly modular chiplet architecture that disaggregated compute from I/O and memory, and could be added, removed, and reused depending on the kind of CPU it was trying to make.

After faffing about for three generations trying to make chiplets work for them, Intel appears to have found the right glue to stitch everything together and is now having its own Rome moment.

Intel ups its advanced packaging game

Cutline: Forget your glasses or squint hard enough and you'd think you were looking at an old Eypc circa 2019.

DMR's resemblance to Rome is hard to miss, but the resemblance is only skin deep. Intel really is using a different kind of glue to build its chips.

The chip comprises up to 22 chiplets, which can be added or subtracted depending on the ratio of compute, cache, and memory Intel needs — just like AMD has been doing for the past seven years.

Those chiplets can be broken into three core groups. There are up to 16 core compute dies built on Intel's bleeding-edge 18A-P process tech, four compute building block (CBB) base dies fabbed on Intel 3-T, and two fabric hub dies built on Intel 3 that tie everything together.

We've discussed 18A-P in the past so we won't rehash too much here, but it's a refined version of the 2 nm-class process tech Intel started churning out earlier this year with its Panther Lake mobile processors.

Intel claims the tech delivers the same performance at 18 percent less power, or up to 9 percent higher performance at the same power, and that's before taking into account microarchitectural improvements from DMR's all new performance cores.

Working our way down the stack, each core chiplet contains up to 16 cores and their respective L2 caches. Up to four of these core chiplets are stacked atop the CBB base die using its Foveros 3D direct hybrid bonding packaging tech. This base die is where all of the chip's L3 cache lives, with 320 MB per CBB, or 1.28 GB for the top-specced DMR SKUs.

While Intel is using 3D packaging to assemble its CBBs, it's not using its 2.5D  Embedded Multi-Die Interconnect Bridge (EMIB) packaging tech to connect the compute assemblies to the two I/O and memory dies or Scalable Fabric Hub (SFH) dies, as the x86 giant is calling them now.

Much like the I/O dies found on AMD's last few generations of Epyc datacenter CPUs, Intel's SFH dies combine the chip's memory controllers and I/O interfaces into a single piece of silicon. This die can then be duplicated for added connectivity or channels, which is exactly what we see with DMR.

Combined, the two SFH dies offer up to 16 channels of DDR5 at 8,000 MT/s, or 12,800 MT/s when using MRDIMMs, and 128 lanes of PCIe 6.0 / CXL 3 / UPI 3, along with an additional eight PCIe 4.0 lanes, presumably for ancillary I/O like onboard networking, USB, or baseboard controllers.

This is a pretty big departure for Intel, which has either crammed everything into one homogeneous die or disaggregated I/O into standalone chiplets while keeping memory on its compute assemblies.

For Xeon 7, Intel wanted to deliver a chip with uniform memory access (UMA) rather than the three non-UMA (NUMA) nodes that Granite Rapids-AP showed up as by default.

This is part of the reason why Intel skipped EMIB in favor of a more traditional UCIe-S type interconnect, where data is carried over the organic package rather than silicon bridges. EMIB would have required edge-to-edge connectivity between the compute and I/O and would have prevented the CBBs from talking directly to both SFH dies without additional hops.

In any case, Intel is calling this new chiplet architecture "fan-out-fabric" and it's what allows it to have UMA while also keeping packaging cost under control, something we strongly suspect may have become an issue with Intel's Xeon-6+ products, known as Clearwater Forest.

Questions remain

Beyond the physical makeup of the chip, there's a lot we still don't know about the platform. For instance, we have no idea how high the chip's cores will clock nor how much of an IPC gain we can expect from DMR's all-new performance cores relative to Granite. We also don't know anything about the cache hierarchy beyond the fact this thing is going to have a lot of L3.

We do know that the cores will feature Intel's updated AMX instructions, which add FP8 support for on-CPU machine learning workloads, and that it'll be one of the first parts with full support for AVX 10.2. We explored the latter in detail a few years back, but in a nutshell, the revised advanced vector extensions aim to address some of the shortfalls faced by Intel's original AVX-512 implementation. HPC and supercomputing enthusiasts, eat your heart out.

But on the topic of HPC, we still don't know how Intel plans to handle NUMA beyond the fact that UMA will probably be the default config. That's fine, but there are a lot of HPC workloads that benefit from carving up high-core count CPUs into smaller pieces. While UMA was obviously a major design consideration, we don't see any reason why one wouldn't be able to carve the chip up into two, four, or potentially more sub-NUMA clusters if you wanted to. As far as we can tell, Intel would just need to enable it in the BIOS.

Product segmentation is another open question. We already know to expect 256- and 192-core variants of the part, but we don't yet know how far down the stack Intel will go. Presumably Intel could cut the chip down to a single CBB base die with one 16-core chiplet if it thought someone would buy it. More realistically, we expect DMR to bottom out at somewhere between 64 and 128 cores.

Taking DMR to Venice

Without a full picture of what DMR will offer, it's tough to say how the part will stack up against the competition.

Having said that, the specs Intel has already disclosed suggest this is going to be Chipzilla's most competitive fight in years. Both AMD and Intel parts will offer up to 256 cores, 16 memory channels operating at the same speeds, and a gigabyte or more of L3 cache on board. Both parts will offer similar connectivity, while TDPs are expected to land around 600 watts for both parts.

As we mentioned before, the one thing Intel doesn't have is hyperthreading, a fact that could give AMD a double-digit percent advantage in certain thread-happy workloads regardless of which chip slinger ends up having the more potent cores.

Intel's saving grace is that this is an HPC-centric part and a lot of HPC apps — including the HPL benchmark used to rank the world's most powerful supercomputers – don't always benefit from hyperthreading.

In fact, Arm claims that simultaneous multithreading actually hurts more than it helps. If true, we could see Intel position DMR as yet another CPU for low-latency agentic sandboxes (containers where AI-generated Python code is executed or C and Rust are compiled and run).

But unless that happens, the reality is DMR probably won't be a high volume product for Intel. Certainly not with Nvidia pushing its in-house Vera CPUs as its preferred AI head node.

DMR's success ultimately is going to come down to price and value. When AMD launched Rome, it couldn't touch Intel on core-for-core performance. What it could do is deliver more cores, more I/O, and more memory for the dollar than anything in Intel's lineup, and that largely remained true up until AMD's 5th-gen Epycs forced Intel to concede it was no longer top dog and slash prices.

If Intel can deliver more cores for less than AMD with its Xeon 7 processors, for the right buyer, it may not matter who has the better performance. ®