Huawei's next-gen Ascend NPUs could become China's best option

At its annual Connect conference, Huawei unveiled a new generation of AI accelerators that promise performance far beyond anything Nvidia can currently sell in the Middle Kingdom. This could be a boon for Chinese developers used to purchasing lesser weapons from everyone's favorite AI arms dealer.

If that weren’t enough, the chip, codenamed the Ascend 960DT — the DT here apparently stands for decode/training — is slated to arrive a full three quarters ahead of schedule, launching in the first quarter of 2027.

With up to 288 GB of what we assume is Huawei’s custom HiZQ memory tech, a homegrown alternative to the high-bandwidth memory used by the rest of the world, and up to four petaFLOPS of FP4 performance (half that at FP8), the accelerator offers twice the performance and memory capacity of the company’s 950-series parts, launched earlier this year.

Compared to American GPUs, the 960DT offers similar memory and bandwidth to Nvidia’s B300 family of chips launched last year, but only about half the FP8 and a third the FP4 compute.

While a big step up for the Chinese chip designer, it still has a long way to go to catch up with Nvidia’s Rubin and AMD's recently launched MI455X which promise substantially higher compute and bandwidth. Rubin boasts between 35 and 50 petaFLOPS of FP4 performance, 288 GB of HBM4 memory, and 22 TB/s of bandwidth, putting it in an entirely different league.

Having said that, neither of those chips is available for sale in the Middle Kingdom, so it's not like Chinese model devs have better options. The best GPU Nvidia can sell in China offers near-identical dense floating-point performance, twice the memory, double the memory bandwidth, and support for much larger scale-up domains.

Scaling up

The performance gap may not be nearly as damning as it sounds, either. AI models aren’t trained, and for the most part aren’t run on a single GPU or NPU anymore. The more important factor is often how efficiently the platform scales.

Nvidia's and AMD's latest systems pack 72 GPUs into a single rack-scale compute platform, which can be expanded to 576 using optical interconnects for scale up and scale out networking.

For its upcoming 960-series accelerators, Huawei is going a similar route using near packaged optics (NPO) to scale its compute domain to as many as 4,096 chips capable of delivering up to 16 exaFLOPS of FP4 compute.

What Huawei lacks in compute density, it makes up for in scale, a tactic we’ve previously seen with Google’s TPU pods. This shouldn’t surprise anyone considering that high-speed networking is Huawei’s bread and butter.

We looked at Huawei’s NPO tech in greater detail earlier this week, but, in a nutshell, it addresses a major scaling challenge and failure point inherent to earlier Huawei designs — pluggable optics are power hungry, prone to failure, and take up a ton of space.

Huawei says NPO allowed it to consolidate 48,000 800 Gbps optical pluggables into 5,500 of its Hi-One NPO units, cutting power consumption by 550 kilowatts and failure rates in half. Using the new tech, it claims it can now achieve 99.8 percent uptime.

This is likely welcome news for Chinese model builders under pressure to adopt domestic alternatives to Western kit. DeepSeek had reportedly run into trouble using Huawei’s earlier NPUs for training and ultimately was forced to switch back to Nvidia GPUs.

The Ascend 960PR

Alongside Huawei’s high-bandwidth 960DT, the company is also prepping a compute optimized version of the chip called the 960PR for later (Q3) in 2027.

In many respects, the chip fills a role similar to the one Nvidia's Rubin CPX accelerators were intended to serve before they were canned earlier this year.

The PR in the 960PR apparently refers to prefill and recommenders, and sees Huawei trade memory capacity and bandwidth for a substantial increase in low-precision compute performance. Specifically, the PR variant will boast up to 8 petaFLOPS of FP4 performance, 192 GB of what we believe to be its custom HiBL memory, which is good for 2.4 TB/s of bandwidth.

As we understand it, the chip is designed to work in conjunction with Huawei’s DT chips in order to boost LLM inference performance. The compute-heavy prefill phase of the inference pipeline, where prompts are processed, is handed off to the 960PR, while the memory-intensive decode phase, where the output tokens are actually generated, runs on the 960DT.

This kind of heterogeneous architecture has proven to be quite effective at boosting performance, and is not unlike the arrangement used by Nvidia's Vera Rubin and Groq LPX configs we looked at back at GTC this spring. The main difference is that Huawei doesn’t have an SRAM heavy decode accelerator just yet.

Huawei eyes million NPU clusters, ever faster AI

At Connect, Huawei signaled its intent to scale its compute clusters to half a million or more NPUs. 

And in a proof of concept, it announced it’d successfully scaled its Ascend 950-based TaiShan superpod to 4,096 accelerators. Using a two-tier, four-plane Clos network topology, the company expects it’ll soon be able to support clusters with up to 512,000 NPUs, and, by moving to a multi-rail topology, it believes million-NPU superclusters are within the realm of possibility.

From what we can tell, this is only a theoretical configuration, not something the company has actually proven in the wild.

Huawei also teased the next two generations of its Ascend accelerators, which as you might expect, promise steady performance gains, more memory, and faster interconnects to tie them all together.

Slated for some time in 2028, the Ascend 970-series parts promise up to 3.6 petaFLOPS FP8 or 14 petaFLOPS at FP4 — Huawei appears to be dedicating more die area to low precision data types, something we have seen with AMD and Nvidia’s latest generation of chips as well.

At 288 GB, memory capacity won’t change much from the 960-series, but Huawei claims it expects to boost memory bandwidth to 14.4 TB/s, which should benefit AI inference considerably.

By 2029, Huawei expects to double performance again to 7.2 petaFLOPS FP8 and 28 petaFLOPS FP4 while boosting memory capacity to 384 GB and 38.4 TB/s of memory bandwidth. Having said that, three years is a long time to wait and we wouldn’t be surprised to see significant changes to Huawei’s roadmap before then. ®