NPU vs GPU: What the 40 TOPS Number on Your Laptop Actually Measures

Written by

in

10 min read

Laptop listings advertise “40 TOPS NPU.” Phone launches promise AI that runs several times faster than last year. TOPS stands for trillion operations per second, the unit vendors use for the throughput of an AI acceleration block. Forty TOPS means forty trillion operations in a second.

An NPU (neural processing unit) is a chip built to do only the multiply-and-add work that AI models run on. That work means repeating the same small arithmetic billions of times. A CPU handles many kinds of tasks one after another, which makes it a poor fit. A GPU runs thousands of identical calculations side by side, since that is how graphics are drawn, and the same ability transfers to AI work. The NPU narrows the job further and spends less power doing it. Everything below comes from vendor spec pages and documentation checked on November 5, 2026.

40 TOPS is a Microsoft badge threshold

The number comes from the hardware bar Microsoft set for the Copilot+ PC tier, not from an industry performance grade. The requirements page lists an NPU of 40+ TOPS, 16GB of DDR5 or LPDDR5 memory, and 256GB of SSD or UFS storage[1].

Requirement Copilot+ PC
NPU 40+ TOPS
Memory 16GB DDR5/LPDDR5
Storage 256GB SSD/UFS
Qualifying silicon AMD Ryzen AI 300 and 400 series, Intel Core Ultra 200V and 300V series, Snapdragon X series[1]

All three conditions have to be met. The same page states that a machine with a 40+ TOPS NPU but under 16GB of RAM is not a Copilot+ PC[1]. Neither that page nor the developer guide explains where the number 40 came from.

The features most people actually use ask for far less. Windows Studio Effects (automatic framing, background blur, eye contact correction, voice focus) runs on any NPU rated at 10+ TOPS, per the same page[1]. Forty TOPS works as a tier threshold rather than as the minimum any particular feature needs.

Having an NPU does not mean the work stays on the machine either. Paint’s Cocreator, which generates images from a text prompt, is a Copilot+ feature that still requires a Microsoft account and an internet connection[1]. Whether a given feature stays local has to be checked feature by feature.

How the NPU, GPU and CPU split the work

Microsoft’s developer guide describes the NPU as using energy on AI tasks more efficiently than a CPU or GPU, which extends battery life, and says the NPU works alongside the CPU and GPU while Windows 11 assigns each task to the most appropriate place[2]. The three parts divide the work rather than compete for it.

What lands on the NPU tends to be short, continuous processing that runs all day: background blur on a video call, eye contact correction, live captions, which Microsoft itself cites when explaining what an NPU is for[1]. Windows Live Captions translates subtitles into English from more than 40 languages, and into Simplified Chinese from 27[1].

Large language models work the other way. The whole model has to live in memory and be read back quickly, so memory capacity and memory speed decide the outcome. The numbers for that come below.

TOPS is a peak figure under ideal conditions

A TOPS rating is a ceiling reached when the circuitry runs perfectly, not a measured throughput. AMD writes the caveat into its own press release footnote: TOPS for a Ryzen processor “is the maximum number of operations per second that can be executed in an optimal scenario and may not be typical,” and varies with system configuration, AI model and software version[7]. Intel labels the figure a peak on its product spec pages and states the INT8 basis next to it[5].

INT8 means each number is squeezed into a single byte as an integer. Microsoft’s developer documentation notes that many NPU devices only support integer math in low-bit formats such as INT8, so a model trained in 32-bit floating point has to be quantized before it can run there[2]. That compression step has its own piece in the quantization explainer.

The same TOPS number does not mean the same thing

Each vendor measures a different block in a different numeric format. Taking the published figures exactly as written:

Product Published figure What was counted
Qualcomm Snapdragon X2 Elite up to 85 TOPS NPU; no data format stated on the product page[3]
Qualcomm Snapdragon X Elite (prior gen) up to 45 TOPS NPU[4]
Intel Core Ultra X9 388H NPU 50 TOPS, GPU 122 TOPS Both INT8 peak, listed as separate rows[5]
Intel Panther Lake platform up to 180 platform TOPS CPU, GPU and NPU added together[6]
AMD Ryzen AI 400 series (desktop) up to 50 TOPS NPU[7]
AMD Ryzen AI PRO 400 series up to 60 TOPS NPU[7]
NVIDIA GeForce RTX 5090 3,352 AI TOPS GPU tensor cores, FP4 basis[8]
Apple M5 none published 16-core Neural Engine; Apple cites 153GB/s memory bandwidth instead[9]
Samsung Exynos 2600 none published NPU described only as 113% better than the previous chip[14]

NVIDIA’s 3,352 and Intel’s 50 cannot share a scale. The first is measured in FP4 (4-bit floating point) and the second in INT8, and fewer bits per number let the same circuitry push more numbers through per second, which inflates the operations-per-second figure. Intel’s 180 is a third kind of number: it adds the CPU and GPU to the NPU, so it sits apart from the NPU 50 the same company publishes on its spec page.

Apple and Samsung skip the unit entirely. Apple says the Neural Engine got faster without giving a throughput figure[9], and Samsung describes the Exynos 2600 NPU as 113% better than its predecessor[14]. The second is a ratio against the previous chip, so the absolute throughput behind it stays unknown and other vendors’ numbers have nothing to sit beside.

Published TOPS figures measure different things
Published TOPS figures measure different things

Memory bandwidth sets the token rate

A language model works in tokens, the small chunks it cuts text into, roughly a short word or less in English. For every token it produces, the machine reads the model’s weights (the numbers left behind by training, one per parameter) out of memory once. The rate at which tokens appear therefore follows how fast data moves out of memory rather than how much arithmetic the chip can do. NVIDIA’s developer blog puts it the same way: model execution is frequently memory-bandwidth bound, specifically bound on the weights[11].

Splitting the problem into two ceilings shows the gap. These figures come from published specs, calculated directly[12].

Assumptions: 8-billion-parameter (8B) model quantized to 4 bits → weights of about 4.0GB
             Memory from the Intel Core Ultra X9 388H spec (LPDDR5X 9,600MT/s, 2 channels = 128-bit)
             Compute at 50 and 85 TOPS, the low and high ends of current laptop NPU claims

Memory bandwidth   = 9,600 × 128 ÷ 8 = 153.6 GB/s
Bandwidth ceiling  = 153.6 ÷ 4.0  ≈ 38 tokens/sec
Compute ceiling    = 50T ÷ (8B × 2) ≈ 3,125 tokens/sec
                   = 85T ÷ (8B × 2) ≈ 5,313 tokens/sec

The compute ceiling assumes two operations per parameter, a multiply and an add. It lands more than 80 times above the bandwidth ceiling. The two sides are not held to identical units: a 4-bit model is mapped onto INT8 throughput, and the 85 TOPS figure goes through the same formula even though its data format is never stated. Real speeds fall below both because of software. A gap of that order still settles which side binds first, and whether the NPU is rated 50 or 85, the 38 tokens per second stays put.

Running an 8B model in 4-bit: which ceiling hits first
Running an 8B model in 4-bit: which ceiling hits first

Bandwidth changes the answer. The same arithmetic across several devices[12]:

Device (memory bandwidth) 3B in 4-bit (1.5GB) 8B in 4-bit (4.0GB) 30B in 4-bit (15.0GB)
DDR5-5600 dual channel laptop (89.6GB/s) 60 tokens/sec 22 tokens/sec 6 tokens/sec
Apple M5 (153GB/s) 102 tokens/sec 38 tokens/sec 10 tokens/sec
Apple M5 Pro (307GB/s) 205 tokens/sec 77 tokens/sec 20 tokens/sec
Apple M5 Max (614GB/s) 409 tokens/sec 154 tokens/sec 41 tokens/sec

Every row assumes the model fits in memory to begin with. A 15GB model on a 16GB machine fails to load long before speed enters the conversation.

When Apple introduced the M5 Pro and M5 Max in March, it tied the bandwidth increase to “higher token generation for LLMs”[10]. The M5 Pro pairs 307GB/s with up to 64GB of memory, the M5 Max 614GB/s with up to 128GB[10]. Whether the model fits in memory and how fast it can be read back are the two things a buyer feels.

On-device models are small and compressed

Apple has published the size of its own on-device model: roughly 3 billion parameters, with weights compressed to 2 bits per weight through quantization-aware training. The embedding table runs at 4 bits and the cache holding intermediate results at 8 bits[13]. Models that run in a data center are orders of magnitude larger, which is why some requests stay on the phone and others leave it.

A machine without an NPU can still run models locally. Microsoft’s documentation states that the Windows AI APIs require a Copilot+ PC with an NPU, while Foundry Local, Microsoft’s tool for running models on the machine itself, works on any Windows device with a DirectX 12-capable GPU, including devices without an NPU[15]. The same page states that inference input and output never leave the machine[15].

What to do with this

  • Check whether your PC has an NPU in Task Manager: Ctrl+Shift+Esc, Performance tab, NPU in the left panel[15].
  • If local language models are the goal, look at memory size and bandwidth before TOPS. Weights in 4-bit come to roughly half a gigabyte per billion parameters, so an 8B model is about 4GB[12], and the machine needs more than that once the cache and the program itself are loaded.
  • Estimate speed with one division: memory bandwidth in GB/s divided by model size in GB gives the ceiling in tokens per second, and real throughput lands under it.
  • Before comparing two TOPS numbers, check three things: NPU alone or platform total, INT8 or FP4, and peak or measured. If any of the three differ, the numbers are not comparable as printed.
  • Check whether the feature you want needs an NPU at all. Video call effects run at the 10 TOPS level, and some badge-gated features still require an internet connection.
  • A machine without an NPU can still run models locally through the documented DirectX 12 GPU path.

Summary

Forty TOPS is the line Microsoft drew for the Copilot+ PC tier, bundled with 16GB of memory and 256GB of storage. Vendors measure different blocks in different formats, so TOPS figures rarely compare without first matching the conditions. What governs the experience of running a language model locally is memory capacity and bandwidth, and neither appears in a TOPS rating. API pricing and token units are covered in the token cost explainer.

Notes


Notes

  1. Microsoft, Windows 11 specs and feature requirements (checked 2026-11-05) Windows 11 Specs and System Requirements ↩
  2. Microsoft Learn, Copilot+ PCs developer guide (checked 2026-11-05) Develop AI applications for Copilot+ PCs ↩
  3. Qualcomm, Snapdragon X2 Elite product page (checked 2026-11-05) Snapdragon X2 Elite ↩
  4. Qualcomm, Snapdragon X Elite product page (checked 2026-11-05) Snapdragon X Elite ↩
  5. Intel, Core Ultra X9 388H product specifications (checked 2026-11-05: NPU peak 50 INT8 TOPS, GPU peak 122 INT8 TOPS, LPDDR5X 9,600MT/s, 2 channels) Intel Core Ultra X9 388H Specifications ↩
  6. Intel Newsroom, Panther Lake architecture (checked 2026-11-05) Intel Unveils Panther Lake Architecture ↩
  7. AMD press release, expanded Ryzen AI 400 series portfolio (checked 2026-11-05) AMD Gives Consumers and Businesses More AI PC Options ↩
  8. NVIDIA, GeForce RTX 5090 product page (checked 2026-11-05: 5th-generation Tensor Cores, 3,352 AI TOPS, FP4) GeForce RTX 5090 ↩
  9. Apple Newsroom, M5 announcement (2025-10-15) Apple unleashes M5 ↩
  10. Apple Newsroom, M5 Pro and M5 Max announcement (2026-03) Apple debuts M5 Pro and M5 Max ↩
  11. NVIDIA developer blog (checked 2026-11-05) Mastering LLM Techniques: Inference Optimization ↩
  12. AI Signal calculation (2026-11-05, scripts/tmp-npu-bw.ts). Inputs are the published specs in notes 1, 3, 5, 9 and 10; model sizes count only the weights re-read for each token, at 4 bits per parameter. ↩
  13. Apple Machine Learning Research, foundation model updates (published 2025-06-09) Updates to Apple’s On-Device and Server Foundation Language Models ↩
  14. Hankyung, Samsung unveils Exynos 2600 (2025-12-19). Samsung’s own product page blocks automated access, so the figure comes from this report. Samsung Exynos 2600 unveiled, AI performance up 113% ↩
  15. Microsoft Learn, FAQs about using AI in Windows apps (checked 2026-11-05) FAQs about using AI in Windows apps ↩

Source: https://learn.microsoft.com/en-us/windows/ai/npu-devices/