Blackwell isn't a monolith: what happens when you actually measure kernel performance on GB10

NVIDIA's Blackwell architecture isn't one chip. It's a family: B200, GB200, GB10, RTX PRO 6000 Blackwell, and more coming. They share a lot: the same core architecture, the same compute capabilities, the same instruction set. From a distance, they look like variations on a theme.

The software stack that runs models on them mostly treats them that way too. Optimizations shipped for one Blackwell chip get applied to all of them by default, gated on a single "is this Blackwell?" check.

I recently ran a full kernel tuning sweep on GB10 (the Grace Blackwell workstation variant sitting inside HP's ZGX Nano) and found something worth sharing: the assumption that Blackwell is homogeneous doesn't quite hold. GB10 wants different kernel settings than B200 or GB200. Not dramatically different. But consistently different in ways that suggest the shared-heuristic approach is under-serving smaller Blackwell variants.

This piece walks through what I measured, why it matters, and what it says about how the AI infrastructure ecosystem thinks about hardware families. The finding is small, but it's real, and it points at a design question that will only get more important as NVIDIA ships more Blackwell variants.

Why any of this matters

Before getting into what I measured, let me set the stage for why kernel-level performance is a thing worth caring about at all.

When you run a language model, most of the compute happens in a small number of highly-optimized routines called kernels. A kernel is a compact chunk of code that runs on the GPU, does one specific mathematical operation across a large amount of data, and returns a result. For a transformer, the big-ticket kernels are attention and matrix multiplication. For newer state-space models like Mamba, there's a different set of kernels doing different operations.

The math these kernels compute is fixed. What varies is how the GPU executes them. The same computation can be split across the GPU's parallel machinery in many different ways, and different splits run at different speeds. A well-tuned kernel might run 20-30% faster than a naively-launched version, computing bitwise-identical results.

Someone has to figure out the right splits for each combination of operation, input shape, and GPU. NVIDIA does a lot of this work themselves. That's part of what CUDA and cuDNN are for. Open source frameworks like vLLM do more, and typically ship a curated set of pre-tuned configurations for common GPUs.

The chip I was working with, GB10, was new enough that vLLM hadn't shipped a tuned configuration for one specific kernel called selective state update. It's the kernel that does the heavy lifting for Mamba layers. Every time the engine started up, it printed a warning that this kernel was running on a fallback schedule rather than one specifically tuned for GB10: "sub-optimal performance." I wanted to know: how sub-optimal, really?

What Mamba is doing

To understand what I was tuning, you need a rough picture of what Mamba is doing that's different from a standard transformer.

Transformers use attention. When they process a new token, they look at every previous token and compute how much each of them influences what should come next. This produces the model's characteristic behavior (reading a long document coherently, following instructions from many tokens ago) but it has a cost: the "attention" work grows quadratically with context length. Double the context, quadruple the compute.

Mamba is a state-space model. Instead of storing every previous token and looking back at all of them, it maintains a fixed-size internal memory called a state. Every time a new token arrives, the state gets updated: the model reads the current state, combines it with the new token, produces an output, and writes an updated state back. The state stays the same size no matter how much context has passed. This makes Mamba much more efficient at long contexts.

The model (soon to be released Nemotron variant) I was working with is a hybrid: it has some Mamba layers (using state) and some attention layers (using the classic key-value cache). Both memory structures coexist. In each Mamba layer, for each of its attention heads, there's a fixed-size state matrix that has to be read, updated, and written back on every single token the model generates.

That state matrix has a specific shape. For the model I was using, each head's state is 128 rows by 64 columns of numbers, 8,192 numbers total. There are many heads per layer, and many layers in the model. On every token, the GPU updates all of them. This is where the "selective state update" kernel lives, and this is what I was tuning.

The warehouse analogy

Here's an analogy that might help. Feel free to skip if it slows you down.

Imagine each state matrix as a warehouse. The warehouse has 128 shelves, and each shelf has 64 boxes on it. When a new token arrives, the model has to touch every box: read it, modify it, put it back.

To do this work, you have a bunch of workers organized into crews. Each crew gets assigned a section of the warehouse to handle. Two questions determine how efficiently the work gets done:

How wide is each crew's section? If each crew handles all 64 boxes on their assigned shelf, that's one thing. If each crew handles only 4 boxes and you have 16 crews per shelf, that's another. Different section widths mean different amounts of parallelism.

How many workers are in each crew? More workers per crew means more parallel effort inside each section, but larger crews get in each other's way. Smaller crews are more efficient per worker but rely on having many sections to keep everyone busy.

The trade-off isn't obvious. Small sections with big crews swarm the warehouse in parallel but risk chaos. Big sections with small crews are orderly but might leave workers idle. The right answer depends on how big the warehouse is (that's dstate × headdim), how many warehouses need work at once (that's the concurrency), and, critically, what the warehouse layout looks like (that's the specific GPU).

Different GPUs have different layouts. An H100 is a big warehouse with wide corridors: lots of aisles for crews to move through in parallel. A B200 is even bigger. GB10 is smaller, with narrower corridors. What works efficiently in a big warehouse doesn't necessarily work in a small one.

The two decisions I described map onto real Triton kernel parameters:

• The section width is called BLOCK_SIZE_M, the number of feature-axis elements each GPU thread block processes as its tile
• The crew size is called num_warps, the number of 32-thread warps working together within each thread block

These are the two knobs the kernel tuner searches over.

The tuning process

vLLM ships tuned configurations for the selective state update kernel for six GPUs: H100, H200, A100, B200, GB200, and RTX PRO 6000 Blackwell. Each configuration is a JSON file mapping "workload sizes" (represented as a value called effective_batch, which is roughly the number of independent state updates happening concurrently) to the winning (BLOCK_SIZE_M, num_warps) pair for that workload size on that chip.

When the engine starts up, it looks for a config file matching the current GPU. If found: use it. If not: fall back to a generic heuristic and print the "sub-optimal performance" warning.

GB10 didn't have a shipped config. I generated one by running vLLM's built-in tuning script:

python benchmarks/kernels/benchmark_selective_state_update.py \
    --save-configs --validate --compare --verbose

The tuner does something conceptually simple:

1. Pick a workload size (say effective_batch=4096)
2. Try every combination of BLOCK_SIZE_M and num_warps from a small search space
3. Time each combination
4. Record which one was fastest
5. Write it to a JSON file

The search space isn't large. There are 5 valid BLOCK_SIZE_M values (4, 8, 16, 32, 64), 4 num_warps values (1, 2, 4, 8), and 15 or so workload sizes to test. Around 300 kernel launches total, plus some warmup runs. The whole thing finishes in about an hour on GB10.

The environment was fussier than the tuning itself. Two vLLM installations on the box conflicted with each other, the CUDA toolkit path had to be exported correctly for JIT compilation to work, and the tuner script imports from a tests/ directory that isn't in the pip package, so you have to copy it in manually. None of this was hard, but each step needed to be figured out. It took most of a day to get from "we want to tune this" to "the tuner is running." The tuning itself was the easy part.

Once everything was configured, the run produced a validated JSON file. All 12 workload sizes passed the built-in correctness check: for each winning configuration, the tuner runs the kernel with those settings and compares the output against a CPU reference implementation. Any discrepancy above a small tolerance would flag as a failure. Nothing failed. The tuned configurations were producing correct results, just slightly faster than the fallback heuristic.

What the numbers showed

Here's the comparison the tuner produced. "Heuristic" is what vLLM's fallback logic picks when no tuned config exists. "Tuned" is what the tuner found by empirical search.

The speedups are small. At the smallest workload size, tuning found a 1.16× improvement. Across most of the range, it's 1.02×, a couple of percent. (End-to-end throughput on a real serving workload turned out to be a different story, which I'll come back to in the caveats.)

If the goal was "make the model run dramatically faster," this is a disappointment. vLLM's fallback heuristic for GB10 was already pretty good. The "sub-optimal performance" warning was overstating the situation. The heuristic sits close to optimal, and the tuner found a modestly better configuration in the same neighborhood.

But if you look past the speedup numbers to the "winning config" column, something else appears.

The finding

Look at the winners at large workload sizes:

• effective_batch ≥ 4096: BLOCK_SIZE_M=4, num_warps=8

That configuration wins on GB10, consistently, across the entire large-workload range. Now compare to what vLLM ships for other Blackwell chips:

• B200: BLOCK_SIZE_M=4, num_warps=1 for essentially every workload size
• GB200: BLOCK_SIZE_M=4, num_warps=1 similarly
• H100 (for comparison, not Blackwell): Shows genuine per-shape variation with BLOCK_SIZE_M values ranging from 16 to 64 and num_warps from 1 to 8

GB10 and B200 both use small tile sizes (BLOCK_SIZE_M=4), but they diverge sharply on the number of warps per block. B200 wants one. GB10 wants eight. That's not a rounding difference. Those are opposite ends of the num_warps search space.

Meanwhile, vLLM's fallback heuristic (the one that fires when no tuned config exists) has this structure:

if dstate <= 16:    BLOCK_SIZE_M, num_warps = 32, 4
elif dstate <= 32:  BLOCK_SIZE_M, num_warps = 16, 4
elif dstate <= 64:  BLOCK_SIZE_M, num_warps = 8, 4
else:
    if is_blackwell:
        BLOCK_SIZE_M, num_warps = 32, 8
    elif dstate <= 128:
        BLOCK_SIZE_M, num_warps = 4, 4

For Nemotron's shape (dstate=128) on any Blackwell chip, the heuristic returns BLOCK_SIZE_M=32, num_warps=8. That's the "Heuristic" column in the table above.

But this heuristic branches on a single boolean: is this a Blackwell chip or not? It doesn't distinguish between datacenter Blackwell (B200, GB200, with lots of SMs and high memory bandwidth), workstation Blackwell (GB10, with fewer SMs and lower bandwidth per SM), or consumer Blackwell (RTX PRO 6000 and others). All of them get the same fallback.

The reason this matters: the underlying hardware isn't quite the same. GB10 is a workstation-class chip. It has fewer streaming multiprocessors than B200 and GB200. Its memory subsystem is optimized for a different design point. The kernel configurations that work well on B200 don't necessarily work well on GB10, even though both are "Blackwell."

The measured result confirms this. GB10's optimal config at large workloads (M=4, w=8) is different from B200's shipped config (M=4, w=1), and different from the shared Blackwell fallback (M=32, w=8). Three different answers for three different situations that the current heuristic treats as one.

The intuition, if you want it: GB10's smaller SM count means each thread block needs to do more work internally to keep the GPU busy, which favors more warps per block. Its narrower memory paths mean bigger tiles waste bandwidth on suboptimal transfers, which favors smaller BLOCK_SIZE_M. Different tuning point, different winner.

What we did about it

I contributed the tuned GB10 configuration upstream to vLLM. The pull request is a single JSON file drop plus a discussion of what I found. The PR message notes the observation about the is_blackwell heuristic branch potentially being too coarse, but doesn't propose a code change. It just flags it for maintainers to consider.

There's a reason for that framing. This isn't a bug. vLLM's current architecture is entirely reasonable: as new Blackwell chips arrive, contributors tune configs for them and merge those configs upstream. The shared fallback is a placeholder that kicks in until someone does the tuning for a specific chip. That's how the H100/A100/B200/etc. configs got there. It's how the GB10 config is getting there now.

What the finding suggests is a design question worth thinking about, not a fix that needs to happen immediately: should the fallback distinguish between Blackwell variants? Or is the ecosystem better served by keeping the fallback simple and letting per-chip tuning fill in the gaps as chips arrive?

Reasonable people can disagree. Fine-grained fallbacks make the code more complex and require more knowledge about which chips fall into which memory tiers. Simple fallbacks with per-chip overrides are easier to maintain but silently under-serve chips nobody has tuned yet. There's no obvious right answer.

What's worth calling out is that the trade-off exists and has real consequences. Anyone deploying vLLM on a Blackwell chip that isn't in the currently-shipped list will get the shared-Blackwell fallback, which (based on GB10's experience) may sit close to optimal or may sit meaningfully off. Without measurement, there's no way to know which. And measurement takes time.

Caveats worth knowing

A few honest limitations of what I measured.

This is one workload on one chip. The tuning was done for Nemotron's specific Mamba shape (headdim=64, dstate=128). Other models with different Mamba parameters might see different results. Other workloads on GB10 (image models, non-Mamba language models, chunked sequences) might have completely different kernel behavior.

The comparison to B200/GB200 assumes their shipped configs are near-optimal. They might not be. Both use identical uniform configurations across every workload size, which could indicate either "genuinely optimal at this hardware profile" or "someone dropped a placeholder that never got properly tuned." The claim about Blackwell heterogeneity depends on those configs being trustworthy references. If they aren't, the story gets more complicated: maybe nobody has properly tuned Mamba on any Blackwell chip except at the heuristic level, and GB10 just happens to be the first place where someone did the measurement.

Testing this would be straightforward: run the same tuner on B200 or GB200 and see if the shipped configs match what the tuner finds. I don't have access to those chips, but it's on my list to encourage someone who does.

The end-to-end speedup was zero. I re-ran the full benchmark sweep with the tuned config in place: 24 cells (4 speculative depths × 6 concurrency levels), 20 repeats each. Compared cell-by-cell against the untuned baseline, the deltas bounced between -2.1% and +1.3% with a mean of -0.2%. That's a textbook null result. The 2% kernel-level speedup did not translate to a measurable aggregate throughput change on this workload.

There's a mechanical reason. Selective state update is one kernel among many that run per decode step. Attention, MoE routing, expert MLP, layer normalization, and token sampling all contribute. Speeding up the SSU kernel by 2% translates to less than 2% end-to-end lift, proportional to how much of decode time that kernel accounts for. On this workload (Nemotron 3.5 Lightning at short-to-medium context lengths), Mamba's share of decode time is small enough that any translation of the kernel gain lands below run-to-run noise.

Workloads with a heavier Mamba share should see larger end-to-end lift from the same tuning: long-context generation where Mamba's linear scaling really matters, models with more Mamba layers relative to attention, or higher SSM group counts. This piece doesn't measure those regimes. If you're running such a workload on GB10 and want to know what the tuning is worth, the tuner runs in about an hour and produces the answer directly.

If you're picking a GB10 for its memory bandwidth and unified-memory model, kernel tuning is a rounding error compared to those platform decisions.

What's next: NVFP4

The whole reason kernel-level tuning has such small headroom on GB10 is that the model is already close to a fundamental limit. At BF16, each decode step reads all the active model weights from memory once. The Nemotron variant I was using has ~3 billion active parameters per token, which at 2 bytes each is 6 GB of memory traffic per generated token. GB10's memory bandwidth caps how fast that traffic can move. Kernel tuning can trim overhead around that ceiling; it can't raise the ceiling itself.

NVFP4 quantization changes the math. Weights get stored in 4-bit format instead of 16-bit. Same model, one-quarter the memory footprint per token. Blackwell hardware has native FP4 tensor cores, so this isn't a software emulation. It's the hardware doing what it was designed for.

The projected impact is substantial. Single-user throughput could jump from ~50 tokens/second to somewhere in the range of 140-160 tokens/second, approaching cloud API speeds on a workstation. And the improvements stack: Mamba kernel tuning, speculative decoding, and NVFP4 attack different bottlenecks, so their gains compound rather than compete.

That's the follow-on work. When NVIDIA ships the NVFP4 build of this model, I'll benchmark it the same way, and probably discover new surprises. Kernel-level tuning under a different memory regime might look completely different. The is_blackwell heuristic branch might turn out to be even more wrong at that operating point. Or the whole picture might simplify: memory-bound problems that go away have a way of surfacing compute-bound problems that were hiding underneath.

We'll see. For now: one small config file merged upstream, one small observation about how the ecosystem thinks about hardware families, and a reminder that when someone tells you a family of chips is homogeneous, it's worth taking your own measurements.

---

The GB10 tuning configuration for selective state update and the raw benchmark output are being contributed upstream to vLLM. If you're deploying a Mamba or hybrid-Mamba model on a Blackwell chip that isn't currently in the shipped configuration list for that kernel, the same tuning process is available: one script, one afternoon, one JSON file, and the results will tell you whether the shared-Blackwell fallback is serving your specific hardware well or not.