Three dtype examples · BF16 output
Same sequence length, head dimension, tile size, head counts, and GPU specifications
| Q/K/V storage | Tiled AI FLOPs/B | Ideal AI FLOPs/B | Tiled ceiling TFLOP/s | Roofline regime | Preset |
|---|
The comparison keeps the same entered GPU TFLOP/s for all three dtypes; hardware throughput for BF16, FP8 and MXFP4 may differ. Tiled HBM assumes repeated K/V traffic, while the ideal model assumes one HBM read per Q/K/V element.
GPU Roofline Model
Log–log axes · HBM bandwidth roof · model predictions
Tiled attention · repeated KV reads
- Matrix FLOPs (QKᵀ + PV)
- —
- HBM traffic
- —
- Roofline performance
- —
- Bandwidth lower-bound time
- —
- Compute lower-bound time
- —
- Predicted bottleneck
- —
Ideal HBM reuse · tensor read/write once
- Matrix FLOPs (QKᵀ + PV)
- —
- HBM traffic
- —
- Roofline performance
- —
- Bandwidth lower-bound time
- —
- Compute lower-bound time
- —
- Predicted bottleneck
- —
How the model is calculated
Matrix arithmetic (non-causal)
Ridge point
Ideal HBM bytes
Tiled HBM bytes
N = shared Q/KV sequence length; d = head dimension; s* = stored bytes per element, including modeled MX scale metadata; r = modeled KV sharing across query heads (1 by default, or Hq/Hkv if enabled). The tiled estimate assumes KV blocks are re-fetched from HBM for each query tile per sharing group; actual L2 reuse can make HBM AI significantly higher. Head dimension cancels from AI in this simplified model but still changes FLOPs, bytes, and predicted time. In practice, GPU utilization, softmax, and on-chip memory effects can reduce achieved performance far below the roof.