1
Rotate Q and K
A fixed Hadamard transform redistributes large outliers across the head dimension. Q and K mean query and key.
Q′ = Q · H
K′ = K · H
K′ = K · H
Quantize the whole attention path without breaking softmax.
MXAttention combines outlier smoothing, a better MXFP4 scaling boundary, and normalization-aware probability quantization so both matrix multiplications can use 4-bit data while preserving the meaning of attention.
Hadamard rotation
Spreads Q/K outliers before quantization.
UOS · Universal Optimal Scaling
Uses the analytically selected MXFP4 boundary Qmax = 7.25 to balance clipping and underflow.
PNQ · Pre-Normalization Quantization
Quantizes before the row-sum update, so numerator and denominator use the same values.
End-to-end flow
The core pipeline
The important idea is not just “use FP4.” Each quantization point is placed where it protects the semantics of online softmax.
→
2
Apply UOS
Choose the shared E8M0 scale with the universal FP4 boundary 7.25. E8M0 stores the shared power-of-two block scale.
e = ceil(log₂(M / 7.25))
scale = 2ᵉ
scale = 2ᵉ
→
3
Compute FP4 scores
Q and K are consumed by the first low-precision matrix multiplication.
S = Q̂K̂ᵀ / √d
→
4
Quantize before normalization
PNQ reuses the quantized exponential tile for both the row sum and PV, where PV means probability × value.
P̃ = exp(S − m)
P̂ = MXFP4(P̃)
l += ΣP̂
O += P̂V̂
P̂ = MXFP4(P̃)
l += ΣP̂
O += P̂V̂
Why PNQ matters
Naïve FP4 versus normalization-aware FP4
Quantizing only the PV branch creates a mismatch: the numerator uses FP4 probabilities while the denominator still uses higher-precision probabilities.
Two different probability tensors
The row sum and output accumulator disagree about what “P” is.
Denominator
l += Σ P̃
≠
Numerator
O += MXFP4(P̃) · V̂
Effective attention row sum
≈ 0.9266
One shared quantized probability tensor
The same FP4 values define both normalization and value aggregation.
Denominator
l += Σ P̂
=
Numerator
O += P̂ · V̂
Normalized effective row sum
1.0000
Interactive UOS intuition
Clipping versus underflow
Move the block maximum to see how the UOS rule selects a power-of-two scale. The cells are illustrative E2M1 projections, not a full hardware emulator.
Block maximum M
UOS computes the shared scale from ceil(log₂(M / 7.25)).
164
Block max
14.5
Exponent e
1
Shared scale
2
Example 32-value MX block
Red cells indicate saturation pressure; amber cells indicate values likely to collapse toward zero after coarse FP4 projection.
Mental model
What each component fixes
MXAttention preserves attention semantics by aligning the quantization path with the softmax algorithm.
UOS improves numerical representation, but PNQ is what prevents online softmax from normalizing one probability tensor while aggregating with another.
Hadamard: suppress localized Q/K outliers.
UOS: improve MXFP4 block scaling with Qmax = 7.25.
PNQ: preserve row normalization after probability quantization.
Fused kernel: avoid materializing the full attention matrix.
Glossary
Abbreviations and notation
Definitions for the names, formats, tensors, and symbols used in the diagrams.
MXAttentionMicroscaling attention design
The complete low-precision method: Hadamard rotation, UOS, and PNQ inside a fused tiled-attention kernel.
MXFP4Microscaling 4-bit floating point
Each element uses a 4-bit floating-point value, while a small block of values shares a separate power-of-two scale.
UOSUniversal Optimal Scaling
A data-free MXFP4 scale rule using Qmax = 7.25 to balance large-value clipping against small-value underflow and rounding.
PNQPre-Normalization Quantization
Quantizes unnormalized softmax probabilities before calculating their row sum, then reuses the same values for normalization and PV.
E2M12 exponent bits, 1 mantissa bit
The element-level FP4 format used by MXFP4. Its small set of representable values makes scale selection important.
E8M08 exponent bits, 0 mantissa bits
The shared block-scale format. With no mantissa, the scale is a power of two: efficient, but relatively coarse.
OCPOpen Compute Project
The organization behind the standard microscaling format specification that defines MXFP4 and related formats.
Q, K, VQuery, Key, Value
QKᵀ produces attention scores. Normalized probabilities then multiply V to produce the output.
PVProbability × Value
The second attention matrix multiplication, which combines value vectors using their softmax probabilities.
FP44-bit floating point
A compact floating-point representation using four bits per element. It is faster and smaller, but less precise than FP16 or BF16.
Online softmaxTiled, numerically stable softmax
Maintains a running maximum, row sum, and output accumulator so the full attention matrix does not need to be stored.
Hadamard rotationOrthogonal mixing transform
Spreads localized Q/K outliers across channels before quantization while preserving the full-precision dot product.
How to read the equations
M — maximum absolute value in one MX block.Qmax — normalized boundary used to choose the block scale.e — exponent of the shared power-of-two scale.H — Hadamard or orthogonal rotation matrix.d — attention head dimension.S — score tile from QKᵀ / √d.m — running row maximum in online softmax.P̃ — unquantized, unnormalized softmax values.P̂ — MXFP4-quantized softmax values.l — running softmax row sum.O — running attention output accumulator.Q̂, K̂, V̂ — quantized query, key, and value tensors.