INT4 vs FP4 — How 4 Bits Encode a Number

Same 16 codes, very different value placement. Uniform steps vs. floating-point spacing.

Sign bit

Exponent bits

Mantissa bits

INT4

Uniform · Additive

Signed integer, symmetric range. Every step is the same size.

Ssign

b2²

b2¹

b2⁰

1 sign 3 magnitude bits

value = (−1)^S × integer × scale

codes

−7…7

range (×scale)

const

step size

Representable values (× scale)

Equal spacing everywhere — the gap near 0 equals the gap near the max.

📐

Uniform grid. Great when values are spread evenly. Wastes resolution if most weights cluster near zero.

FP4 (E2M1)

Non-uniform · Multiplicative

1 sign · 2 exponent · 1 mantissa. Steps grow with magnitude.

Ssign

E2¹

E2⁰

Mfrac

1 sign 2 exponent 1 mant.

value = (−1)^S × 2^E−bias × (1.M) × scale

codes

±6

max (×scale)

var

step size

Representable values (× scale)

Dense near 0, sparse far out — fine steps where weights actually live.

🎯

Floating grid. More resolution near zero (where most weights are), wider dynamic range with the same 4 bits.

Side by Side — Same Bits, Different Grid

Both use 16 codes. INT4 ticks (top) are evenly spaced; FP4 ticks (bottom) bunch up near zero.

INT4

FP4

Property	INT4	FP4 (E2M1)
Bit layout	1 sign + 3 magnitude	1 sign + 2 exp + 1 mantissa
Spacing	Uniform (additive)	Non-uniform (multiplicative)
Step near 0	Same as everywhere	Small — high resolution
Step far from 0	Same as everywhere	Large — coarse
Dynamic range	Limited (linear)	Wider (exponential)
Best when…	Values spread evenly	Values cluster near 0 + outliers
Used in	W4A16 INT kernels, GPTQ/AWQ	MXFP4 / NVFP4, next-gen HW

🔑 Key idea: INT spends its 16 codes on an even ruler; FP spends them on a logarithmic ruler. Neural-net weights pile up near zero, so FP4's fine-near-zero grid often fits the distribution better at the same bit-width — which is why MXFP4/NVFP4 are gaining traction for low-bit inference.