Module fp8 

F8_E4M3/F8_E5M2 weight quantization.

Both are natively supported safetensors dtypes (see the pinned safetensors 0.7’s Dtype::F8_E4M3/Dtype::F8_E5M2), so unlike INT4 (see crate::quant::pack4) no packing convention is needed – this module only has to do the bit-level f32 <-> f8 conversion and per-group amax-based scale computation, since the crate depends on neither the half crate (f16/bf16 only, no f8) nor any other f8 implementation.

Encoding follows the OCP FP8 spec: E4M3 is the “FN” variant (no infinities; the single exponent=1111/mantissa=111 bit pattern is reserved for NaN, freeing up the rest of that exponent for finite values up to 448). E5M2 is IEEE-754-like (has infinities). Rounding is round-to-nearest-even; out-of-range magnitudes saturate (to 448/57344 for E4M3/E5M2 respectively, or to infinity for E5M2, which has one). Subnormal outputs are flushed to zero rather than rounded into the target format’s subnormal range – an accepted simplification for weight quantization, where values within about one ULP of the smallest normal (2^-9 for E4M3, 2^-16 for E5M2) are negligible relative to the scale factor applied before conversion. Decoding handles subnormals and NaN/Inf fully, since it also needs to correctly read back bytes this module didn’t itself produce.

Structs

QuantizedFp8

An f32 tensor quantized to f8 bytes, one f32 scale per group (the tensor is scaled by 1 / scale before conversion, matching the affine schemes’ convention of storing a multiplicative dequantization scale).

Enums

Fp8Variant

Which OCP FP8 encoding to target.

Functions

dequantize_fp8

Reconstructs f32 values from a QuantizedFp8.

f8_to_f32

Decodes an f8 byte (variant) back to f32. Unlike f32_to_f8, this fully handles subnormals, since it may be asked to decode bytes this module didn’t itself produce.

f32_to_f8

Encodes x as an f8 byte in variant. See the module docs for rounding/saturation behavior.

quantize_fp8

Quantizes data (row-major, shape shape) to variant, scaling each group so its amax maps to the format’s largest finite value.