Skip to main content

quantize_fp8

Function quantize_fp8 

Source
pub fn quantize_fp8(
    data: &[f32],
    shape: &[usize],
    granularity: Granularity,
    variant: Fp8Variant,
) -> QuantizedFp8
Expand description

Quantizes data (row-major, shape shape) to variant, scaling each group so its amax maps to the format’s largest finite value.