Enum Scheme 

pub enum Scheme {
    Int8 {
        symmetric: bool,
    },
    Int4 {
        symmetric: bool,
    },
    Fp8 {
        variant: Fp8Variant,
    },
}

Which quantization scheme to apply to a tensor.

Variants

Int8

8-bit affine integer quantization.

Fields

symmetric: bool

Symmetric (zero_point = 0) vs asymmetric.

Int4

4-bit affine integer quantization, nibble-packed into U8 (see pack4).

Fields

symmetric: bool

Symmetric (zero_point = 0) vs asymmetric.

Fp8

8-bit floating point.

Fields

variant: Fp8Variant

Which OCP FP8 encoding.

Implementations

impl Scheme

pub fn type_name(self) -> &'static str

The compressed-tensors weights.type field for this scheme.

pub fn num_bits(self) -> u8

The compressed-tensors weights.num_bits field for this scheme.

pub fn is_symmetric(self) -> bool

Whether this scheme is symmetric (zero_point = 0 / no zero-point tensor written).

pub fn short_name(self) -> String

A short, stable name used in CLI output and file names (e.g. int8, int4, fp8_e4m3).

Trait Implementations

impl Clone for Scheme

fn clone(&self) -> Scheme

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Scheme

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Scheme

fn eq(&self, other: &Scheme) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for Scheme

impl Eq for Scheme

impl StructuralPartialEq for Scheme