Enum Granularity 

pub enum Granularity {
    PerTensor,
    PerChannel {
        axis: usize,
    },
    Group {
        axis: usize,
        group_size: usize,
    },
}

Quantization granularity: how a tensor’s elements are partitioned into groups that each get their own scale/zero-point.

Variants

PerTensor

One scale/zero-point for the whole tensor.

PerChannel

One scale/zero-point per index along axis (e.g. per output channel of a Linear or Conv weight). Equivalent to Granularity::Group with group_size: 1.

Fields

axis: usize

Axis to quantize independently along (typically the output-channel axis, 0).

Group

One scale/zero-point per contiguous group_size-element block along axis (the GPTQ/AWQ/compressed-tensors convention; axis is typically the reduction/input-channel axis).

Fields

axis: usize

Axis the grouping runs along.

group_size: usize

Number of elements per group along axis.

Implementations

impl Granularity

pub fn strategy_name(self) -> &'static str

A short, stable name for the compressed-tensors strategy field.

Trait Implementations

impl Clone for Granularity

fn clone(&self) -> Granularity

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Granularity

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Granularity

fn eq(&self, other: &Granularity) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for Granularity

impl Eq for Granularity

impl StructuralPartialEq for Granularity