Module format 

Output convention: vllm-project/compressed-tensors layered on plain .safetensors, so quantized checkpoints stay loadable by existing HF/vLLM tooling for INT8/FP8. For a quantized weight tensor named foo.weight:

• foo.weight itself becomes the quantized values (I8, packed U8 for INT4, or F8_E4M3/F8_E5M2).
• foo.weight_scale holds one F32 scale per group (flattened to 1-D, length = number of groups – see crate::quant::compute_groups for how elements map to groups).
• foo.weight_zero_point holds one I32 zero-point per group, only for asymmetric schemes (symmetric schemes have an implicit zero-point of 0 and omit this tensor).
• A quantization_config JSON blob in the file’s __metadata__ header describes the scheme (config_groups), which tensors were left unquantized (ignore), and – since this crate’s INT4 packing (see crate::quant::pack4) doesn’t match compressed-tensors’ own int32-based pack-quantized layout – a teenygrad_packed_int4 extension recording each packed tensor’s true logical shape.

Tensors are only quantized if they’re rank >= 2 (see should_quantize) – 1-D tensors (biases, norm weights) are passed through unchanged and listed in ignore, matching common PTQ tooling’s default of leaving those alone.

Structs

ConfigGroup

One compressed-tensors config group: a scheme plus the module types it applies to.

PackedTensorInfo

Logical shape of a nibble-packed INT4 tensor, since the packed U8 tensor’s own shape ([ceil(n / 2)]) doesn’t reflect it. See crate::quant::pack4.

QuantizationConfig

The full quantization_config metadata blob embedded in the output .safetensors header.

WeightsConfig

The compressed-tensors weights block for one scheme.

Constants

QUANTIZATION_CONFIG_KEY

The metadata key quantization_config is stored under in the .safetensors header.

Functions

build_config

Assembles the full QuantizationConfig for a checkpoint quantized uniformly with scheme/granularity.

config_from_metadata

Parses a quantization_config blob previously produced by config_to_metadata.

config_to_metadata

Serializes config into the .safetensors string-metadata map under QUANTIZATION_CONFIG_KEY.

dequantize_tensor

Dequantizes a tensor previously written by quantize_tensor, given its already-decoded scale (and, for asymmetric schemes, zero-point) values. Used by crate::validate.

quantize_tensor

Quantizes one tensor per scheme/granularity, returning the output tensors to insert (quantized weight, _scale, and – for asymmetric schemes – _zero_point), keyed by their final names. name is the original (unsuffixed) tensor name.

scale_tensor_name

foo.weight -> foo.weight_scale.

should_quantize

Whether a tensor should be quantized: rank >= 2 (so biases/norm weights are left alone) and a dtype crate::read::read_f32 can upcast from.

zero_point_tensor_name

foo.weight -> foo.weight_zero_point.