Struct DynamicCache 

pub struct DynamicCache {}

Tracks per-layer KV cache state (sequence length, sliding-window/compileability) across generation steps.

Implementations

impl DynamicCache

pub fn new() -> Self

Creates an empty cache.

pub fn get_sequence_length(&self) -> usize

Returns the number of tokens currently cached.

Not yet implemented.

pub fn is_sliding(&self) -> Option<&[(usize, bool)]>

Per-layer sliding-window state: (window_size, is_sliding) for each layer, or None if no layer uses a sliding window.

Not yet implemented.

pub fn is_compileable(&self) -> bool

Whether this cache’s current state is safe to pass into a compiled (AOT/JIT) kernel path.

Not yet implemented.

Trait Implementations

impl Clone for DynamicCache

fn clone(&self) -> DynamicCache

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DynamicCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DynamicCache

fn default() -> DynamicCache

Returns the “default value” for a type.