Struct LayerNorm 

pub struct LayerNorm<D: Dtype, IT, OT, const RANK: usize> {
    pub normalized_shape: Vec<usize>,
    pub eps: f64,
    pub affine: bool,
    /* private fields */
}

Layer normalisation over the last normalized_shape.len() dimensions.

Input shape: [*, normalized_shape[0], ..., normalized_shape[n-1]] Output shape: same as input.

Parameters:

• normalized_shape — the shape of the axes to normalise over (e.g. [C] for 2-D input [N, C], or [H, W] for spatial normalisation)
• eps — numerical stability constant (default 1e-5)
• affine — if true, learns per-element γ and β (default true)

Fields

normalized_shape: Vec<usize>

The shape of the trailing axes to normalize over.

eps: f64

Numerical stability constant added to the variance.

affine: bool

Whether to learn per-element scale (gamma) and shift (beta) parameters.

Implementations

impl<D: Dtype, IT, OT, const RANK: usize> LayerNorm<D, IT, OT, RANK>

pub fn new(normalized_shape: impl Into<Vec<usize>>) -> Self

Creates a new layer with default eps, affine on.

pub fn with_eps(self, eps: f64) -> Self

Sets the numerical stability constant.

pub fn with_affine(self, affine: bool) -> Self

Sets whether to learn per-element scale/shift parameters.

Trait Implementations

impl<D: Dtype, IT: Tensor<D, RANK> + EagerTensor, OT: Tensor<D, RANK>, const RANK: usize> Layer<IT> for LayerNorm<D, IT, OT, RANK>

type Output = OT

This layer’s output type.

fn call(&self, _input: IT) -> Self::Output

Applies this layer to input.

impl<D: Dtype, const RANK: usize> Layer<SymTensor> for LayerNorm<D, SymTensor, SymTensor, RANK>

type Output = SymTensor

This layer’s output type.

fn call(&self, input: SymTensor) -> SymTensor

Applies this layer to input.