Skip to main content

cross_entropy_loss_backward

teeny_kernels::nn::loss::nll

Function cross_entropy_loss_backward 

Source 
pub fn cross_entropy_loss_backward<T: Triton, const BLOCK_SIZE: i32>(
    dy_ptr: T::Pointer<f32>,
    input_ptr: T::Pointer<f32>,
    targets_ptr: T::Pointer<i32>,
    dx_ptr: T::Pointer<f32>,
    _n_rows: i32,
    n_cols: i32,
)
where
    T::I32Tensor: Tensor<i32, 1> + Comparison<i32, BoolTensor = T::BoolTensor>,
    T::Pointer<f32>: AddOffsets<i32, 1, T::I32Tensor, Output = T::Tensor<T::Pointer<f32>>> + AddOffsets<i32, 1, T::Tensor<i32>, Output = T::Tensor<T::Pointer<f32>>>,
    T::Pointer<i32>: AddOffsets<i32, 1, T::I32Tensor, Output = T::Tensor<T::Pointer<i32>>>,
    T::Tensor<i32>: Tensor<i32, 1>,
Expand description

Cross-entropy loss backward.

dx[n, c] = dy[n] * (softmax(x[n])[c] - indicator(c == target[n]))

Grid: [n_rows, 1, 1] — one CTA per row. BLOCK_SIZE must equal next_power_of_two(n_cols).