Module sigmoid 

Structs

LogsigmoidBackward

Backward: dx = dy * sigmoid(-x) = dy / (1 + exp(x))

LogsigmoidForward

Forward: y = log(sigmoid(x)) = -log(1 + exp(-x))

LogsigmoidForwardDispatch

Forward: y = log(sigmoid(x)) = -log(1 + exp(-x))

LogsigmoidOp

SigmoidBackward

Backward: dx = dy * y * (1 - y) = dy * (y - y²)

SigmoidForward

Forward: y = 1 / (1 + exp(-x))

SigmoidForwardDispatch

Forward: y = 1 / (1 + exp(-x))

SigmoidOp

SiluBackward

Backward: dx = dy * (sigmoid(x) + y * (1 - sigmoid(x))) = dy * (s + y - y*s) where s = sigmoid(x)

SiluForward

Forward: y = x * sigmoid(x)

SiluForwardDispatch

Forward: y = x * sigmoid(x)

SiluOp

Functions

logsigmoid_backward

Backward: dx = dy * sigmoid(-x) = dy / (1 + exp(x))

logsigmoid_forward

Forward: y = log(sigmoid(x)) = -log(1 + exp(-x))

sigmoid_backward

Backward: dx = dy * y * (1 - y) = dy * (y - y²)

sigmoid_forward

Forward: y = 1 / (1 + exp(-x))

silu_backward

Backward: dx = dy * (sigmoid(x) + y * (1 - sigmoid(x))) = dy * (s + y - y*s) where s = sigmoid(x)

silu_forward

Forward: y = x * sigmoid(x)