Expand description
Activation layers (ReLU, GELU, sigmoid, etc).
Modules§
- elu
- ELU and SELU.
- gelu
- GELU (exact and tanh-approximate).
- hard
- “Hard” (piecewise-linear) activation approximations: hardtanh, hardsigmoid, hardswish.
- misc
- Other activations that don’t fit the other modules.
- relu
- ReLU and leaky ReLU.
- sigmoid
- Sigmoid and related activations.
- softmax
- Softmax and log-softmax.
- tanh
- Tanh and related activations.