Activation Functions
SiLU
Sigmoid Linear Unit
SiLU(x) = x * sigmoid(x) = x * (1 / (1 + e^(-x)))GELU
Gaussian Error Linear Unit
GELU(x) = x * Φ(x)Sigmoid Linear Unit
SiLU(x) = x * sigmoid(x) = x * (1 / (1 + e^(-x)))Gaussian Error Linear Unit
GELU(x) = x * Φ(x)