AI GlossaryActivation FunctionENTRY — Neural Networks
[Neural Networks]
Activation Function.
A mathematical function that determines the output of a neuron based on its weighted input sum.
In-depth explanation
01Activation functions introduce non-linearity, enabling neural networks to learn complex patterns. Without them, a network would just be a linear transformation. Common functions include ReLU (max(0, x)), sigmoid (1/(1+e^-x)), tanh, and softmax (for multi-class output). Choice of activation function affects training dynamics and model capabilities.
Examples
02EX. 01
ReLU in hidden layers
EX. 02
Softmax in classification output
More in Neural Networks
0301BackpropagationThe algorithm for calculating gradients of the loss function with respect to network weights.02Batch SizeThe number of training examples used in one iteration of model training.03EpochOne complete pass through the entire training dataset during model training.04Neural NetworkA computing system inspired by biological neural networks, consisting of interconnected nodes (neurons).05NeuronA basic computational unit in a neural network that receives inputs, applies weights and activation, and produces output.
[NEXT] — APPLY THE CONCEPT
Master Activation Function.
Learn how to apply this concept with hands-on projects in our comprehensive AI programs.