AI GlossaryBatch SizeENTRY — Neural Networks
[Neural Networks]
Batch Size.
The number of training examples used in one iteration of model training.
In-depth explanation
01Instead of updating weights after each example (stochastic) or after all examples (batch), mini-batch gradient descent updates after a fixed number of examples. Larger batches provide more stable gradients but require more memory; smaller batches add noise but can help escape local minima. Common sizes range from 16 to 256.
Examples
02EX. 01
Batch size of 32
EX. 02
Batch size of 128
More in Neural Networks
0301Activation FunctionA mathematical function that determines the output of a neuron based on its weighted input sum.02BackpropagationThe algorithm for calculating gradients of the loss function with respect to network weights.03EpochOne complete pass through the entire training dataset during model training.04Neural NetworkA computing system inspired by biological neural networks, consisting of interconnected nodes (neurons).05NeuronA basic computational unit in a neural network that receives inputs, applies weights and activation, and produces output.
[NEXT] — APPLY THE CONCEPT
Master Batch Size.
Learn how to apply this concept with hands-on projects in our comprehensive AI programs.