What is an epoch?
An epoch is one full pass over the training data, with every example shown to the model once. Training data is split into batches, the model updates its parameters after each batch, and when the last batch has been processed one epoch is complete. With fifty thousand examples and a batch size of fifty, an epoch is a thousand updates. Training usually runs for several epochs, reshuffling the data each time so the batches differ.
The number of epochs decides how long the model keeps learning from the same examples. Too few and the model is underfit, having not yet captured the patterns. Too many and it begins to memorize individual examples, which is overfitting, and performance on new data gets worse even as training loss keeps falling. Early stopping watches the validation loss after each epoch and halts training when it stops improving.
Epoch in practice
A hospital’s billing team fine-tunes a language model to suggest procedure codes from clinical notes. The dataset has eight thousand labeled notes. The team trains for five epochs and checks the validation loss after each one. It falls through epoch three, then rises. The checkpoint from epoch three is kept and the later ones are discarded. Pre-training a large language model, by contrast, often uses less than one epoch, because the dataset is so large that the model never needs to see most of it twice.