AI Glossary
Overfitting
overfitting, over-learning, over-training
Overfitting is when a model memorizes its training data instead of learning general patterns, so it performs excellently on familiar data but poorly on new, previously unseen examples.
- The model learns the specifics and noise of the training set rather than patterns that generalize.
- Symptom: high accuracy on training data alongside markedly weaker accuracy on test data.
- A common risk when fine-tuning on a small dataset — eased by larger data and a separate validation set.
Overfitting is one of the fundamental problems in machine learning. It arises when, instead of capturing general rules, a model starts to memorize specific examples from the training set — including their incidental features and noise. The result is a model that reproduces its training data almost flawlessly but gets lost on new data, because it learned the answers by heart rather than a pattern that transfers to other cases.
The warning sign is a large gap between accuracy on the training data and accuracy on a separate test set. That is why model evaluation is done on data the model did not see during training, with progress tracked on a held-out validation set. Misleading benchmarks can hide overfitting when the test set leaks into the training data — in that case a high score does not reflect any real ability to generalize.
In deployment, the risk of overfitting grows especially when fine-tuning a model on a small, narrow set of examples. The model can then learn the style of a few dozen samples so literally that it stops handling variants outside them. You counter this with larger and more varied data, fewer training steps, and testing on real-world cases rather than only on data resembling the training set.
Related terms