Foundation model

A foundation model is a large model that first goes through expensive pre-training on very broad and varied training data, and only then is adapted to narrower tasks. The name captures the role: it is the foundation on which many different applications are built, instead of training a separate model for each problem from scratch. Underneath, it almost always relies on deep learning over networks with many layers.

It is easy to confuse this concept with an LLM. An LLM is one specific type of foundation model — one specialized in language — whereas foundation models also include architectures that work on images, audio or mixed data. In other words, every large language model is a foundation model, but not every foundation model is a language model. A small language model, in turn, is a smaller, lighter variant of a language model, designed with lower cost and operation close to the data in mind, rather than a broad, universal base.

In practice, a company rarely trains a foundation model on its own — that requires enormous datasets and compute. Instead, it takes a ready-made base and adapts it to its needs through fine-tuning or suitable prompts. As a result, a single well-chosen base model can serve many processes — from customer service to document analysis — at a far lower cost than building solutions from the ground up.