Small language model (SLM)

A small language model (Small Language Model, SLM) is a compact variant of a large language model, designed so that it can run on modest hardware — a single graphics card, a company server, sometimes even a laptop. The line between an SLM and an LLM is a matter of convention and shifts over time, but the idea is models with one or two orders of magnitude fewer parameters than the largest systems.

Unlike a large model, which prioritizes broad general knowledge and the ability to handle arbitrary tasks, a small model gives up some of that versatility in exchange for low cost, speed and the ability to operate without an external API. Its effectiveness is boosted further by fine-tuning on a specific company's data and by quantization, which reduces hardware requirements even more.

From a deployment perspective, an SLM is the natural choice wherever data privacy, predictable inference cost and independence from a vendor matter. It works well for narrow, repeatable tasks — classifying documents, extracting data, handling routine queries — where broad general knowledge is not needed and the priorities are control and cost across a high volume of calls.