AI Glossary
Neural network
neural network, artificial neural network
A neural network is a machine-learning model built from layers of connected units (neurons) that progressively transform the input data and learn relationships by tuning the connection weights during training.
- It consists of layers of neurons connected by weights.
- The weights are tuned during training to reduce prediction error.
- The transformer architecture that LLMs are built on is a kind of neural network.
A neural network processes data through successive layers, in which each neuron combines the signals from the previous layer with the weights assigned to them. Those weights are the model parameters, and tuning them during training lets the network model the relationships in the data ever more accurately.
The more layers, the deeper the network and the greater its ability to capture complex patterns — though its appetite for data and compute grows too. Modern language models are built on the transformer architecture, which is a particular type of neural network.
Related terms
Related articles