What are model parameters?
Parameters are the numbers inside a model that training adjusts. In a neural network they are the weights, which scale the connections between units, and the biases, which shift each unit’s output. Before training they are random; after training they hold everything the model has learned. A linear regression with ten input features has eleven parameters. A modern language model has billions, arranged in large matrices inside each layer.
Parameters are distinct from hyperparameters, which are the settings a person chooses before training and which the model does not learn. The learned parameters, saved to disk, are the model weights that a vendor distributes or a company keeps.
Why parameter count matters
Parameter count is the standard shorthand for model size. A “7B” model has about seven billion parameters; a “70B” model has about seventy billion. More parameters generally mean more capacity to store knowledge and handle subtle instructions, but also more memory, slower generation, and higher serving cost. Each parameter occupies two bytes at the 16-bit precision most models are trained in, so a seven-billion-parameter model needs roughly fourteen gigabytes of memory on a graphics processing unit (GPU) for its weights alone, before the working memory that inference requires. Quantization reduces that by storing parameters in fewer bits.
Parameter count is an imperfect guide to quality. Training data quality and quantity, architecture, and post-training matter as much, and a newer small model often outperforms an older large one. Mixture of experts models complicate the count further: a model may have hundreds of billions of parameters in total but use only a fraction of them for any given token, so its speed resembles a much smaller model while its memory footprint does not.
Parameters in practice
A specialty pharmacy wants an on-site assistant that answers staff questions about its own procedures without sending patient information outside the building. The IT lead compares a seven-billion-parameter model that runs on a workstation GPU already in the server room with a seventy-billion-parameter model that would need two data center GPUs. On the pharmacy’s own test questions, the smaller model with retrieval over the procedure manuals answers nearly as well, and the hardware decision is made on that evidence rather than on the parameter count.
What to watch for
- Ask for active parameters as well as total parameters when a vendor quotes a mixture of experts model.
- Translate parameter count into GPU memory before committing to hardware, including room for the KV (key-value) cache.
- Judge models on your own evaluation set, since parameter count does not predict performance on a specific task.