What is a generative model?
A generative model is a machine learning model that learns the underlying distribution of its training data so it can produce new examples that resemble it. Where a discriminative model learns the boundary between classes, a generative model learns what the data itself looks like, which lets it generate new samples, complete partial ones, or score how plausible a given example is.
The main families are autoregressive models, which predict the next token and underlie large language models such as GPT; diffusion models, which learn to turn noise into images step by step; generative adversarial networks, in which a generator and a critic train against each other; and variational autoencoders, which compress data into a compact representation and decode it back. Generative AI products are applications built on these models.
Generative models in practice
Text and image generation get the attention, but generative models have quieter uses. A manufacturer trains one on thousands of images of good parts only. At inspection time, a part the model considers implausible is flagged for review, which catches defect types that nobody had labeled. The same idea supports synthetic data for testing, scenario simulation in planning, and anomaly detection in transaction streams.
What to watch for
- A generative model produces plausible output rather than verified output, which is the root of hallucination.
- Evaluation is harder than for a classifier, because there is no single right answer to score against.
- Check the provenance and license of training data, since the model can reproduce fragments of what it saw.