What is GPT?
GPT stands for Generative Pre-Trained Transformer, and each word in the name describes part of how the model works. It is generative because it produces new text one token at a time. It is pre-trained because it first learns general patterns of language from a very large body of text before any task-specific adjustment. It is a transformer because it uses the transformer neural network architecture, whose attention mechanism lets the model weigh every earlier token in the input when choosing the next one.
The term began as the name of a specific series of models from OpenAI, starting with the original GPT and followed by GPT-2, GPT-3, GPT-4, and later versions, each trained on more data with more parameters. The same recipe, pre-training followed by instruction tuning and alignment, is now the basis of most large language models, so “GPT” is often used loosely to mean any model of this kind. In precise usage it refers to OpenAI’s model series.
How GPT works
Training happens in stages. During pre-training the model reads enormous amounts of text and learns to predict each next token from the tokens before it, adjusting billions of model weights until its predictions are good. This stage costs the most compute and produces a base model that can continue any text but does not follow instructions well. Instruction tuning then teaches the base model to answer requests, and reinforcement learning from human feedback shapes its answers toward what people rate as helpful and safe. At inference time the model takes a prompt, splits it into tokens, and generates a response within its context window.
Why GPT matters
GPT-style models are what most companies mean when they talk about generative AI for text. The pre-training approach is why one model can draft an email, summarize a contract, write code, and classify support tickets without separate training for each. For a business the practical consequences are cost and dependence. A hosted GPT model is rented per token; an open-weight model built on the same principles can run on hardware the company controls. The choice between them depends on data sensitivity, volume, and how much of the capability the company wants to own.
What to watch for
- A product that says it is “built on GPT” is usually a thin layer over a hosted model, so ask what the vendor adds and where your data goes.
- Version names change often and capabilities differ between them, so contracts and evaluations should name the exact model.
- Pre-training gives a model broad knowledge with a knowledge cutoff; it knows nothing about your company until you supply that through retrieval-augmented generation or fine-tuning.