What is a benchmark?
A benchmark is a fixed collection of test items, each with a known correct answer or a scoring rule, that is run against many models so their results can be compared. In classic machine learning a benchmark might be a labeled image dataset such as ImageNet with accuracy as the score. For large language models, benchmarks cover multiple-choice knowledge questions, grade school and competition math, code generation checked by running unit tests, reading comprehension, instruction following, and tool use. Vendors report scores in model cards and announcements, and public leaderboards aggregate them.
A benchmark score measures a model on that benchmark’s distribution of questions, under that benchmark’s prompting and scoring choices. It is useful as a first filter when choosing which models to evaluate, and it is unreliable as the only basis for a decision.
Why benchmarks matter
Benchmarks let a team shortlist models without testing every one in depth, and they track progress across model generations. They also drive the market: vendors tune for them, which is why a few points of difference on a popular leaderboard rarely translate to a noticeable difference in a business application. Two problems recur. Contamination occurs when benchmark questions leak into a model’s training data, a form of information leakage that inflates scores. Saturation occurs when most current models score near the top, so the benchmark no longer separates them.
Benchmark in practice
A medical device manufacturer wants an assistant that answers engineers’ questions from its quality management documents. Public benchmarks narrow the field to a handful of open-weight models that fit the available GPU memory. The team then builds its own benchmark: three hundred real questions collected from engineers, each with an answer and the source passage verified by the quality manager. Every candidate model is run against this set with the same retrieval pipeline, and the scores are compared with an LLM-as-a-judge check and a human review of the disagreements. The model that leads the public leaderboards finishes second on the company’s own questions.
What to watch for
- Ask which version of a benchmark a score comes from, how many attempts the model was allowed, and whether the prompt format matched other vendors’ reports.
- Build a private test set from real tasks in your business and treat it like test data, kept out of any prompts or fine-tuning.
- Re-run your benchmark whenever a hosted vendor updates a model behind the same name, because capability can shift without notice.
- Weigh latency, cost per task, and context window alongside the quality score, since the best-scoring model is often not the most practical one.