What are embeddings?
Embeddings are lists of numbers, called vectors, that represent the meaning of a piece of data. An embedding model reads a sentence, a document chunk, a product description, or an image and produces a vector of several hundred to a few thousand numbers. Items with similar meaning produce vectors that are close together, measured by a distance such as cosine similarity. “Invoice overdue by 30 days” and “payment past due for a month” land near each other even though they share no words.
Embeddings turn meaning into geometry that software can measure. Once text is a point in a space, a computer can find the nearest points, group points into clusters, and measure how far apart two items are. That is the foundation of semantic search, retrieval-augmented generation (RAG), recommendation systems, and much of modern natural language processing.
How embeddings work
An embedding model is a neural network, usually a transformer, trained so that related inputs map to nearby vectors. Training pairs come from sources such as questions and their answers, duplicate posts, or captions and their images. The model learns to pull matching pairs together and push unrelated pairs apart. After training, the network is frozen and used as a function: data in, vector out.
The number of dimensions is a design choice. More dimensions capture finer distinctions but cost more to store and compare. Some models support several output sizes from one model. The embedding model used to index documents must be the same one used to embed queries, because vectors from different models are not comparable.
Types of embeddings
- Text embeddings cover words, sentences, and passages and are the workhorse of search and RAG.
- Image and multimodal embeddings place pictures and their descriptions in the same space, so a text query can retrieve images.
- Sparse embeddings keep a weight per vocabulary term and behave like an improved keyword index, which is why hybrid search pairs them with dense vectors.
- Domain-specific embeddings are trained or fine-tuned on legal, medical, financial, or code text where general models misread the vocabulary.
Embeddings in practice
A retailer with a catalog assembled from a dozen acquired brands had thousands of near-duplicate products listed under different names. Embedding every product description and clustering the vectors surfaced groups of items that were the same product with different wording, which merchandisers then merged. The same embeddings now drive the storefront search, so a shopper who types “waterproof jacket for hiking” finds rain shells whose listings never used the word waterproof.
A managed service provider used embeddings of ticket descriptions to cluster a year of help desk tickets. Three clusters accounted for a large share of volume and pointed to a single misconfigured email policy, which the team fixed.
What to watch for
- Changing the embedding model requires re-embedding every stored item, so treat the choice as a schema decision and record the model version with each vector.
- General-purpose models can miss domain vocabulary such as part numbers, drug names, or legal terms of art, so test retrieval quality on your own documents.
- Embeddings of confidential text are confidential data, because the original content can often be approximately reconstructed from the vector.
- Embeddings capture similarity of meaning rather than truth, so two passages that contradict each other on the same topic will sit close together.
- Store metadata such as source, date, and permissions alongside each vector so retrieval can be filtered before the model sees anything.