What is reranking?
Reranking is a second stage in a retrieval pipeline that re-scores the candidates returned by the first search and puts the most relevant ones at the top. The first stage, whether vector search, keyword search, or hybrid search, is built for speed across millions of items and produces a rough ordering. The reranker is a slower, more accurate model, usually a cross-encoder, that reads the query and each candidate passage together and outputs a relevance score. Because it only sees a few dozen candidates, it can afford the extra computation.
Why reranking matters
A language model answers from the passages it is given, and its context window is finite. If the best passage sits in position twenty of the retrieved list, it may be cut off or buried among weaker material, and the answer suffers. Reranking raises the share of questions where the right evidence is in the first three to five positions, which is often the largest quality improvement available in a retrieval-augmented generation system once the chunking strategy is settled.
Reranking in practice
A credit union’s compliance team searches regulatory guidance and internal procedures. Vector search on a question about overdraft fee disclosures returned twenty passages that all mentioned overdrafts, with the actual disclosure requirement in position eleven. Adding a reranker moved it to position one. The team now passes only the top five reranked passages to the model, which shortened responses and reduced cost per question.