What is a reranker and when should you use one in RAG?

A reranker is a model that takes the top-k results from a first-stage retriever, typically 50 to 200 candidates, and reorders them by true relevance to the query. Rerankers are usually cross-encoders that deliver much higher precision than the bi-encoder used in first-stage retrieval, at the cost of higher latency per candidate. Use a reranker when precision at the top-3 or top-5 matters, such as in RAG where only a few chunks fit into the language model prompt. Typical lift is 10 to 30 percent on retrieval metrics.