What is the difference between a bi-encoder and a cross-encoder?

A bi-encoder processes the query and document independently, producing two vectors that are compared with cosine similarity or dot product, which makes it fast enough to index millions of documents offline. A cross-encoder processes the query and document together through a single transformer pass, producing a direct relevance score that captures fine-grained interactions between terms. Cross-encoders score much better but require a full model inference per pair, so they only work on a pre-filtered candidate set of dozens to hundreds of items, not millions.