Chapter 3: Reranking and Filtering

What is Reranking?

Reranking uses a more powerful model to re-score initial retrieval results:

Cross-Encoder Reranking

Cross-encoders score query-document pairs directly:

from sentence_transformers import CrossEncoder

# Load reranker
reranker = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')

# Initial retrieval
candidates = retriever.get_relevant_documents(query)

# Score pairs
pairs = [(query, doc.page_content) for doc in candidates]
scores = reranker.predict(pairs)

# Sort by reranked scores
reranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)

Cohere Rerank

Production-ready reranking API:

import cohere

co = cohere.Client("your-api-key")

# Rerank documents
results = co.rerank(
    model="rerank-english-v3.0",
    query=query,
    documents=[doc.page_content for doc in candidates],
    top_n=5
)

# Get reranked documents
for result in results.results:
    print(f"Score: {result.relevance_score}, Index: {result.index}")

Metadata Filtering

Filter before or after retrieval:

# Pre-retrieval filtering (more efficient)
retriever = vectorstore.as_retriever(
    search_kwargs={
        "k": 10,
        "filter": {
            "department": "HR",
            "year": {"$gte": 2023}
        }
    }
)

# Post-retrieval filtering
results = retriever.get_relevant_documents(query)
filtered = [doc for doc in results if doc.metadata.get('verified')]

Complete Retrieval Pipeline

def advanced_retrieve(query, vectorstore, reranker, k=5):
    # Step 1: Initial retrieval (more candidates)
    candidates = vectorstore.similarity_search(query, k=50)
    
    # Step 2: Rerank
    pairs = [(query, doc.page_content) for doc in candidates]
    scores = reranker.predict(pairs)
    
    # Step 3: Sort and return top k
    ranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)
    return [doc for doc, score in ranked[:k]]

Comparison

Method	Latency	Quality	Cost
No reranking	Low	Good	Free
Cross-encoder	Medium	Better	Self-host
Cohere Rerank	Medium	Best	API cost

---

Next Steps

Test your knowledge with the Module 5 Quiz!