In RAG, the more context(content) you provide📚, the more the LLM will be distracted😵, leading to less focused, less accurate outputs and overlooking key details🔑❌. On the other hand, the less context you provide, the less relevant the answer could be, leading to hallucinations🌀.

Here’s how to easily solve it:

You have to pass the most specific and relevant context as per the user query. Here using a reranker is an evergreen solution suggested by many.

Example:

When the user makes a query to your LLM App, the retriever finds relevant documents or passages from the huge dataset. But, not all of these documents are equally helpful. Thus, the reranker scores the retrieved documents to find and focus on those that will give the best response.

Suppose you selected top {k} documents from the vector DB, suppose k = 10. So, the reranker will then compare those 10 docs with the user query and will rank each doc. Then you can select top-ranked docs like top 3 docs🔝.

In summary, Reranker helps in selecting the most relevant pieces of information for the user query. As a result, the LLM responses become better enough to impress your boss👔.

Optional Info: Internally, The reranker uses a model, trained on a large set of Q&A data, to assess documents for features like keywords and semantic similarity to the query, thereby assigning each a relevance score.

If you want to delve more, the below studies collectively validate the crucial role of rerankers in refining the information retrieval process within RAG systems:

1. [2308.07107] Large Language Models for Information Retrieval: A Survey (arxiv.org)

2. Simple and Effective Retrieve-Edit-Rerank Text Generation - ACL Anthology

3. [2303.08559] Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples! (arxiv.org)

You can try any of these widely used rerankers like bge-reranker, mxbai-rerank, cross-encoder for your application. Code Example 💻:

That’s all. I post knowledge about making better Generative AI apps 🤖 every week using real-life examples💡. Connect if you have a growth mindset 🌱.

Please show your support by liking 👍, commenting 💬, and sharing🔄 .