RAG Powered Doc QnA & Semantic Caching with Gemini AI

Subsequent, let’s construct our semantic cache service for storing LLM responses and utilizing them as responses for comparable queries. Beneath is the code for the CustomGPTCache software:

from typing import Checklist
import config as cfg

from langchain.schema import Doc
from langchain.vectorstores.deeplake import DeepLake
from langchain.embeddings.sentence_transformer import SentenceTransformerEmbeddings


class CustomGPTCache:
    def __init__(self) -> None:
        # Initialize the embeddings mannequin and cache vector retailer
        self.embeddings = SentenceTransformerEmbeddings(
            model_name="all-MiniLM-L12-v2"
        )
        self.response_cache_store = DeepLake(
            dataset_path="database/cache_vectorstore",
            embedding=self.embeddings,
            read_only=False,
            num_workers=4,
            verbose=False,
        )

    def cache_query_response(self, question: str, response: str):
        # Create a Doc object utilizing question because the content material and it is 
        # response as metadata
        doc = Doc(
            page_content=question,
            metadata={"response": response},
        )

        # Insert the Doc object into cache vectorstore
        _ = self.response_cache_store.add_documents(paperwork=[doc])

    def find_similar_query_response(self, question: str, threshold: int):
        attempt:
            # Discover comparable question primarily based on the enter question
            sim_response = self.response_cache_store.similarity_search_with_score(
                question=question, okay=1
            )

            # Return the response from the fetched entry if it is rating is extra 
            # than threshold
            return [
                {
                    "response": res[0].metadata["response"],
                }
                for res in sim_response
                if res[1] > threshold
            ]
        besides Exception as e:
            increase Exception(e)

Let’s stroll by way of how we’re constructing the cache system. We’re utilizing the all-MiniLM-L12-v2 mannequin from Sentence Transformers for creating embeddings for the cache. The motivation for this strategy was that: for embeddings of upper dimension, we get the next vary of similarity rating between dissimilar queries which leads to mis-triggering of the cache.

From intensive analysis, I’ve noticed that utilizing embeddings of decrease dimensions, say 256, we get a wider vary of similarity scores between 0.3 to 1.0. Coming again to the code, we initialize the embeddings utilizing Langchain’s SentenceTransformerEmbeddings by specifying the mannequin identify. Subsequent, we initialize the Deeplake vector retailer the place the question and responses can be saved.

Notice that when initializing cache vector retailer we don’t set the overwrite parameter to True as that might be counter-intuitive. We’ll add two strategies to the CustomGPTCache class: cache_query_response and find_similar_query_response. Let’s talk about about them.

1. cache_query_response: This technique is known as within the QAChain when an LLM response is generated. A Doc object is created with the question within the page_content and the response within the metadata area. Then, that Doc object is added to the vector retailer.

2. find_similar_query_response: This technique is known as within the QAChain for each run. It returns an inventory of dictionaries containing responses. The person question is handed into this technique for each run. It makes use of the question to do a similarity search on the cache vector retailer to search out essentially the most comparable question that’s cached. We solely fetch one comparable entry from the vector retailer. Then we create an inventory of these responses and put a threshold on the similarity rating to forestall returning irrelevant responses.

Step 4: Doc Questions and Answering Utilizing QAChain

Lastly, we are going to have a look at how QAChain makes use of the cache vector retailer and docstore to generate responses to person queries. We initialize Gemini embeddings and Gemini mannequin within the class constructor utilizing the required parameters like mannequin identify, API key, and many others. We additionally initialize the cache right here which can be used within the different strategies. Then, we are going to add two strategies to the QAChain class for dealing with person queries: generate_response and ask_question.

1. generate_response: This technique handles the LLM response a part of the QAChain. It merely takes a person question and generates a response for the question utilizing Langchain’s RetrievalQA Chain. After the response is generated, it’s cached utilizing the cache_query_response technique of CustomGPTCache, and the response is returned.

2. ask_question: This technique is known as for each person response. First, the find_similar_query_response technique of CustomGPTCache is known as utilizing the person question. If it returns an inventory of responses, the response is returned, in any other case, the generate_response technique is known as utilizing the person question. Moreover, the generate_response technique is known as if any exception is raised from the cache aspect. Beneath is the codebase for the QAChain pipeline for reference.

import config as cfg

from src.cache import CustomGPTCache
from langchain.chains import RetrievalQA
from langchain.prompts import PromptTemplate
from langchain.vectorstores.deeplake import DeepLake
from langchain_google_genai import (
    GoogleGenerativeAIEmbeddings, 
    ChatGoogleGenerativeAI
)


class QAChain:
    def __init__(self) -> None:
        # Initialize Gemini Embeddings
        self.embeddings = GoogleGenerativeAIEmbeddings(
            mannequin="fashions/embedding-001",
            google_api_key=cfg.GOOGLE_API_KEY,
            task_type="retrieval_query",
        )

        # Initialize Gemini Chat mannequin
        self.mannequin = ChatGoogleGenerativeAI(
            mannequin="gemini-pro",
            temperature=0.3,
            google_api_key=cfg.GOOGLE_API_KEY,
            convert_system_message_to_human=True,
        )

        # Initialize GPT Cache
        self.cache = CustomGPTCache()
        self.text_vectorstore = None
        self.text_retriever = None

    def ask_question(self, question):
        attempt:
            # Seek for comparable question response in cache
            cached_response = self.cache.find_similar_query_response(
                question=question, threshold=cfg.CACHE_THRESHOLD
            )

            # If comparable question response is current,vreturn it
            if len(cached_response) > 0:
                print("Utilizing cache")
                consequence = cached_response[0]["response"]
            # Else generate response for the question
            else:
                print("Producing response")
                consequence = self.generate_response(question=question)
        besides Exception as _:
            print("Exception raised. Producing response.")
            consequence = self.generate_response(question=question)

        return consequence

    def generate_response(self, question: str):
        # Initialize the vectorstore and retriever object
        vstore = DeepLake(
            dataset_path="database/text_vectorstore",
            embedding=self.embeddings,
            read_only=True,
            num_workers=4,
            verbose=False,
        )
        retriever = vstore.as_retriever(search_type="similarity")
        retriever.search_kwargs["distance_metric"] = "cos"
        retriever.search_kwargs["fetch_k"] = 20
        retriever.search_kwargs["k"] = 15

        # Write immediate to information the LLM to generate response
        prompt_template = """
        <YOUR PROMPT HERE>
        Context: {context}
        Query: {query}

        Reply:
        """

        PROMPT = PromptTemplate(
            template=prompt_template, input_variables=["context", "question"]
        )

        chain_type_kwargs = {"immediate": PROMPT}

        # Create Retrieval QA chain
        qa = RetrievalQA.from_chain_type(
            llm=self.mannequin,
            retriever=retriever,
            verbose=False,
            chain_type_kwargs=chain_type_kwargs,
        )

        # Run the QA chain and retailer the response in cache
        consequence = qa({"question": question})["result"]
        self.cache.cache_query_response(question=question, response=consequence)
        
        return consequence

Now that we have now your entire pipeline prepared, let’s use them to check our software. Step one is to ingest the doc. We initialize the Ingestion object and name the ingest_document technique utilizing the file path for the doc that’s to be saved.

from src.ingestion import Ingestion

ingestion = Ingestion()

file = "Apple 10k.pdf"

ingestion.ingest_documents(
    file=file
)

After the doc is ingested, we are going to initialize the QAChain object. Then we name the ask_question technique utilizing the person question to generate a response.

from src.qachain import QAChain

qna = QAChain()

%%time

question = "What have been the highlights for 2nd quarter of FY2023?"

outcomes = qna.ask_question(
    question=question
)

print(outcomes)

# OUTPUT:
# Producing response
#         The highlights for the second quarter of FY2023 have been:
#         •	MacBook Professional 14”, MacBook Professional 16” and Mac mini; and
#         •	Second-generation HomePod.
# CPU instances: whole: 2.94 s
# Wall time: 11.4 s

%%time

question = "Second quarter highlights, FY2023."

outcomes = qna.ask_question(
    question=question
)

print(outcomes)

# OUTPUT:
# Utilizing cache
#         The highlights for the second quarter of FY2023 have been:
#         •	MacBook Professional 14”, MacBook Professional 16” and Mac mini; and
#         •	Second-generation HomePod.
# CPU instances: whole: 46.9 ms
# Wall time: 32.8 ms

Utility Efficiency and Limitations

There are a number of RAG analysis libraries that can be utilized to judge the efficiency of the applying. The 2 hottest ones are RAGAS and Tonic Validate Metrics. Each supply comparable metrics like Reply similarity rating, Retrieval precision, Augmentation precision, Augmentation accuracy/relevance, and Retrieval k-recall, which assist consider the applying efficiency. Though few exceptions are dealt with within the above codebases, there are just a few instances the place the applying may crash. The PDFReader presently has no examine on the enter file kind which must be dealt with.

This software is extensively reusable in a variety of enterprise use instances like medical, monetary, industrial, e-commerce, and many others. In medical industries, RAG pipelines will help in rapidly troubleshooting instrument failures, deciding the proper drugs for a medical situation, and many others. RAG pipelines will help reply queries from a big corpus of economic paperwork within the blink of an eye fixed. Their fast retrieval and answering capabilities make them excellent for query answering over massive datasets. RAG pipelines may also allow fast overview of earlier analysis developments of a subject and counsel additional analysis scope.

Whereas our present pipeline is prepared for use for any use case, there are specific limitations. Gemini professional mannequin presently has a context size of 32k which could not be appropriate for giant information bases. An alternate suggestion could be to make use of the Gemini Professional 1.5 mannequin which helps a context size of as much as 1 million. The present pipeline has no chat reminiscence, so, the mannequin won’t have the context to earlier conversations. Langchain offers a number of reminiscence choices (Reminiscence Docs) that may be built-in into the pipeline effortlessly.

Code Reusability

• The Ingestion pipeline code could be reused to construct ingestion methods for different functions that require doc storage and retrieval.
• The PDF Reader software could be repurposed for different functions that require processing and metadata extraction from PDF paperwork.
• The CustomGPTCache code can be utilized in different initiatives to implement environment friendly caching and retrieval of LLM responses or comparable use instances.
• The QAChain class could be tailored for different question-answering methods that use doc shops and caching mechanisms.

Conclusion

So, people, that’s how one can talk along with your paperwork by way of RAG-powered query answering and semantic caching utilizing Gemini Professional and Langchain!

On this article, we’ve developed a pipeline that showcases a complete strategy to dealing with paperwork for clever QnA. By integrating Google’s Gemini embeddings, a customized PDF reader, and semantic caching, this RAG system maintains effectivity and accuracy in offering solutions to person queries. Furthermore, the codebase is extremely adaptable scalable, and dependable for quite a lot of QnA use instances with minimal code adjustments!

Key Takeaways

• The RAG system includes constructing an Ingestion pipeline to add and retailer paperwork for question-answering.
• The Ingestion pipeline makes use of Gemini embeddings for doc embeddings and a customized PDF reader for metadata extraction.
• A customized semantic cache service is carried out to retailer and retrieve LLM responses for comparable queries effectively.
• The QAChain class is answerable for producing responses to person queries and using the cache and vector retailer.