Cleric Call Logs Retrival Augmented Generation Agent

This is a call logs RAG agent implemented with LangChain and LLamaIndex. The application uses FewShotPromptTemplate and PromptTemplate to prompt LLM and create a list of facts extracted from the given documents. LLamaIndex plays the role to store data chunks to a VectorStore for the Agent to retrieve relevant information according to the question. You can test it here.

• Application URL [closed]
• API Endpoint [closed]
  • /submit_question_and_documents
  • /get_question_and_facts

Implemented Architecture

The figure below shows the architecture of my implementation. The core component is packaged as a container and hosted within GCP (Google Cloud Platform), the API endpoint is here. The UI is hosted with HuggingFace Spaces.

Run with Docker

Pull the image from this container repo, the latest version is v0.3, you can use latest as well:

docker pull mitchhsu/cleric-agent:<version> 

Run container, the port of the application is set at 8000:

docker run -p <host_port>:8000 mitchhsu/cleric-agent:<version>

To run the UI, just run:

python app/ui.py

Remember to install the dependencies by running pip install -r requirements.txt.

Implementation Details

• app/app.py: Implements the API using Python Flask, there are two endpoints provided, details of the schemas:

  • /submit_question_and_documents: Handles user submissions of documents and question. Respond "sucess" if submission was successful. Error handling are covered when request schema mismatch, URLs format error, document load error, and no logs found.

  • /get_question_and_facts: Handles user query of question and facts. Respond respective status of the query ("done", "processing", "No data found").

    The endpoint also creates a VectorStoreIndex to split data into chunks, then passes a retriever for the Agent to retrieve relevant information.

• app/agent.py: Implements the agent logic, including prompt engineering. The agent is responsible for constructing a response given the call logs. Here I use FewShotPromptTemplate and PromptTemplate to format the prompt pass to gpt.
• app/ui.py: Implements the user interface using gradio.
• app/utils.py: Implements helper functions, such as URL validation, document retrieving, etc.
• app/configs.py: Stores configurations such as prompt templates and model selection.
• app/models.py: Defines the pydantic models for API request and response.

Design Thinking Process

I initially developed a basic agent using a PromptTemplate, but I noticed random responses and lengthy statements, so I refined the prompt template and implemented a FewShotTemplate (3-shot) for the better structure (see here). In anticipating challenge where the call logs might be extensive, therefore affecting the agent's accuracy and increase computational demands. I leveraged LLamaIndex and VectorStoreIndex to manage data chunks and facilitate the retrieval of essential information for the agent's extraction process. And to preserve the order of the call logs, I sorted the retrieve data nodes with their original order to ensure chronological decision-making.

Here is the example from the validator: