PDF Bot with Ollama

A bot that accepts PDF docs and lets you ask questions on it.

The LLMs are downloaded and served via Ollama.

Table of Contents

• Requirements
• How to run
• Demo
• Improvements
• Contributing
• Credits

Requirements

How to run

CPU version

make start

GPU version

make start-gpu

When the server is up and running, access the app at: http://localhost:8501

Switch to a different model by changing the MODEL env variable in the docker-compose.yaml. Check out the available models from here.

Note:

• It takes a while to start up since it downloads the specified model for the first time.
• If your hardware does not have a GPU and you choose to run only on CPU, expect high response time from the bot.
• Only Nvidia is supported as mentioned in Ollama's documentation. Others such as AMD isn't supported yet. Read how to use GPU on Ollama container and docker-compose.
• Make sure to have Nvidia drivers setup on your execution environment for the best results.

Image on DockerHub: https://hub.docker.com/r/amithkoujalgi/pdf-bot

Demo

PDF.Bot.Demo.mp4

Sample PDFs

Hl-L2351DW v0522.pdf

HL-B2080DW v0522.pdf

Improvements

• Expose model params such as temperature, top_k, top_p as configurable env vars

Contributing

Contributions are most welcome! Whether it's reporting a bug, proposing an enhancement, or helping with code - any sort of contribution is much appreciated.

Requirements

Setup Ollama server for development

docker run -it -v ~/ollama:/root/.ollama -p 11434:11434 ollama/ollama

Install the libs

pip install -r requirements.txt

Start the app

streamlit run pdf_bot/app.py

Credits

Thanks to the incredible Ollama, Langchain and Streamlit projects.