This project is a Retrieval-Augmented Generation (RAG) based Command Line Interface (CLI) chatbot built using the LlamaIndex framework. The chatbot interacts with documents, uses the Qdrant vector database for document indexing, and leverages the Llama 3.1 language model provided by Ollama framework for generating responses. The chatbot system is designed to remember previous conversations within a session.
-
Clone the repository:
git clone https://github.com/SuramyaP/RAG-based-CLI-Chatbot.git -
Install the required dependencies:
pip install -r requirements.txt -
Pull Qdrant Docker Image and Run the Container.
Pull:
docker run -p 6333:6333 qdrant/qdrantRun:
docker run -p 6333:6333 qdrant/qdrant -
Install and Run Ollama
Install Ollama by following the instructions in their README.
Then, run Ollama by:
ollama run llama3.1
Run the code by:
python main.py
The program currently acts as a chatbot for a book called "Girl Meets Boy" by Derek Strange. To add your documents, go to the documents folder and add your pdf files.
The BAAI/bge-base-en-v1.5 model of Huggingface has been used to convert documents into vector representations suitable for indexing and retrieval. Likewise, llama 3.1 has been used for generating responses. If you want to change the model, go to chatbot.py and update the model parameter in line llm = Ollama(model="llama3.1:latest", request_timeout=1500.0) with any other model available in Ollama framework. (You can check their README to take a look at all the models available.) Also, feel free to tinker with the request_timeout parameter which has currently been set to 1500 seconds.
At times, the bot may respond with "Empty Response" in which case, try asking the question again. Likewise, at times the bot may take a little long to respond, in which case please be a little patient.
- Thanks to the LlamaIndex framework and Qdrant for providing the tools that make this project possible.
- Special thanks to the developers and contributors of HuggingFace and Ollama for their incredible models.