First of all, congrats on your great work, the code is incredibly light and powerful!!
Wel, about the models, to use another AI model is very simple, change these lines on app.py.
Select model
Change this:
from langchain.chat_models import ChatOpenAI
For this:
from langchain.chat_models import ChatOllama
Using the model
Change this:
llm=ChatOpenAI(temperature=0.2)
For this:
# Keep the temperature is important because this parameter will make your model more assertive about the documents, and don't start to create information that doesn't exist
# The temperature of the model. Increasing the temperature will make the model answer more creatively. (Default: 0.8)
llm=ChatOllama(
temperature=0.2,
model="llama3.2:1b"
)
Pls see below a list of possible models to use:
| Model |
Parameters |
Size |
Download |
| DeepSeek-R1 |
7B |
4.7GB |
ollama run deepseek-r1 |
| DeepSeek-R1 |
671B |
404GB |
ollama run deepseek-r1:671b |
| Llama 3.3 |
70B |
43GB |
ollama run llama3.3 |
| Llama 3.2 |
3B |
2.0GB |
ollama run llama3.2 |
| Llama 3.2 |
1B |
1.3GB |
ollama run llama3.2:1b |
| Llama 3.2 Vision |
11B |
7.9GB |
ollama run llama3.2-vision |
| Llama 3.2 Vision |
90B |
55GB |
ollama run llama3.2-vision:90b |
| Llama 3.1 |
8B |
4.7GB |
ollama run llama3.1 |
| Llama 3.1 |
405B |
231GB |
ollama run llama3.1:405b |
| Phi 4 |
14B |
9.1GB |
ollama run phi4 |
| Phi 3 Mini |
3.8B |
2.3GB |
ollama run phi3 |
| Gemma 2 |
2B |
1.6GB |
ollama run gemma2:2b |
| Gemma 2 |
9B |
5.5GB |
ollama run gemma2 |
| Gemma 2 |
27B |
16GB |
ollama run gemma2:27b |
| Mistral |
7B |
4.1GB |
ollama run mistral |
| Moondream 2 |
1.4B |
829MB |
ollama run moondream |
| Neural Chat |
7B |
4.1GB |
ollama run neural-chat |
| Starling |
7B |
4.1GB |
ollama run starling-lm |
| Code Llama |
7B |
3.8GB |
ollama run codellama |
| Llama 2 Uncensored |
7B |
3.8GB |
ollama run llama2-uncensored |
| LLaVA |
7B |
4.5GB |
ollama run llava |
| Solar |
10.7B |
6.1GB |
ollama run solar |
First of all, congrats on your great work, the code is incredibly light and powerful!!
Wel, about the models, to use another AI model is very simple, change these lines on app.py.
Select model
Change this:
For this:
Using the model
Change this:
For this:
Pls see below a list of possible models to use:
ollama run deepseek-r1ollama run deepseek-r1:671bollama run llama3.3ollama run llama3.2ollama run llama3.2:1bollama run llama3.2-visionollama run llama3.2-vision:90bollama run llama3.1ollama run llama3.1:405bollama run phi4ollama run phi3ollama run gemma2:2bollama run gemma2ollama run gemma2:27bollama run mistralollama run moondreamollama run neural-chatollama run starling-lmollama run codellamaollama run llama2-uncensoredollama run llavaollama run solar