Skip to content

NOT ISSUE, how to run with another AI models #10

Description

@jfbvm

First of all, congrats on your great work, the code is incredibly light and powerful!!

Wel, about the models, to use another AI model is very simple, change these lines on app.py.

Select model
Change this:

from langchain.chat_models import ChatOpenAI

For this:

from langchain.chat_models import ChatOllama

Using the model
Change this:

llm=ChatOpenAI(temperature=0.2)

For this:

# Keep the temperature is important because this parameter will make your model more assertive about the documents, and don't start to create information that doesn't exist
# The temperature of the model. Increasing the temperature will make the model answer more creatively. (Default: 0.8)
llm=ChatOllama(
    temperature=0.2,
    model="llama3.2:1b"
)

Pls see below a list of possible models to use:

Model Parameters Size Download
DeepSeek-R1 7B 4.7GB ollama run deepseek-r1
DeepSeek-R1 671B 404GB ollama run deepseek-r1:671b
Llama 3.3 70B 43GB ollama run llama3.3
Llama 3.2 3B 2.0GB ollama run llama3.2
Llama 3.2 1B 1.3GB ollama run llama3.2:1b
Llama 3.2 Vision 11B 7.9GB ollama run llama3.2-vision
Llama 3.2 Vision 90B 55GB ollama run llama3.2-vision:90b
Llama 3.1 8B 4.7GB ollama run llama3.1
Llama 3.1 405B 231GB ollama run llama3.1:405b
Phi 4 14B 9.1GB ollama run phi4
Phi 3 Mini 3.8B 2.3GB ollama run phi3
Gemma 2 2B 1.6GB ollama run gemma2:2b
Gemma 2 9B 5.5GB ollama run gemma2
Gemma 2 27B 16GB ollama run gemma2:27b
Mistral 7B 4.1GB ollama run mistral
Moondream 2 1.4B 829MB ollama run moondream
Neural Chat 7B 4.1GB ollama run neural-chat
Starling 7B 4.1GB ollama run starling-lm
Code Llama 7B 3.8GB ollama run codellama
Llama 2 Uncensored 7B 3.8GB ollama run llama2-uncensored
LLaVA 7B 4.5GB ollama run llava
Solar 10.7B 6.1GB ollama run solar

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions