Using langchain for text to SQL using custom llm API

Question

I am trying to use my llama2 model (exposed as an API using ollama). I want to chat with the llama agent and query my Postgres db (i.e. generate text to sql). I was able to find langchain code that uses open AI to do this. However, I am unable to find anything out there which fits my situation.

Any pointers will be of great help.

Code with openai

# Create connection to postgres
import psycopg2  # Import the library

database = 'postgres'
username = 'postgres'
password = 'password'
server = 'localhost'
port = '5432'

# Establish the connection
conn = psycopg2.connect(
    dbname=database,
    user=username,
    password=password,
    host=server,
    port=port
)

db = SQLDatabase.from_uri(
    "postgresql://postgres:password@localhost:5432/postgres")
toolkit = SQLDatabaseToolkit(db=db, llm=OpenAI(temperature=0))

agent_executor = create_sql_agent(
    llm=OpenAI(temperature=0),
    toolkit=toolkit,
    verbose=True,
    agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
)

agent_executor.run("Describe the transaction table")

I want to make the above code work for my llama2 model exposed via an API at localhost:11434/api/generate

LOCKhart · Accepted Answer · 2023-08-29 09:43:04Z

2

Load your llm like mentioned here https://python.langchain.com/docs/integrations/llms/ollama

and then use that inplace of openai. You'll most probably have to change the prompts to fit llama2 desired format

answered Aug 29, 2023 at 9:43

LOCKhart

362 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Chop Labalagun Over a year ago

Could you please provide more data, I am using Llama 2 and i am able to create the object but when i send the questions it complains about the input text sequence: TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]] Any help is appreciated.

Collectives™ on Stack Overflow

Using langchain for text to SQL using custom llm API

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related