fbpx
Supercharging Large Language Models With Langchain Supercharging Large Language Models With Langchain
During this tutorial, we will explore how to supercharge Large Language Models (LLMs) with LangChain. The focus of this tutorial will be... Supercharging Large Language Models With Langchain

During this tutorial, we will explore how to supercharge Large Language Models (LLMs) with LangChain. The focus of this tutorial will be to build a Modular Reasoning, Knowledge and Language (MRKL) application that uses LLMs + LangChain. This MRKL app will incorporate features such as web search, scientific search, and database search on top of LLMs. With these added capabilities, the application we build will be able to provide answers to a wide range of questions, making it a valuable tool for a variety of use cases.

We will only use services with a free tier or services that provide free credits when signing up, so building this application as a proof of concept will cost $0 🤑

Note: At the time of writing this blog post LangChain’s latest version was 0.0.88. The library is very active (as far as I can tell a new version has been released daily over the past few weeks), by the time you’re reading this there might already be a new version.

All the code for this tutorial can be found in this GitHub repo.

Why is it important?

Although Large Language Models (LLMs) like ChatGPT are remarkable, they come with significant constraints such as knowledge cut-offs and hallucinations. While this may not be an issue for certain tasks, such as using them to generate imaginative content like a novel, there are times when customers and organizations require precise information from LLMs. Rather than conducting direct searches using tools like web search engines, they prefer to access this data via a unified interface, for example in a conversational style. In such cases, the accuracy of the underlying LLM is critical. In this tutorial, we will demonstrate how to overcome these limitations, allowing you to harness the full potential of LLMs.

Scope of this tutorial

In this tutorial, we will build a system that focuses on natural-language-based knowledge tasks, i.e. interacting with different knowledge resources (e.g. internet, database) via natural language. AI21 Labs described this as a Modular Reasoning, Knowledge and Language (MRKL, pronounced miracle) system in their paper from May 2022.

An example would be to ask the system a question about current events. In this case traditional chat-based LLMs will answer incorrectly as they have a knowledge cutoff, i.e. they don’t have access to information after a certain point in time. If we ask ChatGPT who the current prime minister of the UK is it will respond a s follows:

Image by author

The MRKL system we will build in this tutorial will be able to answer this question (and even more complicated ones) correctly 😃

Below is an (admittedly oversimplified) architecture of what we will be building:

Image by author

Getting started

All the development in this tutorial can be done on a local machine, no extensive compute resources required. Since we will build an MRKL system, we will draw inspiration from LangChain’s documentation for MRKL agents. Agents in LangChain use LLMs to determine which actions to take in which order. The LLM we will be using in this tutorial will be OpenAI’s GPT-3 model which we will be connecting to via API access.

Note: In a future tutorial we will explore how we can use other LLMs besides OpenAI’s GPT-3 and test their performance.

Access to GPT-3

To gain access to GPT-3 we need to create an account with OpenAI. When signing up we receive $18 in free credit that can be used during the first three months. Afterwards the Davinci model (which is the model we will be using) will cost $0.02 for the generation of 1,000 tokens (~750 words). You can learn more about OpenAI’s pricing on their website.

After we sign up, we need to create an API key that allows us to access GPT-3 programmatically. Do not share this API key with others as it might allow them to use OpenAI services on your bill!

Image by author

Integrating web search

Most of LangChain’s example notebooks will use the SerpAPI service. It’s a great service and allows 100 searches per month within the free tier. However, it turns out when experimenting extensively with LangChain 100 free searches are exhausted quite quickly. And their next tier costs already $50 per month:

Image by author

Luckily, there exist other implementations for web searches. This is (not always) clear from the documentation, but a quick look into LangChain’s GitHub repo reveals another option:

Image by author

Serper is very similar to SerpAPI, with the significant difference that it allows 1,000 free searches when signing up 🤗 So for the purpose of this tutorial we will use Serper.

Image by author

Just like we did with OpenAI, we need to sign up and create an API key:

Image by author

First test run

We have all the building blocks for the first version of our MRKL system in place, so let’s take it for a test run.Let’s use the example from above and ask our app about the current PM of the UK. To do so we need to “tell” the LLM that is has access to a search API and that it should access it when needed.

Note: We will explore exactly how this works under the hood in a future blog post

import os
from langchain.utilities import GoogleSerperAPIWrapper
from langchain.llms import OpenAI
from langchain.agents import initialize_agent, Tool

os.environ["SERPER_API_KEY"] = "<YOUR_SERPER_API_KEY>"
os.environ["OPENAI_API_KEY"] = "<YOUR_OPENAI_API_KEY>"

search = GoogleSerperAPIWrapper()
llm = OpenAI(temperature=0)

tools = [
    Tool(
        name="Search",
        func=search.run,
        description="Useful for when you need to answer questions about current events. You should ask targeted questions"
    )
]

mrkl = initialize_agent(tools, llm, agent="zero-shot-react-description")
print(mrkl.run("Who is the prime minister of the UK?"))

Image by author

This worked nicely 🤗 Now let’s see how far we can push this MRKL system that only has access to Google web search:

Image by author

That’s not bad at all, but often in real-world applications, we need to be more precise than that. So let’s see how we can improve the answer to get the exact distance between Southampton and London.

Integrating Wolfram Alpha

Adding Wolfram Alpha (WA) to LLMs is similarly exciting as adding web search. So much so that the founder of WA has explored this himself in his blog post in January 2023.

In order to integrate WA, we again need to sign up for a free account. Once we have done that, this time one extra step is needed: We need to create an app in in the developer portal in order to receive API keys. With this we will be able to conduct 2,000 searches per month for free. To create this app we need to go to the WA product page. Once there we need to get a new AppID (i.e. create a new app):

Image by author

Once we have created this new app we can copy the AppID and use that to access WA in our MRKL app. To do so we only need to add WA as a tool for the LLM like so:

from langchain.utilities.wolfram_alpha import WolframAlphaAPIWrapper
from langchain.utilities import GoogleSerperAPIWrapper

os.environ["SERPER_API_KEY"] = "<YOUR_SERPER_API_KEY>"
os.environ["WOLFRAM_ALPHA_APPID"] = "<YOUR_WA_APP_ID>"

search = GoogleSerperAPIWrapper()
wolfram = WolframAlphaAPIWrapper()

tools = [
    Tool(
        name="Search",
        func=search.run,
        description="Useful for when you need to answer questions about current events. You should ask targeted questions."
    ),
    Tool(
        name="Wolfram",
        func=wolfram.run,
        description="Useful for when you need to answer questions about math, science, geography."
    )
]

Now that we have that, let’s try the above question again:

Image by author

This looks much better, let’s verify that this is actually true:

Image by author

Looking good 😊

Integrating a database

As mentioned at the start of this tutorial, the real power of an MRKL system will be unlocked if our agent may not only query the internet for information but also proprietary, internal information of an organization. To demonstrate how this can work we will add one more tool to our system, which is a database search.

To do so, we first need to create a database. In the repo for this tutorial, I created a folder called foo_db. In that folder is an SQL script called Chinook_Sqlite.sql. If we run this script with SQLite (instructions) it will create the Chinook database, which represents a digital media store, including tables for artists, albums, media tracks, invoices, and customers.

Let’s test that the database is up and running:

Image by author

Great! Now that we have a database we can add a DB search to the available tools for the MRKL agent:

import os
from langchain.utilities.wolfram_alpha import WolframAlphaAPIWrapper
from langchain.utilities import GoogleSerperAPIWrapper
from langchain SQLDatabase

os.environ["SERPER_API_KEY"] = "<YOUR_SERPER_API_KEY>"
os.environ["WOLFRAM_ALPHA_APPID"] = "<YOUR_WA_APP_ID>"

search = GoogleSerperAPIWrapper()
wolfram = WolframAlphaAPIWrapper()
db = SQLDatabase.from_uri("sqlite:///foo_db/Chinook.db")
db_chain = SQLDatabaseChain(llm=llm, database=db, verbose=True)

tools = [
    Tool(
        name="Search",
        func=search.run,
        description="Useful for when you need to answer questions about current events. You should ask targeted questions."
    ),
    Tool(
        name="Wolfram",
        func=wolfram.run,
        description="Useful for when you need to answer questions about math, science, geography."
    ),
    Tool(
        name="FooBar DB",
        func=db_chain.run,
        description="Useful for when you need to answer questions about FooBar. Input should be in the form of a question containing full context"
    )
]

Now let’s try that with quite a convoluted question that requires our MRKL system to use the web search as well as the DB search:

Image by author

Let’s also double check this result:

Image by author

Image by author

Looks like our MRKL app also got this one right! 🤗

Conclusion

In this tutorial, we have created an MRKL app using LangChain on top of an LLM (in this case GPT-3). There are still many questions to answer, for example how LangChain works under the hood and how other LLMs besides GPT-3, which I will dive deeper into in future blog posts.

Article originally posted here. Reposted with permission.

ODSC Community

The Open Data Science community is passionate and diverse, and we always welcome contributions from data science professionals! All of the articles under this profile are from our community, with individual authors mentioned in the text itself.

1