skip navigation
skip mega-menu

LangChain Chat with Custom Tools, Functions and Memory

In this story we are going to explore how you can create a simple web-based chat application that communicates with a private REST API, uses OpenAI functions and conversational memory.

We will be using again the LangChain framework which provides a great infrastructure for interacting with Large Language Models (LLM).

The Agent we are going to describe in this post is going to use these tools:

The agent will have two user interfaces:

  • A Streamlit based tweb client

  • A command line interface


OpenAI functions

One of the main problems when dealing with responses from a LLM like ChatGPT is that the responses are not completely predictable. When you try to parse the responses there might be slight variations in the output which make parsing by programmatic tools error prone. Since LangChain agents send user input to an LLM and expect it to route the output to a specific tool (or function), the agents need to be able to parse predictable output.

To address this problem OpenAI introduced on June 13th of 2023 “Function Calling” which allows developers to describe the JSON output describing which functions (in LangChain speak: tools) to call based on a specific input.

The approach initially used by LangChain agents — like e.g. the ZERO_SHOT_REACT_DESCRIPTION agent — uses prompt engineering to route messages to the right tools. This approach is potentially less accurate and slower than the approach using OpenAI functions.

As of the time of writing this blog the models which support this feature are:

  • gpt-4–0613

  • gpt-3.5-turbo-0613 (this includes gpt-3.5-turbo-16k-0613, which we used for your playground chat agent)

Here are some examples given by OpenAI on how function calling works:

Convert queries such as “Email Anya to see if she wants to get coffee next Friday” to a function call like send_email(to: string, body: string), or “What’s the weather like in Boston?” to get_current_weather(location: string, unit: 'celsius' | 'fahrenheit')

Internally the instructions about the function and its parameters are injected into the system message.

The API endpoint is:

POST https://api.openai.com/v1/chat/completions

And you can find the low level API details here:

OpenAI Platform

Explore developer resources, tutorials, API docs, and dynamic examples to get the most out of OpenAI's platform.

platform.openai.com


The Agent Loop

As far as the author of the blog could see the agent loop is the same as for most agents. The basic agents code is similar to the ZERO_SHOT_REACT_DESCRIPTION agent. So the agent loop still could be depicted with this diagramme:

The agent loop


Custom and LangChain Tools

A LangChain agent uses tools (corresponds to OpenAPI functions). LangChain (v0.0.220) comes out of the box with a plethora of tools which allow you to connect to all kinds of paid and free services or interactions, like e.g:

  • arxiv (free)

  • azure_cognitive_services

  • bing_search

  • brave_search

  • ddg_search

  • file_management

  • gmail

  • google_places

  • google_search

  • google_serper

  • graphql

  • human interaction

  • jira

  • json

  • metaphor_search

  • office365

  • openapi

  • openweathermap

  • playwright

  • powerbi

  • pubmed

  • python

  • requests

  • scenexplain

  • searx_search

  • shell

  • sleep

  • spark_sql

  • sql_database

  • steamship_image_generation

  • vectorstore

  • wikipedia (free)

  • wolfram_alpha

  • youtube

  • zapier

We will show in this blog how you can create a custom tool to access a custom REST API.


Conversational Memory

This type of memory comes in handy when you want to remember items from previous inputs. For example: if you ask “Who is Albert Einstein?” and then subsequently “Who were his mentors?”, then conversational memory will help the agent to remember that “his” refers to “Albert Einstein”.

Here is LangChain’s documentation on Memory.


An Agent with Functions, Custom Tool and Memory

Our agent can be found in a Git repository:

GitHub - gilfernandes/chat_functions: Simple playground chat app that interacts with OpenAI's…

Simple playground chat app that interacts with OpenAI's functions with memory and custom tools. - GitHub …

github.com

In order to get it to run, please install Conda first.

Then create the following environment and install the following libraries:

conda activate langchain_streamlit

pip install langchain

pip install prompt_toolkit

pip install wikipedia

pip install arxiv

pip install python-dotenv

pip install streamlit

pip install openai

pip install duckduckgo-search

Then create an .env file with this content:

OPENAI_API_KEY=<key>

Then you can run the command line version of the agent using:

python .\agent_cli.py

And the Streamlit version can be run with this command on port 8080:

streamlit run ./agent_streamlit.py --server.port 8080


Here are some screenshots of the tool:

Using a custom agent


The UI shows which tool (OpenAI function) is being used and which input was sent to it.

Using Arxiv Tool


Using the calculator

Using Wikipedia


Code Details

https://github.com/gilfernandes/chat_functions?source=post_page-----e34daa331aa7--------------------------------

Summary

OpenAI functions can now be easily used with LangChain. And this seems to be a much better method (faster and more accurate) to create agents compared to prompt based approaches.

OpenAI functions can also be easily integrated with memory and custom tools too. There is no excuse to not use OpenAI functions for the type of agent described in this blog.

Subscribe to our newsletter

Sign up here