How to Train a Custom AI Chatbot Using the ChatGPT API and a Knowledge Base

In our in-advance article, we established the way to construct an AI chatbot with the ChatGPT API and assign a position to personalize it. But what if you need to educate the AI on your very own records? For instance, you may have a book, financial records, or a huge set of databases, and also you want to search them effectively. In this article, we carry you an easy-to-comply academic on the way to educate an AI chatbot together with your custom expertise base with LangChain and ChatGPT API. We are deploying LangChain, GPT Index, and other effective libraries to educate the AI chatbot on the use of OpenAI’s Large Language Model (LLM). So on that observation, let’s test out how to train and create an Custom AI Chatbot with the use of your very own dataset.

Set Up the Software Environment to Train an AI Chatbot

Install Python and Pip

1. First off, you need to install Python along with Pip on your pc by following our linked manual. Make certain to enable the checkbox for “Add Python.Exe to PATH” at some stage in setup.

2. To test if Python is properly installed, open the Terminal on your laptop. Once here, run the under commands one at a time, and it will output their version wide variety. On Linux and macOS, you may use python3 in preference to python from now onwards.

python --version

pip --version

3. Run the beneath command to update Pip to the brand new model.

python -m pip install -U pip

Top 7 News API Data Feeds for Real-Time Updates

Install OpenAI, GPT Index, PyPDF2, and Gradio Libraries

1. Open the Terminal and run the below command to put it in the OpenAI library.

pip install openai

2. Next, let’s set up the GPT Index.

pip install gpt_index==0.4.24

3. Now, deploy Langchain by using strolling the underneath command.

pip install langchain==0.0.148

4. After that, installation PyPDF2 and PyCryptodome to parse PDF documents.

pip install PyPDF2

pip install PyCryptodome

5. Finally, set up the Gradio library. This is meant for creating an easy UI to engage with the trained AI chatbot.

pip install gradio

Download a Code Editor

Finally, we want a code editor to edit a number of the code. On Windows, I would advocate Notepad   (Download). Simply download and install the program through the connected hyperlink. You also can use VS Code on any platform if you are snug with powerful IDEs. Other than VS Code, you may deploy Sublime Text (Download) on macOS and Linux.

For ChromeOS, you could use the extraordinary Caret app (Download) to edit the code. We are nearly finished setting up the software environment, and it’s time to get the OpenAI API key.

Get the OpenAI API Key For Free

1. Head to OpenAI’s internet site (visit) and log in. Next, click on “Create new mystery key” and duplicate the API key. Do note that you can’t reproduction or view the entire API key afterward. So it’s advocated to copy and paste the API key to a Notepad record for later use.

2. Next, visit platform.Openai.Com/account/utilization and take a look at if you have enough credit left. If you have exhausted all your unfastened credit score, you want to feature a charge method on your OpenAI account.

Add Your Documents to Train the AI Chatbot

1. First, create a brand new folder called medical doc in a reachable location like the Desktop. You can pick any other region as well according to your desire. However, hold the folder name medical doc.

2. Next, flow the files for training inside the “doc” folder. You can add more than one text or PDF file (even scanned ones). If you’ve got a huge table in Excel, you can import it as a CSV or PDF record after which upload it to the “doc” folder. You can also upload SQL database files, as defined in this Langchain AI tweet. I haven’t tried many record formats besides the cited ones, but you may upload and test on your own. For this text, I am including one of my articles on NFT in PDF format.

Note: If you have a large document, it will take a longer time to process the data, depending on your CPU and GPU. In addition, it will quickly use your free OpenAI tokens. So in the beginning, start with a small document (30-50 pages or < 100MB files) to understand the process.

Make the Code Ready

1. Now, open a code editor like Sublime Text or launch Notepad++ and paste the code. Once more, I have taken outstanding help from armrrs on Google Colab tweaked the code to make it well-matched with PDF files, and created a Gradio interface on Pinnacle.

2. Next, click on “File” in the pinnacle menu and pick out “Save As…”. After that, set the record call app.Py and alternate the “Save as type” to “All types”. Then, save the document to the location where you created the “medical doc” folder (in my case, it’s the Desktop).

3. Make certain the “docs” folder and “app.Py” are in the same area, as shown in the screenshot underneath. The “app.Py” file might be out of doors in the “doc” folder and now not inner.

4. Come again to the code once more in Notepad++. Here, replace Your API Key with the one that you generated above on OpenAI’s website.

5. Finally, press “Ctrl+S” to save the code. You are now equipped to run the code.

Create ChatGPT AI Bot with a Custom Knowledge Base

1. First, open the Terminal and run the beneath command to move to the Desktop. It’s where I saved the “medical doctors” folder and “app.Py” report.

cd Desktop

2. Now, run the beneath command.

python app.py

3. It will begin indexing the file with the usage of the OpenAI LLM model. Depending on the report size, it’ll make the effort to technique the file. Once it’s accomplished, an “index.Json” report could be created on the Desktop. If the Terminal is not showing any output, don’t worry, it’d nevertheless be processing the information. For your information, it takes around 10 seconds to technique a 30MB file.

4. Once the LLM has processed the records, you may find a neighborhood URL. Copy it.

5. Now, paste the copied URL into the net browser, and there you have it. Your custom-trained ChatGPT-powered AI chatbot is ready. To begin, you could ask the AI chatbot when the document is ready.

6. You can ask further questions, and the ChatGPT bot will solutions from the records you provided to the AI. So that is how you could build a custom-trained AI chatbot along with your personal dataset. You can now teach and create an AI chatbot primarily based on any kind of information you need.

Manage the Custom AI Chatbot

1. You can reproduce the general public URL and percentage it along with your buddies and family. The hyperlink could be live for 72 hours, but you also need to keep your computer turned on for the reason that server example is strolling in your computer.

2. To forestall the custom-skilled AI chatbot, press “Ctrl C” in the Terminal window. If it does no longer work, press “Ctrl   C” again.

3. To restart the AI chatbot server, genuinely flow to the Desktop place once more and run the under command. Keep in thoughts, the nearby URL may be equal, however, the public URL will trade after each server restart.

4. If you need to educate the AI chatbot with new statistics, delete the files within the “medical doctors” folder and upload new ones. You can also upload a couple of documents, but make certain to feature clean statistics to get a coherent reaction.

5. Now, run the code once more inside the Terminal, and it’ll create a new “index.Json” document. Here, the antique “index.Json” file will be replaced mechanically.

python app.py

6. To keep song of your tokens, head over to OpenAI’s online dashboard and take a look at how an awful lot loose credit score is left.

7. Lastly, you don’t want to the touch the code except you need to trade the API key or the OpenAI model for further customization.