How to Build Your Own Chatbot Using Open Source LLM'S

In this article, we will learn how to build a chatbot using machine learning techniques, mainly focusing on the BlenderBot model from Hugging Face. We will discuss chatbots, transformers, large language models (LLMs), and how to set up and use our chatbot with Gradio, an easy-to-use interface for machine learning applications. This guide is meant for beginners so that we will explain each concept and step in the process.

Key component

1. ChatBot is a computer program designed to engage in conversation with human users, taking text input and responding with corresponding output. Chatbots leverage advanced algorithms to simulate human-like responses, providing assistance or information based on user queries.

2. Transformers Transformers are a special kind of model architecture that serves as the backbone of many state-of-the-art NLP applications, including chatbots. They can process input text and generate human-like responses. Here are some key aspects of transformers:

   • Input Processing

     When a user interacts with a chatbot, their message undergoes tokenization, where the text is divided into smaller units called tokens. For instance, the sentence "Hello, how are you?" is split into tokens like "Hello," "how," "are," "you," and "?". Each token is then transformed into a numerical representation through embedding, capturing its meaning in a format the model can process.

   • Understanding Context

     After processing the input, the transformer moves to understanding the context. It looks at the entire sentence and previous conversation, not just individual words. Through a self-attention mechanism, the model identifies key parts of the input that are most relevant. This helps the chatbot track the ongoing conversation and respond meaningfully by considering both current and past interactions.

   • Generating Responses

     Once the model interprets the input, it predicts the most suitable response based on its training data. This involves selecting the next word or phrase that aligns with the context, ensuring the reply is both appropriate and natural-sounding.

   • Iterative Conversation

     Chatbots engage in iterative conversations by continuously processing user inputs and generating responses. With each exchange, the model updates its understanding of the context, allowing for dynamic and interactive dialogues that adapt to the user's inputs.

3. Large Language Models (LLMs)

   Inside transformers, we find large language models (LLMs). It is a type of neural network architecture designed to understand and generate human-like text. These models are trained on massive datasets, enabling them to grasp the complexities of language. This allows chatbots to engage users in meaningful conversations.

4. Hugging Face

   Hugging Face is a popular platform for sharing and using transformer models. It provides a user-friendly interface for accessing a variety of pre-trained models, making it easier to implement NLP tasks without extensive knowledge of the underlying algorithms.

Prerequisites

1. Python: Install Python 3.6 or higher from the official website.

2. Basic Python Knowledge: Familiarity with Python programming.

3. IDE or Code Editor: Use an IDE like Visual Studio Code or PyCharm.

4. Command Line Interface (CLI): Basic knowledge of command line operations for creating directories and activating environments.

5. Hugging Face Account: Sign up at Hugging Face for API access.

Steps to Build the Chatbot

Before diving into building your chatbot, ensure you have a suitable development environment set up. For this project, I recommend using Visual Studio or any other preferred Integrated Development Environment (IDE) that supports Python. Follow the steps below to create your chatbot using Gradio and Transformers.

1. Generating Hugging Face Tokens

   Hugging Face tokens are essential for API authentication, providing secure access to models, managing usage limits, and enabling access to restricted resources on their platform. To ensure your chatbot can interact with the Hugging Face models effectively, follow these steps to obtain your tokens:

   • Head to your profile in hugging face platform

   • In settings, click on access token option.

     

   • Click on Create New Token and generate your tokens

   • Copy the code displayed in the popup and save it securely for future use.

2. Setting up project Environment (Visual studio)

   If you are not using Visual Studio skip this part.

   • Create a Project Folder

     Start by creating a dedicated folder for your chatbot project. This will help you keep all related files organized.

       mkdir chatbot_project
       cd chatbot_project
     

   • Create a Python File: Inside the project folder, create a Python file where you'll write your chatbot code. You can name it something like chat_bot.py.

       touch chat_bot.py
     

   • Set Up a Virtual Environment: It’s a good practice to use a virtual environment to manage your project’s dependencies. This helps prevent conflicts with other projects.

       python -m venv venv
     

   • Activate the Virtual Environment: Once the virtual environment is created, activate it to start working within it.

     Windows Command

       venv\Scripts\activate
     

     Linux/Mac Command:

       source venv/bin/activate
     

3. Installing the Required Libraries

   To build your chatbot, you'll need to install two essential libraries: Gradio and Transformers. These libraries provide the necessary tools for creating a user-friendly interface and leveraging powerful pre-trained models.

   • Open your terminal (or the terminal within Visual Studio).

   • Run the following command:

    pip install gradio transformers

This command uses pip, the Python package manager, to download and install the required libraries. Once the installation completes, you can proceed with your chatbot development.

4. Importing Libraries

   After installing the libraries, you need to import them into your Python script. This step allows you to utilize the functionalities provided by Gradio and Transformers. Add the following lines at the beginning of your script

    import gradio as gr from transformers 
    import BlenderbotTokenizer, BlenderbotForConditionalGeneration
   

   In this snippet:

   • gradio is used for creating the web interface.

   • BlenderbotTokenizer and BlenderbotForConditionalGeneration are essential for working with the BlenderBot model.

5. Using Hugging Face Tokens in Your Chatbot

   We need to incorporate hugging face token it into our chatbot project to authenticate and access the models securely. Here's how to do it, choose any one option:

   • Store it as an environment variable: (Highly preferable)

     You can do this by adding the following line to your terminal or command prompt before running your script.

       export HUGGINGFACE_TOKEN='your_token_here'
     

     Make sure you have the required libraries installed to access environment variables

       import os
       from dotenv import load_dotenv
       # Load environment variables from a .env file
       load_dotenv()
       # Retrieve the Hugging Face token
       hugging_face_token = os.getenv('HUGGINGFACE_TOKEN')
     

(OR)

• Configure the Token in Your Code: (least preferred)

  Use the retrieved token when initializing the Hugging Face model. Here’s an example:

    use_auth_token=hugging_face_token
    tokenizer = BlenderbotTokenizer.from_pretrained(model_name, use_auth_token)
    model = BlenderbotForConditionalGeneration.from_pretrained(model_name, use_auth_token)
  

6. Choosing and Fetching the Model

   For this project, we will utilize the BlenderBot model from Hugging Face, as this serves best when it comes to conversions like text. you can choose any model of your choice that suites best for your requirement.

    # Specify the model name for the pre-trained BlenderBot.
    model_name = "facebook/blenderbot-400M-distill"
    # Load the tokenizer form the BlenderBot model to convert text into tokens.
    tokenizer = BlenderbotTokenizer.from_pretrained(model_name)
    # Load the BlenderBot model itself for generating responses based on tokens.
    model = BlenderbotForConditionalGeneration.from_pretrained(model_name)
   

   Here model_name specifies the version of BlenderBot you want to use. The from_pretrained method downloads the model and tokenizer, enabling them for use in your chatbot.

7. Building the Chatbot Function

   Next, define a function that processes user input and generates responses from the chatbot model. Here’s how you can implement it:

    def chat_with_bot(input_text, history):
        # Initialize context for conversation.
        context = ""
        # Build context from history.
        for msg in history:
            context += f"User: {msg[0]}\nAssistant: {msg[1]}\n"
        # Add current user input to context.
        context += f"User: {input_text}\nAssistant:"
        # Tokenize context for model input.
        inputs = tokenizer(context, return_tensors="pt", truncation=True)
        # Generate response from the model.
        bot_output = model.generate(inputs["input_ids"], max_length=100, num_beams=4, early_stopping=True)
        # Decode the generated tokens to a string.
        response = tokenizer.decode(bot_output[0], skip_special_tokens=True)
        # Append input and response to history.
        history.append((input_text, response))
        # Return the response and updated history.
        return response, history
   

   This function, chat_with_bot, compiles the conversation history to provide context for the chatbot, generates a response based on the user’s input, and returns both the response and updated conversation history.

8. Creating the Gradio Interface

   Set up the Gradio interface to allow users to interact with the chatbot

    # Gradio Interface with plain color background
    with gr.Blocks(css="""
    .main-container {
        background-color: #C4DFDF;  /* Set your desired background color */
        height: 100vh;  /* Full height */
    }
   
    .chatbox {
        border: 1px solid #ccc;
        border-radius: 10px;
        padding: 10px;
        background-color: rgba(249, 249, 249, 0.8);  /* Slightly transparent white background for chatbox */
    }
    h1 {
        text-align: center;
        color: #1a1a1a;
        font-family: 'Lobster', cursive;
    }
    """ ) as demo:
   
        # Interface elements
        gr.Markdown("<h1>Keerthi's Chatbot</h1>")
        gr.Markdown("Welcome! Type below to chat with the chatbot that you built.")
   
        with gr.Column(elem_classes="main-container"):  # Apply the main-container class
            chatbot = gr.Chatbot(label="Chatbot", elem_classes="chatbox")
            msg = gr.Textbox(placeholder="Enter your message...", label="Your Message")
   
            with gr.Row():
                submit_btn = gr.Button("Send")
                clear_btn = gr.Button("Clear Chat")
   
            # Chatbot response function
            def respond(message, history=[]):
                response, updated_history = chat_with_bot(message, history)
                return updated_history
   
            # Button functionality
            submit_btn.click(respond, inputs=[msg, chatbot], outputs=chatbot)
            clear_btn.click(lambda: None, None, chatbot)
   

   This code creates a simple yet elegant interface where users can type messages and receive responses from the chatbot. The Send button triggers the response function, while the Clear Chat button resets the conversation.

9. Launching the App

   Finally, launch the Gradio app with this command

    # Launch the app
    demo.launch(share=True)
   

   The share=True parameter allows you to create a public link for others to access your chatbot easily.

Full copyable code

import gradio as gr
from transformers import BlenderbotTokenizer, BlenderbotForConditionalGeneration

# Load the model and tokenizer
model_name = "facebook/blenderbot-400M-distill" 
tokenizer = BlenderbotTokenizer.from_pretrained(model_name)
model = BlenderbotForConditionalGeneration.from_pretrained(model_name)

# Chatbot function
def chat_with_bot(input_text, history):
    context = ""
    for msg in history:
        context += f"User: {msg[0]}\nAssistant: {msg[1]}\n"

    context += f"User: {input_text}\nAssistant:"

    inputs = tokenizer(context, return_tensors="pt", truncation=True)
    bot_output = model.generate(inputs["input_ids"], max_length=100, num_beams=4, early_stopping=True)
    response = tokenizer.decode(bot_output[0], skip_special_tokens=True)

    history.append((input_text, response))
    return response, history

# Gradio Interface with plain color background
with gr.Blocks(css="""
.main-container {
    background-color: #C4DFDF;  
    height: 100vh;  /* Full height */
}

.chatbox {
    border: 1px solid #ccc;
    border-radius: 10px;
    padding: 10px;
    background-color: rgba(249, 249, 249, 0.8);  
}
h1 {
    text-align: center;
    color: #1a1a1a;
    font-family: 'Lobster', cursive;
}
""" ) as demo:

    # Interface elements
    gr.Markdown("<h1>Keerthi's Chatbot</h1>")
    gr.Markdown("Welcome! Type below to chat with the chatbot that you built.")

    with gr.Column(elem_classes="main-container"):  # Apply the main-container class
        chatbot = gr.Chatbot(label="Chatbot", elem_classes="chatbox")
        msg = gr.Textbox(placeholder="Enter your message...", label="Your Message")

        with gr.Row():
            submit_btn = gr.Button("Send")
            clear_btn = gr.Button("Clear Chat")

        # Chatbot response function
        def respond(message, history=[]):
            response, updated_history = chat_with_bot(message, history)
            return updated_history

        # Button functionality
        submit_btn.click(respond, inputs=[msg, chatbot], outputs=chatbot)
        clear_btn.click(lambda: None, None, chatbot)

# Launch the app
demo.launch(share=True)

Now simply run this python code using python chat_bot.py it should give a result something like this.

click on the link, it will open your chatbot on web browser. If you used the code above, you will see this as a output

Congratulations! 🎉 You have successfully created your own chatbot. Let's explore how to build chatbots on AWS Cloud in our upcoming articles.

Conclusion

we have successfully built a chatbot using the BlenderBot model from Hugging Face, leveraging the power of machine learning to create engaging interactions. This guide has walked you through essential concepts, from the architecture of chatbots and transformers to practical implementation using Gradio. let’s see how to build chatbots on AWS Cloud in our next article. Happy coding!! 💻