Zara's Blog

After spending hours on conversation design, training, and testing your Rasa chatbot, you are finally ready to deploy it as a backend API. You start looking at the deployment documentation, and there is no easy step-by-step guide to deploy your beloved chatbot. I know because I have been there! That’s why I am sharing how I deployed my chatbot here.

First, I built a custom connector to interact with my chatbot through a webhook. Next, I deployed my chatbot to a cloud virtual machine (VM). I used the Google Cloud Platform (GCP) compute instance. However, the steps outlined here are applicable to other cloud providers. Below you can find the details of the step-by-step guide on how I deployed my Rasa chatbot on GCP.

Step 1: Create a Custom Connector

A custom connector is an interface between the chatbot users and Rasa chatbot. Rasa has prebuilt integrations with many channel connectors such as Slack, Telegram, Facebook Messenger, etc. However, to make a custom connector we need to implement a class that subclasses rasa.core.channels.channel.InputChannel and implements name and blueprint methods. More details on this can be found here.

In my chatbot repo, I created a folder called addons that contains my custom connector class, custom_channel.py:

import asyncio
import inspect
from sanic import Sanic, Blueprint, response
from sanic.request import Request
from sanic.response import HTTPResponse
from typing import Text, Dict, Any, Optional, Callable, Awaitable, NoReturn

import rasa.utils.endpoints
from rasa.core.channels.channel import (
    InputChannel,
    CollectingOutputChannel,
    UserMessage,
)

class MyIO(InputChannel):
    def name(name) -> Text:
        return "myio"

    def blueprint(
        self, on_new_message: Callable[[UserMessage], Awaitable[None]]
    ) -> Blueprint:

        custom_webhook = Blueprint(
            "custom_webhook_{}".format(type(self).__name__),
            inspect.getmodule(self).__name__,
        )

        @custom_webhook.route("/", methods=["GET"])
        async def health(request: Request) -> HTTPResponse:
            return response.json({"status": "ok"})

        @custom_webhook.route("/webhook", methods=["POST"])
        async def receive(request: Request) -> HTTPResponse:
            sender_id = request.json.get("sender") 
            text = request.json.get("text") 
            input_channel = self.name() 
            metadata = self.get_metadata(request)

            collector = CollectingOutputChannel()
            
            await on_new_message(
                UserMessage(
                    text,
                    collector,
                    sender_id,
                    input_channel=input_channel,
                    metadata=metadata,
                )
            )

            return response.json(collector.messages)

        return custom_webhook

Additionally, I appended my credentials.yml file with:

addons.custom_channel.MyIO:
  username: "user_name"
  another_parameter: "some value"

Also, in the domain.yml file, I included the channel name myio after the first response, i.e, utter_greet:

responses:
  utter_greet:
    - text: Hello!
      channel: myio

And that is all we need to do on the chatbot side to enable the custom connection. To run the chatbot, we will need to specify the path to credentials.yml:

rasa run --credentials ./credentials.yml

Now that we have the Rasa chatbot server running on port 5005 locally, multiple users can communicate with it simultaneously, each within their own session using the http://localhost:5005/webhooks/myio/webhook webhook and a payload similar to the following:

{
  "sender": "test_user_1",
  "text": "Hi there!",
  "metadata": {}
}

Vola! We have our custom webhook working. Let’s now move on to deploying our VM.

Step 2: Deploy on a VM

I used a private GitHub repository to store my Rasa chatbot. To enable access to this private repository from within the VM, I set up a GitHub repository SSH deploy key by following the steps below:

1. Created a pair of private and public ssh keys using ssh-keygen command
2. Copied the public key (found in the file with the .pub extension) to deploy keys of the repository. If you have never done this before, navigate to the repo > click on “Settings” on the top right > “Deploy keys” on the left menu > “Add deploy key”
3. Copied the private key to Google Secret Manager. To accomplish this, navigate to the secret manager > click on “+ CREATE SECRET” > simply assign a name to the secret and upload or copy the key to the secret value. To access the secret from within the VM,
   1. the scope needs to authenticate with cloud-platform (see below), and
   2. the service account needs to have the Secret Manager Secret Accessor permissions.

To avoid retraining the chatbot on the VM, I stored my trained model weights in cloud storage. I also created a requirements.txt configuration file for my Python packages including rasa, google-cloud-core, google-cloud-storage, etc.

Finally, I used a startup bash script to deploy the Rasa chatbot remotely. The script

1. downloads the Rasa chatbot from a private GitHub repository
2. sets up the Python environment
3. downloads the trained model weights from the cloud storage
4. starts the Rasa server (and action server if exists)

Below is my startup_script.sh:

sudo apt-get update
sudo apt -y install jq

curl "https://secretmanager.googleapis.com/v1/projects/<PROJECT-NAME>/secrets/<SECRET-NAME>/versions/<SECRET-VERSION>:access" \
    --request "GET" \
    --header "authorization: Bearer $(gcloud auth print-access-token)" \
    --header "content-type: application/json" | jq -r '.payload.data' | base64 --decode > ~/.ssh/id_rsa

echo -e '\n' >> ~/.ssh/id_rsa
chmod 400 ~/.ssh/id_rsa
echo -e 'Host github.com\n\tHostname github.com\n\tIdentityFile ~/.ssh/id_rsa' >> ~/.ssh/config
eval `ssh-agent -s`
ssh-add ~/.ssh/id_rsa
ssh-keyscan -t rsa github.com > known_hosts.github
cp known_hosts.github ~/.ssh/known_hosts

git clone git@github.com:<GITHUB-USERNAME>/<REPOSITORY-NAME>.git ~/app
cd ~/app

yes | sudo apt install python3.8-venv
yes | sudo apt install pip
pip install --upgrade pip
python3 -m pip install --user virtualenv
python3 -m venv venv
source ./venv/bin/activate
yes | pip install -r requirements.txt

mkdir ./models
gsutil cp gs://<PATH-TO-MODEL>/<MODEL-NAME>.tar.gz ./models

rasa run --credentials ./credentials.yml & rasa run actions -p 5055

Running the script below, I spun the VMs and opened port 5005 for the Rasa chatbot:

MY_INSTANCE_NAME=<RASA-CHATBOT-NAME>
ZONE=<ZONE>

gcloud compute instances create $MY_INSTANCE_NAME \
    --image-family=ubuntu-2004-lts \
    --image-project=ubuntu-os-cloud \
    --machine-type=<MACHINE-TYPE> \
    --scopes="https://www.googleapis.com/auth/cloud-platform" \
    --service-account <SERVICE-ACCOUNT> \
    --metadata-from-file startup-script=startup_script.sh \
    --zone $ZONE \
    --tags rasa


gcloud compute firewall-rules create port-5005 \
    --allow tcp:5005 \
    --source-ranges 0.0.0.0/0 \
    --target-tags rasa \
    --description "Allow port 5005 access to server"

If you made it this far with me, you now have your Rasa chatbot accessible on the http://<VM-IP-ADDRESS>:5005/webhooks/myio/webhook. Enjoy chatting away with your in-cloud bot!

The other day, I was exploring the different effects I can apply to live pictures on my iPhone and noticed that the Long Exposure effect removes moving objects and people from the frame. If you are a pro photographer, this is not news to you. You have probably already used a Neutral Density (ND) filter to remove crowds of people from your photographs. With the long exposure effect, if an object moves fast across different frames, it looks like the object never existed.

The long exposure effect got me thinking about what sticks and what fades away in life. Let’s say, as babies, each of us is a picture with intrinsic traits. Going through life, we gradually embellish our picture with new skills, habits, connections, etc. The colors we add to our picture only persist if we steadily get exposed to them. In the long exposure effect, fast-moving objects get removed, and the slow, steady ones remain. Likewise, new habits and skills can enrich our picture only if we are consistent with them. This is the same old boring advice of “repetition makes perfect.” As for our connections, the long-term friendships and relationships that we attend to and nurture are the ones that last.

But there is another side to this. The skills, habits, or connections that are not nurtured are forgotten. Some may leave a faint ghost-like trace on our picture, but they are barely identifiable. This reminds me of the “use it or lose it” concept that you may have heard in the context of body muscles deteriorating when not used enough. I can personally name many of my deteriorated skills. Off the top of my head, I used to be great at geometry and arithmetic, but I am not anymore. I used to be a much better cook, but now I am reliant on recipes. I am not even going to bother with the lost connections. There are, unfortunately, too many beautiful people that I’ve lost touch with.

With the long exposure effect, our pictures weather. So, now the question is, what do we want to stick and what are we willing to compromise? How can we become the most authentic, the highest expression of ourselves?

I recently learned about Toltec ancient wisdom^I, and it really inspired me. Toltec is a culture that ruled southern Mexico before the Aztecs ( 900–1521 AD). They were scientists and artists known for their wisdom and knowledge.

Exploring the Toltec culture, I realized how little human struggles have changed over the centuries, and interestingly how much modern psychology resembles ancient wisdom. Despite all our advancements, it seems that we are still suffering from the same things our ancestors did. In some cases, our technology has not only not reduced our suffering but has exacerbated our pain. In this article, I attempt to apply Totlec’s wisdom to social media.

Toltec believed that domestication takes away an individual’s power by introducing the individual to a man-made “rule book” overflowing with shoulds and shouldn’ts:

• “you should follow your passion,”
• “you should work hard,”
• “you are thirty, and you should be in a stable relationship already,”
• “you shouldn’t make a fool out of yourself,”
• and many, many more.

All are engrained in our flesh because we have heard these dreams of society repeatedly. We are taught from an early age how to behave, so we are not ostracized, rejected, or hurt by others. Toltec called this rule book the first parasite of the mind. A parasite, by definition, is anything that lives off of another organism and at the expense of the host. The rule book is a parasite of the mind because hosting it imposes immense pain on us. Thanks to the following two parasites of the mind, the “judge” and the “victim,” the rule book is highly effective in making us miserable. The judge relentlessly enforces the rule book and is critical of us every time we rebel against it. The victim is the master of self-pity and the martyr of domestication. The victim immaculately completes the cycle of suffering (If these characters remind of the psychology-based animated film “Inside Out,” you are not alone).

Toltec ancient wisdom reveals the path to living an empowered, free, and fulfilled life by taking control back from the three parasites of the mind and by not feeding these parasites. The more we provide these parasites, the stronger they get and the more autonomy they will demand. Every day, we either feed the parasites or starve them with the choices we make: how to spend our time, who to hang out with, where to go, and what to invest in.

These days, many of us spend hours on social media consuming random, carefully curated content. Based on a recent study by Uswitch, an average adult spent three hours a day on social media in 2020; a 2x increase compared to 2012. But why? It’s no surprise that social media’s promise to connect the world is only marginally fulfilled. If not connecting, what are we doing on social media? The time spent on social media is not bringing us joy; thus, it is likely feeding the parasites. So how is social media feeding the parasites?

1. Every time we see someone’s adventures and compare ourselves to them, the judge prevails, and the victim’s back gets more bent.
2. When we portray ourselves as “perfect” to the outside world and seek the approval of spectators, the rule book is enforced.
3. When we fall for the ads, brands’ promises for happiness, the rule book is amended.

Long before the advent of social media, computer-less Toltec knew that a fulfilled life is only achieved by taking power away from the parasites of the mind. By taking a good look in the mirror and honestly evaluating our choices every day, we can overcome the domestication parasites. We can gain our freedom by pondering on this question: “Are the parasites dominating now?”

I “The Four Agreements: A Practical Guide to Personal Freedom” by Don Miguel Ruiz

Natural Language Processing (NLP) is a set of methods that enable the machine or a computer to understand the human language. Many organizations invest in NLP-driven technologies such as chatbots to address their needs in customer service, virtual assistants, publicity monitoring, issue tracking, etc. Both opportunities and market pressures drive this adoption:

• (opportunity) substantial cost savings due to operational efficiency and scale. For example, chatbots virtual assistants are expected to reduce managers’ workload by 70% by 2024.
• (opportunity) companies don’t always have visibility into the grievances raised on social media and cannot act quickly enough. NLP systems can monitor the social media sites for any grievances raised by customers and triage actionable complaints to the proper internal teams or invoke the chatbot system to interact with the customer and help them resolve the issue.
• (pressure) market demand for speedy, always-available service has increased. NLP agents offer time savings and often enhanced quality of service to consumers at any time of the day.

The main reason for this rising interest is that NLP is finally delivering its promise of understanding human language in context. Among many involved factors, the following two have played a significant role in the popularity of NLP:

• The advent of powerful deep-learning NLP models. The impact of these models have been so revolutionary that experts refer to this category of NLP methods as Natural Language Understanding (NLU)
• Various open-source libraries, affordable cloud services, or commercial products have democratized advanced NLP products.

To this date, most NLP applications were limited to managing customer relations and making financial investments. We are, however, seeing emerging adoption in other industries, such as healthcare, to organize electronic health records and to catalog information in a way that is easily searchable using custom knowledge bases. The NLP applications are vast, and we are just at the beginning of the path.