4 FIFA World Cup Twilio Bot¶

It was World Cup season just a couple of months ago, and everyone was rooting for their favorite team. For this project, why not create a bot that can help people stay updated on how the World Cup is progressing? And along the way, we might learn something new. This project is a Twilio application, hosted on Heroku. It is a chat (SMS) bot of sorts. You will be able to send various special messages to this bot and it will respond with the latest World Cup updates.

Here are some screenshots to give you a taste of the final product:

4.1 Getting your tools ready¶

Let’s begin by setting up the directory structure. There will be four files in total in your root folder:

Procfile
app.py
requirements.txt
runtime.txt

You can quickly create them by running the following command in the terminal:

$ touch Procfile app.py requirements.txt runtime.txt

Don’t worry about these files for now. Their purpose will be explored when you start populating them later on.

Create a new Python virtual environment to work in. If you don’t know what a virtual environment is and why it is useful to use it, check out this chapter of the Intermediate Python book. You can create the virtual environment by running the following commands in the terminal:

$ python -m venv env
$ source env/bin/activate

You can deactivate the virtual environment at any time by running:

$ deactivate

You will need to use Python 3.6 or higher and the following Python libraries:

Flask==1.1.2
python-dateutil==2.8.1
requests==2.24.0
twilio==6.45.2

Add these four lines to your requirements.txt file and run

$ pip install -r requirements.txt

We will be using Flask to create the web app. We will be using the Twilio library to interface with Twilio, requests library to consume web APIs and get latest World Cup information and dateutil to handle date-time conversions.

You may wonder, why mention the specific versions of these libraries? Well, these are the latest versions of these libraries at the time of writing. Listing the version numbers keeps this tutorial somewhat future-proof, so if future versions of these libraries break backward compatibility, you will know which versions will work. You can find the versions of libraries installed on your system by running:

$ pip freeze

4.2 Defining the project requirements¶

It is a good idea to list down the features/requirements of our SMS bot. We want it to be able to respond to five different kinds of messages:

“today” should return the details of all games happening today
“tomorrow” should return the details of all games happening tomorrow
“complete” should return the complete group stage details
A country code (like “BRA” for Brazil) should return information related to that particular country
“list” should return all of the FIFA country codes

Suitable responses for these endpoints are:

today

England vs Panama at 08:00 AM
Japan vs Senegal at 11:00 AM
Poland vs Colombia at 02:00 PM

tomorrow

Saudi Arabia vs Egypt at 10:00 AM
Uruguay vs Russia at 10:00 AM
Spain vs Morocco at 02:00 PM
Iran vs Portugal at 02:00 PM

complete

--- Group A ---
Russia Pts: 6
Uruguay Pts: 6
Egypt Pts: 0
Saudi Arabia Pts: 0


--- Group B ---
Spain Pts: 4
Portugal Pts: 4
Iran Pts: 3
Morocco Pts: 0

...

ARG (Argentina’s FIFA code)

--- Past Matches ---
Argentina 1 vs Iceland 1
Argentina 0 vs Croatia 3


--- Future Matches ---
Nigeria vs Argentina at 02:00 PM on 26 Jun

list

KOR
PAN
MEX
ENG
COL
JPN
POL
SEN
RUS
EGY
POR
...

Since the World Cup is an event watched by people all over the world, we must consider date/time information. The API we will be using provides us with the UTC time. This can be converted to your local time zone, such as America/New York so that you don’t have to do mental time calculations. This will also provide you with an opportunity to learn how to do basic time manipulations using dateutil.

With these requirements in mind, let’s move on.

4.3 Finding and exploring the FIFA API¶

Now you need to find the right API which you can use to receive up-to-date information. This tutorial uses this website. The specific endpoints we are interested in are:

urls = {'group': 'https://worldcup.sfg.io/teams/group_results',
        'country': 'https://worldcup.sfg.io/matches/country?fifa_code=',
        'today': 'https://worldcup.sfg.io/matches/today',
        'tomorrow': 'https://worldcup.sfg.io/matches/tomorrow'
}

Instead of using the country codes endpoint available at worldcup.sfg.io, we will be maintaining a local country code list.

countries = ['KOR', 'PAN', 'MEX', 'ENG', 'COL', 'JPN', 'POL', 'SEN',
            'RUS', 'EGY', 'POR', 'MAR', 'URU', 'KSA', 'IRN', 'ESP',
            'DEN', 'AUS', 'FRA', 'PER', 'ARG', 'CRO', 'BRA', 'CRC',
            'NGA', 'ISL', 'SRB', 'SUI', 'BEL', 'TUN', 'GER', 'SWE']

Normally, you would run a Python interpreter to test out APIs before writing final code in a .py file. This provides you with a much quicker feedback loop to check whether or not the API handling code is working as expected.

This is the result of testing the API:

We can get “today” (& “tomorrow”) information by running the following code:

import requests
# ...
html = requests.get(urls['today']).json()
for match in html:
     print(match['home_team_country'], 'vs',
        match['away_team_country'], 'at',
        match['datetime'])

Attention

This endpoint will not return anything now because FIFA world cup is over. The other endpoints should still work.

We can get “country” information by running:

import requests
# ...
data = requests.get(urls['country']+'ARG').json()
for match in data:
    if match['status'] == 'completed':
        print(match['home_team']['country'],
            match['home_team']['goals'],
            "vs", match['away_team']['country'],
            match['away_team']['goals'])
    if match['status'] == 'future':
        print(match['home_team']['country'], "vs",
            match['away_team']['country'],
            "at", match['datetime'])

We can get “complete” information by running:

import requests
# ...
data = requests.get(urls['group']).json()
for group in data:
    print("--- Group", group['letter'], "---")
    for team in group['ordered_teams']:
        print(team['country'], "Pts:",
        team['points'])

And lastly we can get “list” information by simply running:
print('\n'.join(countries))

In order to explore the JSON APIs, you can make use of JSBeautifier. This will help you find out the right node fairly quickly through proper indentation. In order to use this amazing resource, just copy JSON response, paste it on the JSBeautifier website and press “Beautify JavaScript or HTML” button. It will look something like Fig. 4.2

Now that you know which API to use and what code is needed for extracting the required information, you can move on and start editing the app.py file.

4.4 Start writing `app.py`¶

First of all, let’s import the required libraries:

import os
from flask import Flask, request
import requests
from dateutil import parser, tz
from twilio.twiml.messaging_response import MessagingResponse

We are going to use os module to access environment variables. In this particular project, you don’t have to use your Twilio credentials anywhere, but it is still good to know that you should never hardcode your credentials in any code file. You should use environment variables for storing them. This way, even if you publish your project in a public git repo, you won’t have to worry about leaked credentials.

We will be using flask as our web development framework of choice and requests as our preferred library for consuming online APIs. Moreover, dateutil will help us parse dates-times from the online API responses.

We will be using MessagingResponse from the twilio.twiml.messaging_response package to create TwiML responses. These are response templates used by Twilio. You can read more about TwiML here.

Next, you need to create the Flask object:

app = Flask(__name__)

Now, define your local timezone using the tz.gettz method. The example uses America/New_york as the time zone, but you can use any another time zone to better suit your location:

to_zone = tz.gettz('America/New_York')

This app will only have one route. This is the / route. This will accept POST requests. We will be using this route as the “message arrive” webhook in Twilio. This means that whenever someone sends an SMS to your Twilio number, Twilio will send a POST request to this webhook with the contents of that SMS. We will respond to this POST request with a TwiML template, which will tell Twilio what to send back to the SMS sender.

Here is the basic “hello world” code to test this out:

@app.route('/', methods=['POST'])
def receive_sms():
    body = request.values.get('Body', None)
    resp = MessagingResponse()
    resp.message(body or 'Hello World!')
    return str(resp)

Let’s complete your app.py script and test it:

if __name__ == "__main__":
    port = int(os.environ.get("PORT", 5000))
    app.run(host='0.0.0.0', port=port)

At this point, the complete contents of this file should look something like this:

import os
from flask import Flask, request
import requests
from dateutil import parser, tz
from twilio.twiml.messaging_response import MessagingResponse

app = Flask(__name__)
to_zone = tz.gettz('America/New_York')

@app.route('/', methods=['POST'])
def receive_sms():
    body = request.values.get('Body', None)
    resp = MessagingResponse()
    resp.message(body or 'Hello World!')
    return str(resp)

if __name__ == "__main__":
    port = int(os.environ.get("PORT", 5000))
    app.run(host='0.0.0.0', port=port)

Add the following line to your Procfile:

web: python app.py

This will tell Heroku which file to run. Add the following code to the runtime.txt file:

python-3.6.8

This will tell Heroku which Python version to use to run your code.

You’ll want to make use of version control with git and push your code to Heroku by running the following commands in the terminal:

$ git init .
$ git add Procfile runtime.txt app.py requirements.txt
$ git commit -m "Committed initial code"
$ heroku create
$ heroku apps:rename custom_project_name
$ git push heroku master

Note

If you don’t have Heroku CLI installed the above commands with heroku prefix will fail. Make sure you have the CLI tool installed and you have logged in to Heroku using the tool. Other ways to create Heroku projects are explained in later chapters. For now, simply download and install the CLI.

Replace custom_project_name with your favorite project name. This needs to be unique, as this will dictate the URL where your app will be served. After running these commands, Heroku will provide you with a public URL for your app. Now you can copy that URL and sign up on Twilio!

4.5 Getting started with Twilio¶

Go to Twilio and sign up for a free trial account if you don’t have one already (Fig. 4.3).

At this point Twilio should prompt you to select a new Twilio number. Once you do that you need to go to the Console’s “number” page and you need to configure the webhook (Fig. 4.4).

Here you need to paste the server address which Heroku gave you. Now it’s time to send a message to your Twilio number using a mobile phone (it should echo back whatever you send it).

Here’s what that should look like Fig. 4.5.

If everything is working as expected, you can move forward and make your app.py file do something useful.

4.6 Finishing up `app.py`¶

Rewrite the receive_sms function based on this code:

# ...

urls = {'group': 'https://worldcup.sfg.io/teams/group_results',
        'country': 'https://worldcup.sfg.io/matches/country?fifa_code=',
        'today': 'https://worldcup.sfg.io/matches/today',
        'tomorrow': 'https://worldcup.sfg.io/matches/tomorrow'
}

#...

@app.route('/', methods=['POST'])
def receive_sms():
    body = request.values.get('Body', '').lower().strip()
    resp = MessagingResponse()

    if body == 'today':
        data = requests.get(urls['today']).json()
        output = "\n"
        for match in data:
            output += match['home_team_country'] + ' vs ' + \
            match['away_team_country'] + " at " + \
            parser.parse(match['datetime']).astimezone(to_zone)
                .strftime('%I:%M %p') + "\n"
        else:
            output += "No matches happening today"

    elif body == 'tomorrow':
        data = requests.get(urls['tomorrow']).json()
        output = "\n"
        for match in data:
            output += match['home_team_country'] + ' vs ' + \
            match['away_team_country'] + " at " + \
            parser.parse(match['datetime']).astimezone(to_zone)
                .strftime('%I:%M %p') + "\n"
        else:
            output += "No matches happening tomorrow"

    elif body.upper() in countries:
        data = requests.get(urls['country']+body).json()
        output = "\n--- Past Matches ---\n"
        for match in data:
            if match['status'] == 'completed':
                output += match['home_team']['country'] + " " + \
                          str(match['home_team']['goals']) + " vs " + \
                          match['away_team']['country']+ " " + \
                          str(match['away_team']['goals']) + "\n"

        output += "\n\n--- Future Matches ---\n"
        for match in data:
            if match['status'] == 'future':
                output += match['home_team']['country'] + " vs " + \
                          match['away_team']['country'] + " at " + \
                          parser.parse(match['datetime']).astimezone(to_zone)
                            .strftime('%I:%M %p on %d %b') +"\n"

    elif body == 'complete':
        data = requests.get(urls['group']).json()
        output = ""
        for group in data:
            output += "\n\n--- Group " + group['letter'] + " ---\n"
            for team in group['ordered_teams']:
                output += team['country'] + " Pts: " + \
                            str(team['points']) + "\n"

    elif body == 'list':
        output = '\n'.join(countries)

    else:
        output = ('Sorry we could not understand your response. '
            'You can respond with "today" to get today\'s details, '
            '"tomorrow" to get tomorrow\'s details, "complete" to '
            'get the group stage standing of teams or '
            'you can reply with a country FIFA code (like BRA, ARG) '
            'and we will send you the standing of that particular country. '
            'For a list of FIFA codes send "list".\n\nHave a great day!')

    resp.message(output)
    return str(resp)

The code for date-time parsing is a bit less intuitive:

parser.parse(match['datetime']).astimezone(to_zone).strftime('%I:%M %p on %d %b')

Here you are passing match['datetime'] to the parser.parse method. After that, you use the astimezone method to convert the time to your time zone, and, finally, format the time.

%I gives us the hour in 12-hour format
%M gives us the minutes
%p gives us AM/PM
%d gives us the date
%b gives us the abbreviated month (e.g Jun)

You can learn more about format codes from here.

After adding this code, the complete app.py file should look something like this:

import os
from flask import Flask, request
import requests
from dateutil import parser, tz
from twilio.twiml.messaging_response import MessagingResponse

app = Flask(__name__)
to_zone = tz.gettz('America/New_York')

countries = ['KOR', 'PAN', 'MEX', 'ENG', 'COL', 'JPN', 'POL', 'SEN', 
                'RUS', 'EGY', 'POR', 'MAR', 'URU', 'KSA', 'IRN', 'ESP', 
                'DEN', 'AUS', 'FRA', 'PER', 'ARG', 'CRO', 'BRA', 'CRC', 
                'NGA', 'ISL', 'SRB', 'SUI', 'BEL', 'TUN', 'GER', 'SWE']

urls = {'group': 'http://worldcup.sfg.io/teams/group_results',
        'country': 'http://worldcup.sfg.io/matches/country?fifa_code=',
        'today': 'http://worldcup.sfg.io/matches/today',
        'tomorrow': 'http://worldcup.sfg.io/matches/tomorrow'
}

@app.route('/', methods=['POST'])
def receive_sms():
    body = request.values.get('Body', '').lower().strip()
    resp = MessagingResponse()

    if body == 'today':
        html = requests.get(urls['today']).json()
        output = "\n"
        for match in html:
            output += (
                match['home_team_country'] + " vs " +
                match['away_team_country'] + " at " +
                parser.parse(match['datetime']).astimezone(to_zone)
                    .strftime('%I:%M %p') + "\n"
            )
        else:
            output += "No matches happening today"

    elif body == 'tomorrow':
        html = requests.get(urls['tomorrow']).json()
        output = "\n"
        for match in html:
            output += (
                match['home_team_country'] + " vs " +
                match['away_team_country'] + " at " +
                parser.parse(match['datetime']).astimezone(to_zone)
                    .strftime('%I:%M %p') + "\n"
            )
        else:
            output += "No matches happening tomorrow"

    elif body.upper() in countries:
        html = requests.get(urls['country']+body).json()
        output = "\n--- Past Matches ---\n"
        for match in html:
            if match['status'] == 'completed':
                output += (
                    match['home_team']['country'] + " " +
                    str(match['home_team']['goals']) + " vs " +
                    match['away_team']['country'] + " " +
                    str(match['away_team']['goals']) + "\n"
                )

        output += "\n\n--- Future Matches ---\n"
        for match in html:
            if match['status'] == 'future':
                output += (
                    match['home_team']['country'] + " vs " +
                    match['away_team']['country'] + " at " +
                    parser.parse(match['datetime'])
                      .astimezone(to_zone)
                      .strftime('%I:%M %p on %d %b') + "\n"
                )
    
    elif body == 'complete':
        html = requests.get(urls['group']).json()
        output = ""
        for group in html:
            output += "\n\n--- Group " + group['letter'] + " ---\n"
            for team in group['ordered_teams']:
                output += (
                    team['country'] + " Pts: " +
                    str(team['points']) + "\n"
                )

    elif body == 'list':
        output = '\n'.join(countries)

    else:
        output = ('Sorry we could not understand your response. '
            'You can respond with "today" to get today\'s details, '
            '"tomorrow" to get tomorrow\'s details, "complete" to '
            'get the group stage standing of teams or '
            'you can reply with a country FIFA code (like BRA, ARG) '
            'and we will send you the standing of that particular country. '
            'For a list of FIFA codes send "list".\n\nHave a great day!')

    resp.message(output)
    return str(resp)

if __name__ == "__main__":
    port = int(os.environ.get("PORT", 5000)) 
    app.run(host='0.0.0.0', port=port)

Now you just need to commit this code to your git repo and push it to Heroku:

$ git add app.py
$ git commit -m "updated the code :boom:"
$ git push heroku master

Now you can go ahead and try sending an SMS to your Twilio number.

4.7 Troubleshoot¶

If you don’t receive a response to your SMS, you should check your Heroku app logs for errors. You can easily access the logs by running $ heroku logs from the project folder
Twilio also offers an online debug tool which can help troubleshoot issues
Twilio requires you to verify the target mobile number before you can send it any SMS during the trial. Make sure you do that
Don’t feel put off by errors. Embrace them and try solving them with the help of Google and StackOverflow

4.8 Next Steps¶

Now that you have a basic bot working, you can create one for NBA, MLB, or something completely different. How about a bot which allows you to search Wikipedia just by sending a text message? I am already excited about what you will make!

I hope you learned something in this chapter. See you in the next one!

Sign up for updates!