billysoftacademy.com

Learn how to speed up Python - MySQL application session handling with Redis On Linux Ubuntu 22.04 LTS

Introduction

The process of validating user identity during login requests is known as authentication. During this process, users provide their login credentials in the form of usernames and passwords, which are then matched against stored database entries by the application. If the login credentials match, the user is granted access to the system.Typically, login credentials are stored in a relational database such as MySQL or PostgreSQL without any caching mechanism. However, this approach has certain limitations. 

For instance, it can cause database overloading as the application has to make a roundtrip to the database server to verify user credentials every time a login request is submitted. Also, disk-based databases do not perform optimally when faced with a high volume of requests.

To address these challenges, Redis can be used to cache user login credentials. Redis is a fast data store that leverages the computer’s RAM to store data in key-value pairs. By using Redis, the application can speed up session handling and avoid contacting the backend database for each login request. In this guide, we will explore how to use Redis to improve the performance of Python/MySQL applications on an Ubuntu 22.04 server.

Requirements

The following is a list of items needed to complete the configuration successfully:
1) A Linux Ubuntu Server 20.04 or 22.04 LTS
2) MySQL Server 
3) Redis Server
4) An SSH Client such as Putty or the built in MacOS terminal app
5) A basic understanding of Linux commands

Overview

The following is an overview of the steps covered in this tutorial:
1) Installing Python DB Drivers for Redis and MySQL.
2) Configuring a simple MySQL Database.
3) Creating a MySQL gateway module for Python.
4) Creating a Redis module for Python.
5) Creating the Python Apps’ entry point.

Step 1: Installing Python DB Drivers fro Redis and MySQL.

The software saves user information, like usernames and passwords, in a MySQL database server. A Python script cross-references the database when someone logs in and saves their login information in a Redis database for future usage. To make this work, you will require database drivers, which are Python modules that enable communication with the MySQL and Redis servers. Follow these instructions to install the drivers:

1. Update your package information index and install python3-pip, a Python package manager that allows you to install additional modules that are not part of the Python standard library, by running this command:

    sudo apt install python3-pip

2. Install the MySQL driver for Python:

    pip install mysql-connector-python

3. Install the Redis driver for Python:

    pip install redis.

Step 2: Configuring a simple MySQL Database

You will need to create a MySQL table. In a real-world scenario, you may have multiple tables to handle various requests. Follow these steps to create a database and its table:

– To start, log in to your MySQL database server as a root user. Use the command “sudo mysql -u root -p” and enter your MySQL server’s root password when prompted.
– Run the following command to create a sample company database and a company_user account. Change “example-mysql-password” to a strong password.

  CREATE DATABASE company;
CREATE USER 'company_user'@'localhost' IDENTIFIED WITH mysql_native_password BY 'example-mysql-password';
GRANT ALL PRIVILEGES ON company.* TO 'company_user'@'localhost';
FLUSH PRIVILEGES;

– Confirm that the previous commands ran successfully by checking the output.
– Switch to the new company database by entering “USE company” in the command line.
– Create a system_users table with the necessary columns to store user data. Use the command “CREATE TABLE system_users…” to create the table.
– Verify that the table was created successfully by checking the output.
– Populate the system_users table with sample data using the command “INSERT INTO system_users…”.
– Verify that the data was inserted correctly by querying the system_users table with “SELECT user_id, first_name, last_name, password FROM system_users;”.
– Log out from the MySQL database by entering “QUIT” in the command line.

Now that you have created the necessary MySQL table, you can move on to building a Python module that interacts with the database.

Step 3: Creating a MySQL gateway module for Python

In order to ensure code reusability, it is recommended that you create separate modules for each task when working on a Python project. To set up a central module that can communicate with and query a MySQL database from a Python script, you can follow the steps below:

1. Create a new project directory that will store your Python source code files separately from other system files. For example:

mkdir project

2. Navigate to the new project directory using the command:

cd project

3. Use a text editor like nano to create a new file called mysql_db.py. This file will contain the Python module that interacts with the MySQL database.

nano mysql_db.py

4. Enter the following details into the mysql_db.py file, replacing “example-mysql-password” with the correct MySQL password for the “company_user” account:

~/project/mysql_db.py

import mysql.connector

class MysqlDb:

    def db_con(self):

        mysql_con = mysql.connector.connect(
            host="localhost",
            user="company_user",
            password="example-mysql-password",
            database="company",
            port="3306"
        )

        return mysql_con

    def query(self, username, password):

        db = self.db_con()
        db_cursor = db.cursor()

        db_query = "select username, password from system_users where username = %s and password = md5(%s)"
        db_cursor.execute(db_query, (username, password))

        result = db_cursor.fetchone()
        row_count = db_cursor.rowcount

        if row_count < 1:
            return False
        else:
            return result[1]

5. Save and close the mysql_db.py file.

The mysql_db.py module contains a single class (MysqlDb:) with two methods:

– db_con(self): This method connects to the MySQL database that you created earlier and returns a reusable MySQL connection using the “return mysql_con” statement.

– query(self, username, password): This method accepts a username and password, and queries the system_users table to find a match. If a user doesn’t exist in the table, the method returns the boolean False value. If a match is found, the method returns the user’s password (result[1]).

Now that the MySQL module is ready, you can move on to setting up a Redis module that communicates with the Redis key-value store.

Step 4: Creating a Redis module for Python

For this stage of the process, you will be developing a module that links up with the Redis server. Here are the steps to follow:

1. Begin by creating a new file called redis_db.py using the nano editor.
2. In the redis_db.py file, input the following details, making sure to replace “example-redis-password” with the correct password for the Redis server:

~/project/redis_db.py

 
    import redis
    
    class RedisDb:
    
        def db_con(self):
    
            r_host = 'localhost'
            r_port = 6379
            r_pass = 'example-redis-password'
            redis_con = redis.Redis(host = r_host, port = r_port, password = r_pass)
    
            return redis_con
 

3. Once you’ve entered all the necessary information, save and exit the redis_db.py file.
4. The RedisDb class in the above file consists of one method – db_con(self): – that uses the provided credentials to connect to the Redis server. This method then returns a reusable connection using the “return redis_con” statement.
5. With the Redis class now set up, you can move on to creating the main file for your project in the next step

Step 5: Creating the Python Apps' entry point.

It is essential for every Python application to have an initial point, also known as the primary file, that executes as soon as the application runs. In this file, you will develop code that displays the authenticated users’ current server time. The custom MySQL and Redis modules developed earlier are used in this file to authenticate users.

To create the file, follow the instructions below:

1. Open a new file named index.py using nano index.py command.
2. Enter the following details in the index.py file:

~/project/index.py

    from encodings import utf_8
import base64
from hashlib import md5
import json

import datetime

import http.server
from http import HTTPStatus
import socketserver

import mysql_db
import redis_db

class HttpHandler(http.server.SimpleHTTPRequestHandler):

def do_GET(self):

self.send_response(HTTPStatus.OK)
self.send_header('Content-type', 'application/json')
self.end_headers()

authHeader = self.headers.get('Authorization').split(' ');
auth_user, auth_password = base64.b64decode(authHeader[1]).decode('utf8').split(':')

mysql_server = mysql_db.MysqlDb()
redis_server = redis_db.RedisDb()

redis_client = redis_server.db_con()

now = datetime.datetime.now()
current_time = now.strftime("%Y-%m-%d %H:%M:%S")

resp = {}

if redis_client.exists(auth_user):

if md5(auth_password.encode('utf8')).hexdigest() != redis_client.get(auth_user).decode('utf8'):
resp = {"error": "Invalid username/password."}
else:
resp = {"time": current_time, "authorized by": "Redis server"}
else:

mysql_resp = mysql_server.query(auth_user, auth_password)

if mysql_resp == False:
resp = {"error": "Invalid username/password."}
else:
resp = {"time": current_time, "authorized by": "MySQL server"}
redis_client.set(auth_user, mysql_resp)

self.wfile.write(bytes(json.dumps(resp, indent = 2) + "\r\n", "utf8"))

httpd = socketserver.TCPServer(('', 8080), HttpHandler)
print("Web server is running on port 8080...")

try:
httpd.serve_forever()
except KeyboardInterrupt:
httpd.server_close()
print("Web server has stopped runing.")

3. Save and close the index.py file.

The import section in the index.py file adds various modules, including utf_8, base64, md5, and json, which are used to encode and format text, and http.server, HTTPStatus, and socketserver that are web server modules.

The datetime module is used to obtain the system’s date and time, while mysql_db and redis_db are custom modules that were created earlier to access the MySQL and Redis servers.

The HttpHandler class is a handler class for the HTTP server, and under the class, the do_GET function serves the HTTP GET requests and displays the system’s date/time for authenticated users.

The Python script runs the if…else… logic to verify whether the user’s details exist in the Redis server. If the user details exist and the Redis stored password doesn’t match the user’s submitted password, the application returns the {“error”: “Invalid username/password.”} error.

If the user details do not exist in the Redis server, the application queries the MySQL database server using the mysql_resp = mysql_server.query(auth_user, auth_password) statement. In case the user’s supplied password doesn’t match the database stored value, the application returns the {“error”: “Invalid username/password.”} error. Otherwise, the application caches the user’s details in the Redis server using the redis_client.set(auth_user, mysql_resp) statement.

In all cases where the user’s credentials match the Redis/MySQL details, the application displays the system’s current date/time using the {“time”: current_time, …} statement. The authorized by entry in the output allows you to see the database server that authenticates the users in the application.

You have now set up the main file for the application. In the next step, you’ll test the application.

Step 6: Running tests for the application

The next step is to test if the Redis caching mechanism is functioning properly. To do this, you can run the commands below:

First, run the application using the following python3 command:

python3 index.py

Make sure that the custom web server of the application is running correctly by checking the output:

Output
Web server is running on port 8080…

To test the Redis caching mechanism, you will need to open a new terminal window and establish a new SSH connection to your server. Then, use the following curl commands to send four GET requests using the credentials of john_doe. To send the four requests using a single command, append [1-4] to the end of the http://localhost:8080/ URL:

curl -X GET -u john_doe:password_1 http://localhost:8080/[1-4]

After executing the commands above, you should verify the output to ensure that the MySQL server serves only the first authentication request and the Redis database serves the next three requests:

[1/4]:...
{
"time": "2023-11-07 10:04:38",
"authorized by": "MySQL server"
}

...
[4/4]...
{
"time": "2023-11-07 10:04:38",
"authorized by": "Redis server"
}

By following the steps above, you can confirm that your application logic is functioning as expected.

Conclusion

In conclusion, integrating Redis into Python/MySQL applications can significantly enhance session handling and overall application performance. Here’s a recap of the key takeaways:

Key Benefits:

Reduced database load: Redis alleviates pressure on the MySQL server by caching frequently accessed login credentials, minimizing database roundtrips.
Faster response times: Redis, as an in-memory data store, retrieves data much faster than disk-based databases, leading to swifter response times.
Improved scalability: Redis can handle a high volume of concurrent requests efficiently, enabling applications to scale gracefully under load.


Implementation Steps:

1. Install Python drivers: Install mysql-connector-python and redis for database interactions.
2. Configure MySQL database: Create a table to store user credentials.
3. Create MySQL gateway module: Establish reusable MySQL connections for queries.
4. Create Redis module: Connect to Redis for caching operations.
5. Develop application entry point: Integrate modules and handle authentication logic.
6. Test the application: Verify Redis caching is functioning as intended.

Redis proves to be a valuable tool for optimizing session handling and enhancing the performance of Python/MySQL applications. By following these guidelines, you can effectively integrate Redis and reap its performance benefits for your applications.

Scroll to Top