Knowledgebase

Introduction

Redis Pub/Sub is a computer architecture that allows publishers to send information to subscribers through channels. You can use this technology to build real-time applications that rely on Inter-Process Communications(IPCs). For instance, webchats, financial systems, and more. In these systems, speed is the name of the game and when end-users connect to a channel, they expect to receive messages with the shortest latency possible. When designing highly responsive applications with traditional SQL databases, you might face scalability issues. However, Redis stores data in your server's RAM and can execute many read/write cycles per second.

Another great advantage of implementing the Pub/Sub paradigm with Redis is the ability to detach the publishers and subscribers(decoupling). You simply send data to a channel without the need to define the recipients. Similarly, subscribers can show interest in any channel and wait for messages without any knowledge about the publishers.

In that kind of a decoupled system, your frontend and backend engineers can design and test their builds independently reducing the overall time needed to complete a project since everything is done in a parallel manner. This improves the freedom of developers and streamlines your recruitment process because the loosely coupled components reduce the complexity of the final application.

In this guide, you'll create a decoupled water billing application with PHP and the Redis Server. In this system, you'll use a frontend script(publisher) to accept water consumption data from homeowners and send the information to a billings channel. Under the hood, you'll use a backend script(subscriber) to process the data from the billings channel and save it permanently to a MySQL database.

Prerequisites

To follow along with this Redis Pub/Sub tutorial, ensure you've the following:

• An Ubuntu 20.04 server.
• A sudo user.
• A Lamp Stack.
• A Redis Server

1. Install the PHP Redis Exension

To connect to the Redis Server via a PHP script, you need to install the php-redis extension. SSH to your server and update the package information index.

$ sudo apt update

Next, run the following command to install the extension.

$ sudo apt install -y php-redis

Restart the Apache webserver to load the new changes.

$ sudo systemctl restart apache2

Once you've set up the right environment for connecting to the Redis server from PHP, you'll create a MySQL database next.

2. Create a Sample Database and User

Redis is an in-memory database, and although it can persist data to disk, it was not designed for that purpose. It is mainly meant to handle performance-focused data. For instance, in this sample water billing application, the Redis server will ingest water consumption usage from data clerks in a timely manner. Then, you'll process and save the data permanently to a disk on a MySQL database.

To create the database, log in to your MySQL database server as root.

$ sudo mysql -u root -p

Next, enter the root password for your MySQL server and press Enter to proceed. Then, run the following statements to create a billing_db database and a privileged user to connect to it. Replace EXAMPLE_PASSWORD with a strong value.

mysql> CREATE DATABASE billing_db;
       CREATE USER 'billing_db_user'@'localhost' IDENTIFIED WITH mysql_native_password BY 'EXAMPLE_PASSWORD';
       GRANT ALL PRIVILEGES ON billing_db.* TO 'billing_db_user'@'localhost';           
       FLUSH PRIVILEGES;

Output.

...
Query OK, 0 rows affected (0.00 sec)

Switch to the new database.

mysql> USE billing_db;

Make sure you've selected the billing_db database by confirming the output below.

Database changed

Next create a customers table to store clients' data including the customer_id, first_name, and last_name.

mysql> CREATE TABLE customers (
           customer_id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
           first_name VARCHAR(50),
           last_name VARCHAR(50)           
       ) ENGINE = InnoDB;

Make sure you get the following output to confirm that you've created the table.

Query OK, 0 rows affected (0.01 sec)

Next, use the following INSERT commands to enter sample data into the customers table.

mysql> INSERT INTO customers (first_name, last_name) values ('JOHN', 'DOE');
       INSERT INTO customers (first_name, last_name) values ('MARY', 'SMITH');
       INSERT INTO customers (first_name, last_name) values ('STEVE', 'JONES');

After creating each record in the table, you should receive the following confirmation message.

...  
Query OK, 1 row affected (0.01 sec)

Next, set up a table to handle monthly bills for the customers. Once you receive the water consumption data for each customer, you'll store the information in a billings table. In this table, the customer_id refers back to the same field in the customers table. To identify each bill, you'll use the ref_id column. The billing_period field stores the last date of the month when the bill is incurred. The units_consumed is the actual cubic meters of water the homeowner has used during the period. The cost_per_unit represents the amount your sample company is charging per unit in dollars($) per month.

Execute the command below to create the billings table.

mysql> CREATE TABLE billings (
           ref_id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
           customer_id BIGINT,
           billing_period VARCHAR(50),
           units_consumed DOUBLE,
           cost_per_unit  DOUBLE          
       ) ENGINE = InnoDB;

Make sure you get the output below.

Query OK, 0 rows affected (0.02 sec)

Exit from the MySQL command-line interface.

mysql> QUIT;

Output.

Bye

You've now created the database, set up tables, and entered some sample data. In the next step, you'll code a frontend script that accepts water usage data from data clerks for billing purposes.

3. Create a Frontend Script on PHP

In the billing system, water consumption for homeowners is measured in cubic meters. Unless smart devices are installed in those homes, data clerks have to physically visit and manually record the readings from the water meters and relay the same information to the company's database.

In the modern world, this information can be sent via mobile applications that connect to an API hosted on a central cloud server. In this step, you'll create a frontend script that accepts such data using PHP. Use nano to create a new frontend.php file in the root directory of your web server.

$ sudo nano /var/www/html/frontend.php

Next, enter the following information into the /var/www/html/frontend.php file.

<?php 

    try {   
             
        $redis = new Redis(); 
        $redis->connect('127.0.0.1', 6379);

        $channel      = 'billings';
        $billing_data = file_get_contents('php://input');          

        $redis->publish($channel, $billing_data); 

        echo "Data successfully sent to the billings channel.\n";         

        } catch (Exception $e) {
            echo $e->getMessage();
        }

Save and close the file when you're through with editing. Before you proceed with the rest of the guide, here is the /var/www/html/frontend.php explained:

1. You're connecting to the Redis server at localhost on port 6379 using the code below.

    $redis = new Redis(); 
    $redis->connect('127.0.0.1', 6379);
    ...

2. Then, you're initializing a new $channel variable and retrieving incoming JSON data into the script and assigning it to a $billing_data variable using the PHP file_get_contents('php://input'); statement.

    ...
    $channel      = 'billings';
    $billing_data = file_get_contents('php://input'); 
    ...

3. Finally, you're publishing the new JSON input data into the billings channel using the $redis->publish statement command and echoing out a success message. Also, you're capturing any errors that you might encounter using the PHP catch(...){...} block as shown below.

    ...
   
    $redis->publish($channel, $billing_data); 
   
    echo "Data successfully sent to the billings channel.\n";         
   
    } catch (Exception $e) {
        echo $e->getMessage();
    }
    ...

You've now created a frontend.php script that receives customers' billing data and routes it to a Redis channel. In the next step, you'll create a backend script to process the data.

4. Create a Backend Script on PHP

To process the data from the frontend.php script and save it in the MySQL database as you receive it, you'll SUBSCRIBE to the billings channel through a backend.php script. Use nano to open a new /var/www/html/backend.php file.

$ sudo nano /var/www/html/backend.php

Then, enter the information below into the file.

<?php 

    try {   
             
        $redis = new Redis(); 
        $redis->connect('127.0.0.1', 6379);
        $redis->setOption(Redis:: OPT_READ_TIMEOUT, -1);       

        $redis->subscribe(['billings'], function($instance, $channelName, $message) { 
         
            $billing_data = json_decode($message, true); 

            $db_name     = 'billing_db';
            $db_user     = 'billing_db_user';
            $db_password = 'EXAMPLE_PASSWORD';
            $db_host     = 'localhost';

            $pdo = new PDO('mysql:host=' . $db_host . '; dbname=' . $db_name, $db_user, $db_password);
            $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);   

            $sql = 'insert into billings(
                        customer_id, 
                        billing_period, 
                        units_consumed,
                        cost_per_unit
                    ) 
                    values(
                        :customer_id, 
                        :billing_period, 
                        :units_consumed,
                        :cost_per_unit
                    )'; 

            $data = [];

            $data = [
                    'customer_id'    => $billing_data['customer_id'],
                    'billing_period' => $billing_data['billing_period'],
                    'units_consumed' => $billing_data['units_consumed'],
                    'cost_per_unit'  => 2.5                                 
                    ];

            $stmt = $pdo->prepare($sql);
            $stmt->execute($data); 

            echo "The Redis data was sent to the MySQL database successfully.\n" ;               
                    
        });         

        } catch (Exception $e) {
            echo $e->getMessage();
        }

Save and close the file when you're through with editing. In the above file, you're connecting to the Redis server and subscribing to the billings channel using the $redis->subscribe(['billings'], function(...) {...}) statement. You've used the line $redis->setOption(Redis:: OPT_READ_TIMEOUT, -1); to force the script to never timeout. Once you receive the data from the channel in real-time, you're sending it to the MySQL server through a prepared statement. Your backend script is now in place. Next, you'll test the Redis Pub/Sub logic.

5. Test the Redis Pub/Sub Application

Connect to your Linux server on two different terminal windows. On the first window, run the following backend.php script.

$ php /var/www/html/backend.php

Please note, the script above has a blocking function that listens and waits for messages from a Redis server in real-time. Therefore, you should not try to execute any other command once you run the script. For now, don't expect any output from the script. On the second terminal window, run the curl commands below to send data to the frontend.php script.

$ curl -X POST http://localhost/frontend.php -H 'Content-Type: application/json' -d '{"customer_id":1, "billing_period": "2021-08-31", "units_consumed":12.36}'
$ curl -X POST http://localhost/frontend.php -H 'Content-Type: application/json' -d '{"customer_id":2, "billing_period": "2021-08-31", "units_consumed":40.20}'
$ curl -X POST http://localhost/frontend.php -H 'Content-Type: application/json' -d '{"customer_id":3, "billing_period": "2021-08-31", "units_consumed":24.36}'

You're using curl to send data to the frontend script in this guide to prove the concept. In a production environment, data clerks might capture the information through mobile apps, desktop applications, or web applications. After executing each curl command, you should receive the following output showing data has been successfully sent to the Redis billings channel.

...
Data successfully sent to the billings channel.

On the second window where your /var/www/html/backend.php script is subscribed to the billings channel and listening for messages, you should receive the following output showing that data is automatically sent to the MySQL database as soon as it is received.

...
The Redis data was sent to the MySQL database successfully.

To verifying whether the data was successfully routed from the Redis channel to the MySQL database, log in to the MySQL server as `billing_db_user'.

$ mysql -u billing_db_user -p

Enter your billing_db_user password(For instance, EXAMPLE_PASSWORD) and press Enter to proceed. Then, run the USE statement to switch to the billing_db database.

mysql> USE billing_db;

Output.

Database changed.

Next, execute the following SELECT statement to retrieve the records. To get valuable information about the customers' bills showing their names, link the billings and the customers tables by executing the following JOIN statement.

mysql> SELECT
           billings.ref_id as bill_reference_no,
           customers.customer_id,
           customers.first_name,
           customers.last_name,
           billings.billing_period,
           billings.units_consumed,
           billings.cost_per_unit,
          CONCAT('$', (billings.units_consumed * billings.cost_per_unit)) as bill_amount
       FROM billings
       LEFT JOIN customers
       ON billings.customer_id = customers.customer_id;

You should now see all the processed customers' bills in the table as shown below.

+-------------------+-------------+------------+-----------+----------------+----------------+---------------+-------------+
| bill_reference_no | customer_id | first_name | last_name | billing_period | units_consumed | cost_per_unit | bill_amount |
+-------------------+-------------+------------+-----------+----------------+----------------+---------------+-------------+
|                 1 |           1 | JOHN       | DOE       | 2021-08-31     |          12.36 |           2.5 | $30.9       |
|                 2 |           2 | MARY       | SMITH     | 2021-08-31     |           40.2 |           2.5 | $100.5      |
|                 3 |           3 | STEVE      | JONES     | 2021-08-31     |          24.36 |           2.5 | $60.9       |
+-------------------+-------------+------------+-----------+----------------+----------------+---------------+-------------+
3 rows in set (0.00 sec)

The above output confirms that the Redis Pub/Sub logic is working as expected.

Conclusion

In this guide, you've used the Redis Pub/Sub feature to publish water billing data to a channel on your Ubuntu 20.04 server with PHP. You've then subscribed to the channel to process and save data to the MySQL database. This guide shows you the versatility of the fast Redis in-memory database server in decoupling a system.