Knowledgebase

Introduction

Redis is an open-source in-memory data store that can work as a quick-response database. It's capable of handling millions of queries per second with high availability and scalability. A Redis Cluster is a group of Redis instances used to achieve high availability, improved fault tolerance, and horizontal scaling in data management systems.

A Redis cluster uses a full mesh topology where every node interconnects with the main node using a TCP connection. In Kubernetes, pods interconnect to each other making it possible to deploy Redis for high availability within a cluster.

Benefits of deploying Redis in a Kubernetes cluster include:

• High Availability

• Scalability

• Load Balancing and Service Discovery

• Simplified Deployment and Management

• Monitoring and Logging

This article explains how to deploy a Redis Cluster on a RCS Kubernetes Engine (VKE) cluster.

Prerequisites

Before you begin

• Deploy a RCS Kubernetes Engine (VKE) cluster with at least three nodes

• Install Kubectl on your local machine to access the cluster

Create the Redis Namespace

By default, Kubernetes adds all cluster components such as services, pods and ConfigMaps to the default namespace. By creating a separate namespace, you are able to manage pods and services more efficiently in the cluster. In this section, create the Redis namespace as described in the steps below.

1. Create a new namespace for the Redis cluster.

   $ kubectl create ns redis
   

2. Verify that the new namespace is available.

   $ kubectl get ns
   

   Output:

   NAME              STATUS   AGE
   
   default           Active   12m
   
   kube-node-lease   Active   12m
   
   kube-public       Active   12m
   
   kube-system       Active   12m
   
   redis             Active   7s
   

Define a Storage Class

A storage class is a Kubernetes resource that allows you to reserve disk space or attach volumes from a cloud provider to your cluster. By default, pods do not store the data permanently. When a pod gets deleted or restarts, data inside the pods is permanently lost.

In this section, mount a RCS Block Storage instance to your Kubernetes cluster to store data permanently, and avoid any data loss as described below.

1. Using a text editor such as Nano, create a new storage class YAML file.

   $ nano sc.yaml
   

2. Add the following configurations to the file.

   apiVersion: storage.k8s.io/v1
   
   kind: StorageClass
   
   metadata:
   
     name: redis-storage
   
   provisioner: block.csi.rcs.is
   
   volumeBindingMode: WaitForFirstConsumer
   
   allowVolumeExpansion: true
   
   reclaimPolicy: Delete
   

   Save and close the file.

3. Apply the storage class to your cluster.

   $ kubectl apply -f sc.yaml
   

4. Verify that the storage class is available.

   $ kubectl get sc
   

   Your output should look like the below:

   NAME                             PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
   
   redis-storage                    block.csi.rcs.is            Delete          WaitForFirstConsumer   true                   6s
   
   RCS-block-storage (default)    block.csi.rcs.is            Delete          Immediate              true                   12m
   
   RCS-block-storage-hdd          block.csi.rcs.is            Delete          Immediate              true                   12m
   
   RCS-block-storage-hdd-retain   block.csi.rcs.is            Retain          Immediate              true                   12m
   
   RCS-block-storage-retain       block.csi.rcs.is            Retain          Immediate              true                   12m
   

   As displayed in the above output, a new storage class redis-storage is available and it's attached to a RCS Block Storage volume.

Create a Persistent Volume

Data durability is essential for Redis deployments. Persistent volumes store Redis data on the cluster to achieve data durability in case a pod restarts. A Persistent volume requests a specified amount of storage from the Kubernetes cluster, and it's dynamically provisioned by a StorageClass.

In this section, create a persistent volume as described below.

1. Create a new manifest file to configure three persistent volumes using the RCS-block-storage provisioner.

   $ nano pv.yaml
   

2. Add the following configurations to the file.

   apiVersion: v1
   
   kind: PersistentVolume
   
   metadata:
   
     name: redis-pv1
   
   spec:
   
     storageClassName: RCS-block-storage
   
     capacity:
   
       storage: 10Gi
   
     accessModes:
   
       - ReadWriteOnce
   
     hostPath:
   
       path: "/storage/data1"
   
   ---
   
   apiVersion: v1
   
   kind: PersistentVolume
   
   metadata:
   
     name: redis-pv2
   
   spec:
   
     storageClassName: RCS-block-storage
   
     capacity:
   
       storage: 10Gi
   
     accessModes:
   
       - ReadWriteOnce
   
     hostPath:
   
       path: "/storage/data2"
   
   ---
   
   apiVersion: v1
   
   kind: PersistentVolume
   
   metadata:
   
     name: redis-pv3
   
   spec:
   
     storageClassName: RCS-block-storage
   
     capacity:
   
       storage: 10Gi
   
     accessModes:
   
       - ReadWriteOnce
   
     hostPath:
   
       path: "/storage/data3"
   

   Save and close the file.

   The above configuration creates three Persistent Volumes with a size of 10 GB using the RCS-block-storage provisioner. ReadWriteOnce means that the PVC is only mounted as read-write by a single node at a time.

3. Apply the configuration to the cluster

   $ kubectl apply -f pv.yaml
   

4. Verify the new persistent volumes

   $ kubectl get pv
   

   Your output should look like the one below:

   NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY    STATUS      CLAIM   STORAGECLASS         REASON   AGE
   
   redis-pv1   10Gi        RWO            Retain           Available           RCS-block-storage            6s
   
   redis-pv2   10Gi        RWO            Retain           Available           RCS-block-storage            5s
   
   redis-pv3   10Gi        RWO            Retain           Available           RCS-block-storage            4s
   

Create the ConfigMap

The ConfigMap is a key-value store in a Kubernetes cluster in which you can define the Redis configuration information. In this section, create a ConfigMap for the Redis cluster as described below.

1. Create a new ConfigMap YAML file

   $ nano redis-config.yaml
   

2. Add the following configurations to the file. Replace your_secure_password with a secure password for your cluster

   apiVersion: v1
   
   kind: ConfigMap
   
   metadata:
   
     name: redis-config
   
   data:
   
     redis.conf: |
   
       masterauth your_secure_password
   
       requirepass your_secure_password
   
       # Define the Redis listening interfaces
   
       bind 0.0.0.0
   
       # Enable or disable the protected mode
   
       protected-mode no
   
       # Define the Redis listening port
   
       port 6379
   
       tcp-backlog 511
   
       # Close the connection after a client is idle for N seconds (0 to disable)
   
       timeout 0
   
   
   
       tcp-keepalive 300
   
       daemonize no
   
       supervised no
   
       # Define Redis PID file
   
       pidfile "/var/run/redis_6379.pid"
   
       loglevel notice
   
       logfile ""
   
       databases 16
   
       always-show-logo yes
   
       save 900 1
   
       save 300 10
   
       save 60 10000
   
   
   
       stop-writes-on-bgsave-error yes
   
       rdbcompression yes
   
       rdbchecksum yes
   
       dbfilename "dump.rdb"
   
       rdb-del-sync-files no
   
       # Define the database storage location
   
       dir "/data"
   
       replica-serve-stale-data yes
   
       replica-read-only yes
   
       repl-diskless-sync no
   
       repl-diskless-sync-delay 5
   
       repl-diskless-load disabled
   
       repl-disable-tcp-nodelay no
   
       replica-priority 100
   
       acllog-max-len 128
   
       lazyfree-lazy-eviction no
   
       lazyfree-lazy-expire no
   
       lazyfree-lazy-server-del no
   
       replica-lazy-flush no
   
   
   
       lazyfree-lazy-user-del no
   
       appendonly yes
   
       appendfilename "appendonly.aof"
   
       appendfsync everysec
   
       no-appendfsync-on-rewrite no
   
       auto-aof-rewrite-percentage 100
   
       auto-aof-rewrite-min-size 64mb
   
   
   
       aof-load-truncated yes
   
       aof-use-rdb-preamble yes
   
       lua-time-limit 5000
   
       slowlog-log-slower-than 10000
   
       slowlog-max-len 128
   
       latency-monitor-threshold 0
   
       notify-keyspace-events ""
   
       hash-max-ziplist-entries 512
   
       hash-max-ziplist-value 64
   
       list-max-ziplist-size -2
   
       list-compress-depth 0
   
       set-max-intset-entries 512
   
       zset-max-ziplist-entries 128
   
       zset-max-ziplist-value 64
   
       hll-sparse-max-bytes 3000
   
       stream-node-max-bytes 4kb
   
       stream-node-max-entries 100
   
   
   
       activerehashing yes
   
       client-output-buffer-limit normal 0 0 0
   
       client-output-buffer-limit replica 256mb 64mb 60
   
       client-output-buffer-limit pubsub 32mb 8mb 60
   
       hz 10
   
       dynamic-hz yes
   
       aof-rewrite-incremental-fsync yes
   
       rdb-save-incremental-fsync yes
   
   
   
       jemalloc-bg-thread yes
   

   Save and close the file

   In the above configuration, the master and slavepasswords must be the same to establish a connection in the Redis cluster

3. Apply your configuration to the Kubernetes cluster

   $ kubectl apply -n redis -f redis-config.yaml
   

4. Verify that the ConfigMap is available in the Redis namespace

   $ kubectl get configmap -n redis
   

   Output:

   NAME               DATA   AGE 
   
   kube-root-ca.crt   1      110s
   
   redis-config       1      4s
   

   To view the full Redis code for a ConfigMap, visit the GitHub repository to fork or download the file.

Scale Redis in the Kubernetes Cluster using a StatefulSet

A StatefulSet deploys stateful applications and clustered applications that save data to persistent storage. It's suitable for deploying Redis and other applications that require persistent identities and stable hostnames. In this section, create a StatefulSet as below.

1. Create a new StatefulSet YAML file to scale the Redis cluster

   $ nano redis-statefulset.yaml
   

2. Add the following configurations to the file

   apiVersion: apps/v1
   
   kind: StatefulSet
   
   metadata:
   
     name: redis
   
   spec:
   
     serviceName: redis
   
     # Specify the number of Redis replicas
   
     replicas: 3
   
     selector:
   
       matchLabels:
   
         app: redis
   
     template:
   
       metadata:
   
         labels:
   
           app: redis
   
       spec:
   
         initContainers:
   
         - name: config
   
           # Specify the Redis docker image version
   
           image: redis:6.2.3-alpine
   
           command: [ "sh", "-c" ]
   
           args:
   
             - |
   
               # Copy the Redis configuration file to each Redis pod.
   
               cp /tmp/redis/redis.conf /etc/redis/redis.conf
   
   
   
               echo "finding master..."
   
               MASTER_FDQN=`hostname  -f | sed -e 's/redis-[0-9]\./redis-0./'`
   
               if [ "$(redis-cli -h sentinel -p 5000 ping)" != "PONG" ]; then
   
                 echo "master not found, defaulting to redis-0"
   
   
   
                 if [ "$(hostname)" == "redis-0" ]; then
   
                   echo "this is redis-0, not updating config..."
   
                 else
   
                   echo "updating redis.conf..."
   
                   echo "slaveof $MASTER_FDQN 6379" >> /etc/redis/redis.conf
   
                 fi
   
               else
   
                 echo "sentinel found, finding master"
   
                 MASTER="$(redis-cli -h sentinel -p 5000 sentinel get-master-addr-by-name mymaster | grep -E '(^redis-\d{1,})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})')"
   
                 echo "master found : $MASTER, updating redis.conf"
   
                 echo "slaveof $MASTER 6379" >> /etc/redis/redis.conf
   
               fi
   
           # Specify the Redis volume name and mount path
   
           volumeMounts:
   
           - name: redis-config
   
             mountPath: /etc/redis/
   
           - name: config
   
             mountPath: /tmp/redis/
   
         # Specify the Redis Docker image version, listening port, volume name, and mount path.
   
          containers:
   
         - name: redis
   
           image: redis:6.2.3-alpine
   
           command: ["redis-server"]
   
           args: ["/etc/redis/redis.conf"]
   
           ports:
   
           - containerPort: 6379
   
             name: redis
   
           volumeMounts:
   
           - name: data
   
             mountPath: /data
   
           - name: redis-config
   
             mountPath: /etc/redis/
   
         volumes:
   
         - name: redis-config
   
           emptyDir: {}
   
         - name: config
   
           configMap:
   
             name: redis-config
   
     # Specify the RCS storage class and requested storage space from the Kubernetes cluster.
   
      volumeClaimTemplates:
   
     - metadata:
   
         name: data
   
       spec:
   
         accessModes: [ "ReadWriteOnce" ]
   
         storageClassName: "RCS-block-storage"
   
         resources:
   
           requests:
   
             storage: 500Mi
   

   Save and close the file

3. Apply the above StatefulSet configuration to deploy the Redis cluster

   $ kubectl apply -n redis -f redis-statefulset.yaml
   

4. Verify the list of running pods in the cluster

   $ kubectl get pods -n redis
   

   Output:

   NAME      READY   STATUS    RESTARTS   AGE
   
   redis-0   1/1     Running   0          31s
   
   redis-1   1/1     Running   0          25s
   
   redis-2   1/1     Running   0          20s
   

   As displayed in the above output, all Redis cluster pods are available and running.

Create a Headless Service Resource

To access Redis internally on your cluster, create a headless service object in the Kubernetes cluster to access the application internally as described below.

1. Create a new headless service resource file

   $ nano redis-service.yaml
   

2. Add the following configurations to the file

   apiVersion: v1
   
   kind: Service
   
   metadata:
   
     name: redis
   
   spec:
   
     clusterIP: None
   
     ports:
   
     - port: 6379
   
       targetPort: 6379
   
       name: redis
   
     selector:
   
       app: redis
   

   Save and close the file

3. Apply the service resource to the Kubernetes cluster

   $ kubectl apply -n redis -f redis-service.yaml
   

4. Verify that the Redis service is running

   $ kubectl get service -n redis
   

   Output:

   NAME    TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
   
   redis   ClusterIP   None         <none>        6379/TCP   8s
   

Verify the Redis Master Slave Replication

You have deployed the Redis cluster with three pods named redis-0, redis-1, and redis-2. The pod redis-0 acts as a master while other pods work as slaves in the cluster.

1. View the master pod redis-0 logs.

   $ kubectl -n redis logs redis-0
   

   Output:

   1:C 17 Jul 2023 15:17:12.968 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
   
   1:C 17 Jul 2023 15:17:12.981 # Redis version=6.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
   
   1:C 17 Jul 2023 15:17:12.981 # Configuration loaded
   
   1:M 17 Jul 2023 15:17:12.982 * monotonic clock: POSIX clock_gettime
   
                   _._                                                  
   
              _.-``__ ''-._                                             
   
         _.-``    `.  `_.  ''-._           Redis 6.2.3 (00000000/0) 64 bit
   
     .-`` .-```.  ```\/    _.,_ ''-._                                  
   
    (    '      ,       .-`  | `,    )     Running in standalone mode
   
    |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
   
    |    `-._   `._    /     _.-'    |     PID: 1
   
     `-._    `-._  `-./  _.-'    _.-'                                   
   
    |`-._`-._    `-.__.-'    _.-'_.-'|                                            |    `-._`-._        _.-'_.-'    |           https://redis.io       
   
     `-._    `-._`-.__.-'_.-'    _.-'                                   
   
    |`-._`-._    `-.__.-'    _.-'_.-'|                                  
   
    |    `-._`-._        _.-'_.-'    |                                  
   
     `-._    `-._`-.__.-'_.-'    _.-'                                   
   
         `-._    `-.__.-'    _.-'                                       
   
             `-._        _.-'                                           
   
                 `-.__.-'                                               
   
   
   
   1:M 17 Jul 2023 15:17:12.989 # Server initialized
   
   1:M 17 Jul 2023 15:17:12.989 * Ready to accept connections
   
   1:M 17 Jul 2023 15:17:57.464 * Replica 10.244.9.4:6379 asks for synchronization
   
   1:M 17 Jul 2023 15:17:57.465 * Full resync requested by replica 10.244.9.4:6379
   
   1:M 17 Jul 2023 15:17:57.465 * Replication backlog created, my new replication IDs are 'b70dca12298759349d656cfe77273767af7384af' and  '0000000000000000000000000000000000000000'
   
   1:M 17 Jul 2023 15:17:57.465 * Starting BGSAVE for SYNC with target: disk
   
   12:C 17 Jul 2023 15:17:58.034 * DB saved on disk
   
   12:C 17 Jul 2023 15:17:58.037 * RDB: 0 MB of memory used by copy-on-write
   
   1:M 17 Jul 2023 15:17:58.073 * Background saving terminated with success
   
   1:M 17 Jul 2023 15:17:58.073 * Synchronization with replica 10.244.85.131:6379 succeeded
   

2. To view detailed information about the Redis master node, use the describe command as below

   $ kubectl -n redis describe pod redis-0
   

   Output:

         /etc/redis/ from redis-config (rw)
   
         /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5czkj (ro)
   
   Conditions:
   
     Type              Status
   
     Initialized       True 
   
     Ready             True 
   
     ContainersReady   True 
   
     PodScheduled      True 
   
   Volumes:
   
     data:
   
       Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
   
       ClaimName:  data-redis-0
   
       ReadOnly:   false
   
     redis-config:
   
       Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
   
       Medium:     
   
       SizeLimit:  <unset>
   
     config:
   
       Type:      ConfigMap (a volume populated by a ConfigMap)
   
       Name:      redis-config
   
       Optional:  false
   
     kube-api-access-5czkj:
   
       Type:                    Projected (a volume that contains injected data from multiple sources)
   
       TokenExpirationSeconds:  3607
   
       ConfigMapName:           kube-root-ca.crt
   
       ConfigMapOptional:       <nil>
   
       DownwardAPI:             true
   
   QoS Class:                   BestEffort
   
   Node-Selectors:              <none>
   
   Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
   
                                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
   

3. Connect to the redis-0 pod to fetch the replication information

   $ kubectl -n redis exec -it redis-0 -- sh
   

4. Log in to the Redis shell

   # redis-cli 
   

5. Authenticate with the replica using your master password

   > auth your_secure_password
   

6. Verify the available replication information

   > info replication
   

   Your output should look like the one below:

    # Replication
   
    role:master
   
    connected_slaves:2
   
    slave0:ip=10.244.9.4,port=6379,state=online,offset=112,lag=0
   
    slave1:ip=10.244.85.131,port=6379,state=online,offset=112,lag=0
   
    master_failover_state:no-failover
   
    master_replid:b70dca12298759349d656cfe77273767af7384af
   
    master_replid2:0000000000000000000000000000000000000000
   
    master_repl_offset:112
   
    second_repl_offset:-1
   
    repl_backlog_active:1
   
    repl_backlog_size:1048576
   
    repl_backlog_first_byte_offset:1
   
    repl_backlog_histlen:112
   

7. Exit the Redis replica instance

    $ exit
   

Test Replication Across the Redis Cluster Nodes

To test the Redis replication process, write sample data on the master pod and verify that the same data replicates on the slave pods

1. Connect to the master pod redis-0

   $ kubectl -n redis exec -it redis-0 -- sh
   

2. Log in to Redis using Redis CLI

   # redis-cli 
   

3. Enter the master password to gain full access to the Redis cluster

   127.0.0.1:6379> auth your_secure_password
   

4. Add some data to the master node

   > SET test1 june
   
   > SET test2 july
   
   > SET test3 august
   

5. Verify the added data

   > KEYS *
   

   Output:

   1) "test3"
   
   2) "test2"
   
   3) "test1"
   

6. Connect to the slave pod redis-1

   $ kubectl -n redis exec -it redis-1 -- sh
   

7. Log in to the Redis shell

   # redis-cli
   

8. Authenticate using the slave password you created earlier

   > auth your_secure_password
   

9. Verify that data successfully replicates from the master node

   > KEYS *
   

   Your output should look like the one below.

   1) "test1"
   
   2) "test2"
   
   3) "test3"
   

Conclusion

In this article, you have deployed a Redis cluster to a RCS Kubernetes Engine (VKE) cluster. You have added a key-value store on the master node and verified the replicated data on the slave node. For more information about the Redis cluster, visit the official documentation.