Knowledgebase

Introduction

Redis is an open-source in-memory data store that can work as a quick-response database. It's capable of handling millions of queries per second with high availability and scalability. A Redis Cluster is a group of Redis instances used to achieve high availability, improved fault tolerance, and horizontal scaling in data management systems.

A Redis cluster uses a full mesh topology where every node interconnects with the main node using a TCP connection. In Kubernetes, pods interconnect to each other making it possible to deploy Redis for high availability within a cluster.

Benefits of deploying Redis in a Kubernetes cluster include:

• High Availability
• Scalability
• Load Balancing and Service Discovery
• Simplified Deployment and Management
• Monitoring and Logging

This article explains how to deploy a Redis Cluster on a Rcs Kubernetes Engine (VKE) cluster.

Prerequisites

Before you begin

• Deploy a Rcs Kubernetes Engine (VKE) cluster with at least three nodes
• Install Kubectl on your local machine to access the cluster

Create the Redis Namespace

By default, Kubernetes adds all cluster components such as services, pods and ConfigMaps to the default namespace. By creating a separate namespace, you are able to manage pods and services more efficiently in the cluster. In this section, create the Redis namespace as described in the steps below.

1. Create a new namespace for the Redis cluster.

    $ kubectl create ns redis

2. Verify that the new namespace is available.

    $ kubectl get ns

   Output:

    NAME              STATUS   AGE
    default           Active   12m
    kube-node-lease   Active   12m
    kube-public       Active   12m
    kube-system       Active   12m
    redis             Active   7s

Define a Storage Class

A storage class is a Kubernetes resource that allows you to reserve disk space or attach volumes from a cloud provider to your cluster. By default, pods do not store the data permanently. When a pod gets deleted or restarts, data inside the pods is permanently lost.

In this section, mount a Rcs Block Storage instance to your Kubernetes cluster to store data permanently, and avoid any data loss as described below.

1. Using a text editor such as Nano, create a new storage class YAML file.

    $ nano sc.yaml

2. Add the following configurations to the file.

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: redis-storage
    provisioner: block.csi.vultr.com
    volumeBindingMode: WaitForFirstConsumer
    allowVolumeExpansion: true
    reclaimPolicy: Delete

   Save and close the file.

3. Apply the storage class to your cluster.

    $ kubectl apply -f sc.yaml

4. Verify that the storage class is available.

    $ kubectl get sc

   Your output should look like the below:

    NAME                             PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
    redis-storage                    block.csi.vultr.com            Delete          WaitForFirstConsumer   true                   6s
    vultr-block-storage (default)    block.csi.vultr.com            Delete          Immediate              true                   12m
    vultr-block-storage-hdd          block.csi.vultr.com            Delete          Immediate              true                   12m
    vultr-block-storage-hdd-retain   block.csi.vultr.com            Retain          Immediate              true                   12m
    vultr-block-storage-retain       block.csi.vultr.com            Retain          Immediate              true                   12m

   As displayed in the above output, a new storage class redis-storage is available and it's attached to a Rcs Block Storage volume.

Create a Persistent Volume

Data durability is essential for Redis deployments. Persistent volumes store Redis data on the cluster to achieve data durability in case a pod restarts. A Persistent volume requests a specified amount of storage from the Kubernetes cluster, and it's dynamically provisioned by a StorageClass.

In this section, create a persistent volume as described below.

1. Create a new manifest file to configure three persistent volumes using the vultr-block-storage provisioner.

    $ nano pv.yaml

2. Add the following configurations to the file.

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: redis-pv1
    spec:
      storageClassName: vultr-block-storage
      capacity:
        storage: 10Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/storage/data1"
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: redis-pv2
    spec:
      storageClassName: vultr-block-storage
      capacity:
        storage: 10Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/storage/data2"
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: redis-pv3
    spec:
      storageClassName: vultr-block-storage
      capacity:
        storage: 10Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/storage/data3"

   Save and close the file.

   The above configuration creates three Persistent Volumes with a size of 10 GB using the vultr-block-storage provisioner. ReadWriteOnce means that the PVC is only mounted as read-write by a single node at a time.

3. Apply the configuration to the cluster

    $ kubectl apply -f pv.yaml

4. Verify the new persistent volumes

    $ kubectl get pv

   Your output should look like the one below:

    NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY    STATUS      CLAIM   STORAGECLASS         REASON   AGE
    redis-pv1   10Gi        RWO            Retain           Available           vultr-block-storage            6s
    redis-pv2   10Gi        RWO            Retain           Available           vultr-block-storage            5s
    redis-pv3   10Gi        RWO            Retain           Available           vultr-block-storage            4s

Create the ConfigMap

The ConfigMap is a key-value store in a Kubernetes cluster in which you can define the Redis configuration information. In this section, create a ConfigMap for the Redis cluster as described below.

1. Create a new ConfigMap YAML file

    $ nano redis-config.yaml

2. Add the following configurations to the file. Replace your_secure_password with a secure password for your cluster

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: redis-config
    data:
      redis.conf: |
        masterauth your_secure_password
        requirepass your_secure_password
        # Define the Redis listening interfaces
        bind 0.0.0.0
        # Enable or disable the protected mode
        protected-mode no
        # Define the Redis listening port
        port 6379
        tcp-backlog 511
        # Close the connection after a client is idle for N seconds (0 to disable)
        timeout 0
   
        tcp-keepalive 300
        daemonize no
        supervised no
        # Define Redis PID file
        pidfile "/var/run/redis_6379.pid"
        loglevel notice
        logfile ""
        databases 16
        always-show-logo yes
        save 900 1
        save 300 10
        save 60 10000
   
        stop-writes-on-bgsave-error yes
        rdbcompression yes
        rdbchecksum yes
        dbfilename "dump.rdb"
        rdb-del-sync-files no
        # Define the database storage location
        dir "/data"
        replica-serve-stale-data yes
        replica-read-only yes
        repl-diskless-sync no
        repl-diskless-sync-delay 5
        repl-diskless-load disabled
        repl-disable-tcp-nodelay no
        replica-priority 100
        acllog-max-len 128
        lazyfree-lazy-eviction no
        lazyfree-lazy-expire no
        lazyfree-lazy-server-del no
        replica-lazy-flush no
   
        lazyfree-lazy-user-del no
        appendonly yes
        appendfilename "appendonly.aof"
        appendfsync everysec
        no-appendfsync-on-rewrite no
        auto-aof-rewrite-percentage 100
        auto-aof-rewrite-min-size 64mb
   
        aof-load-truncated yes
        aof-use-rdb-preamble yes
        lua-time-limit 5000
        slowlog-log-slower-than 10000
        slowlog-max-len 128
        latency-monitor-threshold 0
        notify-keyspace-events ""
        hash-max-ziplist-entries 512
        hash-max-ziplist-value 64
        list-max-ziplist-size -2
        list-compress-depth 0
        set-max-intset-entries 512
        zset-max-ziplist-entries 128
        zset-max-ziplist-value 64
        hll-sparse-max-bytes 3000
        stream-node-max-bytes 4kb
        stream-node-max-entries 100
   
        activerehashing yes
        client-output-buffer-limit normal 0 0 0
        client-output-buffer-limit replica 256mb 64mb 60
        client-output-buffer-limit pubsub 32mb 8mb 60
        hz 10
        dynamic-hz yes
        aof-rewrite-incremental-fsync yes
        rdb-save-incremental-fsync yes
   
        jemalloc-bg-thread yes

   Save and close the file

   In the above configuration, the master and slavepasswords must be the same to establish a connection in the Redis cluster

3. Apply your configuration to the Kubernetes cluster

    $ kubectl apply -n redis -f redis-config.yaml

4. Verify that the ConfigMap is available in the Redis namespace

    $ kubectl get configmap -n redis

   Output:

    NAME               DATA   AGE 
    kube-root-ca.crt   1      110s
    redis-config       1      4s

   To view the full Redis code for a ConfigMap, visit the GitHub repository to fork or download the file.

Scale Redis in the Kubernetes Cluster using a StatefulSet

A StatefulSet deploys stateful applications and clustered applications that save data to persistent storage. It's suitable for deploying Redis and other applications that require persistent identities and stable hostnames. In this section, create a StatefulSet as below.

1. Create a new StatefulSet YAML file to scale the Redis cluster

    $ nano redis-statefulset.yaml

2. Add the following configurations to the file

    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: redis
    spec:
      serviceName: redis
      # Specify the number of Redis replicas
      replicas: 3
      selector:
        matchLabels:
          app: redis
      template:
        metadata:
          labels:
            app: redis
        spec:
          initContainers:
          - name: config
            # Specify the Redis docker image version
            image: redis:6.2.3-alpine
            command: [ "sh", "-c" ]
            args:
              - |
                # Copy the Redis configuration file to each Redis pod.
                cp /tmp/redis/redis.conf /etc/redis/redis.conf
   
                echo "finding master..."
                MASTER_FDQN=`hostname  -f | sed -e 's/redis-[0-9]\./redis-0./'`
                if [ "$(redis-cli -h sentinel -p 5000 ping)" != "PONG" ]; then
                  echo "master not found, defaulting to redis-0"
   
                  if [ "$(hostname)" == "redis-0" ]; then
                    echo "this is redis-0, not updating config..."
                  else
                    echo "updating redis.conf..."
                    echo "slaveof $MASTER_FDQN 6379" >> /etc/redis/redis.conf
                  fi
                else
                  echo "sentinel found, finding master"
                  MASTER="$(redis-cli -h sentinel -p 5000 sentinel get-master-addr-by-name mymaster | grep -E '(^redis-\d{1,})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})')"
                  echo "master found : $MASTER, updating redis.conf"
                  echo "slaveof $MASTER 6379" >> /etc/redis/redis.conf
                fi
            # Specify the Redis volume name and mount path
            volumeMounts:
            - name: redis-config
              mountPath: /etc/redis/
            - name: config
              mountPath: /tmp/redis/
          # Specify the Redis Docker image version, listening port, volume name, and mount path.
           containers:
          - name: redis
            image: redis:6.2.3-alpine
            command: ["redis-server"]
            args: ["/etc/redis/redis.conf"]
            ports:
            - containerPort: 6379
              name: redis
            volumeMounts:
            - name: data
              mountPath: /data
            - name: redis-config
              mountPath: /etc/redis/
          volumes:
          - name: redis-config
            emptyDir: {}
          - name: config
            configMap:
              name: redis-config
      # Specify the Rcs storage class and requested storage space from the Kubernetes cluster.
       volumeClaimTemplates:
      - metadata:
          name: data
        spec:
          accessModes: [ "ReadWriteOnce" ]
          storageClassName: "vultr-block-storage"
          resources:
            requests:
              storage: 500Mi

   Save and close the file

3. Apply the above StatefulSet configuration to deploy the Redis cluster

    $ kubectl apply -n redis -f redis-statefulset.yaml

4. Verify the list of running pods in the cluster

    $ kubectl get pods -n redis

   Output:

    NAME      READY   STATUS    RESTARTS   AGE
    redis-0   1/1     Running   0          31s
    redis-1   1/1     Running   0          25s
    redis-2   1/1     Running   0          20s

   As displayed in the above output, all Redis cluster pods are available and running.

Create a Headless Service Resource

To access Redis internally on your cluster, create a headless service object in the Kubernetes cluster to access the application internally as described below.

1. Create a new headless service resource file

    $ nano redis-service.yaml

2. Add the following configurations to the file

    apiVersion: v1
    kind: Service
    metadata:
      name: redis
    spec:
      clusterIP: None
      ports:
      - port: 6379
        targetPort: 6379
        name: redis
      selector:
        app: redis

   Save and close the file

3. Apply the service resource to the Kubernetes cluster

    $ kubectl apply -n redis -f redis-service.yaml

4. Verify that the Redis service is running

    $ kubectl get service -n redis

   Output:

    NAME    TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
    redis   ClusterIP   None         <none>        6379/TCP   8s

Verify the Redis Master Slave Replication

You have deployed the Redis cluster with three pods named redis-0, redis-1, and redis-2. The pod redis-0 acts as a master while other pods work as slaves in the cluster.

1. View the master pod redis-0 logs.

    $ kubectl -n redis logs redis-0

   Output:

    1:C 17 Jul 2023 15:17:12.968 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
    1:C 17 Jul 2023 15:17:12.981 # Redis version=6.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
    1:C 17 Jul 2023 15:17:12.981 # Configuration loaded
    1:M 17 Jul 2023 15:17:12.982 * monotonic clock: POSIX clock_gettime
                    _._                                                  
               _.-``__ ''-._                                             
          _.-``    `.  `_.  ''-._           Redis 6.2.3 (00000000/0) 64 bit
      .-`` .-```.  ```\/    _.,_ ''-._                                  
     (    '      ,       .-`  | `,    )     Running in standalone mode
     |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
     |    `-._   `._    /     _.-'    |     PID: 1
      `-._    `-._  `-./  _.-'    _.-'                                   
     |`-._`-._    `-.__.-'    _.-'_.-'|                                            |    `-._`-._        _.-'_.-'    |           https://redis.io       
      `-._    `-._`-.__.-'_.-'    _.-'                                   
     |`-._`-._    `-.__.-'    _.-'_.-'|                                  
     |    `-._`-._        _.-'_.-'    |                                  
      `-._    `-._`-.__.-'_.-'    _.-'                                   
          `-._    `-.__.-'    _.-'                                       
              `-._        _.-'                                           
                  `-.__.-'                                               
   
    1:M 17 Jul 2023 15:17:12.989 # Server initialized
    1:M 17 Jul 2023 15:17:12.989 * Ready to accept connections
    1:M 17 Jul 2023 15:17:57.464 * Replica 10.244.9.4:6379 asks for synchronization
    1:M 17 Jul 2023 15:17:57.465 * Full resync requested by replica 10.244.9.4:6379
    1:M 17 Jul 2023 15:17:57.465 * Replication backlog created, my new replication IDs are 'b70dca12298759349d656cfe77273767af7384af' and  '0000000000000000000000000000000000000000'
    1:M 17 Jul 2023 15:17:57.465 * Starting BGSAVE for SYNC with target: disk
    12:C 17 Jul 2023 15:17:58.034 * DB saved on disk
    12:C 17 Jul 2023 15:17:58.037 * RDB: 0 MB of memory used by copy-on-write
    1:M 17 Jul 2023 15:17:58.073 * Background saving terminated with success
    1:M 17 Jul 2023 15:17:58.073 * Synchronization with replica 10.244.85.131:6379 succeeded

2. To view detailed information about the Redis master node, use the describe command as below

    $ kubectl -n redis describe pod redis-0

   Output:

          /etc/redis/ from redis-config (rw)
          /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5czkj (ro)
    Conditions:
      Type              Status
      Initialized       True 
      Ready             True 
      ContainersReady   True 
      PodScheduled      True 
    Volumes:
      data:
        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
        ClaimName:  data-redis-0
        ReadOnly:   false
      redis-config:
        Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
        Medium:     
        SizeLimit:  <unset>
      config:
        Type:      ConfigMap (a volume populated by a ConfigMap)
        Name:      redis-config
        Optional:  false
      kube-api-access-5czkj:
        Type:                    Projected (a volume that contains injected data from multiple sources)
        TokenExpirationSeconds:  3607
        ConfigMapName:           kube-root-ca.crt
        ConfigMapOptional:       <nil>
        DownwardAPI:             true
    QoS Class:                   BestEffort
    Node-Selectors:              <none>
    Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                                  node.kubernetes.io/unreachable:NoExecute op=Exists for 300s

3. Connect to the redis-0 pod to fetch the replication information

    $ kubectl -n redis exec -it redis-0 -- sh

4. Log in to the Redis shell

    # redis-cli 

5. Authenticate with the replica using your master password

    > auth your_secure_password

6. Verify the available replication information

    > info replication

   Your output should look like the one below:

     # Replication
     role:master
     connected_slaves:2
     slave0:ip=10.244.9.4,port=6379,state=online,offset=112,lag=0
     slave1:ip=10.244.85.131,port=6379,state=online,offset=112,lag=0
     master_failover_state:no-failover
     master_replid:b70dca12298759349d656cfe77273767af7384af
     master_replid2:0000000000000000000000000000000000000000
     master_repl_offset:112
     second_repl_offset:-1
     repl_backlog_active:1
     repl_backlog_size:1048576
     repl_backlog_first_byte_offset:1
     repl_backlog_histlen:112

7. Exit the Redis replica instance

     $ exit

Test Replication Across the Redis Cluster Nodes

To test the Redis replication process, write sample data on the master pod and verify that the same data replicates on the slave pods

1. Connect to the master pod redis-0

    $ kubectl -n redis exec -it redis-0 -- sh

2. Log in to Redis using Redis CLI

    # redis-cli 

3. Enter the master password to gain full access to the Redis cluster

    127.0.0.1:6379> auth your_secure_password

4. Add some data to the master node

    > SET test1 june
    > SET test2 july
    > SET test3 august

5. Verify the added data

    > KEYS *

   Output:

    1) "test3"
    2) "test2"
    3) "test1"

6. Connect to the slave pod redis-1

    $ kubectl -n redis exec -it redis-1 -- sh

7. Log in to the Redis shell

    # redis-cli

8. Authenticate using the slave password you created earlier

    > auth your_secure_password

9. Verify that data successfully replicates from the master node

    > KEYS *

   Your output should look like the one below.

    1) "test1"
    2) "test2"
    3) "test3"

Conclusion

In this article, you have deployed a Redis cluster to a Rcs Kubernetes Engine (VKE) cluster. You have added a key-value store on the master node and verified the replicated data on the slave node. For more information about the Redis cluster, visit the official documentation.