Knowledgebase

Introduction

Kubernetes is an open-source container orchestration system designed to automate deployments, scale, and manage applications. Similar to all cloud-powered systems, Kubernetes clusters are open to active security threats that evolve over time. Implementing strong security measures safeguards your cluster services and deployments from common threats that may lead to data loss or corruption.

Cluster deployments are commonly exposed to multiple threats that occur at different levels. For example, vulnerable container images can create an entry point for attackers to other cluster resources. On the other hand, unrestricted network traffic allows attackers to access private network ports and open backdoors to your cluster applications.

This article explains the best practices for Kubernetes security you can implement to protect Pods, Services, Deployments and other resources in your cluster.

Prerequisites

Before you begin:

• Deploy a Rcs Kubernetes Engine cluster.
• Deploy a Rcs Ubuntu server to use as the management machine.
• Access the server using SSH as a non-root user with sudo privileges.
• Install and Configure Kubectl to access the cluster.

Limit Pod Privileges

Security contexts in Kubernetes allow you to define privileges for individual pods or containers. For example, follow the steps below to implement pod-level and container-level contexts.

1. A pod-level security context applies to all containers with a pod. To test the privilege, create a sample pod that specifies users and a timeout value for running containers.

   • Create a new file named security-context-1.yaml.

     console Copy

     $ nano security-context-1.yaml
     

   • Add the following contents to the file.

     yaml Copy

     apiVersion: v1
     kind: Pod
     metadata:
       name: example-pod
     spec:
       securityContext:
         runAsUser: 2000
         runAsGroup: 1000
       containers:
       - name: my-container
         image: busybox
         command: [ "sh", "-c", "sleep 3600" ]
     

     Save and close the file.

     The above configuration assigns the Pod processes to the User ID 2000, and the User Group 1000. It uses the busybox container image that runs for 3600 seconds (1 hour). This allows you to control the Pod users, and active container activity.

   • Apply the resource to your cluster.

     console Copy

     $ kubectl apply -f security-context-1.yaml
     

2. Container-level security contexts apply directly to individual containers within a pod. To test the privilege, create a sample pod that limits process privileges.

   • Create a new file named security-context-2.yaml.

     console Copy

     $ nano security-context-2.yaml
     

   • Add the following contents to the file.

     yaml Copy

     apiVersion: v1
     kind: Pod
     metadata:
       name: example-pod-2
     spec:
       containers:
       - name: container-1
         image: busybox
         command: [ "sh", "-c", "sleep 3600" ]
         securityContext:
           allowPrivilegeEscalation: false
           readOnlyRootFilesystem: true
       - name: container-2
         image: busybox
         command: [ "sh", "-c", "sleep 3600" ]
     

     Save and close the file.

     In the above configuration:

     • allowPrivilegeEscalation: Defines how container processes can elevate privileges. When set to false, container processes cannot escalate privileges. When set to true, processes can escalate privileges.
     • readOnlyRootFileSystem: Mounts the container root file system as read-only. In addition, the security contexts only apply to the container container-1 without affecting container-2.

   • Apply the pod policy to your cluster

     console Copy

     $ kubectl apply -f security-context-2.yaml
     

Apply RBAC (Role-Based Access Control) Policies

Role-Based Access Control (RBAC) is a Kubernetes security mechanism for managing user and group permissions in a cluster. Follow the steps in this section to deploy sample roles, a pod, and a new user example_user with restricted access to specific permissions defined in the role configurations.

1. Create a new namespace resource file my-namespace.yaml.

   console Copy

   $ nano my-namespace.yaml
   

2. Add the following contents to the file. Replace my-namespace with your desired naming scheme.

   yaml Copy

   apiVersion: v1
   kind: Namespace
   metadata:
       name: my-namespace
   

   Save and close the file.

3. Apply the namespace resource to your cluster.

   console Copy

   $ kubectl apply -f my-namespace.yaml
   

4. Create a new role definition file role.yaml.

   console Copy

   $ nano role.yaml
   

   A role includes a set of rules that specify what operations a user or group can perform on specific resources within a namespace. Roles declare the authorized actions a user can perform. These include, create, read, update, and delete operations on Kubernetes objects.

5. Add the following contents to the file to define a new role.

   yaml Copy

   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: my-role
     namespace: my-namespace
   rules:
   - apiGroups: [""]
     resources: ["pods"]
     verbs: ["get", "list", "watch"]
   

   Within the rules section, below are the applied values:

   • resources: Defines the type of resources the role policy affects. The value pods applies the role rules to all Pod resources.
   • verbs: Sets the type of operations Role users can perform on the defined resources. These include get, list, watch, logs, among others.

   Save and exit the file.

6. Apply the role to your cluster.

   console Copy

   $ kubectl apply -f role.yaml
   

7. To apply Role binding, create a new resource file role-binding.yaml.

   console Copy

   $ nano role-binding.yaml
   

   Role binding assigns specific roles to users or groups. Each binding must match an active role in your cluster.

8. Add the following contents to the file to define a new Role binding resource.

   yaml Copy

   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: my-role-binding
     namespace: my-namespace
   subjects:
   - kind: User
     name: example_user
     apiGroup: rbac.authorization.k8s.io
   roleRef:
     kind: Role
     name: my-role
     apiGroup: rbac.authorization.k8s.io
   

   Save and close the file.

   The above configuration creates a new role-binding for the user example_user with the cluster permissions you defined in the role resource earlier.

9. Apply role binding to your cluster.

   console Copy

   $ kubectl apply -f role-binding.yaml
   

10. To test your RBAC policy, create a new Pod resource file my-pod.yaml.

    console Copy

    $ nano my-pod.yaml
    

11. Add the following contents to the file to define an example Nginx Pod.

    yaml Copy

    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod
      namespace: my-namespace
    spec:
      containers:
      - name: nginx
        image: nginx:latest
    

    Save and close the file.

12. Apply the pod to your cluster.

    console Copy

    $ kubectl apply -f my-pod.yaml
    

13. To test your RBAC Role binding, create a new user example_user using the OpenSSL utility to generate a new private key.

    console Copy

    $ openssl genrsa -out example_user.key 2048
    

14. Create a Certificate Signing Request (CSR) for the new Kubernetes user account.

    console Copy

    $ openssl req -new -key example_user.key -out example_user.csr -subj "/CN=example_user"
    

15. Encode the certificate signing request to base64 and copy the generated base64 string to your clipboard.

    console Copy

    $ cat example_user.csr | base64 | tr -d "\n"
    

    Output:

    LS0tLS1CRUdJTiB...

16. Create a new Certificate resource file csr-resource.yaml.

    console Copy

    $ nano csr-resource.yaml
    

17. Add the following contents to the file. Add the base64 string you generated earlier as the request field value to create the Kubernetes CertificateSigningRequest (CSR) resource.

    yaml Copy

    apiVersion: certificates.k8s.io/v1
    kind: CertificateSigningRequest
    metadata:
      name: example_user_csr
    spec:
      request: YOUR-BASE64-STRING
      signerName: kubernetes.io/kube-apiserver-client
      expirationSeconds: 86400
      usages:
      - client auth
    

    Save and close the file.

18. Apply the CSR resource to your cluster.

    console Copy

    $ kubectl apply -f csr-resource.yaml
    

19. View the list of cluster CSR resources to verify that the new user request is available.

    console Copy

    $ kubectl get csr
    

    Output:

    NAME               AGE   SIGNERNAME                            REQUESTOR   REQUESTEDDURATION   CONDITION
    example_user_csr   2s    kubernetes.io/kube-apiserver-client   admin       24h                 Pending

20. Approve the user CSR resource.

    console Copy

    $ kubectl certificate approve example_user_csr
    

21. Export the issued certificate from the cluster CSR to a host file such as example_user_signed_cert.crt.

    console Copy

    $ kubectl get csr example_user_csr -o jsonpath='{.status.certificate}'| base64 -d > example_user_signed_cert.crt
    

22. In Kubernetes, a context is a specific configuration part of the kubectl utility for interacting with a cluster. To modify users, view your available context to reveal the administrative Kubernetes account in use, and copy the generated output to your clipboard.

    console Copy

    $ kubectl config current-context
    

23. Add the new example_user credentials to your cluster using your signed certificate and private key files.

    console Copy

    $ kubectl config set-credentials example_user --client-key=example_user.key --client-certificate=example_user_signed_cert.crt --embed-certs=true
    

24. Add the new example_user context to your kubectl config. Replace example_cluster with the actual available context name you retrieved earlier.

    console Copy

    $ kubectl config set-context example_user_context --cluster=example_cluster --user=example_user
    

25. Activate the new context to start using your cluster as the new user example_user.

    console Copy

    $ kubectl config use-context example_user_context
    

26. To test the new user privileges on the cluster. View all cluster pods in the namespace you created earlier.

    console Copy

    $ kubectl get pods/my-pod -n my-namespace
    

    Viewing cluster pods is successful because the user has get permissions for all cluster pod granted by the namespace Role you created earlier.

27. Try to delete the Nginx pod, and verify that the operation fails due to low privileges.

    console Copy

    $ kubectl delete pods/my-pod -n my-namespace
    

    To allow the user to delete pods, you must edit your Role resource and include the delete operation in your rule verbs.

28. To switch to your privileged cluster account, use the default context you generated earlier. Replace example-context with your default administrative account.

    console Copy

    $ kubectl config use-context example-context
    

You have implemented RBAC using Roles and Role Bindings in a single namespace. To apply cluster wide privileges, use Cluster Roles and Cluster Role Bindings by changing the YAML metadata values.

Implement Network Policies

Network policies allow you to control internal and external network traffic within a cluster. By default, the namespace does not contain any network policies, this enables all ingress and egress traffic between pods in the same namespace.

1. To tighten cluster network security, create a new network policy resource file default-deny-policy.yaml that disables the default policy.

   console Copy

   $ nano default-deny-policy.yaml
   

2. Add the following contents to the file. Replace example-namespace with your target namespace.

   yaml Copy

   apiVersion: networking.k8s.io/v1
   kind: NetworkPolicy
   metadata:
     name: default-deny-ingress
     namespace: example-namespace
   spec:
     podSelector: {}
     policyTypes:
     - Ingress
     - Egress
   

   Save and exit the file.

   The above network policy configuration enforces a default deny rule for ingress and egress traffic on all pods in the example-namespace. This prevents any external or internal traffic from reaching the pods unless additional network policies are created to explicitly allow specific traffic types.

3. Apply the network policy to your cluster to disable the default traffic policy.

   console Copy

   $ kubectl apply -f default-deny-policy.yaml
   

4. To allow network traffic between pods, define new network policies depending on your cluster structure. For example, to allow traffic only from a web deployment named nginx to a database deployment named mysql-db within the namespace example-namespace, create a new allow policy resource file allow-policies.yaml.

   console Copy

   $ nano allow-policies.yaml
   

5. Add the following contents to the file to define two network policies.

   yaml Copy

   apiVersion: networking.k8s.io/v1
   kind: NetworkPolicy
   metadata:
     name: np-allow-ingress
     namespace: example-namespace
   spec:
     podSelector:
       matchLabels:
         app: mysql-db
     policyTypes:
       - Ingress
     ingress:
       - from:
           - podSelector:
               matchLabels:
                 app: nginx
   ---
   apiVersion: networking.k8s.io/v1
   kind: NetworkPolicy
   metadata:
     name: np-allow-egress
     namespace: example-namespace
   spec:
     podSelector:
       matchLabels:
         app: nginx
     policyTypes:
       - Egress
     egress:
       - to:
           - podSelector:
               matchLabels:
                 app: mysql-db - Egress
   

   Save and exit the file.

   In the above configuration, the first network policy allows ingress traffic. Then, the second policy allows egress traffic from the nginx pod to the mysql-db pod within the namespace example-namespace.

6. Apply the new network policy to your cluster.

   console Copy

   $ kubectl apply -f allow-policies.yaml
   

You have disabled the default Kubernetes network policy that allows network traffic across all pods, then, you created a separate policy that only allows communication between the nginx and mysql-db pods within the example-namespace resource.

You can apply multiple network policies and also filter traffic using IP blocks, ports, and protocols. For more information, visit the Kubernetes Network Policies documentation.

Manage Kubernetes Secrets

Kubernetes secrets allow you to securely store small amounts of sensitive data like passwords, Rcs Container Registry credentials, service account tokens, and more within a cluster. You can create secrets to define special files and access them within specific pods or deployments. Follow the steps below to create sample secrets and access them in a specific example pod.

1. Create a new Secret resource file example-secret.yaml.

   console Copy

   $ nano example-secret.yaml
   

2. Add the following contents to the file to define new dbPassword and apiToken example variables with base64 data.

   yaml Copy

   apiVersion: v1
   kind: Secret
   metadata:
     name: example-secret
   data:
     dbPassword: ZXhhbXBsZVB3ZA==
     apiToken: dG9rZW4=
             app: mysql-db - Egress
   

   Save and exit the file.

   The above configuration creates a new example-secret with accessible variables you can apply in specific cluster resources.

3. Apply the secret to your cluster

   console Copy

   $ kubectl apply -f example-secret.yaml
   

Access Secrets in Pods

1. To access cluster Secrets, call the secret and variable as a key in your resource definition file. For example, create a new Pod resource example-pod.yaml to access your new Secret resource.

   console Copy

   $ nano example-pod.yaml
   

2. Add the following contents to the file.

   yaml Copy

   apiVersion: v1
   kind: Pod
   metadata:
     name: example-pod-with-secret
   spec:
     containers:
       - name: my-container
         image: busybox
         command: [ "sh", "-c", "sleep 3600" ]
         env:
           - name: DB_PASSWORD
             valueFrom:
               secretKeyRef:
                 name: example-secret
                 key: dbPassword Egress
   

   Save and close the file.

   Within the above configuration, the example Pod example-pod-with-secret imports the dbPassword variable data from the example-secret to the Pod variable DB_PASSWORD. This allows the Pod to use the Secret values to authenticate with the local variable using values from the Secret resource.

3. Apply the Pod to your cluster.

   console Copy

   $ kubectl apply -f example-pod.yaml
   

You have created a Kubernetes Secret resource and accessed the variable keys in a Pod resource. You can create multiple secrets to store important data such as authentication values to pass directly to pods, deployments and services within the cluster depending on the required variable names.

Pod Security Standards (PSSs)

Kubernetes Pod Security Standards (PSS) are key to maintaining Kubernetes security. PSS contains three profiles:

• Privileged: An unrestricted policy. Should be used only for trusted users performing critical infrastructure workloads, along with additional security measures such as multifactor authentication.
• Baseline: Offers minimal restrictive guardrails while preventing known privilege escalations. Suitable for non-critical applications and typical containerized workloads. It may require different configurations based on specific use cases.
• Restricted: Enforces pod hardening best practices, making it the most secure profile. It's ideal for security-critical applications and vulnerable workloads.

You can configure PSS in your namespace with the following different modes:

• Enforce: Rejects Pods with policy violations.
• Audit: Allows pods with policy violations but includes audit annotations in the event log.
• Warn: Allows pods with policy violations but warns users about the pontetial security risks.

You can use a specific profile and mode in a namespace by setting the corresponding labels. For example, the following command sets the example-namespace namespace Baseline warning, and the warn-version label applies the latest policy versions.

$ kubectl label --overwrite ns example-namespace \
   pod-security.kubernetes.io/warn=baseline \
   pod-security.kubernetes.io/warn-version=latest

The following command sets a Baseline enforcement with an audit for the restricted level on the namespace example-namespace. The audit-version and enforce-version labels apply the latest policy versions.

$ kubectl label --overwrite ns example-namespace \
   pod-security.kubernetes.io/enforce=baseline \
   pod-security.kubernetes.io/audit=restricted \
   pod-security.kubernetes.io/enforce-version=latest \
   pod-security.kubernetes.io/audit-version=latest

Follow the steps below to set the labels in a namespace resource definition file instead of directly using Kubectl commands.

1. Create a new namespace resource file example-namespace.yaml.

   console Copy

   $ nano example-namespace.yaml
   

2. Add the following contents to the file.

   yaml Copy

   apiVersion: v1
   kind: Namespace
   metadata:
     name: example-namespace
     labels:
       pod-security.kubernetes.io/enforce: baseline
       pod-security.kubernetes.io/enforce-version: latest
   
       pod-security.kubernetes.io/audit: restricted
       pod-security.kubernetes.io/audit-version: latest
   
       pod-security.kubernetes.io/warn: restricted
       pod-security.kubernetes.io/warn-version: latest
   

   Save and exit the file.

   The above configuration sets a Baseline enforcement. Then, applies the latest policy versions to the Restricted level audits and warnings.

3. Apply the namespace resource to your cluster.