Introduction
The Kubernetes API Server provides a REST API interface for clients to interact with it and execute operations such as creating Deployments etc. But, a client application does not have to understand and use the lower-level HTTP operations (such as GET, PUT, POST, PATCH etc.) since there are multiple client libraries for different languages.
This guide covers how to manage a Deployment and StatefulSet using the Kubernetes Go client. Before we get into the details, let's briefly understand some of the concepts.
Deployment resource
A Pod is the basic foundation of running an application on Kubernetes. However, simply deploying a single instance of a Pod will not suffice. A Pod is not rescheduled in the case of a Kubernetes node failure and therefore can lead to an outage. A ReplicaSet is a Kubernetes resource that controls multiple, identical instances of a Pod. It can scale the number of Pods (replicas) up or down on-demand. It can also roll out a new version of the application across all replicas.
A Deployment abstracts the functionality of ReplicaSet and manages it. You do not have to create, modify, or delete ReplicaSets directly. The Deployment keeps a history of application versions and can roll back to an older version if required. Just like a ReplicaSet, a Deployment can also scale the number of replicas.
While updating a Deployment, it is possible to pause rollouts for that Deployment before triggering one or more updates. Once you're ready to apply the changes, simply resume rollouts for the Deployment. This approach allows you to apply multiple fixes in between pausing and resuming without triggering unnecessary rollouts. Kubernetes also supports Canary Deployments. If you need to roll out releases to a specific set of users, you can create multiple Deployments (one for each release).
StatefulSet resource
Just like a Deployment, a StatefulSet also manages Pods. However, unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. Although Pods in a StatefulSet are created from the same spec, they are not interchangeable. Each Pod has a persistent identifier that it maintains across any rescheduling. Although individual Pods in a StatefulSet can fail, the persistent Pod identifiers make possible to match existing volumes to the new Pods that replace any that have failed. That's why you can use StatefulSets are useful for applications that require one or more of the following:
Persistent state store.
Unique identifiers for each application instance.
Ordered, graceful deployment and scaling.
Ordered, automated rolling updates.
StatefulSet Pods have a unique identity that consists of an ordinal, a stable network identity, and stable storage. The identity sticks to the Pod, regardless of which node it's (re)scheduled on. For a StatefulSet with N replicas, each Pod in the StatefulSet will be assigned an integer ordinal, that is unique over the Set. By default, pods will be assigned ordinals from 0 up through N-1. Scaling a StatefulSet creates a new Pod instance with the next unused ordinal index. For example, if you scale up from two to three instances, the new instance will get index 2. StatefulSets scale down only one Pod instance at a time.
For persistence, a StatefulSet has to create PersistentVolumeClaims. The PersistentVolumes for the claims can either be provisioned up-front or just in time through dynamic provisioning of PersistentVolumes. After a scale down operation, only the Pods are deleted, not the PersistentVolumeClaims. You’re need to delete PersistentVolumeClaims manually to release the underlying PersistentVolume.
StatefulSet provides two policies for Pod management:
OrderedReady- This is the default policy.Parallel- When this policy is in effect, allPods are created and deleted in parallel. This policy only applies to scaling operations, not updates.
Kubernetes Go APIs
Kubernetes Go client library is a high-level library that can be used by developers to interact with the Kubernetes API using the Go language. This library brings together the Kubernetes API and along with other libraries, providing a Scheme that is pre-configured with Kubernetes API’s objects and a RESTMapper implementation for the Kubernetes API. It also provides a set of clients to use to execute operations on the resources of the Kubernetes API in a simple way.
Since Kubernetes is backward-compatible it is possible to use older versions of client-go with newer versions of clusters. But bear in mind that bug fixes are back-ported to previous client-go releases, not new features.
In addition to the the Go client library, the other two important Go libraries needed to work with the Kubernetes API are the apimachinery and the api.
The API Machinery - It is a generic library that takes care of serializing data between Go structures and objects written in the
JSONformat. This makes it possible for developers of clients, but also API Servers, to write data using Go structures and transparently use these resources inJSONduring theHTTPexchanges.The API Library - It is a collection of Go structures that are needed to work in Go with the resources defined by the Kubernetes API.
Prerequisites
Before following the steps in this guide, you need to:
Install kubectl on your local workstation. It is a Kubernetes command-line tool that allows us to run commands against Kubernetes clusters.
Deploy a rcs Kubernetes Engine (VKE) cluster using the Reference Guide. Once it's deployed, from the Overview tab, click the Download Configuration button in the upper-right corner to download your
kubeconfigfile and save it to a local directory.Point
kubectlto rcs Kubernetes Engine cluster by setting theKUBECONFIGenvironment variable to the path where you downloaded the clusterkubeconfigfile in the previous step.export KUBECONFIG=<path to VKE kubeconfig file>Verify the same using the following command:
kubectl config current-contextInstall Go programming language (version 1.18 or higher) on your local workstation.
Install a recent version of kubectl on your local workstation.
Initialize the project
Create a directory and switch to it:
mkdir k8s-go-client-example cd k8s-go-client-exampleCreate a new Go module:
go mod init k8s-go-client-exampleThis will create a new
go.modfileCreate a new file
main.go:touch main.go
Import libraries
To import required Go modules, add the following to main.go file:
package main
import (
"context"
"flag"
"fmt"
"log"
"path/filepath"
appsv1 "k8s.io/api/apps/v1"
apiv1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/util/homedir"
)
Add the init function
Add the code below to main.go:
var clientset *kubernetes.Clientset
const deploymentName = "test-deployment"
const statefulSetName = "test-statefulset"
const serviceName = "nginx-service"
func init() {
var kubeconfig *string
if home := homedir.HomeDir(); home != "" {
kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
} else {
kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
}
flag.Parse()
config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
log.Fatal(err)
}
clientset, err = kubernetes.NewForConfig(config)
if err != nil {
log.Fatal(err)
}
}
The init function takes care of the following:
Access the
configfile in the.kubefolder under the user's home directory.Uses the path to kube config file to create a
*rest.Configby invokinclientcmd.BuildConfigFromFlags.Creates a
*kubernetes.Clientsetby invoking thekubernetes.NewForConfigpassing in the*rest.Config
Add the function to create a Deployment
Add the createDeployment function to main.go file:
func createDeployment() {
client := clientset.AppsV1().Deployments(apiv1.NamespaceDefault)
numReplicas := int32(2)
deployment := &appsv1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Name: deploymentName,
},
Spec: appsv1.DeploymentSpec{
Replicas: &numReplicas,
Selector: &metav1.LabelSelector{
MatchLabels: map[string]string{
"app": "demo-app",
},
},
Template: apiv1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{
"app": "demo-app",
},
},
Spec: apiv1.PodSpec{
Containers: []apiv1.Container{
{
Name: "web-app",
Image: "nginx",
Ports: []apiv1.ContainerPort{
{
Name: "http",
Protocol: apiv1.ProtocolTCP,
ContainerPort: 80,
},
},
},
},
},
},
},
}
fmt.Println("creating deployment", deploymentName)
_, err := client.Create(context.Background(), deployment, metav1.CreateOptions{})
if err != nil {
log.Fatal("failed to create deployment", err)
}
fmt.Println("successfully created deployment", deploymentName)
}
If successful, this function creates a Deployment named test-deployment.
First, it gets a
v1.DeploymentInterfaceinstance by callingclientset.AppsV1().Deployments()with the default namespace.Prepares a new
Deploymentdefinition.Invokes
DeploymentInterface.Createto initiateDeploymentcreation.
Add the main function
Finally, add the main function which will invoke the createDeployment function.
func main() {
createDeployment()
}
Run the program to create a new Deployment
Fetch the dependencies:
go mod tidyRun the program:
go run main.goYou should see the following output:
creating deployment test-deployment successfully created deployment test-deploymentConfirm that the
Deploymentwas created:kubectl get deployment/test-deploymentYou should see the following output:
NAME READY UP-TO-DATE AVAILABLE AGE test-deployment 2/2 2 2 31sVerify the
Pods created by the Deployment:kubectl get pods -l=app=demo-appYou should see the following output (the
Podnames might differ in your case):NAME READY STATUS RESTARTS AGE test-deployment-6d67cdd94-q5wgv 1/1 Running 0 3m5s test-deployment-6d67cdd94-srz7m 1/1 Running 0 3m5s
Add the function to delete a Deployment
Add the deleteDeployment function to main.go:
func deleteDeployment() {
client := clientset.AppsV1().Deployments(apiv1.NamespaceDefault)
fmt.Println("deleting deployment", deploymentName)
err := client.Delete(context.Background(), deploymentName, metav1.DeleteOptions{})
if err != nil {
log.Fatal("failed to delete deployment", err)
}
fmt.Println("successfully deleted deployment", deploymentName)
}
Update the main function - comment out the createDeployment function and add the deleteDeployment function
func main() {
//createDeployment()
deleteDeployment()
}
Run the program to delete an existing Deployment
To run the program:
go run main.goYou should see the following output:
deleting deployment test-deployment successfully deleted deployment test-deploymentConfirm that the
Deploymentwas deleted:kubectl get deployment/test-deploymentYou should see the following output:
Error from server (NotFound): deployments.apps "test-deployment" not foundConfirm that
Pods created by theDeploymentwere also deleted:kubectl get pods -l=app=demo-appYou should see the following output:
No resources found in default namespace.
Add the function to create a StatefulSet
Add the createStatefulSet function to main.go:
func createStatefulSet() {
client := clientset.AppsV1().StatefulSets(apiv1.NamespaceDefault)
numReplicas := int32(2)
statefulset := &appsv1.StatefulSet{
ObjectMeta: metav1.ObjectMeta{
Name: statefulSetName,
},
Spec: appsv1.StatefulSetSpec{
ServiceName: serviceName,
Replicas: &numReplicas,
Selector: &metav1.LabelSelector{
MatchLabels: map[string]string{
"app": "nginx-app",
},
},
Template: apiv1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{
"app": "nginx-app",
},
},
Spec: apiv1.PodSpec{
Containers: []apiv1.Container{
{
Name: "web-app",
Image: "nginx",
Ports: []apiv1.ContainerPort{
{
Name: "http",
Protocol: apiv1.ProtocolTCP,
ContainerPort: 80,
},
},
VolumeMounts: []apiv1.VolumeMount{
{Name: "www", MountPath: "/usr/share/nginx/html"},
},
},
},
},
},
VolumeClaimTemplates: []apiv1.PersistentVolumeClaim{
{
ObjectMeta: metav1.ObjectMeta{Name: "www"},
Spec: apiv1.PersistentVolumeClaimSpec{
AccessModes: []apiv1.PersistentVolumeAccessMode{apiv1.ReadWriteOnce},
Resources: apiv1.ResourceRequirements{
Requests: apiv1.ResourceList{apiv1.ResourceStorage: resource.MustParse("1Gi")},
},
},
},
},
},
}
fmt.Println("creating stateful set", statefulSetName)
_, err := client.Create(context.Background(), statefulset, metav1.CreateOptions{})
if err != nil {
log.Fatal("failed to create stateful set", err)
}
fmt.Println("successfully created stateful set", statefulSetName)
}
If successful, this function creates a StatefulSet named test-statefulset.
First, it gets a
v1.StatefulSetInterfaceinstance by callingclientset.AppsV1().StatefulSets()with the default namespace.Prepares a new
StatefulSetdefinition.Invokes
StatefulSetInterface.Createto initiateStatefulSetcreation.
Update the main function - comment out deleteDeployment function and add the createStatefulSet function
func main() {
//createDeployment()
//deleteDeployment()
createStatefulSet()
}
Run the program to create a StatefulSet
To run the program:
go run main.goYou should see the following output:
creating stateful set test-statefulset successfully created stateful set test-statefulsetConfirm that the
StatefulSetwas created:kubectl get statefulset/test-statefulsetYou should see the following output:
NAME READY AGE test-statefulset 2/2 27sVerify the
Pods created by theStatefulSet:kubectl get pods -l=app=nginx-appYou should see the following output:
NAME READY STATUS RESTARTS AGE test-statefulset-0 1/1 Running 0 115s test-statefulset-1 1/1 Running 0 112sVerify the
PersistentVolumes created by theStatefulSet(the volume names might differ in your case):kubectl get pvYou should see the following output:
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-4744bb1f-7342-46c3-8324-0f418ee5ca06 1Gi RWO Delete Bound default/www-test-statefulset-1 standard 33m pvc-ba00706e-bfc8-4a03-885b-282b42afe5ad 1Gi RWO Delete Bound default/www-test-statefulset-0 standard 33mVerify the
PersistentVolumeClaims created by theStatefulSet(the volume names might differ in your case):kubectl get pvcYou should see the following output:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE www-test-statefulset-0 Bound pvc-ba00706e-bfc8-4a03-885b-282b42afe5ad 1Gi RWO standard 34m www-test-statefulset-1 Bound pvc-4744bb1f-7342-46c3-8324-0f418ee5ca06 1Gi RWO standard 34m
Add the function to delete a StatefulSet
Add the deleteStatefulSet function:
func deleteStatefulSet() {
client := clientset.AppsV1().StatefulSets(apiv1.NamespaceDefault)
fmt.Println("deleting stateful set", statefulSetName)
err := client.Delete(context.Background(), statefulSetName, metav1.DeleteOptions{})
if err != nil {
log.Fatal("failed to delete stateful set", err)
}
fmt.Println("successfully deleted stateful set", statefulSetName)
}
Update the main function - comment out createStatefulSet function and add the deleteStatefulSet function
func main() {
//createDeployment()
//deleteDeployment()
//createStatefulSet()
deleteStatefulSet()
}
Run the program to delete an existing StatefulSet
Run the program:
go run main.goYou should see the following output:
deleting stateful set test-statefulset successfully deleted stateful set test-statefulsetConfirm that the
StatefulSetwas deleted:kubectl get statefulset/test-statefulsetYou should see the following output:
Error from server (NotFound): statefulsets.apps "test-statefulset" not foundConfirm that
Pods created by theStatefulSetwas also deleted:kubectl get pods -l=app=nginx-appYou should see the following output:
No resources found in default namespace.
Conclusion
This article covered how to create and delete a Deployment and StatefulSet using the Kubernetes Go client library.
You can also learn more in the following documentation:
