Let’s build a custom kubectl plugin in Golang

This blog is about how to extend the capabilities of kubectlusing plugins. These plugins are way of utilizing the existing framework to build more useful features.

Why:

Inspite of kubectl being a great tool for users, the capabilities are still limited and the plugins can be used to add sub-commands specific to one’s development needs, and they can be written in popular languages like python, golang, rust and so on.

Prerequisites:

1. kubectl → Kubernetes command-line tool
2. A kind cluster (or build any other clusters using tools like minikube, kubespray, microk8s, k3s and so on)

Goal:

The aim is to build a plugin which can query the pods if a certain port is listening or not. The inspiration behind this project came when I was working on installing Prometheus, and I was having difficulties finding if the port was being used by some other processes or not, and it was difficult because, there were a lot of microservices running on the nodes and hence the challenge.

Libraries Used:

1. GoLang Client for kubernetes
2. GoLang SSH Terminal library
3. Kubernetes Core V1 API Golang Library

Our Plugin:

The below written code utilizes the bufio lib of golang to read the terminal output

func readLine(reader *bufio.Reader) (strLine string, err error) {
    buffer := new(bytes.Buffer) 
    for {  
          var line []byte  
          var isPrefix bool  
          line, isPrefix, err = reader.ReadLine()  
          if err != nil && err != io.EOF {   
              return "", err  
          }  
          buffer.Write(line)  
          if !isPrefix {   
              break  
          } 
     } 
     return buffer.String(), err}

The below given block of code creates a working kube client using the clientcmd package of client-go . The kubeconfig.ClientConfig() returns the Kubernetes API client config. The corev1client.NewForConfig() returns a corev1 client for accessing the pods.

cmd defines the command to be executed inside the pod terminal, in this case, it’s a netstat command, which filters the LISTEN ports.

Creates a post request for executing the netstat command inside the pod

The remotecommand module under client-go , makes the use of NewSPDYExecutor function which connects to the provided server, which in this case is the req provided above.

Here, MakeRaw , puts the terminal connected to the given file descriptor into raw mode and returns the previous state of the terminal so that it can be restored. We are also creating custom stdout and stderr buffers for capturing the output and any errors.

exec.Stream , is used for executing the command inside the pod

We then create a reader for reading the terminal output line by line, and parse the integers and strings separately for extracting the port number and their respective status, i.e., whether the port is listening or not

Finally, in the main function, we define the arguments needed to call the plugins.

Using the plugin:

1. Build the program: go build
2. Make the golang binary an executable: sudo chmod +x ./kubectl-portScanner
3. Place it in the PATH : sudo mv ./kubectl-portScanner /usr/local/bin
4. You can use the plugin as: kubectl portScanner "podName" "port number" "pod namespace"

Examples:

Below given are some examples of the plugins:

100 port is not listening for any process

9100 port is listening for a particular process