Kubernetes API client

Introduction

This is an overview and notes on how to use the client-go Kubernetes API.

Kubernetes API resources

Accessing via API

The API is versioned for extensability propose, the levels are Alpha level (v1alpha1), Beta (b2beta3) stabel (v1).

You can notice the groups of the resources exists inside the version. To access the endpoints directly start the proxy with 

$ kubectl proxy --port=8080

$ curl http://localhost:8080/apis/

# To dig deeper in versions and existent resources
$ kubectl api-resources && kubectl api-version

Accessing via client-go

Its possible to access a Pod via a clientset and fetch attributes from it, in the example bellow, it must fetch the Image from a CoreDNS container in the Kube-system namespace.

package main

import (
	"fmt"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/tools/clientcmd"
	"k8s.io/client-go/kubernetes"
)

func errPanic(err error) {
	if err != nil {
		panic(err)
	}
}

func main() {
	config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)

	clientset, err := kubernetes.NewForConfig(config)
	errPanic(err)

	pod, err := clientset.CoreV1().Pods("kube-system").Get("coredns-fb8b8dccf-r5sgd", metav1.GetOptions{})
	errPanic(err)

	for _, container :=  range pod.Spec.Containers {
		fmt.Println(container.Image)
	}
}

To have a better visualization of the resource type check the API definition on kubernetes/api

https://github.com/kubernetes/api/blob/master/core/v1/types.go#L3513

A generated documentation is available at:

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/

Watch, informers and caching

Client interfaces clientset includes the Watch verb, which offers an event interface that reacts to changes (add, removes, updated) of objects, but instead of being used directly we must use the Informer inteface for the most common use cases, this pattern offered by the client gives an indexed lookup of objects and in-memory cache.

If the following example we have a callback happening for delete a Pod.

	informersFactory := informers.NewSharedInformerFactory(clientset, 0)
	podInformer := informersFactory.Core().V1().Pods().Informer()

	pods := make(chan *v1.Pod, 1)
	podInformer.AddEventHandler(&cache.ResourceEventHandlerFuncs{
		DeleteFunc: func(obj interface{}) {
			pod := obj.(*v1.Pod)
			pods <- pod
		},
	})
	informersFactory.Start(ctx.Done())
	select {
		case pod := <-pods:
			fmt.Println(pod.Namespace, pod.Name)
	}

Custom Resources

Kubernetes let you extend your API via Custom resources and programmaticaly acccess it. There're two ways of accesing it.

A resource is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind. A custom resrouce is an extension of a Kubernetes API that is not necessarily avaialble in a default installation. They can appear and disappear in a running cluster through dynamic registration.

https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/

On this example this CRD sawtooth.hyperledger.org/example-sawtooth has the following characteristics:

apiVersion: sawtooth.hyperledger.org/v1alpha1
kind: Sawtooth
metadata:
  generation: 1
  name: example-sawtooth
  namespace: default
spec:
  size: 3
There's two ways of accessing it, they are:

Dynamic client access

Using client-go/dynamic, we have an agnotic way to access the attributes of the objects created, so it's possible to access the spec on this way.

	sawtoothRes := schema.GroupVersionResource{
		Group: "sawtooth.hyperledger.org",
		Version: "v1alpha1",
		Resource: "sawtooths",
	}


	list, err := client.Resource(sawtoothRes).Namespace("default").List(metav1.ListOptions{})
	errPanic(err)

	for _, r := range list.Items {
		size, _, err := unstructured.NestedInt64(r.Object, "spec", "size")
		errPanic(err)

		fmt.Println(size)
	}

Typed

But the first thing you need to do is to have a pkg/apis/sawtooth with the following files:

pkg
└── apis
    └── sawtooth
        ├── register.go
        └── v1alpha1
            ├── doc.go
            ├── register.go
            ├── types.go

You can follow the sample-controller or the links bellow to understand what each file means or goes inside, but the spec stays at types.go, pay attention in the tags these are being used to generate our custom spec.

package v1alpha1

import (
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// +genclient
// +genclient:noStatus
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// Sawtooth spec
type Sawtooth struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   SawtoothSpec   `json:"spec,omitempty"`
}

// SawtoothSpec is a spec
type SawtoothSpec struct {
	Size int64  `json:"size"`

}

Now, you can generate the clients and accessors automatically with:

$ generate-groups.sh all \
   github.com/knabben/kube-audit/pkg/generated \
   github.com/knabben/kube-audit/pkg/apis "sawtooth:v1alpha1"

import (
    client "github.com/knabben/kube-audit/pkg/generated/clientset/versioned"
)

...

clientset, err := client.NewForConfig(config)
errPanic(err)

list := clientset.SawtoothV1alpha1().Sawtooths("default")
sawtooth, err := list.Get("example-sawtooth", metav1.GetOptions{})
errPanic(err)

fmt.Println(sawtooth.Spec.Nodes)

For more information:

https://blog.openshift.com/kubernetes-deep-dive-code-generation-customresources/

Listening