Introducing kwctl to Kubernetes Administrators
Author:
Published:
Updated:
We are pleased to announce the availability of a new tool within the Kubewarden project: kwctl.
kwctl is a command line utility designed to help both policy authors and Kubernetes administrators.
This blog post focuses on the user experience of Kubernetes administrators. Future ones will cover the policy developer side of the story.
A Real-World Example: Controlling Container Capabilities
The main character of today’s story is Alice. Alice is a Kubernetes administrator who wants to keep her Kubernetes cluster secure.
Alice uses many solutions to accomplish that, including Kubernetes PSP.
Pod Security Policies are currently deprecated and are going to be removed from Kubernetes 1.25. Due to that, Alice wants to find an alternative to Kubernetes PSPs.
Today Alice will work to replace the container capabilities PSP with a Kubewarden policy.
We will tag along with Alice, watch all the steps she has to perform
and learn how kwctl
will help her.
Finding the Right Policy
As a first step, Alice visits Kubewarden’s Policy Hub, the place where Kubewarden policies can be made discoverable.
Alice enters the “capabilities” query and finds a policy that seems to be doing exactly what she’s looking for. The name of the policy is “psp-capabilities”.
Obtaining the Policy
Next Alice has to obtain the policy. This can be done with a simple command:
$ kwctl pull registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4
This downloads the policy and stores it under Alice’s home directory.
Understanding How the Policy Works
Alice now needs to understand how to use the policy. Each policy listed on Kubewarden’s Policy Hub has links pointing to their documentation.
Instead of opening one of these links, Alice prefers to consult the metadata provided by the policy:
$ kwctl inspect registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4
This command produces the following output:
Note well: a Kubewarden policy is a regular “
.wasm
” file that, in addition to contain the policy’s bytecode, it also embeds additional metadata.This allows to have everything in a single place, versioned together and available off-line.
Evaluating the Policy
Now Alice wants to assess the reliability of the policy, plus find the right configuration values to satisfy her requirements.
There are some good news for Alice: she doesn’t need to deploy the policy into
a Kubernetes cluster to perform this kind of rapid iterations. Once again, kwctl
can help her with the run
subcommand.
The kwctl run
command takes the following parameters:
-r <request-path>
: path to a file containing the Kubernetes Admission Request to be evaluated--settings-json "<JSON document>"
: a string containing the JSON representation of policy’s settings. It’s also possible to use the “-s <settings-path>
” flag to read the policy settings from a local file<URI of the policy>
: the URI pointing to the policy to be used
Note well: unfortunately, obtaining Kubernetes Admission Requests requires some extra work. This is a “Kubernetes problem”, we plan to address it in the near future.
In the meantime, we will assume Alice has access to files like these ones.
Alice will use this request;
that attempts to create a Pod with a container that has the
following securityContext
configuration:
- Add the
NET_ADMIN
and theSYS_TIME
capabilities - Drop the
SYS_PTRACE
capability
As a first step, Alice defines a policy that allows containers to add only the SYS_TIME
capability:
$ kwctl run registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4 \
-r req_pod_with_container_with_capabilities_added.json \
--settings-json '{"allowed_capabilities": ["SYS_TIME"]}'
The output of the command, piped into jq
, is the following one:
{
"uid": "1299d386-525b-4032-98ae-1949f69f9cfc",
"allowed": false,
"status": {
"message": "PSP capabilities policies doesn't allow these capabilities to be added: {\"NET_ADMIN\"}"
}
}
This is the behavior expected by Alice: the request is rejected because the
NET_ADMIN
capability is not part of the allow list.
This policy is capable of mutating incoming requests, Alice wants to try it out using the following command:
$ kwctl run registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4 \
-r req_pod_with_container_with_capabilities_added.json \
--settings-json '{"default_add_capabilities": ["AUDIT_READ"]}'
The command will fail with the following output:
{"valid":false,"message":"These capabilities cannot be added by default because they are not allowed: {\"AUDIT_READ\"}"}
Error: Provided settings are not valid: Some("These capabilities cannot be added by default because they are not allowed: {\"AUDIT_READ\"}")
Nice, the author of the policy implemented some validation of the settings
provided by the end users!
The error message points Alice in the right direction: the AUDIT_READ
capability
cannot be added to all the containers unless it’s also on the list of the
allowed capabilities.
Alice runs the policy again, this time with a different configuration:
$ kwctl run registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4 \
-r req_pod_with_container_with_capabilities_added.json \
--settings-json '{"default_add_capabilities": ["AUDIT_READ"], "allowed_capabilities": ["AUDIT_READ"]}'
This time the configuration is correct, but the request is rejected with the following explanation:
{
"uid": "1299d386-525b-4032-98ae-1949f69f9cfc",
"allowed": false,
"status": {
"message": "PSP capabilities policies doesn't allow these capabilities to be added: {\"NET_ADMIN\", \"SYS_TIME\"}"
}
}
That’s actually a correct behavior, Alice forgot to allow the capabilities the container is already adding to itself.
Alice now runs the kwctl command one last time:
$ kwctl run registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4 \
-r req_pod_with_container_with_capabilities_added.json \
--settings-json '{"default_add_capabilities": ["AUDIT_READ"], "allowed_capabilities": ["AUDIT_READ", "NET_ADMIN", "SYS_TIME"]}'
This time the policy accepts the incoming request, plus it mutates it. This is the output returned:
{
"uid": "1299d386-525b-4032-98ae-1949f69f9cfc",
"allowed": true,
"patchType": "JSONPatch",
"patch": "W3sib3AiOiJhZGQiLCJwYXRoIjoiL3NwZWMvY29udGFpbmVycy8wL3NlY3VyaXR5Q29udGV4dC9jYXBhYmlsaXRpZXMvYWRkLzIiLCJ2YWx1ZSI6IkFVRElUX1JFQUQifV0="
}
Alice can see what the policy is going to change by looking at the contents of
the patch
attribute.
As requested by Kubernetes, the patch operation is encoded using base64
:
$ kwctl run registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4 \
-r req_pod_with_container_with_capabilities_added.json \
--settings-json '{"default_add_capabilities": ["AUDIT_READ"], "allowed_capabilities": ["AUDIT_READ", "NET_ADMIN", "SYS_TIME"]}' \
| jq -r .patch | base64 -d
This produces the following output:
[{"op":"add","path":"/spec/containers/0/securityContext/capabilities/add/2","value":"AUDIT_READ"}]
This JSONPatch document is composed by the following parts:
op
: this defines the operation to perform against the original requestvalue
: the string to be added to the original requestpath
: the JSON Pointer that defines which part of the original request is going to be changed
Hence, this JSONPatch adds “AUDIT_READ” to the original Pod object, at
/spec/containers/0/securityContext/capabilities/add/2
location.
Let’s take a closer look at the JSON Pointer:
/spec/containers
: this references thecontainers
section of the Podspec
/0
: sincespec.containers
is an array, this fragment of the path points to the first element of it. Note well, arrays elements are referenced using zero-based numbering/securityContext/capabilities/add
: this navigates into thesecurityContext
of the container, then into thecapabilities
object and finally into theadd
section. This is the place where container capabilities to be added are specified/2
: the original container has already two capabilities to be added,NET_ADMIN
andSYS_TIME
. Using zero-based numbering,/2
points to the third entry of the array.
The policy is working as expected; it leads to the creation of a Pod with
a container that has the NET_ADMIN
, SYS_TIME
, AUDIT_READ
capabilities
added.
Deploying the Policy
After more experiments with kwctl run
, Alice wants to deploy the policy
inside of a Kubernetes cluster.
Kubewarden policies are defined using the ClusterAdmissionPolicy, a Custom Resource provided by Kubewarden.
Alice can quickly scaffold the YAML file defining this resource using kwctl:
$ kwctl manifest -t ClusterAdmissionPolicy registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4
The command will produce the following output:
apiVersion: policies.kubewarden.io/v1alpha2
kind: ClusterAdmissionPolicy
metadata:
name: generated-policy
spec:
module: "registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4"
settings: {}
rules:
- apiGroups:
- ""
apiVersions:
- v1
resources:
- pods
operations:
- CREATE
mutating: true
kwctl just saved quite some time for Alice 😊
Mirroring Policies
Kubewarden policies are distributed via OCI container registries, the very same pieces of infrastructure already used to distribute container images.
Alice doesn’t want her cluster to pull the Kubewarden policy straight from
ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4
, she rather prefers to
fetch it from a local registry she controls.
This can be done using the following commands:
$ kwctl push -p registry://ghcr.io/kubewarden/policies/psp-capabilities:v0.1.4
registry.alice.corp.lan/policies/psp-capabilities:v0.1.4
Alice can now use the registry.alice.corp.lan/policies/psp-capabilities:v0.1.4
url inside of the ClusterAdmissionPolicy
definition.
The policy can now be enforced inside of the local Kubernetes cluster with
a kubectl apply
command 🎉
Recap
kwctl allows us to download, test and deploy Kubewarden policies. The UX of kwctl mimics the one of docker, hence cloud native users should quickly feel at home with it.
We hope kwctl will be able to simplify the process of interacting with Kubewarden policies. We have many ideas about how to further expand its capabilities, but we would love to hear what you would like to see addressed or changed.
What are you waiting for? Do like Alice and give kwctl a spin!