Skip to main content

Creating a Simple Policy

Kyverno has two kinds of Policy resources: ClusterPolicy used for Cluster-Wide Resources and Policy used for Namespaced Resources. To gain an understanding of Kyverno policies, we'll start our lab with a simple Pod label requirement. As you may know, labels in Kubernetes are used to tag resources in the cluster.

Below is a sample ClusterPolicy which will block any Pod creation that doesn't have the label CostCenter:

~/environment/eks-workshop/modules/security/kyverno/simple-policy/require-labels-policy.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-labels
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-team
      match:
        any:
          - resources:
              kinds:
                - Pod
      validate:
        message: "Label 'CostCenter' is required to deploy the Pod"
        pattern:
          metadata:
            labels:
              CostCenter: "?*"
A

spec.validationFailureAction tells Kyverno if the resource being validated should be allowed but reported (Audit) or blocked (Enforce). The default is Audit, but in our example it is set to Enforce

B

The rules section contains one or more rules to be validated

C

The match statement sets the scope of what will be checked. In this case, it's any Pod resource

D

The validate statement attempts to positively check what is defined. If the statement, when compared with the requested resource, is true, it's allowed. If false, it's blocked

E

The message is what gets displayed to a user if this rule fails validation

F

The pattern object defines what pattern will be checked in the resource. In this case, it's looking for metadata.labels with CostCenter

Create the policy using the following command:

~$kubectl apply -f ~/environment/eks-workshop/modules/security/kyverno/simple-policy/require-labels-policy.yaml
 
clusterpolicy.kyverno.io/require-labels created


Next, take a look at the Pods running in the ui Namespace and notice the applied labels:


~$kubectl -n ui get pods --show-labels
NAME                  READY   STATUS    RESTARTS   AGE   LABELS
ui-67d8cf77cf-d4j47   1/1     Running   0          9m    app.kubernetes.io/component=service,app.kubernetes.io/created-by=eks-workshop,app.kubernetes.io/instance=ui,app.kubernetes.io/name=ui,pod-template-hash=67d8cf77cf


Notice that the running Pod doesn't have the required Label, and Kyverno didn't terminate it. This is because Kyverno operates as an AdmissionController and won't interfere with resources that already exist in the cluster.


However, if you delete the running Pod, it won't be able to be recreated since it doesn't have the required Label. Go ahead and delete the Pod running in the ui Namespace:


~$kubectl -n ui delete pod --all
pod "ui-67d8cf77cf-d4j47" deleted
~$kubectl -n ui get pods
No resources found in ui namespace.


As mentioned, the Pod was not recreated. Try to force a rollout of the ui Deployment:


~$kubectl -n ui rollout restart deployment/ui
deployment.apps/ui restarted


The rollout command itself succeeds because Kyverno's admission webhook validates the Deployment patch, not the restart annotation. However, when the Deployment controller attempts to create the new Pod, Kyverno blocks it at the Pod level since it lacks the required CostCenter label. The new Pod will never come up.


You can confirm this by checking the events in the ui Namespace:


~$kubectl -n ui get events | grep PolicyViolation
9m         Warning   PolicyViolation     replicaset/ui-67d8cf77cf   policy require-labels/autogen-check-team fail: validation error: Label 'CostCenter' is required to deploy the Pod. rule autogen-check-team failed at path /spec/template/metadata/labels/CostCenter/
9m         Warning   PolicyViolation     deployment/ui              policy require-labels/autogen-check-team fail: validation error: Label 'CostCenter' is required to deploy the Pod. rule autogen-check-team failed at path /spec/template/metadata/labels/CostCenter/


The PolicyViolation events show that Kyverno blocked the Pod and ReplicaSet from being created due to the require-labels policy.


Now add the required label CostCenter to the ui Deployment, using the Kustomization patch below:


~/environment/eks-workshop/modules/security/kyverno/simple-policy/ui-labeled/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ui
spec:
  template:
    metadata:
      labels:
        CostCenter: IT


~$kubectl apply -k ~/environment/eks-workshop/modules/security/kyverno/simple-policy/ui-labeled
namespace/ui unchanged
serviceaccount/ui unchanged
configmap/ui unchanged
service/ui unchanged
deployment.apps/ui configured
~$kubectl -n ui rollout status deployment/ui
deployment "ui" successfully rolled out
~$kubectl -n ui get pods --show-labels
NAME                  READY   STATUS    RESTARTS   AGE   LABELS
ui-5498685db8-k57nk   1/1     Running   0          60s   CostCenter=IT,app.kubernetes.io/component=service,app.kubernetes.io/created-by=eks-workshop,app.kubernetes.io/instance=ui,app.kubernetes.io/name=ui,pod-template-hash=5498685db8


As you can see, the admission webhook successfully validated the Policy and the Pod was created with the correct Label CostCenter=IT!


Mutating Rules​


In the above examples, you checked how Validation Policies work in their default behavior defined in validationFailureAction. However, Kyverno can also be used to manage Mutating rules within the Policy, to modify any API Requests to satisfy or enforce the specified requirements on the Kubernetes resources. The resource mutation occurs before validation, so the validation rules will not contradict the changes performed by the mutation section.


Below is a sample Policy with a mutation rule defined:


~/environment/eks-workshop/modules/security/kyverno/simple-policy/add-labels-mutation-policy.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: add-labels
spec:
  rules:
    - name: add-labels
      match:
        any:
          - resources:
              kinds:
                - Pod
      mutate:
        patchStrategicMerge:
          metadata:
            labels:
              CostCenter: IT
A
match.any.resources.kinds: [Pod] targets this ClusterPolicy to all Pod resources cluster-wide
B
mutate modifies resources during creation (vs. validate which blocks/allows). patchStrategicMerge.metadata.labels.CostCenter: IT automatically adds CostCenter: IT label to every Pod




Go ahead and create the above Policy using the following command:


~$kubectl apply -f  ~/environment/eks-workshop/modules/security/kyverno/simple-policy/add-labels-mutation-policy.yaml
 
clusterpolicy.kyverno.io/add-labels created


To validate the Mutation Webhook, let's roll out the carts Deployment without explicitly adding a label:


~$kubectl -n carts rollout restart deployment/carts
deployment.apps/carts restarted
~$kubectl -n carts rollout status deployment/carts
deployment "carts" successfully rolled out


Validate that the label CostCenter=IT was automatically added to the Pod to meet the policy requirements, resulting in a successful Pod creation even though the Deployment didn't have the label specified:


~$kubectl -n carts get pods --show-labels
NAME                     READY   STATUS    RESTARTS   AGE   LABELS
carts-bb88b4789-kmk62   1/1     Running   0          25s   CostCenter=IT,app.kubernetes.io/component=service,app.kubernetes.io/created-by=eks-workshop,app.kubernetes.io/instance=carts,app.kubernetes.io/name=carts,pod-template-hash=bb88b4789


Note that only the carts pod has the CostCenter=IT label. Any other pods in the namespace that were already running before the mutation policy was created (such as carts-dynamodb) will not have the label applied. Kyverno mutation only fires on admission. So existing pods must be restarted to go through the webhook and receive the mutation.


It's also possible to mutate existing resources in your Amazon EKS Clusters with Kyverno Policies using patchStrategicMerge and patchesJson6902 parameters in your Kyverno Policy.


This was just a simple example of labels for our Pods with Validating and Mutating rules. This can be applied to various scenarios such as restricting images from unknown registries, adding data to ConfigMaps, setting tolerations, and much more. In the upcoming labs, you will explore some more advanced use-cases.