Generating Kubernetes ValidatingAdmissionPolicies from Kyverno Policies

Generating Kubernetes ValidatingAdmissionPolicies from Kyverno Policies
By Mariam Fahmy |

In the previous blog post, we discussed writing Common Expression Language (CEL) expressions in Kyverno policies for resource validation. CEL was first introduced to Kubernetes for the Validation rules for CustomResourceDefinitions, and then it was used by Kubernetes ValidatingAdmissionPolicies in 1.26.

ValidatingAdmissionPolicies offer a declarative, in-process alternative to validating admission webhooks.

ValidatingAdmissionPolicies use the Common Expression Language (CEL) to declare the validation rules of a policy. Validation admission policies are highly configurable, enabling policy authors to define policies that can be parameterized and scoped to resources as needed by cluster administrators.

This post will show you how to generate Kubernetes ValidatingAdmissionPolicies and their bindings from Kyverno policies.

Prerequisite

Generating Kubernetes ValidatingAdmissionPolicies require the following:

  1. A cluster with Kubernetes 1.26 or higher.
  2. Enable the ValidatingAdmissionPolicy feature gate.
  3. Enable the admissionregistration.k8s.io/v1beta1 API for v1.28 and v1.29. OR Enable the admissionregistration.k8s.io/v1alpha1 API for v1.26 and v1.27.
  4. Set the --generateValidatingAdmissionPolicy flag in the Kyverno admission controller.
  5. Grant the admission controller service account the required permissions to generate ValidatingAdmissionPolicies and their bindings.

In this post, we will use the beta version of Kubernetes 1.29.

Installation & Setup

  1. Create a local cluster
 1kind create cluster --image "kindest/node:v1.28.0" --config - <<EOF
 2kind: Cluster
 3apiVersion: kind.x-k8s.io/v1alpha4
 4featureGates:
 5  ValidatingAdmissionPolicy: true
 6runtimeConfig:
 7  admissionregistration.k8s.io/v1beta1: true
 8  admissionregistration.k8s.io/v1alpha1: true
 9nodes:
10  - role: control-plane
11  - role: worker
12EOF
  1. Add the Kyverno Helm repository.
1helm repo add kyverno https://kyverno.github.io/kyverno/
2helm repo update
  1. Create a new file that overrides the values in the chart.
 1cat << EOF > new-values.yaml
 2features:
 3  generateValidatingAdmissionPolicy:
 4    enabled: true
 5
 6admissionController:
 7  rbac:
 8    clusterRole:
 9      extraResources:
10      - apiGroups:
11          - admissionregistration.k8s.io
12        resources:
13          - validatingadmissionpolicies
14          - validatingadmissionpolicybindings
15        verbs:
16          - create
17          - update
18          - delete
19          - list
20EOF
  1. Deploy Kyverno
1helm install kyverno kyverno/kyverno -n kyverno --create-namespace --version v3.1.4 --values new-values.yaml

We are now ready to generate Kubernetes ValidatingAdmissionPolicies from Kyverno policies.

Generating Kubernetes ValidatingAdmissionPolicies

In this section, we will create a Kyverno policy that ensures no hostPath volumes are in use for Deployments, and then we will have a look at the generated ValidatingAdmissionPolicy and its binding. Finally, we will create a Deployment that violates the policy.

Let’s start with creating the Kyverno policy.

 1kubectl apply -f - <<EOF
 2apiVersion: kyverno.io/v1
 3kind: ClusterPolicy
 4metadata:
 5  name: disallow-host-path
 6spec:
 7  validationFailureAction: Enforce
 8  background: false
 9  rules:
10    - name: host-path
11      match:
12        any:
13        - resources:
14            kinds:
15              - Deployment
16      validate:
17        cel:
18          expressions:
19            - expression: "!has(object.spec.template.spec.volumes) || object.spec.template.spec.volumes.all(volume, !has(volume.hostPath))"
20              message: "HostPath volumes are forbidden. The field spec.template.spec.volumes[*].hostPath must be unset."
21EOF

You can check whether a ValidatingAdmissionPolicy is generated or not from the Kyverno policy status.

 1$ kubectl get cpol disallow-host-path -o jsonpath='{.status}'
 2
 3{
 4   "autogen":{
 5      
 6   },
 7   "conditions":[
 8      {
 9         "lastTransitionTime":"2023-09-12T11:42:13Z",
10         "message":"Ready",
11         "reason":"Succeeded",
12         "status":"True",
13         "type":"Ready"
14      }
15   ],
16   "ready":true,
17   "rulecount":{
18      "generate":0,
19      "mutate":0,
20      "validate":1,
21      "verifyimages":0
22   },
23   "validatingadmissionpolicy":{
24      "generated":true,
25      "message":""
26   }
27}

Let’s try getting the ValidatingAdmissionPolicy and its binding.

1$ kubectl get validatingadmissionpolicy
2NAME                                 VALIDATIONS   PARAMKIND   AGE
3disallow-host-path   1                        <unset>        8m12s
4
5$ kubectl get validatingadmissionpolicybindings
6NAME                                                   POLICYNAME                    PARAMREF   AGE
7disallow-host-path-binding   disallow-host-path   <unset>      8m30s

You may notice that the ValidatingAdmissionPolicy and the ValidatingAdmissionPolicyBinding share the same name as the Kyverno policy they originate from, with the binding having a “-binding” suffix.

Let’s have a look at the ValidatingAdmissionPolicy and its binding in detail.

 1$ kubectl get validatingadmissionpolicy disallow-host-path -o yaml
 2apiVersion: admissionregistration.k8s.io/v1beta1
 3kind: ValidatingAdmissionPolicy
 4metadata:
 5  creationTimestamp: "2023-09-12T11:42:13Z"
 6  generation: 1
 7  labels:
 8    app.kubernetes.io/managed-by: kyverno
 9  name: disallow-host-path
10  ownerReferences:
11  - apiVersion: kyverno.io/v1
12    kind: ClusterPolicy
13    name: disallow-host-path
14    uid: e540d96b-c683-4380-a84f-13411384241a
15  resourceVersion: "11294"
16  uid: 9f3e0161-d010-4a6f-bd28-bf9c87151795
17spec:
18  failurePolicy: Fail
19  matchConstraints:
20    matchPolicy: Equivalent
21    namespaceSelector: {}
22    objectSelector: {}
23    resourceRules:
24    - apiGroups:
25      - apps
26      apiVersions:
27      - v1
28      operations:
29      - CREATE
30      - UPDATE
31      resources:
32      - deployments
33      scope: '*'
34  validations:
35  - expression: '!has(object.spec.template.spec.volumes) || object.spec.template.spec.volumes.all(volume,
36      !has(volume.hostPath))'
37    message: HostPath volumes are forbidden. The field spec.template.spec.volumes[*].hostPath
38      must be unset.
39  variables: null
40status:
41  observedGeneration: 1
42  typeChecking: {}

 1$ kubectl get validatingadmissionpolicybindings disallow-host-path-binding -o yaml
 2apiVersion: admissionregistration.k8s.io/v1beta1
 3kind: ValidatingAdmissionPolicyBinding
 4metadata:
 5  creationTimestamp: "2023-09-12T11:42:13Z"
 6  generation: 1
 7  labels:
 8    app.kubernetes.io/managed-by: kyverno
 9  name: disallow-host-path-binding
10  ownerReferences:
11  - apiVersion: kyverno.io/v1
12    kind: ClusterPolicy
13    name: disallow-host-path
14    uid: e540d96b-c683-4380-a84f-13411384241a
15  resourceVersion: "11292"
16  uid: 2fec35c3-8a8c-42a7-8a02-a75e8882a01e
17spec:
18  policyName: disallow-host-path
19  validationActions:
20  - Deny

Now, let’s try deploying an app that uses a hostPath:

 1kubectl apply -f - <<EOF
 2apiVersion: apps/v1
 3kind: Deployment
 4metadata:
 5  name: nginx
 6spec:
 7  replicas: 2
 8  selector:
 9    matchLabels:
10      app: nginx
11  template:
12    metadata:
13      labels:
14        app: nginx
15    spec:
16      containers:
17      - name: nginx-server
18        image: nginx
19        volumeMounts:
20          - name: udev
21            mountPath: /data
22      volumes:
23      - name: udev
24        hostPath:
25          path: /etc/udev
26EOF

As expected, the deployment creation is rejected by the API server and not by the Kyverno admission controller.

1The deployments "nginx" is invalid:  ValidatingAdmissionPolicy 'disallow-host-path' with binding 'disallow-host-path-binding' denied request: HostPath volumes are forbidden. The field spec.template.spec.volumes[*].hostPath must be unset.

If either the ValidatingAdmissionPolicy or the binding is deleted/updated for some reason, the controller is responsible for reverting it.

Let’s try deleting the ValidatingAdmissionPolicy.

1$ kubectl delete validatingadmissionpolicy disallow-host-path 
2validatingadmissionpolicy.admissionregistration.k8s.io "disallow-host-path" deleted
3
4$ kubectl get validatingadmissionpolicy 
5NAME                                  VALIDATIONS   PARAMKIND   AGE
6disallow-host-path    1                        <unset>        11s

In addition, you can update the Kyverno policy, and the controller will re-generate the ValidatingAdmissionPolicy accordingly. For example, you can change the Kyverno policy to match statefulsets too.

patch.yaml:

 1spec:
 2  rules:
 3    - name: host-path
 4      match:
 5        any:
 6        - resources:
 7            kinds:
 8              - Deployment
 9              - StatefulSet
10      validate:
11        cel:
12          expressions:
13            - expression: "!has(object.spec.template.spec.volumes) || object.spec.template.spec.volumes.all(volume, !has(volume.hostPath))"
14              message: "HostPath volumes are forbidden. The field spec.template.spec.volumes[*].hostPath must be unset."

1kubectl patch cpol disallow-host-path --type merge --patch-file patch.yaml

The ValidatingAdmissionPolicy will be updated to match StatefulSets too.

 1apiVersion: admissionregistration.k8s.io/v1beta1
 2kind: ValidatingAdmissionPolicy
 3metadata:
 4  creationTimestamp: "2023-09-12T12:54:48Z"
 5  generation: 2
 6  labels:
 7    app.kubernetes.io/managed-by: kyverno
 8  name: disallow-host-path
 9  ownerReferences:
10  - apiVersion: kyverno.io/v1
11    kind: ClusterPolicy
12    name: disallow-host-path
13    uid: e540d96b-c683-4380-a84f-13411384241a
14  resourceVersion: "29208"
15  uid: 9325e2b7-9131-4ff4-9e56-244129cb625e
16spec:
17  failurePolicy: Fail
18  matchConstraints:
19    matchPolicy: Equivalent
20    namespaceSelector: {}
21    objectSelector: {}
22    resourceRules:
23    - apiGroups:
24      - apps
25      apiVersions:
26      - v1
27      operations:
28      - CREATE
29      - UPDATE
30      resources:
31      - deployments
32      - statefulsets
33      scope: '*'
34  validations:
35  - expression: '!has(object.spec.template.spec.volumes) || object.spec.template.spec.volumes.all(volume,
36      !has(volume.hostPath))'
37    message: HostPath volumes are forbidden. The field spec.template.spec.volumes[*].hostPath
38      must be unset.
39  variables: null
40status:
41  observedGeneration: 2
42  typeChecking: {}

Conclusion

In this blog, we discussed how to generate Kubernetes ValidatingAdmissionPolicies from Kyverno policies. You can use CEL expressions in Kyverno policies to validate resources through either the Kyverno engine or the API server. In the next blog, we will discuss how to generate BackgroundScan reports for ValidatingAdmissionPolicies.