The two most common extreme mistakes in Kubernetes security are:
- “Let’s leave it open for now and revisit later” → risk that piles up for years
- “Let’s enforce everything” → one morning half the deploys break and everything gets rolled back
The most sustainable field-tested path is phased hardening: visibility first, then warning, and only at the end mandatory enforcement. PSA (Pod Security Admission) provides a solid foundation for this; Kyverno covers the practical security needs that PSA does not address by encoding them into policy.
1) Position PSA correctly: “baseline” is a starting point
PSA is generally framed around three levels:
privileged(the most relaxed)baseline(a reasonable minimum)restricted(stricter)
PSA’s strength: it operates on a per-namespace basis using a standardized framework. Its weakness: organization-specific needs (registry allowlist, mandatory labels, etc.) cannot be solved by PSA alone.
2) The strategy: audit → warn → enforce
In production I recommend the following ordering:
- Audit: log only (no impact)
- Warn: alert the user (still works)
- Enforce: reject (the actual guardrail)
The aim of this sequence is not “being soft”; it is to collect signal and prevent blast.
3) Activating PSA via namespace labels
Example: first turn on baseline in audit/warn mode across all namespaces:
apiVersion: v1
kind: Namespace
metadata:
name: app-prod
labels:
pod-security.kubernetes.io/audit: baseline
pod-security.kubernetes.io/warn: baseline
Then, once the reports settle, switch to enforce:
metadata:
labels:
pod-security.kubernetes.io/enforce: baseline
4) Where PSA falls short: the need for organization-level guardrails
PSA’s levels do not cover every organization’s needs. For example:
- Pull images only from approved registries
- Allow privileged containers only in specific namespaces
- Keep
hostPathexceptions tightly controlled - Make practices like
runAsNonRootandreadOnlyRootFilesystemmandatory
This is where policy engines like Kyverno step in.
5) “Highest-value” policies with Kyverno
A) Privileged and hostPath restrictions
Sample approach: deny by default, allow only in the “infra” namespace.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: disallow-privileged-except-infra
spec:
validationFailureAction: Audit
rules:
- name: privileged-only-in-infra
match:
any:
- resources:
kinds: ["Pod"]
validate:
message: "Privileged is only permitted in the infra namespace."
deny:
conditions:
any:
- key: "{{ request.object.metadata.namespace }}"
operator: NotEquals
value: "infra"
- key: "{{ request.object.spec.containers[].securityContext.privileged || `false` }}"
operator: Equals
value: true
Note: adapt these Kyverno examples to your version and CRD layout. The principle is constant: gather signal first via Audit.
B) Registry allowlist (supply chain control)
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: image-registry-allowlist
spec:
validationFailureAction: Audit
rules:
- name: only-approved-registries
match:
any:
- resources:
kinds: ["Pod"]
validate:
message: "Images may only be pulled from approved registries."
pattern:
spec:
containers:
- image: "registry.example.com/* | ghcr.io/org/*"
C) readOnlyRootFilesystem + runAsNonRoot (gradual tightening)
Begin with warn/audit and later switch to enforce:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-nonroot-and-readonly
spec:
validationFailureAction: Audit
rules:
- name: nonroot-readonly
match:
any:
- resources:
kinds: ["Pod"]
validate:
message: "Containers must run non-root with a read-only root filesystem."
pattern:
spec:
containers:
- securityContext:
runAsNonRoot: true
readOnlyRootFilesystem: true
6) Exception management: not “policy bypass” but “policy contract”
Exceptions are unavoidable (node-exporter, CNI, storage drivers, etc.). What matters here is:
- The exception must answer “to whom, why, and for how long”
- It must be time-bound (e.g. 30 days)
- Owner and risk note must be clearly captured
A practical solution:
- YAML exception files under an
exceptions/folder - Approval via PR
- Expiry and owner via labels/annotations
7) Runbook: the “Pod rejected” incident
- Was the failure from PSA or Kyverno? (event and admission message)
- What is the namespace’s policy level? (label check)
- Which field tripped it? (
privileged,hostPath,capabilities, etc.) - Resolution:
- Fix the workload (preferred)
- Temporary exception (time-bound + owner)
- Misconfigured policy (correct the rule, then return to Audit)
Closing
PSA + Kyverno transforms Kubernetes security from “negotiation at every deploy” into a productized set of guardrails. The key to success is not the technology; it is phased rollout, exception discipline, and measurement.