The hardest “right problem” in GitOps is secret handling. Application manifests live in Git; the secret itself, you don’t want there. One pragmatic answer to that dilemma: keep the secret encrypted in Git and only decrypt it where it actually belongs (CI or cluster).
In this guide we’ll build a working secrets flow with SOPS + age — one that fits real life: PR review stays intact, and nobody falls back to the “paste the secret somewhere” reflex.
0) Prerequisites and decisions
This article assumes:
- Secrets stay in Git in encrypted form
- The decryption key never lives in Git
- Decryption only happens inside CI or the cluster, in a controlled way
Why age?
- Simple, fast, well-suited to a file-oriented encryption flow
- A “key file” model makes operations easier
Note: SOPS with a cloud KMS is also a strong choice. I’m using age here specifically because it gives you a “lightweight and portable” starting point.
1) Generating an age key (and the storage discipline around it)
Step one is producing the age key itself:
age-keygen -o age.key
Extract the public key:
age-keygen -y age.key
Reasonable storage options:
- A secret manager (Vault, a cloud secret store)
- The CI secret store (repository / environment secrets)
- “Wrapping the key” with an HSM/KMS (advanced)
2) SOPS configuration: .sops.yaml
Drop a .sops.yaml at the repo root to standardize which files are encrypted and how.
Example:
creation_rules:
- path_regex: '.*\\.enc\\.ya?ml$'
age: 'age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
This way the team stops debating “which command do we encrypt with?” — the filename itself signals the flow.
3) Producing an encrypted secret file
Start with a plaintext Kubernetes secret manifest:
apiVersion: v1
kind: Secret
metadata:
name: app-secrets
type: Opaque
stringData:
DATABASE_URL: "postgres://user:pass@db:5432/app"
Encrypt it:
sops -e app-secrets.yaml > app-secrets.enc.yaml
Then move the plaintext file out of the repo or delete it.
4) “Decrypt + apply” inside CI (the simplest starting point)
Even if you aren’t running a GitOps tool, decrypting and applying inside CI is a reasonable first step.
A sample flow:
- CI pulls
age.keyfrom the secret store SOPS_AGE_KEY_FILEenv var is set- Decryption happens via
sops -d kubectl applyruns
export SOPS_AGE_KEY_FILE=./age.key
sops -d app-secrets.enc.yaml | kubectl apply -f -
5) Decryption with GitOps tools (operational notes)
If you’re decrypting inside the cluster, get clarity on two risk classes:
- Where does the key live? (in the controller pod, in the node secret store?)
- Which namespace/service can reach which secret?
The balance I aim for in production:
- Instead of “one key opens everything”, scoped keys
- For the component holding the key:
- node affinity / toleration (critical nodes)
- resource limits
- audit logging
6) Rotation: not “we’ll need it someday” — plan it today
Without a rotation plan, secret management isn’t sustainable.
Practical rotation steps:
- Generate a new age key
- Add the new key to
.sops.yaml(two keys during the transition) - Re-encrypt existing files:
sops -r -i app-secrets.enc.yaml
- Update the key source in CI / the cluster
- Remove the old key and revoke its access
7) Runbook: “decryption isn’t working”
The most common causes:
SOPS_AGE_KEY_FILEpointing at the wrong path- A file encrypted with the wrong public key
- CI secret store updated but the runner is cached
- SOPS metadata corrupted by a merge conflict in the file
Checklist:
- Does
sops -d file.enc.yamlwork locally (in a safe environment)? - Is the metadata healthy via
sops --status file.enc.yaml? - In CI, is
age.keyactually arriving with the right contents? (verify via hash, never log the contents)
Closing
SOPS + age moves GitOps secret handling out of the realm of “rules and exceptions” into a repeatable, auditable process. The real value isn’t the encryption itself; it’s the standardization, rotation, and fast diagnosis during incidents that comes with it.