Kubernetes Control Plane Certificate Expiry: A Runbook
When API Server access suddenly breaks with x509 errors; certificate renewal and safe recovery steps for kubeadm-based clusters.
280 posts
Step-by-step guides, practical examples and hands-on tutorials.
When API Server access suddenly breaks with x509 errors; certificate renewal and safe recovery steps for kubeadm-based clusters.
A runbook that turns firmware upgrade work into a repeatable maintenance rhythm with inventory, ring/wave approach, validation metrics, and a rollback…
Reduce risk while moving production firewall rule sets from iptables to nftables using observability, wave-based rollout, and fast rollback.
A practical approach that turns load testing from a peak-RPS race into an SLO-driven (latency/error) capacity baseline and a CI release gate.
Practical steps for building a WORM (Write Once Read Many) layer against ransomware and accidental deletion using S3 Object Lock, retention policies, and…
A practical chrony runbook for enterprise servers covering secure NTP (NTS), access restrictions, verification commands, and alarm thresholds.
A practical SOPS + age setup and operational discipline for keeping encrypted secrets in Git and decrypting them safely inside CI/CD and the cluster.
A practical RBAC framework for role design, identity integration, and time-boxed emergency access (break-glass) without depending on cluster-admin.
Turn 'what's on which server?' into a living inventory; a guide for scaling osquery queries with FleetDM into operational and security signal.
Roll out security guardrails in production clusters gradually with Pod Security Admission (PSA) and Kyverno: an audit→warn→enforce plan.
A TACACS+ approach that reduces local admin sprawl on network devices and turns session traces into proof through roles, command authorization, and accounting.
A runbook for quickly diagnosing ETCD quorum during API 5xx/timeout storms and walking through safe recovery steps via snapshot restore.
Hardening admin access with OpenSSH security keys (ed25519-sk) using PIN + touch confirmation, while keeping break-glass scenarios intact.
Field runbook to rapidly triage hung deploys caused by Validating/Mutating webhook latency and apply a risk-controlled mitigation.
A practical Batfish flow that validates routing/ACL changes before they reach production via 'snapshot + question set,' catching human error early.
A guide to wiring service-to-service mTLS through SPIFFE identities and SPIRE-issued short-lived certificates instead of relying on IPs and static secrets.
An OpenSSH CA-based approach to set up auditable, time-bound SSH access in place of shared bastion accounts and long-lived keys.
Constrain services into a tighter permission set without changing the application itself: filesystem, capability, syscall, and network limits.
A practical APF setup that prioritizes critical traffic and fairly queues noisy callers, lowering the risk of API server overload.
Detect configuration drift, approve fixes through Git, and apply them under control: source of truth → report → PR → rollout.
Roll out node patches in maintenance waves rather than all-at-once: drain, PDB, parallelism, and a safe rollback path.
PSI, systemd-oomd policy, testing, and recovery steps to catch a node OOM crisis early and evict workloads in a controlled way.
An approach to enabling BFD with FRR (BGP/OSPF) to generate fast signals when the link looks up but traffic isn't flowing (blackhole).
A practical approach that makes privileged operations observable and auditable in production using sudo, auditd rules, and log forwarding.