Kubernetes Production Readiness Checklist

Most Kubernetes incidents are not caused by Kubernetes. They are caused by workloads that were deployed with defaults that are fine on a laptop and dangerous in a cluster. This checklist is what we walk through during a Kubernetes review.

Workload configuration

• Resource requests set on every container; limits set on memory, and used deliberately on CPU.
• Liveness, readiness and startup probes distinguished — readiness gates traffic, liveness restarts, startup protects slow boots.
• Graceful shutdown: SIGTERM handled, terminationGracePeriodSeconds longer than the longest in-flight request.
• PodDisruptionBudget for every workload with more than one replica.
• Anti-affinity or topology spread constraints so replicas do not share a node or zone.
• No :latest tags; images pinned by digest or immutable tag.

readinessProbe:
  httpGet: { path: /readyz, port: 8080 }
  periodSeconds: 5
  failureThreshold: 3
lifecycle:
  preStop:
    exec: { command: ["sleep", "5"] }
terminationGracePeriodSeconds: 45

Cluster hardening

• RBAC scoped per namespace; no cluster-admin bound to application service accounts.
• Workload identity (IRSA on EKS, workload identity on AKS) instead of static cloud credentials in secrets.
• Default-deny NetworkPolicy per namespace, with explicit allows.
• Pod Security Admission set to restricted where the workload permits; exceptions documented.
• Admission policy enforcing required labels, resource requests and image provenance.
• Secrets from an external manager, encrypted at rest, never committed to the manifest repo.

Scaling and capacity

• HPA driven by a metric that correlates with user-visible load, not just CPU.
• Cluster autoscaling (Cluster Autoscaler or Karpenter) with tested scale-up latency.
• Known capacity ceiling: you have load-tested to the point where something breaks and know what breaks first.
• Separate node pools for system, general and burst or GPU workloads.

Observability

• Golden signals per service: latency, traffic, errors, saturation.
• Cluster-level alerts: node pressure, pending pods, failing controllers, certificate expiry.
• Traces propagated across service boundaries with OpenTelemetry.
• Logs structured and correlated with trace IDs.
• Alerts tied to SLOs; anything that pages must be actionable.

Delivery and upgrades

• GitOps: cluster state reconciled from a reviewed repository, not applied from laptops.
• Rollback tested, not assumed.
• Deprecated API inventory checked before each control-plane upgrade.
• Node image and control plane upgraded on a schedule, rehearsed in a non-production cluster.
• etcd or managed control-plane backup, and a restore that has actually been performed.

The uncomfortable question

Before working through the list, ask whether you need a cluster at all. If you run a handful of stateless services with modest scaling requirements, a managed container runtime removes most of this checklist from your responsibility. Kubernetes earns its operational cost when you have many services, real multi-tenancy, or workloads with scheduling requirements — GPUs, batch, mixed profiles — that simpler platforms handle badly.