Kubernetes Workloads: Pods, Deployments, StatefulSets & Jobs

Q: Why does my Pod keep restarting?

Check kubectl describe pod and kubectl logs --previous. The usual causes are an app crash on startup, an OOM kill from a memory limit set too low, or a liveness probe that fails before the app is ready. Add a startup probe or raise initialDelaySeconds for slow boots.

Kubernetes Workloads

A workload is an application running on Kubernetes. You almost never create bare containers or even bare Pods; instead you declare a controller — a Deployment, StatefulSet, DaemonSet, Job, or CronJob — and Kubernetes continuously reconciles the cluster so the real state matches what you declared. Pick the wrong controller and you fight the platform: stateful databases that lose their identity on restart, batch jobs that run forever, or log agents that skip half your nodes.

This page covers what each workload resource guarantees, the Pod-level settings (probes, resources, lifecycle) that every workload shares, and how rollouts actually work.

TL;DR

A Pod is the smallest schedulable unit: one or more containers sharing a network namespace and volumes. Pods are disposable — never manage them directly.
Deployment → stateless, interchangeable replicas with rolling updates. The default for web services and APIs.
StatefulSet → stable names (db-0, db-1), ordered startup, and a persistent volume per replica. For databases and clustered systems.
DaemonSet → exactly one Pod per node (or per matching node). For log shippers, node exporters, CNI agents.
Job / CronJob → run to completion, once or on a schedule. For migrations, batch processing, and reports.
Every workload needs resource requests/limits and readiness/liveness probes to behave well in production.

Quick Example

A production-shaped Deployment for a stateless API:

kubectl apply -f api.yaml creates a ReplicaSet that keeps three Pods running. Changing the image tag triggers a rolling update that never drops below three ready replicas.

Core Concepts

Pods

A Pod wraps one or more containers that are scheduled together on the same node, share an IP address and localhost, and can mount the same volumes. The common multi-container patterns are sidecars (a proxy or log forwarder next to the app) and init containers (run to completion before the app starts, e.g. waiting for a dependency or running a schema check).

Pods are ephemeral. When a node dies, its Pods are gone; a controller creates replacements elsewhere with new names and IPs. That is why you address Pods through a Service, not directly.

The Workload Controllers

A Deployment manages ReplicaSets, which manage Pods. Each rollout creates a new ReplicaSet and scales it up while scaling the old one down; the old ReplicaSets are kept (up to revisionHistoryLimit) so kubectl rollout undo can revert instantly.

A StatefulSet gives each replica a sticky identity: postgres-0 always gets the same persistent volume and the same DNS name through a headless Service, even after rescheduling. Pods start and stop in order by default, which matters for leader election and quorum-based systems.

Probes

Kubernetes uses three probes to decide what to do with a container:

Readiness — "can this Pod receive traffic?" A failing readiness probe removes the Pod from Service endpoints but does not restart it.
Liveness — "is this process stuck?" A failing liveness probe restarts the container.
Startup — "has the app finished booting?" Suspends the other probes until it passes, for slow-starting apps.

Resource Requests and Limits

Requests are what the scheduler reserves when placing the Pod; limits are the ceiling enforced at runtime. Exceeding a memory limit gets the container OOM-killed; exceeding a CPU limit gets it throttled. Requests and limits together determine the Pod's QoS class (Guaranteed, Burstable, BestEffort), which decides eviction order under node pressure.

Rollout Strategies

Deployments support two built-in strategies:

RollingUpdate (default) — replace Pods gradually. maxSurge controls how many extra Pods may exist during the rollout; maxUnavailable controls how many may be missing. maxUnavailable: 0 with maxSurge: 1 is the safest zero-downtime setting.
Recreate — kill all old Pods, then start new ones. Only for apps that can't run two versions at once.

Blue-green and canary releases aren't native Deployment strategies; they're built from multiple Deployments plus Service or ingress routing, or with tools like Argo Rollouts and Flagger. See deployment strategies.

Best Practices

Always Set Requests (and Memory Limits)

Without requests, the scheduler packs Pods blindly and noisy neighbors starve each other. Set CPU and memory requests from observed usage, and set a memory limit to contain leaks. Many teams deliberately omit CPU limits because throttling hurts latency more than it helps; that's a reasonable default when requests are accurate.

Make Readiness and Liveness Different

Readiness should check whether the app can serve (dependencies reachable, caches warm). Liveness should check only whether the process itself is healthy. If liveness checks the database, a database blip restarts every Pod at once and turns a small outage into a large one.

Handle SIGTERM Gracefully

On shutdown Kubernetes sends SIGTERM, waits terminationGracePeriodSeconds (default 30), then sends SIGKILL. Your app should stop accepting new work, finish in-flight requests, and exit. A short preStop sleep gives load balancers time to stop routing before the process exits.

Use PodDisruptionBudgets

A PDB (minAvailable: 2) stops voluntary disruptions like node drains during upgrades from taking down too many replicas at once. Without one, a cluster upgrade can evict every replica of a service simultaneously.

Spread Replicas

Use topologySpreadConstraints or pod anti-affinity so three replicas don't land on one node or one availability zone.

Common Mistakes

Running a Database as a Deployment

Better still for many teams: use a managed database or a mature operator rather than hand-rolling one.

Using the `latest` Tag

image: api:latest makes rollouts unreproducible and kubectl rollout undo meaningless, because the "old" and "new" specs are identical. Pin immutable tags or digests.

Jobs Without Limits

A Job with a crashing container retries up to backoffLimit (default 6), and one that hangs runs forever. Set backoffLimit, activeDeadlineSeconds, and ttlSecondsAfterFinished so completed Jobs are cleaned up. For CronJobs, set concurrencyPolicy: Forbid if runs must not overlap.

FAQ

What's the difference between a Pod and a container?

A container is a single running image. A Pod is the Kubernetes wrapper around one or more containers that share networking and storage and are always scheduled together. Most Pods hold one main container, sometimes with sidecars or init containers.

Should I use a Deployment or a StatefulSet?

Use a Deployment unless replicas need a stable identity or their own persistent storage. If any replica can handle any request and state lives elsewhere (a database, object storage, a cache), it's a Deployment. If replicas are distinguishable, like a primary and replicas or Kafka brokers with their own logs, use a StatefulSet.

How do I run a database migration before a rollout?

Common options: a Job run by your CD pipeline before updating the Deployment, a Helm pre-upgrade hook, or an init container. The Job approach is the clearest because it runs once and its success gates the rollout. Keep migrations backward-compatible so old and new Pods can run side by side during the rolling update.

Why does my Pod keep restarting?

Check kubectl describe pod and kubectl logs --previous. The usual causes are an app crash on startup, an OOM kill from a memory limit set too low, or a liveness probe that fails before the app is ready. Add a startup probe or raise initialDelaySeconds for slow boots.