# Fix Kubernetes CrashLoopBackOff: Causes and Solutions

- URL: https://karandeepsingh.ca/posts/fix-kubernetes-crashloopbackoff/
- Author: Karandeep Singh
- Published: 2026-06-18
- Series: Kubernetes and Containers
- Tags: kubernetes, containers, devops
- Summary: A practical playbook for Kubernetes CrashLoopBackOff — the diagnosis workflow (describe, previous logs, events) plus fixes for the six causes you will actually hit, including OOMKilled, failing liveness probes, and missing config.

`CrashLoopBackOff` is the error every Kubernetes user meets sooner or later. A pod starts, the container dies, Kubernetes restarts it, it dies again — and the kubelet backs off, waiting longer between each restart (10s, 20s, 40s, up to 5 minutes). The status itself is not the problem; it is a *symptom*. This guide gives you a repeatable workflow to find the real cause of Kubernetes CrashLoopBackOff and fix it.

The mistake most people make is staring at the CrashLoopBackOff status and guessing. Don't guess. The cluster already recorded exactly why the container died — you just need the right three commands to read it.

![Fix Kubernetes CrashLoopBackOff — diagnosis workflow and common causes](https://karandeepsingh.ca/images/posts/fix-kubernetes-crashloopbackoff-card.jpg)

## The CrashLoopBackOff Diagnosis Workflow

Whatever the underlying cause, you start the same way every time. These three commands resolve the vast majority of cases (the full list is in the [kubectl cheat sheet](https://karandeepsingh.ca/posts/kubectl-cheat-sheet/)):

```bash
kubectl get pods                       # confirm the status and restart count
kubectl describe pod <pod-name>        # read Events and 'Last State' (reason + exit code)
kubectl logs <pod-name> --previous     # the crashed container's actual output
```

```mermaid
graph TD
  A[Pod in CrashLoopBackOff] --> B[kubectl describe pod]
  B --> C{Last State reason?}
  C -->|OOMKilled / exit 137| D[Raise memory limit / fix leak]
  C -->|Error / exit 1| E[kubectl logs --previous]
  C -->|Liveness probe failed| F[Fix probe path, port, delay]
  C -->|ImagePullBackOff first| G[Fix image name / registry auth]
  E --> H{What does the log say?}
  H -->|Stack trace / panic| I[Fix app bug]
  H -->|Missing env / cannot connect| J[Fix ConfigMap / Secret / dependency]
```

**info:**
`kubectl logs --previous` is the single most important command here. While a pod is crash-looping, the *current* container has barely started, so `kubectl logs` is often empty. The `--previous` flag shows the container that actually died — that is where the stack trace lives.

## Cause 1: The Application Crashes on Startup

The most common cause is the simplest: your code throws on boot. An unhandled exception, a panic, a failed migration. The `--previous` logs show it directly:

```bash
kubectl logs <pod-name> --previous
# e.g. panic: runtime error: invalid memory address or nil pointer dereference
```

**Fix:** it is an application bug, not a Kubernetes problem. Reproduce locally with the same image and arguments, fix the code, rebuild, and redeploy.

## Cause 2: Missing or Wrong Configuration

A container that runs fine locally often crashes in-cluster because an environment variable, ConfigMap, or Secret is missing or wrong — think a `DATABASE_URL` that points nowhere. The logs usually say something like `connection refused` or `required env var not set`.

```bash
kubectl describe pod <pod-name> | grep -A10 Environment
kubectl get configmap,secret -n <namespace>
```

**Fix:** confirm the ConfigMap/Secret exists in the same namespace and that the keys match what the app expects. A typo'd key name is a classic culprit.

## Cause 3: A Failing Liveness Probe

This one fools people because the app is healthy. If a liveness probe checks the wrong path or port, or its `initialDelaySeconds` is too short for a slow-starting app, the kubelet decides the container is unhealthy and kills it — forever.

```yaml
livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 20   # give the app time to boot before probing
  periodSeconds: 10
```

**Fix:** verify the probe path and port actually respond, and raise `initialDelaySeconds` for apps that need warm-up time. `kubectl describe pod` will show `Liveness probe failed` in Events when this is the cause.

## Cause 4: OOMKilled (Exit Code 137)

If `kubectl describe pod` shows `Last State: Terminated, Reason: OOMKilled` or exit code **137**, the container exceeded its memory limit and was killed.

```bash
kubectl describe pod <pod-name> | grep -i -A2 "last state"
```

**Fix:** raise the memory limit, or fix the leak that makes the app exceed it. Set realistic requests and limits:

```yaml
resources:
  requests:
    memory: "128Mi"
  limits:
    memory: "256Mi"
```

**warning:**
Setting the memory limit too low is a self-inflicted CrashLoopBackOff. If an app legitimately needs 300Mi and you cap it at 128Mi, it will be OOMKilled on every startup no matter how clean the code is.

## Cause 5: Wrong Command or Entrypoint

Exit code **127** means "command not found" and **126** means "not executable." These point to a bad `command`/`args` override or a binary that isn't where the container expects it.

**Fix:** check the `command` and `args` in your manifest against the image's real entrypoint. Run the image locally with `docker run` to confirm the command works before deploying.

## Cause 6: A Dependency Isn't Ready

Sometimes the app is correct but starts before its database or an upstream API is reachable, crashes, and loops. The fix is to make startup resilient rather than fragile.

- Add retry-with-backoff logic to the app's startup connection.
- Use an **init container** to wait for the dependency before the main container starts.
- Use a **readiness probe** so traffic only arrives once the app is truly ready.

**Quick init-container example that waits for a database**
```yaml
initContainers:
- name: wait-for-db
  image: busybox:1.36
  command: ['sh', '-c', 'until nc -z db 5432; do echo waiting for db; sleep 2; done']
```
The main container will not start until the `db` service accepts connections on port 5432.

## A Reusable Mental Model

CrashLoopBackOff almost always reduces to one of two questions: *did the container die because of something inside it* (app bug, OOM, bad command) *or something around it* (config, probes, dependencies)? `kubectl describe pod` plus `kubectl logs --previous` answers that in under a minute.

If you are still learning the objects involved — pods, probes, and the controllers that restart them — start with [Kubernetes fundamentals: pods, deployments, and services](https://karandeepsingh.ca/posts/kubernetes-fundamentals-pods-deployments-services/).

**Question:** What is the strangest root cause of a CrashLoopBackOff you have had to track down?

## References and Further Reading

- Kubernetes Authors. [Debug Running Pods](https://kubernetes.io/docs/tasks/debug/debug-application/debug-running-pod/). Kubernetes Documentation.
- Kubernetes Authors. [Configure Liveness, Readiness and Startup Probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/). Kubernetes Documentation.
- Kubernetes Authors. [Pod Lifecycle](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/). Kubernetes Documentation.
- Kubernetes Authors. [Assign Memory Resources to Containers](https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/). Kubernetes Documentation.

## Quiz

1. Where do you find the logs of a container that has already crashed and restarted?
   a) kubectl logs POD
   b) kubectl logs POD --previous
   c) kubectl describe POD
   d) kubectl get POD -o yaml
   Answer: b. `kubectl logs POD --previous` prints the logs of the previous, terminated container instance — where the actual crash and stack trace live. Plain `kubectl logs` shows the new container, which has not failed yet.

2. A container is reported as terminated with exit code 137. What usually caused it?
   a) A syntax error in the app
   b) It was OOMKilled (ran out of memory)
   c) The image failed to pull
   d) The liveness probe passed
   Answer: b. Exit code 137 = 128 + 9 (SIGKILL). In Kubernetes it almost always means the container exceeded its memory limit and was OOMKilled. Raise the memory limit or fix the leak.

3. What can cause CrashLoopBackOff even when the application code is perfectly fine?
   a) Too many replicas
   b) A failing liveness probe repeatedly killing the container
   c) A Service with no selector
   d) Running on the wrong namespace
   Answer: b. A misconfigured liveness probe (wrong path, port, or too-short initialDelaySeconds) makes the kubelet kill a healthy container over and over, producing a crash loop that has nothing to do with your code.

4. CrashLoopBackOff is a symptom. What is the right first diagnostic command?
   a) kubectl delete pod and retry
   b) kubectl scale the deployment up
   c) kubectl describe pod (read Events and Last State)
   d) kubectl edit the deployment
   Answer: c. `kubectl describe pod` shows the Events and the container Last State (reason + exit code), which point you to the real cause before you touch anything.