NOTES INDEX This page is part of the collection of "notes" - these can be considered to be micro-blogs.
Kubernetes Isolate Crashed Pod

Kubernetes Isolate Crashed Pod

See also full blog post: Kubernetes: Isolate Crashed Pod

If you are running your pod (services / containers) on a Kubernetes cluster, and the pod runs into problems, you can decide to take it out of the load, but keep the pod running for further investigations.

To park or isolate a pod:

  • Find your pod name.
  • Find the pod selector labels in use.
  • Update one of the selector labels.
  • Debug.
  • Delete the broken pod.

# find exact pod name:
kubectl --context=YOURCONTEXT get pods -n NAMESPACE \
  | grep -e NAME -e PARTIAL-POD-NAME

# find labels:
kubectl --context=YOURCONTEXT get svc -n NAMESPACE SERVICE-NAME -o wide

# update label:
kubectl --context=YOURCONTEXT label -n NAMESPACE \
  --overwrite pods FULL-POD-NAME somelabel=crashed-OLD-LABEL-VALUE

# do some debugging on the broken pod...

# and when all done, delete the broken pod:
kubectl --context=YOURCONTEXT delete -n NAMESPACE pod FULL-POD-NAME
Code language: Bash (bash)

In summary; by changing one of the labels on your pod, the Kubernetes service will no longer route traffic to that pod, and it will start a fresh instance to take it’s place to help the end users.

For more info, see the full blog post as mentioned at the top of this page.

The one thing to remember:

kubectl help labelCode language: Bash (bash)
January 20, 2022

Leave a Reply

Your email address will not be published. Required fields are marked *