PDB issue while draining the nodes

Question

What is the problem with this config.

Whenever I run kubectl drain this error is coming: "Cannot evict pod as it would violate the pod's disruption budget". I can clearly see that 3 pods are created on 3 different nodes. Now while draining one node this error is coming

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment2
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 34%
      maxUnavailable: 34%
  selector:
    matchLabels:
      app: nginx2
  template:
    metadata:
      labels:
        app: nginx2
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: dedicated
                operator: In
                values:
                - mynodes
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app.kubernetes.io/instance
                  operator: In
                  values:
                  - mytestapps
              topologyKey: kubernetes.io/hostname
            weight: 100
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: pdb-nginx2
spec:
  maxUnavailable: 1
  selector:
    matchLabels:
      app: nginx2

Answer

Hi Tanul,

Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

The error "Cannot evict pod as it would violate the pod's disruption budget" happens because your PodDisruptionBudget (PDB) only allows 1 pod to be unavailable at a time (maxUnavailable: 1). Since you have 3 replicas, Kubernetes is preventing the eviction to avoid violating this limit.

To fix this:

Increase maxUnavailable in the PDB:

You can update the PDB to allow more pods to be unavailable, for example:

maxUnavailable: 2

Temporarily Remove the PDB:

If you need to drain the node quickly, you can delete the PDB, drain the node, and then reapply it:

kubectl delete poddisruptionbudget pdb-nginx2
kubectl drain 
kubectl apply -f .yaml

For further details, you can review these documents: PodDisruptionBudget documentation, Troubleshoot the "UnsatisfiablePDB" error during an AKS cluster upgrade.

If you have any further queries, do let us know. If the comment is helpful, please click "Upvote".

Share via

PDB issue while draining the nodes

1 answer

Your answer