Tuesday, 9 June 2020

MountVolume.SetUp failed for volume "default-token-t7d5k" : failed to sync secret cache: timed out waiting for the condition

If you get the error "MountVolume.SetUp failed for volume "default-token-t7d5k" : failed to sync secret cache: timed out waiting for the condition":

[root@unicamaster software]# kubectl describe pod unica-platform-b5688dcb5-4wptl
Name:         unica-platform-b5688dcb5-4wptl
Namespace:    default
Priority:     0
Node:         unicanode1/10.0.2.5
Start Time:   Wed, 10 Jun 2020 07:16:42 +0530
Labels:       app.kubernetes.io/instance=unica
              app.kubernetes.io/name=unica-platform
              pod-template-hash=b5688dcb5
Annotations:  <none>
Status:       Pending
IP:           10.44.0.4
IPs:
  IP:           10.44.0.4
Controlled By:  ReplicaSet/unica-platform-b5688dcb5
Init Containers:
  install:
    Container ID:  docker://aff6ed9523034dfc36232aba8a5e9f8dacbfd1fa6ce4f697a1b23d57dfd31c18
    Image:         10.0.2.6:5000/unica-install:68
    Image ID:      docker-pullable://10.0.2.6:5000/unica-install@sha256:db069e6cbcd3370bbaa0e7bd80a29a83ab82e70b03a2537572ef27a6e4046233
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
      -c
    Args:
      chmod 755 /docker/unica && ./entrypoint.sh
    State:          Running
      Started:      Wed, 10 Jun 2020 07:17:50 +0530
    Ready:          False
    Restart Count:  0
    Environment Variables from:
      unica-common-env    ConfigMap  Optional: false
      unica-platform-env  ConfigMap  Optional: false
      unica-campaign-env  ConfigMap  Optional: false
      unica-interact-env  ConfigMap  Optional: false
    Environment:          <none>
    Mounts:
      /docker/unica from volume-mount (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Containers:
  unica-platform:
    Container ID:  
    Image:          10.0.2.6:5000/tomcat-unica-platform:12
    Image ID:      
    Port:           9123/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     2
      memory:  1600Mi
    Requests:
      cpu:      1500m
      memory:   1300Mi
    Readiness:  exec [/bin/bash -ce ] delay=20s timeout=30s period=40s #success=1 #failure=3
    Environment Variables from:
      unica-common-env    ConfigMap  Optional: false
      unica-platform-env  ConfigMap  Optional: false
      unica-campaign-env  ConfigMap  Optional: false
      unica-interact-env  ConfigMap  Optional: false
    Environment:
      CONTAINER_NAME:  unica-platform
      WLP_HTTP_PORT:   9123
      WLP_HTTPS_PORT:  9445
    Mounts:
      /docker/unica from volume-mount (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Conditions:
  Type              Status
  Initialized       False
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  volume-mount:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  unica
    ReadOnly:   false
  default-token-t7d5k:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-t7d5k
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason       Age    From                 Message
  ----     ------       ----   ----                 -------
  Normal   Scheduled    5m54s  default-scheduler    Successfully assigned default/unica-platform-b5688dcb5-4wptl to unicanode1
  Warning  FailedMount  5m52s  kubelet, unicanode1  MountVolume.SetUp failed for volume "default-token-t7d5k" : failed to sync secret cache: timed out waiting for the condition
  Normal   Pulling      5m51s  kubelet, unicanode1  Pulling image "10.0.2.6:5000/unica-install:68"
  Normal   Pulled       4m47s  kubelet, unicanode1  Successfully pulled image "10.0.2.6:5000/unica-install:68"
  Normal   Created      4m46s  kubelet, unicanode1  Created container install
  Normal   Started      4m46s  kubelet, unicanode1  Started container install
[root@unicamaster software]# kubectl describe pod unica-platform-b5688dcb5-4wptl
Name:         unica-platform-b5688dcb5-4wptl
Namespace:    default
Priority:     0
Node:         unicanode1/10.0.2.5
Start Time:   Wed, 10 Jun 2020 07:16:42 +0530
Labels:       app.kubernetes.io/instance=unica
              app.kubernetes.io/name=unica-platform
              pod-template-hash=b5688dcb5
Annotations:  <none>
Status:       Pending
IP:           10.44.0.4
IPs:
  IP:           10.44.0.4
Controlled By:  ReplicaSet/unica-platform-b5688dcb5
Init Containers:
  install:
    Container ID:  docker://aff6ed9523034dfc36232aba8a5e9f8dacbfd1fa6ce4f697a1b23d57dfd31c18
    Image:         10.0.2.6:5000/unica-install:68
    Image ID:      docker-pullable://10.0.2.6:5000/unica-install@sha256:db069e6cbcd3370bbaa0e7bd80a29a83ab82e70b03a2537572ef27a6e4046233
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
      -c
    Args:
      chmod 755 /docker/unica && ./entrypoint.sh
    State:          Running
      Started:      Wed, 10 Jun 2020 07:17:50 +0530
    Ready:          False
    Restart Count:  0
    Environment Variables from:
      unica-common-env    ConfigMap  Optional: false
      unica-platform-env  ConfigMap  Optional: false
      unica-campaign-env  ConfigMap  Optional: false
      unica-interact-env  ConfigMap  Optional: false
    Environment:          <none>
    Mounts:
      /docker/unica from volume-mount (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Containers:
  unica-platform:
    Container ID:  
    Image:          10.0.2.6:5000/tomcat-unica-platform:12
    Image ID:      
    Port:           9123/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     2
      memory:  1600Mi
    Requests:
      cpu:      1500m
      memory:   1300Mi
    Readiness:  exec [/bin/bash -ce ] delay=20s timeout=30s period=40s #success=1 #failure=3
    Environment Variables from:
      unica-common-env    ConfigMap  Optional: false
      unica-platform-env  ConfigMap  Optional: false
      unica-campaign-env  ConfigMap  Optional: false
      unica-interact-env  ConfigMap  Optional: false
    Environment:
      CONTAINER_NAME:  unica-platform
      WLP_HTTP_PORT:   9123
      WLP_HTTPS_PORT:  9445
    Mounts:
      /docker/unica from volume-mount (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Conditions:
  Type              Status
  Initialized       False
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  volume-mount:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  unica
    ReadOnly:   false
  default-token-t7d5k:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-t7d5k
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason       Age    From                 Message
  ----     ------       ----   ----                 -------
  Normal   Scheduled    7m28s  default-scheduler    Successfully assigned default/unica-platform-b5688dcb5-4wptl to unicanode1
  Warning  FailedMount  7m26s  kubelet, unicanode1  MountVolume.SetUp failed for volume "default-token-t7d5k" : failed to sync secret cache: timed out waiting for the condition
  Normal   Pulling      7m25s  kubelet, unicanode1  Pulling image "10.0.2.6:5000/unica-install:68"
  Normal   Pulled       6m21s  kubelet, unicanode1  Successfully pulled image "10.0.2.6:5000/unica-install:68"
  Normal   Created      6m20s  kubelet, unicanode1  Created container install
  Normal   Started      6m20s  kubelet, unicanode1  Started container install





Check the node:
[root@unicanode1 docker]# kubectl get secrets
error: stat /root/.kube/config: not a directory

Means setup was incomplete, need to do the following:
[root@unicanode1 ~]# mkdir -p $HOME/.kube

On master:
[root@unicamaster .kube]# scp -r config unicanode1:/root/.kube
root@unicanode1's password:
config

On node:
[root@unicanode1 .kube]# ls -la
total 8
drwxr-xr-x  2 root root   20 Jun 10 07:35 .
dr-xr-x---. 6 root root  251 Jun 10 07:33 ..
-rw-------  1 root root 5444 Jun 10 07:35 config
[root@unicanode1 .kube]# vi config
[root@unicanode1 .kube]# kubectl get secrets
NAME                                      TYPE                                  DATA   AGE
chart-unica-tls                           Opaque                                2      18m
default-token-t7d5k                       kubernetes.io/service-account-token   3      2d23h
nginx-nginx-ingress-backend-token-jmnzw   kubernetes.io/service-account-token   3      17h
nginx-nginx-ingress-token-9q5b5           kubernetes.io/service-account-token   3      17h

Should be fine now..


No comments:

Post a Comment

PVC in terminating state

Referencee:https://veducate.co.uk/kubernetes-pvc-terminating/  I still had errors performing deployment, till I released that although I pre...