I think this is one of the hardest (maybe I don't know enough). But here is a good start:
https://gravitational.com/blog/troubleshooting-kubernetes-networking/
Basically, curl is a very important tool since most pods have very limited commands you can run.
And here are some very basic services and how it works within pods:
https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies
For discovering services, check out this DNS docs:
https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
Wednesday, 10 June 2020
Deleting images in docker
docker images
docker rmi -f 1ee7db3527cf
docker images
docker rmi -f 2006138de05d
docker rmi -f 743901726c7f
docker rmi -f 093aaee064e4
docker rmi -f 1ee7db3527cf
docker images
docker rmi -f 2006138de05d
docker rmi -f 743901726c7f
docker rmi -f 093aaee064e4
Tuesday, 9 June 2020
About /etc/docker/daemon.json and /etc/default/docker
It needs to be on nodes as well.
/etc/docker/daemon.json
{
"insecure-registries": ["10.0.2.6:5000"]
}
/etc/default/docker
DOCKER_OPTS="--insecure-registry 10.0.2.6:5000"
/etc/docker/daemon.json
{
"insecure-registries": ["10.0.2.6:5000"]
}
/etc/default/docker
DOCKER_OPTS="--insecure-registry 10.0.2.6:5000"
MountVolume.SetUp failed for volume "default-token-t7d5k" : failed to sync secret cache: timed out waiting for the condition
If you get the error "MountVolume.SetUp failed for volume "default-token-t7d5k" : failed to sync secret cache: timed out waiting for the condition":
[root@unicamaster software]# kubectl describe pod unica-platform-b5688dcb5-4wptl
Name: unica-platform-b5688dcb5-4wptl
Namespace: default
Priority: 0
Node: unicanode1/10.0.2.5
Start Time: Wed, 10 Jun 2020 07:16:42 +0530
Labels: app.kubernetes.io/instance=unica
app.kubernetes.io/name=unica-platform
pod-template-hash=b5688dcb5
Annotations: <none>
Status: Pending
IP: 10.44.0.4
IPs:
IP: 10.44.0.4
Controlled By: ReplicaSet/unica-platform-b5688dcb5
Init Containers:
install:
Container ID: docker://aff6ed9523034dfc36232aba8a5e9f8dacbfd1fa6ce4f697a1b23d57dfd31c18
Image: 10.0.2.6:5000/unica-install:68
Image ID: docker-pullable://10.0.2.6:5000/unica-install@sha256:db069e6cbcd3370bbaa0e7bd80a29a83ab82e70b03a2537572ef27a6e4046233
Port: <none>
Host Port: <none>
Command:
/bin/sh
-c
Args:
chmod 755 /docker/unica && ./entrypoint.sh
State: Running
Started: Wed, 10 Jun 2020 07:17:50 +0530
Ready: False
Restart Count: 0
Environment Variables from:
unica-common-env ConfigMap Optional: false
unica-platform-env ConfigMap Optional: false
unica-campaign-env ConfigMap Optional: false
unica-interact-env ConfigMap Optional: false
Environment: <none>
Mounts:
/docker/unica from volume-mount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Containers:
unica-platform:
Container ID:
Image: 10.0.2.6:5000/tomcat-unica-platform:12
Image ID:
Port: 9123/TCP
Host Port: 0/TCP
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 2
memory: 1600Mi
Requests:
cpu: 1500m
memory: 1300Mi
Readiness: exec [/bin/bash -ce ] delay=20s timeout=30s period=40s #success=1 #failure=3
Environment Variables from:
unica-common-env ConfigMap Optional: false
unica-platform-env ConfigMap Optional: false
unica-campaign-env ConfigMap Optional: false
unica-interact-env ConfigMap Optional: false
Environment:
CONTAINER_NAME: unica-platform
WLP_HTTP_PORT: 9123
WLP_HTTPS_PORT: 9445
Mounts:
/docker/unica from volume-mount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
volume-mount:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: unica
ReadOnly: false
default-token-t7d5k:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-t7d5k
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 5m54s default-scheduler Successfully assigned default/unica-platform-b5688dcb5-4wptl to unicanode1
Warning FailedMount 5m52s kubelet, unicanode1 MountVolume.SetUp failed for volume "default-token-t7d5k" : failed to sync secret cache: timed out waiting for the condition
Normal Pulling 5m51s kubelet, unicanode1 Pulling image "10.0.2.6:5000/unica-install:68"
Normal Pulled 4m47s kubelet, unicanode1 Successfully pulled image "10.0.2.6:5000/unica-install:68"
Normal Created 4m46s kubelet, unicanode1 Created container install
Normal Started 4m46s kubelet, unicanode1 Started container install
[root@unicamaster software]# kubectl describe pod unica-platform-b5688dcb5-4wptl
Name: unica-platform-b5688dcb5-4wptl
Namespace: default
Priority: 0
Node: unicanode1/10.0.2.5
Start Time: Wed, 10 Jun 2020 07:16:42 +0530
Labels: app.kubernetes.io/instance=unica
app.kubernetes.io/name=unica-platform
pod-template-hash=b5688dcb5
Annotations: <none>
Status: Pending
IP: 10.44.0.4
IPs:
IP: 10.44.0.4
Controlled By: ReplicaSet/unica-platform-b5688dcb5
Init Containers:
install:
Container ID: docker://aff6ed9523034dfc36232aba8a5e9f8dacbfd1fa6ce4f697a1b23d57dfd31c18
Image: 10.0.2.6:5000/unica-install:68
Image ID: docker-pullable://10.0.2.6:5000/unica-install@sha256:db069e6cbcd3370bbaa0e7bd80a29a83ab82e70b03a2537572ef27a6e4046233
Port: <none>
Host Port: <none>
Command:
/bin/sh
-c
Args:
chmod 755 /docker/unica && ./entrypoint.sh
State: Running
Started: Wed, 10 Jun 2020 07:17:50 +0530
Ready: False
Restart Count: 0
Environment Variables from:
unica-common-env ConfigMap Optional: false
unica-platform-env ConfigMap Optional: false
unica-campaign-env ConfigMap Optional: false
unica-interact-env ConfigMap Optional: false
Environment: <none>
Mounts:
/docker/unica from volume-mount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Containers:
unica-platform:
Container ID:
Image: 10.0.2.6:5000/tomcat-unica-platform:12
Image ID:
Port: 9123/TCP
Host Port: 0/TCP
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 2
memory: 1600Mi
Requests:
cpu: 1500m
memory: 1300Mi
Readiness: exec [/bin/bash -ce ] delay=20s timeout=30s period=40s #success=1 #failure=3
Environment Variables from:
unica-common-env ConfigMap Optional: false
unica-platform-env ConfigMap Optional: false
unica-campaign-env ConfigMap Optional: false
unica-interact-env ConfigMap Optional: false
Environment:
CONTAINER_NAME: unica-platform
WLP_HTTP_PORT: 9123
WLP_HTTPS_PORT: 9445
Mounts:
/docker/unica from volume-mount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
volume-mount:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: unica
ReadOnly: false
default-token-t7d5k:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-t7d5k
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 7m28s default-scheduler Successfully assigned default/unica-platform-b5688dcb5-4wptl to unicanode1
Warning FailedMount 7m26s kubelet, unicanode1 MountVolume.SetUp failed for volume "default-token-t7d5k" : failed to sync secret cache: timed out waiting for the condition
Normal Pulling 7m25s kubelet, unicanode1 Pulling image "10.0.2.6:5000/unica-install:68"
Normal Pulled 6m21s kubelet, unicanode1 Successfully pulled image "10.0.2.6:5000/unica-install:68"
Normal Created 6m20s kubelet, unicanode1 Created container install
Normal Started 6m20s kubelet, unicanode1 Started container install
Check the node:
[root@unicanode1 docker]# kubectl get secrets
error: stat /root/.kube/config: not a directory
Means setup was incomplete, need to do the following:
[root@unicanode1 ~]# mkdir -p $HOME/.kube
On master:
[root@unicamaster .kube]# scp -r config unicanode1:/root/.kube
root@unicanode1's password:
config
On node:
[root@unicanode1 .kube]# ls -la
total 8
drwxr-xr-x 2 root root 20 Jun 10 07:35 .
dr-xr-x---. 6 root root 251 Jun 10 07:33 ..
-rw------- 1 root root 5444 Jun 10 07:35 config
[root@unicanode1 .kube]# vi config
[root@unicanode1 .kube]# kubectl get secrets
NAME TYPE DATA AGE
chart-unica-tls Opaque 2 18m
default-token-t7d5k kubernetes.io/service-account-token 3 2d23h
nginx-nginx-ingress-backend-token-jmnzw kubernetes.io/service-account-token 3 17h
nginx-nginx-ingress-token-9q5b5 kubernetes.io/service-account-token 3 17h
Should be fine now..
[root@unicamaster software]# kubectl describe pod unica-platform-b5688dcb5-4wptl
Name: unica-platform-b5688dcb5-4wptl
Namespace: default
Priority: 0
Node: unicanode1/10.0.2.5
Start Time: Wed, 10 Jun 2020 07:16:42 +0530
Labels: app.kubernetes.io/instance=unica
app.kubernetes.io/name=unica-platform
pod-template-hash=b5688dcb5
Annotations: <none>
Status: Pending
IP: 10.44.0.4
IPs:
IP: 10.44.0.4
Controlled By: ReplicaSet/unica-platform-b5688dcb5
Init Containers:
install:
Container ID: docker://aff6ed9523034dfc36232aba8a5e9f8dacbfd1fa6ce4f697a1b23d57dfd31c18
Image: 10.0.2.6:5000/unica-install:68
Image ID: docker-pullable://10.0.2.6:5000/unica-install@sha256:db069e6cbcd3370bbaa0e7bd80a29a83ab82e70b03a2537572ef27a6e4046233
Port: <none>
Host Port: <none>
Command:
/bin/sh
-c
Args:
chmod 755 /docker/unica && ./entrypoint.sh
State: Running
Started: Wed, 10 Jun 2020 07:17:50 +0530
Ready: False
Restart Count: 0
Environment Variables from:
unica-common-env ConfigMap Optional: false
unica-platform-env ConfigMap Optional: false
unica-campaign-env ConfigMap Optional: false
unica-interact-env ConfigMap Optional: false
Environment: <none>
Mounts:
/docker/unica from volume-mount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Containers:
unica-platform:
Container ID:
Image: 10.0.2.6:5000/tomcat-unica-platform:12
Image ID:
Port: 9123/TCP
Host Port: 0/TCP
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 2
memory: 1600Mi
Requests:
cpu: 1500m
memory: 1300Mi
Readiness: exec [/bin/bash -ce ] delay=20s timeout=30s period=40s #success=1 #failure=3
Environment Variables from:
unica-common-env ConfigMap Optional: false
unica-platform-env ConfigMap Optional: false
unica-campaign-env ConfigMap Optional: false
unica-interact-env ConfigMap Optional: false
Environment:
CONTAINER_NAME: unica-platform
WLP_HTTP_PORT: 9123
WLP_HTTPS_PORT: 9445
Mounts:
/docker/unica from volume-mount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
volume-mount:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: unica
ReadOnly: false
default-token-t7d5k:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-t7d5k
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 5m54s default-scheduler Successfully assigned default/unica-platform-b5688dcb5-4wptl to unicanode1
Warning FailedMount 5m52s kubelet, unicanode1 MountVolume.SetUp failed for volume "default-token-t7d5k" : failed to sync secret cache: timed out waiting for the condition
Normal Pulling 5m51s kubelet, unicanode1 Pulling image "10.0.2.6:5000/unica-install:68"
Normal Pulled 4m47s kubelet, unicanode1 Successfully pulled image "10.0.2.6:5000/unica-install:68"
Normal Created 4m46s kubelet, unicanode1 Created container install
Normal Started 4m46s kubelet, unicanode1 Started container install
[root@unicamaster software]# kubectl describe pod unica-platform-b5688dcb5-4wptl
Name: unica-platform-b5688dcb5-4wptl
Namespace: default
Priority: 0
Node: unicanode1/10.0.2.5
Start Time: Wed, 10 Jun 2020 07:16:42 +0530
Labels: app.kubernetes.io/instance=unica
app.kubernetes.io/name=unica-platform
pod-template-hash=b5688dcb5
Annotations: <none>
Status: Pending
IP: 10.44.0.4
IPs:
IP: 10.44.0.4
Controlled By: ReplicaSet/unica-platform-b5688dcb5
Init Containers:
install:
Container ID: docker://aff6ed9523034dfc36232aba8a5e9f8dacbfd1fa6ce4f697a1b23d57dfd31c18
Image: 10.0.2.6:5000/unica-install:68
Image ID: docker-pullable://10.0.2.6:5000/unica-install@sha256:db069e6cbcd3370bbaa0e7bd80a29a83ab82e70b03a2537572ef27a6e4046233
Port: <none>
Host Port: <none>
Command:
/bin/sh
-c
Args:
chmod 755 /docker/unica && ./entrypoint.sh
State: Running
Started: Wed, 10 Jun 2020 07:17:50 +0530
Ready: False
Restart Count: 0
Environment Variables from:
unica-common-env ConfigMap Optional: false
unica-platform-env ConfigMap Optional: false
unica-campaign-env ConfigMap Optional: false
unica-interact-env ConfigMap Optional: false
Environment: <none>
Mounts:
/docker/unica from volume-mount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Containers:
unica-platform:
Container ID:
Image: 10.0.2.6:5000/tomcat-unica-platform:12
Image ID:
Port: 9123/TCP
Host Port: 0/TCP
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 2
memory: 1600Mi
Requests:
cpu: 1500m
memory: 1300Mi
Readiness: exec [/bin/bash -ce ] delay=20s timeout=30s period=40s #success=1 #failure=3
Environment Variables from:
unica-common-env ConfigMap Optional: false
unica-platform-env ConfigMap Optional: false
unica-campaign-env ConfigMap Optional: false
unica-interact-env ConfigMap Optional: false
Environment:
CONTAINER_NAME: unica-platform
WLP_HTTP_PORT: 9123
WLP_HTTPS_PORT: 9445
Mounts:
/docker/unica from volume-mount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
volume-mount:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: unica
ReadOnly: false
default-token-t7d5k:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-t7d5k
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 7m28s default-scheduler Successfully assigned default/unica-platform-b5688dcb5-4wptl to unicanode1
Warning FailedMount 7m26s kubelet, unicanode1 MountVolume.SetUp failed for volume "default-token-t7d5k" : failed to sync secret cache: timed out waiting for the condition
Normal Pulling 7m25s kubelet, unicanode1 Pulling image "10.0.2.6:5000/unica-install:68"
Normal Pulled 6m21s kubelet, unicanode1 Successfully pulled image "10.0.2.6:5000/unica-install:68"
Normal Created 6m20s kubelet, unicanode1 Created container install
Normal Started 6m20s kubelet, unicanode1 Started container install
Check the node:
[root@unicanode1 docker]# kubectl get secrets
error: stat /root/.kube/config: not a directory
Means setup was incomplete, need to do the following:
[root@unicanode1 ~]# mkdir -p $HOME/.kube
On master:
[root@unicamaster .kube]# scp -r config unicanode1:/root/.kube
root@unicanode1's password:
config
On node:
[root@unicanode1 .kube]# ls -la
total 8
drwxr-xr-x 2 root root 20 Jun 10 07:35 .
dr-xr-x---. 6 root root 251 Jun 10 07:33 ..
-rw------- 1 root root 5444 Jun 10 07:35 config
[root@unicanode1 .kube]# vi config
[root@unicanode1 .kube]# kubectl get secrets
NAME TYPE DATA AGE
chart-unica-tls Opaque 2 18m
default-token-t7d5k kubernetes.io/service-account-token 3 2d23h
nginx-nginx-ingress-backend-token-jmnzw kubernetes.io/service-account-token 3 17h
nginx-nginx-ingress-token-9q5b5 kubernetes.io/service-account-token 3 17h
Should be fine now..
Kubelet http server gave http response to https client (ImagePullBackOff status on a Kubernetes pod)
[root@unicamaster docker]# kubectl get pods
NAME READY STATUS RESTARTS AGE
listener-0 0/1 Init:0/1 0 46m
nginx-nginx-ingress-controller-7fc9499f7b-96h99 1/1 Running 0 90m
nginx-nginx-ingress-default-backend-679f548db6-tq2vz 1/1 Running 0 90m
unica-campaign-79b746c5b-jb8rz 0/1 Init:0/1 0 46m
unica-interact-fd5d44b49-xbw52 0/1 Init:0/1 0 46m
unica-platform-b5688dcb5-4bzz4 0/1 Init:ImagePullBackOff 0 46m
[root@unicamaster docker]# helm ls --all unica
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
unica 1 Tue Jun 9 15:05:08 2020 DEPLOYED unica-0.5.0 1.0 default
[root@unicamaster software]# kubectl describe pod unica-platform-b5688dcb5-4bzz4
Name: unica-platform-b5688dcb5-4bzz4
Namespace: default
Priority: 0
Node: unicanode1/10.0.2.5
Start Time: Tue, 09 Jun 2020 15:05:09 +0530
Labels: app.kubernetes.io/instance=unica
app.kubernetes.io/name=unica-platform
pod-template-hash=b5688dcb5
Annotations: <none>
Status: Pending
IP: 10.44.0.5
IPs:
IP: 10.44.0.5
Controlled By: ReplicaSet/unica-platform-b5688dcb5
Init Containers:
install:
Container ID:
Image: 10.0.2.6:5000/unica-install:68
Image ID:
Port: <none>
Host Port: <none>
Command:
/bin/sh
-c
Args:
chmod 755 /docker/unica && ./entrypoint.sh
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment Variables from:
unica-common-env ConfigMap Optional: false
unica-platform-env ConfigMap Optional: false
unica-campaign-env ConfigMap Optional: false
unica-interact-env ConfigMap Optional: false
Environment: <none>
Mounts:
/docker/unica from volume-mount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Containers:
unica-platform:
Container ID:
Image: 10.0.2.6:5000/tomcat-unica-platform:12
Image ID:
Port: 9123/TCP
Host Port: 0/TCP
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 2
memory: 1600Mi
Requests:
cpu: 1500m
memory: 1300Mi
Readiness: exec [/bin/bash -ce ] delay=20s timeout=30s period=40s #success=1 #failure=3
Environment Variables from:
unica-common-env ConfigMap Optional: false
unica-platform-env ConfigMap Optional: false
unica-campaign-env ConfigMap Optional: false
unica-interact-env ConfigMap Optional: false
Environment:
CONTAINER_NAME: unica-platform
WLP_HTTP_PORT: 9123
WLP_HTTPS_PORT: 9445
Mounts:
/docker/unica from volume-mount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-t7d5k (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
volume-mount:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: unica
ReadOnly: false
default-token-t7d5k:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-t7d5k
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned default/unica-platform-b5688dcb5-4bzz4 to unicanode1
Normal Pulling 38m (x4 over 39m) kubelet, unicanode1 Pulling image "10.0.2.6:5000/unica-install:68"
Warning Failed 38m (x4 over 39m) kubelet, unicanode1 Failed to pull image "10.0.2.6:5000/unica-install:68": rpc error: code = Unknown desc = Error response from daemon: Get https://10.0.2.6:5000/v2/: http: server gave HTTP response to HTTPS client
Warning Failed 38m (x4 over 39m) kubelet, unicanode1 Error: ErrImagePull
Normal BackOff 14m (x107 over 39m) kubelet, unicanode1 Back-off pulling image "10.0.2.6:5000/unica-install:68"
Warning Failed 4m34s (x151 over 39m) kubelet, unicanode1 Error: ImagePullBackOff
Means you need to set daemon.json:
If you didn't set the daemon.json correctly:
[root@unicamaster docker]# systemctl restart docker
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
[root@unicamaster docker]# vi daemon.json
[root@unicamaster docker]# systemctl restart docker
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
[root@unicamaster docker]# systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2020-06-09 15:48:38 IST; 58s ago
Docs: https://docs.docker.com
Process: 5452 ExecStart=/usr/bin/dockerd -H fd:// (code=exited, status=1/FAILURE)
Main PID: 5452 (code=exited, status=1/FAILURE)
Correct syntax:
{
"insecure-registries" : ["10.0.2.6:5000"]
}
NOTE: Need to restart on node1 as well
Error: could not find tiller - Helm init
My first post on on Cloud Native... this is with Helm.
[root@unicamaster software]# helm install --name nginx stable/nginx-ingress -f ./omnix-unica/extra-configs/nginx-conf.yaml
Error: could not find tiller
[root@unicamaster software]# cd tmp
[root@unicamaster tmp]# l s-l
bash: l: command not found...
[root@unicamaster tmp]# l s-l
bash: l: command not found...
[root@unicamaster tmp]# ls -l
total 0
[root@unicamaster tmp]# gunzip -c ../helm-v2.15.2-linux-amd64.tar.gz | tar -xvf -
linux-amd64/
linux-amd64/tiller
linux-amd64/helm
linux-amd64/README.md
linux-amd64/LICENSE
[root@unicamaster tmp]# ls -l
total 0
drwxr-xr-x 2 root root 64 Oct 30 2019 linux-amd64
[root@unicamaster tmp]# cd *
[root@unicamaster linux-amd64]# ls -l
total 76720
-rwxr-xr-x 1 root root 38789120 Oct 30 2019 helm
-rw-r--r-- 1 root root 11343 Oct 30 2019 LICENSE
-rw-r--r-- 1 root root 3288 Oct 30 2019 README.md
-rwxr-xr-x 1 root root 39755776 Oct 30 2019 tiller
[root@unicamaster linux-amd64]# tiller
bash: tiller: command not found...
[root@unicamaster linux-amd64]# ./tiller
[main] 2020/06/09 14:12:58 Starting Tiller v2.15.2 (tls=false)
[main] 2020/06/09 14:12:58 GRPC listening on :44134
[main] 2020/06/09 14:12:58 Probes listening on :44135
[main] 2020/06/09 14:12:58 Storage driver is ConfigMap
[main] 2020/06/09 14:12:58 Max history per release is 0
^C
[root@unicamaster linux-amd64]# l s-l
bash: l: command not found...
[root@unicamaster linux-amd64]# cp -p tiller /usr/local/bin
[root@unicamaster linux-amd64]# ls -l /usr/local/bin
total 76704
lrwxrwxrwx 1 root root 30 Apr 27 11:19 db2greg -> /opt/ibm/db2/V11.1/bin/db2greg
lrwxrwxrwx 1 root root 32 Apr 27 11:19 db2ls -> /opt/ibm/db2/V11.1/install/db2ls
-rwxr-xr-x 1 root root 38789120 Oct 30 2019 helm
-rwxr-xr-x 1 root root 39755776 Oct 30 2019 tiller
[root@unicamaster linux-amd64]# pwd
/opt/software/tmp/linux-amd64
[root@unicamaster linux-amd64]# cd ..
[root@unicamaster tmp]# cd ..
[root@unicamaster software]# pwd
/opt/software
[root@unicamaster software]# l s-l
bash: l: command not found...
[root@unicamaster software]# pwd
/opt/software
[root@unicamaster software]# ls -l
total 23952
-rwxr-x--- 1 root root 24525846 Apr 27 11:10 helm-v2.15.2-linux-amd64.tar.gz
drwxr-x--- 6 root root 288 Jun 9 13:56 omnix-unica
drwxr-xr-x 3 root root 25 Jun 9 14:12 tmp
drwxr-xr-x 2 root root 135 Jun 9 12:51 unica.12.0.0.0
drwxr-xr-x 2 root root 69 Jun 9 13:26 unica.12.0.0.1
[root@unicamaster software]# helm install --name nginx stable/nginx-ingress -f ./omnix-unica/extra-configs/nginx-conf.yaml
Error: could not find tiller
[root@unicamaster software]# help init
bash: help: no help topics match `init'. Try `help help' or `man -k init' or `info init'.
[root@unicamaster software]# helm init
Creating /root/.helm
Creating /root/.helm/repository
Creating /root/.helm/repository/cache
Creating /root/.helm/repository/local
Creating /root/.helm/plugins
Creating /root/.helm/starters
Creating /root/.helm/cache/archive
Creating /root/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /root/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
[root@unicamaster software]# helm install --name nginx stable/nginx-ingress -f ./omnix-unica/extra-configs/nginx-conf.yaml
Error: could not find a ready tiller pod
[root@unicamaster software]# kubectl create serviceaccount --namespace kube-system tiller
serviceaccount/tiller created
[root@unicamaster software]# helm install --name nginx stable/nginx-ingress -f ./omnix-unica/extra-configs/nginx-conf.yaml
Error: release nginx failed: namespaces "default" is forbidden: User "system:serviceaccount:kube-system:default" cannot get resource "namespaces" in API group "" in the namespace "default"
[root@unicamaster software]# kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
clusterrolebinding.rbac.authorization.k8s.io/tiller-cluster-rule created
[root@unicamaster software]# kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
deployment.apps/tiller-deploy patched
[root@unicamaster software]# helm version
Client: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
[root@unicamaster software]# helm install --name nginx stable/nginx-ingress -f ./omnix-unica/extra-configs/nginx-conf.yaml
NAME: nginx
LAST DEPLOYED: Tue Jun 9 14:20:49 2020
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/ClusterRole
NAME AGE
nginx-nginx-ingress 1s
==> v1/ClusterRoleBinding
NAME AGE
nginx-nginx-ingress 1s
==> v1/ConfigMap
NAME DATA AGE
nginx-nginx-ingress-controller 1 1s
==> v1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-nginx-ingress-controller 0/1 1 0 1s
nginx-nginx-ingress-default-backend 0/1 1 0 1s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
nginx-nginx-ingress-controller-7fc9499f7b-96h99 0/1 ContainerCreating 0 0s
nginx-nginx-ingress-default-backend-679f548db6-tq2vz 0/1 ContainerCreating 0 1s
==> v1/Role
NAME AGE
nginx-nginx-ingress 1s
==> v1/RoleBinding
NAME AGE
nginx-nginx-ingress 1s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-nginx-ingress-controller LoadBalancer 10.97.49.16 <pending> 80:32500/TCP,443:31138/TCP 1s
nginx-nginx-ingress-controller-metrics ClusterIP 10.108.50.227 <none> 9913/TCP 1s
nginx-nginx-ingress-default-backend ClusterIP 10.97.62.215 <none> 80/TCP 1s
==> v1/ServiceAccount
NAME SECRETS AGE
nginx-nginx-ingress 1 1s
nginx-nginx-ingress-backend 1 1s
NOTES:
The nginx-ingress controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace default get services -o wide -w nginx-nginx-ingress-controller'
An example Ingress that makes use of the controller:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
name: example
namespace: foo
spec:
rules:
- host: www.example.com
http:
paths:
- backend:
serviceName: exampleService
servicePort: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls
[root@unicamaster software]# helm install --name nginx stable/nginx-ingress -f ./omnix-unica/extra-configs/nginx-conf.yaml
Error: could not find tiller
[root@unicamaster software]# cd tmp
[root@unicamaster tmp]# l s-l
bash: l: command not found...
[root@unicamaster tmp]# l s-l
bash: l: command not found...
[root@unicamaster tmp]# ls -l
total 0
[root@unicamaster tmp]# gunzip -c ../helm-v2.15.2-linux-amd64.tar.gz | tar -xvf -
linux-amd64/
linux-amd64/tiller
linux-amd64/helm
linux-amd64/README.md
linux-amd64/LICENSE
[root@unicamaster tmp]# ls -l
total 0
drwxr-xr-x 2 root root 64 Oct 30 2019 linux-amd64
[root@unicamaster tmp]# cd *
[root@unicamaster linux-amd64]# ls -l
total 76720
-rwxr-xr-x 1 root root 38789120 Oct 30 2019 helm
-rw-r--r-- 1 root root 11343 Oct 30 2019 LICENSE
-rw-r--r-- 1 root root 3288 Oct 30 2019 README.md
-rwxr-xr-x 1 root root 39755776 Oct 30 2019 tiller
[root@unicamaster linux-amd64]# tiller
bash: tiller: command not found...
[root@unicamaster linux-amd64]# ./tiller
[main] 2020/06/09 14:12:58 Starting Tiller v2.15.2 (tls=false)
[main] 2020/06/09 14:12:58 GRPC listening on :44134
[main] 2020/06/09 14:12:58 Probes listening on :44135
[main] 2020/06/09 14:12:58 Storage driver is ConfigMap
[main] 2020/06/09 14:12:58 Max history per release is 0
^C
[root@unicamaster linux-amd64]# l s-l
bash: l: command not found...
[root@unicamaster linux-amd64]# cp -p tiller /usr/local/bin
[root@unicamaster linux-amd64]# ls -l /usr/local/bin
total 76704
lrwxrwxrwx 1 root root 30 Apr 27 11:19 db2greg -> /opt/ibm/db2/V11.1/bin/db2greg
lrwxrwxrwx 1 root root 32 Apr 27 11:19 db2ls -> /opt/ibm/db2/V11.1/install/db2ls
-rwxr-xr-x 1 root root 38789120 Oct 30 2019 helm
-rwxr-xr-x 1 root root 39755776 Oct 30 2019 tiller
[root@unicamaster linux-amd64]# pwd
/opt/software/tmp/linux-amd64
[root@unicamaster linux-amd64]# cd ..
[root@unicamaster tmp]# cd ..
[root@unicamaster software]# pwd
/opt/software
[root@unicamaster software]# l s-l
bash: l: command not found...
[root@unicamaster software]# pwd
/opt/software
[root@unicamaster software]# ls -l
total 23952
-rwxr-x--- 1 root root 24525846 Apr 27 11:10 helm-v2.15.2-linux-amd64.tar.gz
drwxr-x--- 6 root root 288 Jun 9 13:56 omnix-unica
drwxr-xr-x 3 root root 25 Jun 9 14:12 tmp
drwxr-xr-x 2 root root 135 Jun 9 12:51 unica.12.0.0.0
drwxr-xr-x 2 root root 69 Jun 9 13:26 unica.12.0.0.1
[root@unicamaster software]# helm install --name nginx stable/nginx-ingress -f ./omnix-unica/extra-configs/nginx-conf.yaml
Error: could not find tiller
[root@unicamaster software]# help init
bash: help: no help topics match `init'. Try `help help' or `man -k init' or `info init'.
[root@unicamaster software]# helm init
Creating /root/.helm
Creating /root/.helm/repository
Creating /root/.helm/repository/cache
Creating /root/.helm/repository/local
Creating /root/.helm/plugins
Creating /root/.helm/starters
Creating /root/.helm/cache/archive
Creating /root/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /root/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
[root@unicamaster software]# helm install --name nginx stable/nginx-ingress -f ./omnix-unica/extra-configs/nginx-conf.yaml
Error: could not find a ready tiller pod
[root@unicamaster software]# kubectl create serviceaccount --namespace kube-system tiller
serviceaccount/tiller created
[root@unicamaster software]# helm install --name nginx stable/nginx-ingress -f ./omnix-unica/extra-configs/nginx-conf.yaml
Error: release nginx failed: namespaces "default" is forbidden: User "system:serviceaccount:kube-system:default" cannot get resource "namespaces" in API group "" in the namespace "default"
[root@unicamaster software]# kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
clusterrolebinding.rbac.authorization.k8s.io/tiller-cluster-rule created
[root@unicamaster software]# kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
deployment.apps/tiller-deploy patched
[root@unicamaster software]# helm version
Client: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
[root@unicamaster software]# helm install --name nginx stable/nginx-ingress -f ./omnix-unica/extra-configs/nginx-conf.yaml
NAME: nginx
LAST DEPLOYED: Tue Jun 9 14:20:49 2020
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/ClusterRole
NAME AGE
nginx-nginx-ingress 1s
==> v1/ClusterRoleBinding
NAME AGE
nginx-nginx-ingress 1s
==> v1/ConfigMap
NAME DATA AGE
nginx-nginx-ingress-controller 1 1s
==> v1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-nginx-ingress-controller 0/1 1 0 1s
nginx-nginx-ingress-default-backend 0/1 1 0 1s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
nginx-nginx-ingress-controller-7fc9499f7b-96h99 0/1 ContainerCreating 0 0s
nginx-nginx-ingress-default-backend-679f548db6-tq2vz 0/1 ContainerCreating 0 1s
==> v1/Role
NAME AGE
nginx-nginx-ingress 1s
==> v1/RoleBinding
NAME AGE
nginx-nginx-ingress 1s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-nginx-ingress-controller LoadBalancer 10.97.49.16 <pending> 80:32500/TCP,443:31138/TCP 1s
nginx-nginx-ingress-controller-metrics ClusterIP 10.108.50.227 <none> 9913/TCP 1s
nginx-nginx-ingress-default-backend ClusterIP 10.97.62.215 <none> 80/TCP 1s
==> v1/ServiceAccount
NAME SECRETS AGE
nginx-nginx-ingress 1 1s
nginx-nginx-ingress-backend 1 1s
NOTES:
The nginx-ingress controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace default get services -o wide -w nginx-nginx-ingress-controller'
An example Ingress that makes use of the controller:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
name: example
namespace: foo
spec:
rules:
- host: www.example.com
http:
paths:
- backend:
serviceName: exampleService
servicePort: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls
Subscribe to:
Posts (Atom)
PVC in terminating state
Referencee:https://veducate.co.uk/kubernetes-pvc-terminating/ I still had errors performing deployment, till I released that although I pre...
-
Got this error after I enabled nested virtualization on my vm (was working previously): Solution:https://github.com/GNS3/gns3-gui/issues/30...
-
Kubelet http server gave http response to https client (ImagePullBackOff status on a Kubernetes pod)[root@unicamaster docker]# kubectl get pods NAME READY STATUS RESTART...
-
Got this error: [root@unica12master .kube]# kubectl get nodes NAME STATUS ROLES AGE VERSION unica12master....