问题描述
我对 Kubernetes 很陌生，只是想学习。
# cat flask-deployment.yml
---
apiVersion: apps/v1
kind: Deployment
Metadata:
name: flask
labels:
app: flask
spec:
replicas: 1
selector:
matchLabels:
app: flask
template:
Metadata:
labels:
app: flask
spec:
containers:
- name: container1-flask
image: xxxxx/flaskarmv8:1
ports:
- containerPort: 5000
这是redis部署：
# cat redis-deployment.yml
apiVersion: apps/v1
kind: Deployment
Metadata:
labels:
component: redis
name: redis
spec:
selector:
matchLabels:
component: redis
replicas: 1
template:
Metadata:
labels:
app: redis
role: Metadata
component: redis
spec:
containers:
- name: redis
image: redis:6.0-rc4-alpine
ports:
- containerPort: 6379
restartPolicy: Always
我有 2 个 Pod 的 2 个部署结果。
# kubectl get pods -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READInesS GATES
flask-64544b95f8-gsl49 1/1 Running 0 82m 192.168.42.130 rockpi <none> <none>
redis-596bbd776-vnvkp 1/1 Running 0 82m 192.168.42.129 rockpi <none> <none>
和部署：
# kubectl get deployments -owide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
flask 1/1 1 1 83m container1-flask xxxx/flaskarmv8:1 app=flask
redis 1/1 1 1 83m redis redis:6.0-rc4-alpine component=redis
如果我进入了flask pod，我就不能nslookup redis：
# kubectl exec flask-64544b95f8-gsl49 -it -- sh
/ # nslookup redis
;; connection timed out; no servers Could be reached
这虽然是 coredns pod 未运行的标志。但如果我检查所有豆荚。 coredns 正在运行。
# kubectl get pods -owide -n kube-system
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READInesS GATES
calico-kube-controllers-86bddfcff-qp47x 1/1 Running 0 106m 192.168.7.194 rockpix <none> <none>
calico-node-9sqt8 0/1 Running 0 88m 192.168.5.165 rockpi <none> <none>
calico-node-cv7gh 1/1 Running 0 106m 192.168.5.164 rockpix <none> <none>
calico-node-ltmpb 1/1 Running 0 25m 192.168.5.166 rockpro64 <none> <none>
coredns-74ff55c5b-5jhhw 1/1 Running 0 113m 192.168.7.193 rockpix <none> <none>
coredns-74ff55c5b-f6j7z 1/1 Running 0 113m 192.168.7.195 rockpix <none> <none>
etcd-rockpix 1/1 Running 0 113m 192.168.5.164 rockpix <none> <none>
kube-apiserver-rockpix 1/1 Running 0 113m 192.168.5.164 rockpix <none> <none>
kube-controller-manager-rockpix 1/1 Running 0 113m 192.168.5.164 rockpix <none> <none>
kube-proxy-mf8f5 1/1 Running 0 113m 192.168.5.164 rockpix <none> <none>
kube-proxy-rlvjq 1/1 Running 0 88m 192.168.5.165 rockpi <none> <none>
kube-proxy-xwh69 1/1 Running 0 25m 192.168.5.166 rockpro64 <none> <none>
kube-scheduler-rockpix 1/1 Running 0 113m 192.168.5.164 rockpix <none> <none>
但是我发现 calico 节点没有在运行 flask pod 的同一个 kube 工作节点上运行。 所以我的第一个疑问是这可能是我无法从 Flask 容器内 nslookup 或 ping redis 的原因吗？
然后我启动了flask服务：
kubeclt create -f flask-service.yaml
kind: Service
apiVersion: v1
Metadata:
name: flask
spec:
selector:
run: flask
ports:
- protocol: TCP
port: 5000
targetPort: 5000
并创建了redis ClusterIP服务：
# kubectl expose deploy redis --port=6379 --type=ClusterIP
现在我有这些服务：
# kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
flask ClusterIP 10.106.80.27 <none> 5000/TCP 7m45s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 125m
redis ClusterIP 10.99.81.2 <none> 6379/TCP 5m39s
我的一个新奇发现是 env 没有更新烧瓶和 redis pod 端点： 检查烧瓶舱内。
/ # env
KUBERNETES_PORT=tcp://10.96.0.1:443
KUBERNETES_SERVICE_PORT=443
HOSTNAME=flask-64544b95f8-gsl49
SHLVL=1
HOME=/root
TERM=xterm
KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_SERVICE_HOST=10.96.0.1
为了消除我的怀疑，我检查了 calico not running node：下面你确实看到 status running 但在最底部你也看到 calico/node is not ready: BIRD is not ready: BGP not established with 192.168.5.164
# kubectl describe pod calico-node-9sqt8 -n kube-system
Name: calico-node-9sqt8
Namespace: kube-system
Priority: 2000001000
Priority Class Name: system-node-critical
Node: rockpi/192.168.5.165
Start Time: Sat,27 Feb 2021 22:18:44 +0000
Labels: controller-revision-hash=599f8d748
k8s-app=calico-node
pod-template-generation=1
Annotations: <none>
Status: Running
IP: 192.168.5.165
IPs:
IP: 192.168.5.165
Controlled By: DaemonSet/calico-node
Init Containers:
upgrade-ipam:
Container ID: containerd://b21a2650e29d6cc763128cdbc3ed0ef344235b5cbf10a29960077f755ff3e08a
Image: docker.io/calico/cni:v3.17.2
Image ID: docker.io/calico/cni@sha256:e845f71241ef25fae0dd283d8b46a84e7c0a4bc8faaaccc78d7a854748a5393f
Port: <none>
Host Port: <none>
Command:
/opt/cni/bin/calico-ipam
-upgrade
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sat,27 Feb 2021 22:18:45 +0000
Finished: Sat,27 Feb 2021 22:18:45 +0000
Ready: True
Restart Count: 0
Environment Variables from:
kubernetes-services-endpoint ConfigMap Optional: true
Environment:
KUBERNETES_NODE_NAME: (v1:spec.nodeName)
CALICO_NETWORKING_BACKEND: <set to the key 'calico_backend' of config map 'calico-config'> Optional: false
Mounts:
/host/opt/cni/bin from cni-bin-dir (rw)
/var/lib/cni/networks from host-local-net-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-796gq (ro)
install-cni:
Container ID: containerd://5a81605a65873888244dae450144394b64e0cdaa6d9d8e6e9e55573f0aa42dee
Image: docker.io/calico/cni:v3.17.2
Image ID: docker.io/calico/cni@sha256:e845f71241ef25fae0dd283d8b46a84e7c0a4bc8faaaccc78d7a854748a5393f
Port: <none>
Host Port: <none>
Command:
/opt/cni/bin/install
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sat,27 Feb 2021 22:18:47 +0000
Finished: Sat,27 Feb 2021 22:18:50 +0000
Ready: True
Restart Count: 0
Environment Variables from:
kubernetes-services-endpoint ConfigMap Optional: true
Environment:
CNI_CONF_NAME: 10-calico.conflist
CNI_NETWORK_CONfig: <set to the key 'cni_network_config' of config map 'calico-config'> Optional: false
KUBERNETES_NODE_NAME: (v1:spec.nodeName)
CNI_MTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false
SLEEP: false
Mounts:
/host/etc/cni/net.d from cni-net-dir (rw)
/host/opt/cni/bin from cni-bin-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-796gq (ro)
flexvol-driver:
Container ID: containerd://07daa43eaa753ac53883bb52b7fc8e77e776ea943fa0c7630441d39d10d1d6da
Image: docker.io/calico/pod2daemon-flexvol:v3.17.2
Image ID: docker.io/calico/pod2daemon-flexvol@sha256:4690a45bf5409a7800160125088607a887f7b09f847b9d0e5a2366394bc61de4
Port: <none>
Host Port: <none>
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sat,27 Feb 2021 22:18:51 +0000
Finished: Sat,27 Feb 2021 22:18:51 +0000
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/host/driver from flexvol-driver-host (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-796gq (ro)
Containers:
calico-node:
Container ID: containerd://d2b149f718fa4132728e0f2e92e448eec98a1c819b2ffdf4f427c4817c4658b9
Image: docker.io/calico/node:v3.17.2
Image ID: docker.io/calico/node@sha256:edced8e620e619da50492a45891f5c77915cd3e0aa8381b9b7708c9ec1ddbcc5
Port: <none>
Host Port: <none>
State: Running
Started: Sat,27 Feb 2021 22:18:52 +0000
Ready: False
Restart Count: 0
Requests:
cpu: 250m
Liveness: exec [/bin/calico-node -Felix-live -bird-live] delay=10s timeout=1s period=10s #success=1 #failure=6
Readiness: exec [/bin/calico-node -Felix-ready -bird-ready] delay=0s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
kubernetes-services-endpoint ConfigMap Optional: true
Environment:
DATASTORE_TYPE: kubernetes
WAIT_FOR_DATASTORE: true
NODENAME: (v1:spec.nodeName)
CALICO_NETWORKING_BACKEND: <set to the key 'calico_backend' of config map 'calico-config'> Optional: false
CLUSTER_TYPE: k8s,bgp
IP: autodetect
CALICO_IPV4POOL_IPIP: Always
CALICO_IPV4POOL_VXLAN: Never
Felix_IPINIPMTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false
Felix_VXLANMTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false
Felix_WIREGUARDMTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false
CALICO_disABLE_FILE_LOGGING: true
Felix_DEFAULTENDPOINTTOHOSTACTION: ACCEPT
Felix_IPV6SUPPORT: false
Felix_LOGSEVERITYSCREEN: info
Felix_HEALTHENABLED: true
Mounts:
/lib/modules from lib-modules (ro)
/run/xtables.lock from xtables-lock (rw)
/sys/fs/ from sysfs (rw)
/var/lib/calico from var-lib-calico (rw)
/var/log/calico/cni from cni-log-dir (ro)
/var/run/calico from var-run-calico (rw)
/var/run/nodeagent from policysync (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-796gq (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
var-run-calico:
Type: HostPath (bare host directory volume)
Path: /var/run/calico
HostPathType:
var-lib-calico:
Type: HostPath (bare host directory volume)
Path: /var/lib/calico
HostPathType:
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType: FileOrCreate
sysfs:
Type: HostPath (bare host directory volume)
Path: /sys/fs/
HostPathType: DirectoryOrCreate
cni-bin-dir:
Type: HostPath (bare host directory volume)
Path: /opt/cni/bin
HostPathType:
cni-net-dir:
Type: HostPath (bare host directory volume)
Path: /etc/cni/net.d
HostPathType:
cni-log-dir:
Type: HostPath (bare host directory volume)
Path: /var/log/calico/cni
HostPathType:
host-local-net-dir:
Type: HostPath (bare host directory volume)
Path: /var/lib/cni/networks
HostPathType:
policysync:
Type: HostPath (bare host directory volume)
Path: /var/run/nodeagent
HostPathType: DirectoryOrCreate
flexvol-driver-host:
Type: HostPath (bare host directory volume)
Path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds
HostPathType: DirectoryOrCreate
calico-node-token-796gq:
Type: Secret (a volume populated by a Secret)
SecretName: calico-node-token-796gq
Optional: false
QoS Class: Burstable
Node-Selectors: kubernetes.io/os=linux
Tolerations: :NoSchedule op=Exists
:NoExecute op=Exists
CriticalAddonsOnly op=Exists
node.kubernetes.io/disk-pressure:NoSchedule op=Exists
node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/network-unavailable:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists
node.kubernetes.io/pid-pressure:NoSchedule op=Exists
node.kubernetes.io/unreachable:NoExecute op=Exists
node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 2m50s (x591 over 101m) kubelet (combined from similar events): Readiness probe Failed: 2021-02-27 23:58:48.841 [INFO][18491] confd/health.go 180: Number of node(s) with BGP peering established = 0
calico/node is not ready: BIRD is not ready: BGP not established with 192.168.5.164
然后我检查了 calico 节点状态
# sudo ./calicoctl node status
Calico process is running.
IPv4 BGP status
+---------------+-------------------+-------+----------+-------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+---------------+-------------------+-------+----------+-------------+
| 10.8.0.1 | node-to-node mesh | start | 22:18:53 | Connect |
| 192.168.5.166 | node-to-node mesh | up | 23:31:13 | Established |
+---------------+-------------------+-------+----------+-------------+
IPv6 BGP status
No IPv6 peers found.
这是calico-node pod没有运行的iptable
rockpi:~# netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 192.168.5.1 0.0.0.0 UG 0 0 0 eth0
0.0.0.0 192.168.5.1 0.0.0.0 UG 0 0 0 eth0
0.0.0.0 192.168.5.1 0.0.0.0 UG 0 0 0 eth0
10.8.0.0 0.0.0.0 255.255.255.0 U 0 0 0 tun0
192.168.5.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.5.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.5.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.5.163 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.42.128 0.0.0.0 255.255.255.192 U 0 0 0 *
192.168.42.130 0.0.0.0 255.255.255.255 UH 0 0 0 cali4ae1ba57970
192.168.42.131 0.0.0.0 255.255.255.255 UH 0 0 0 cali16e69c6be33
难道这个rockpi节点也是我在子网10.8.0.0上运行的openvpn服务器？
解决方法
暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！
如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。
小编邮箱:dio#foxmail.com (将#修改为@）