Kubernetes - 按服务名称调用微服务

问题描述

我在 K8S 集群上部署了两个微服务（本地在 3 个虚拟机上 - 1 个主节点和 2 个工作节点）：
1- 货币兑换微服务
2- 货币转换微服务

我正在尝试使用服务名称从 currency-conversion 调用 currency-exchange 微服务：
http:///currency-exchange:8000.

返回错误如下：
{"timestamp":"2021-02-17T08:38:25.590+0000","status":500,"error":"Internal Server Error","message":"currency-exchange executing GET http://currency-exchange:8000/currency-exchange/from/EUR/to/INR","path":"/currency-conversion/from/EUR/to/INR/quantity/10"}

我正在使用 Kubernetes，CentOS8 使用 Calico CNI 并设置 Felix_IPTABLESBACKEND=NFT，基于 this link 以促进 POD-TO-POD 通信。
当前可用的服务：

[root@k8s-master ~]# kubectl get svc
NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE

currency-conversion                  NodePort    10.106.70.108    <none>        8100:32470/TCP               3h40m

currency-exchange                    NodePort    10.110.232.189   <none>        8000:31776/TCP               3h41m

豆荚：

[root@k8s-master ~]# kubectl get pods -o wide
NAME                                        READY   STATUS    RESTARTS   AGE     IP                NODE            NOMINATED NODE   READInesS GATES

currency-conversion-86d9bc4698-rxdkh        1/1     Running   0          5h45m   192.168.212.125   worker-node-1   <none>           <none>
currency-exchange-c79ff888b-c8sdd           1/1     Running   0          5h44m   192.168.19.160    worker-node-2   <none>           <none>
currency-exchange-c79ff888b-nfqpx           1/1     Running   0          5h44m   192.168.212.65    worker-node-1   <none>           <none>

可用的 coredns Pod 列表：

[root@k8s-master ~]# kubectl get pods -o wide -n kube-system | grep coredns
coredns-74ff55c5b-9x5qm                    1/1     Running   8          25d   192.168.235.218   k8s-master      <none>           <none>
coredns-74ff55c5b-zkkn7                    1/1     Running   8          25d   192.168.235.220   k8s-master      <none>           <none>

列出所有 ENV 变量：

[root@k8s-master ~]# kubectl exec -it currency-conversion-86d9bc4698-rxdkh -- printenv

HOSTNAME=currency-conversion-86d9bc4698-rxdkh
CURRENCY_EXCHANGE_SERVICE_HOST=http://currency-exchange
KUBERNETES_SERVICE_HOST=10.96.0.1
CURRENCY_EXCHANGE_SERVICE_PORT=8000

........

nslookup kubernetes.default exec 命令：

[root@k8s-master ~]# kubectl exec -it currency-conversion-86d9bc4698-rxdkh -- nslookup kubernetes.default
    nslookup: can't resolve '(null)': Name does not resolve

nslookup: can't resolve 'kubernetes.default': Try again
command terminated with exit code 1

人们如何解决这样的问题？他们是否配置/调整 DNS 以作为服务注册表正常工作？

提前致谢

已编辑：

[root@k8s-master ~]# kubectl describe service currency-conversion
Name:                     currency-conversion
Namespace:                default
Labels:                   app=currency-conversion
Annotations:              <none>
Selector:                 app=currency-conversion
Type:                     NodePort
IP Families:              <none>
IP:                       10.106.70.108
IPs:                      10.106.70.108
Port:                     <unset>  8100/TCP
TargetPort:               8100/TCP
NodePort:                 <unset>  32470/TCP
Endpoints:                192.168.212.125:8100
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

[root@k8s-master ~]# kubectl describe service currency-exchange
Name:                     currency-exchange
Namespace:                default
Labels:                   app=currency-exchange
Annotations:              <none>
Selector:                 app=currency-exchange
Type:                     NodePort
IP Families:              <none>
IP:                       10.110.232.189
IPs:                      10.110.232.189
Port:                     <unset>  8000/TCP
TargetPort:               8000/TCP
NodePort:                 <unset>  31776/TCP
Endpoints:                192.168.19.160:8000,192.168.212.65:8000
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

我刚刚在所有 coredns PODS 中发现错误行为，很多超时：

[root@k8s-master ~]# kubectl logs coredns-74ff55c5b-zkkn7 -n kube-system
.:53
[INFO] plugin/reload: Running configuration MD5 = db32ca3650231d74073ff4cf814959a7
coredns-1.7.0
linux/amd64,go1.14.4,f59c03d
[ERROR] plugin/errors: 2 6675909625369157619.3582573715596351475. HINFO: read udp 192.168.235.221:59744->192.168.100.1:53: I/O timeout
[ERROR] plugin/errors: 2 6675909625369157619.3582573715596351475. HINFO: read udp 192.168.235.221:53400->192.168.100.1:53: I/O timeout
[ERROR] plugin/errors: 2 6675909625369157619.3582573715596351475. HINFO: read udp 192.168.235.221:58465->192.168.100.1:53: I/O timeout
[ERROR] plugin/errors: 2 6675909625369157619.3582573715596351475. HINFO: read udp 192.168.235.221:58197->192.168.100.1:53: I/O timeout
[ERROR] plugin/errors: 2 6675909625369157619.3582573715596351475. HINFO: read udp 192.168.235.221:57794->192.168.100.1:53: I/O timeout
[ERROR] plugin/errors: 2 6675909625369157619.3582573715596351475. HINFO: read udp 192.168.235.221:43345->192.168.100.1:53: I/O timeout
[ERROR] plugin/errors: 2 6675909625369157619.3582573715596351475. HINFO: read udp 192.168.235.221:57361->192.168.100.1:53: I/O timeout
[ERROR] plugin/errors: 2 6675909625369157619.3582573715596351475. HINFO: read udp 192.168.235.221:51716->192.168.100.1:53: I/O timeout

如何开始追踪问题？

额外细节：

[root@k8s-master ~]# kubectl exec -i -t currency-conversion-86d9bc4698-rxdkh -- sh
/ # wget http://currency-exchange:8000/currency-exchange/from/EUR/to/INR
wget: bad address 'currency-exchange:8000'

解决方法

在我看来，您的 CNI 覆盖网络设置不正确。我检查了您之前的问题以验证节点的 ip 地址，在我看来您的 pod 网络与您的主机网络重叠：

Kubernetes pod-network-cidr 是 Kubernetes 集群中所有 Pod 的 IP prefix。此范围不得与您的 VPC

中的其他网络冲突

Kubernetes pod 网络 documentation 也对此进行了描述：

注意你的 Pod 网络不能与任何主机重叠 网络：如果有任何重叠，您可能会看到问题。 （如果 您发现网络插件的首选 Pod 之间存在冲突 网络和你的一些主机网络，你应该想一个合适的 改为使用 CIDR 块，然后在 kubeadm init 期间使用它 --pod-network-cidr 并作为网络插件中的替代品 YAML）。

在创建 cluster 时，在 calico 说明中也提到了这一点：

注意：如果 192.168.0.0/16 已在您的网络中使用，您必须选择不同的 pod 网络 CIDR，替换 192.168.0.0/16 上面的命令。

附注。您可以随时从 here wget 卷曲。