问题描述
我在 google Endpoint 上有一个 API,后端部署在 GKE 上,我想通过入口公开它,以便我可以在其上使用 IAP。
我正在使用 ESP2。
我首先将我的服务部署为 LoadBalancer 并且它正在运行。
Thing is my ingress 说:
“所有后端服务都处于不健康状态”
我知道健康检查没有通过,但我不明白为什么...
服务和相应的 pod 没有显示错误,但是在我的 pod 事件中我可以看到: “就绪探测失败:获取 http://10.32.1.27:8000/swagger:拨号 tcp 10.32.1.27:8000:连接:连接被拒绝”
我的 pod 和服务配置如下所示:
apiVersion: v1
kind: Service
metadata:
name: devfleet-django-endpoint-service
namespace: my-ns
spec:
# NodePort is mandatory for Ingress to perform load balancer
type: NodePort
ports:
- port: 443
protocol: TCP
targetPort: 9000
name: https
selector:
app: devfleet-django-endpoint
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: devfleet-django-endpoint
namespace: my-ns
spec:
replicas: 2
selector:
matchLabels:
app: devfleet-django-endpoint
template:
metadata:
labels:
app: devfleet-django-endpoint
spec:
containers:
- name: esp
image: gcr.io/endpoints-release/endpoints-runtime:2
args: [
"--listener_port=9000","--backend","127.0.0.1:8080","--service=my-custom-domain.io","--rollout_strategy=managed","-z","healthz","--ssl_server_cert_path","/etc/esp/ssl"
]
volumeMounts:
- mountPath: /etc/esp/ssl
name: esp-ssl
readOnly: true
- mountPath: /etc/nginx/custom
name: nginx-config
readOnly: true
ports:
- containerPort: 9000
- name: devfleet-django-endpoint
image: my-img
ports:
- containerPort: 8000
imagePullPolicy: Always
env:
some_env_data
readinessProbe:
httpGet:
path: /swagger
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 10
failureThreshold: 3
livenessProbe:
httpGet:
path: /swagger
port: 8000
initialDelaySeconds: 15
periodSeconds: 60
timeoutSeconds: 10
failureThreshold: 3
volumeMounts:
- name: devfleet-storage
mountPath: /secrets/cloudstorage
readOnly: true
resources:
requests:
cpu: 30m
memory: 90Mi
limits:
cpu: 200m
memory: 400Mi
- image: b.gcr.io/cloudsql-docker/gce-proxy:1.13
name: cloudsql-proxy
command: ["/cloud_sql_proxy","--dir=/app","-instances=production-213911:europe-west1:dev-postgresql=tcp:5432","-credential_file=/secrets/cloudsql/credentials.json"]
volumeMounts:
- name: prodsql-oauth-credentials
mountPath: /secrets/cloudsql
readOnly: true
resources:
requests:
cpu: 10m
memory: 10Mi
limits:
cpu: 20m
memory: 50Mi
volumes:
- name: prodsql-oauth-credentials
secret:
secretName: secret-name
- name: devfleet-storage
secret:
secretName: secret-name
- name: app
emptyDir:
- name: esp-ssl
secret:
secretName: secret-name
- name: nginx-config
configMap:
name: nginx-config
nodeSelector:
cloud.google.com/gke-nodepool: default-pool
我的入口配置是:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: devapi
namespace: my-ns
annotations:
kubernetes.io/ingress.global-static-ip-name: ingress-devapi
ingress.gcp.kubernetes.io/pre-shared-cert: "my-cert"
kubernetes.io/ingress.class: gce
ingress.kubernetes.io/enable-cors: "true"
kubernetes.io/ingress.allow-http: "false"
spec:
backend:
serviceName: devfleet-django-endpoint-service
servicePort: 443
知道我做错了什么吗?
谢谢
解决方法
因此,为了将您的 API 部署暴露在 esp 容器和您的应用容器在同一个 Pod 中的入口上的 google 端点上,您需要创建具有特定运行状况检查的后端配置。
这是我所做的:
- 添加了后端配置:
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: esp-config
namespace: my-namespace
spec:
healthCheck:
checkIntervalSec: 80
port: the-node-port-on-which-your-service-is-exposed
type: HTTPS
requestPath: /healthz
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: devapi-config
namespace: my-namespace
spec:
healthCheck:
checkIntervalSec: 80
port: the-node-port-on-which-your-service-is-exposed
requestPath: /
- 更新我的服务:
apiVersion: v1
kind: Service
metadata:
name: name
namespace: my-namespace
annotations:
cloud.google.com/backend-config: '{"ports": {
"443":"esp-config","8035":"devapi-config"
}}'
spec:
ports:
- port: Exposed port
targetPort: target-port-you-set-in-deployment
protocol: TCP
name: name
- port: Exposed port
targetPort: target-port-you-set-in-deployment
protocol: TCP
name: another-name
selector:
app: name
type: NodePort
- 更新我的入口
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: name
namespace: my-namespace
annotations:
kubernetes.io/ingress.global-static-ip-name: static-ip-name
ingress.gcp.kubernetes.io/pre-shared-cert: my-cert
kubernetes.io/ingress.class: gce
ingress.kubernetes.io/enable-cors: "true"
spec:
backend:
serviceName: my-service-name
servicePort: The-exposed-port-of-my-application (not esp)