在入口处公开 Google Endpoint API

问题描述

我在 google Endpoint 上有一个 API，后端部署在 GKE 上，我想通过入口公开它，以便我可以在其上使用 IAP。
我正在使用 ESP2。
我首先将我的服务部署为 LoadBalancer 并且它正在运行。

Thing is my ingress 说： “所有后端服务都处于不健康状态”
我知道健康检查没有通过，但我不明白为什么...

服务和相应的 pod 没有显示错误，但是在我的 pod 事件中我可以看到： “就绪探测失败：获取 http://10.32.1.27:8000/swagger：拨号 tcp 10.32.1.27:8000：连接：连接被拒绝”

我的 pod 和服务配置如下所示：

apiVersion: v1
kind: Service
metadata:
 name: devfleet-django-endpoint-service
 namespace: my-ns
spec:
  # NodePort is mandatory for Ingress to perform load balancer
  type: NodePort
  ports:
  - port: 443
    protocol: TCP
    targetPort: 9000
    name: https
  selector:
    app: devfleet-django-endpoint
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: devfleet-django-endpoint
  namespace: my-ns
spec:
  replicas: 2
  selector:
    matchLabels:
      app: devfleet-django-endpoint
   template:
    metadata:
      labels:
        app: devfleet-django-endpoint
    spec:
      containers:
      - name: esp
        image: gcr.io/endpoints-release/endpoints-runtime:2
        args: [
          "--listener_port=9000","--backend","127.0.0.1:8080","--service=my-custom-domain.io","--rollout_strategy=managed","-z","healthz","--ssl_server_cert_path","/etc/esp/ssl"
        ]
        volumeMounts:
        - mountPath: /etc/esp/ssl
          name: esp-ssl
          readOnly: true
        - mountPath: /etc/nginx/custom
          name: nginx-config
          readOnly: true        
        ports:
          - containerPort: 9000
      - name: devfleet-django-endpoint
        image: my-img
        ports:
        - containerPort: 8000
        imagePullPolicy: Always
        env:
         some_env_data
        readinessProbe:
          httpGet:
            path: /swagger
            port: 8000
          initialDelaySeconds: 10
          periodSeconds: 10
          timeoutSeconds: 10
          failureThreshold: 3
        livenessProbe:
          httpGet:
           path: /swagger
            port: 8000
          initialDelaySeconds: 15
          periodSeconds: 60
          timeoutSeconds: 10
          failureThreshold: 3
        volumeMounts:
          - name: devfleet-storage
            mountPath: /secrets/cloudstorage
            readOnly: true
        resources:
          requests:
            cpu: 30m
            memory: 90Mi
          limits:
            cpu: 200m
            memory: 400Mi            
      - image: b.gcr.io/cloudsql-docker/gce-proxy:1.13
        name: cloudsql-proxy
        command: ["/cloud_sql_proxy","--dir=/app","-instances=production-213911:europe-west1:dev-postgresql=tcp:5432","-credential_file=/secrets/cloudsql/credentials.json"]
        volumeMounts:
           - name: prodsql-oauth-credentials
            mountPath: /secrets/cloudsql
            readOnly: true
         resources:
          requests:
            cpu: 10m
            memory: 10Mi
          limits:
            cpu: 20m
            memory: 50Mi          
      volumes:
        - name: prodsql-oauth-credentials
          secret:
            secretName: secret-name
        - name: devfleet-storage
           secret:
            secretName: secret-name
        - name: app
           emptyDir:
        - name: esp-ssl
          secret:
            secretName: secret-name
        - name: nginx-config
          configMap:
            name: nginx-config
          
      nodeSelector:
        cloud.google.com/gke-nodepool: default-pool

我的入口配置是：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: devapi
  namespace: my-ns
  annotations:
    kubernetes.io/ingress.global-static-ip-name: ingress-devapi
    ingress.gcp.kubernetes.io/pre-shared-cert: "my-cert"
    kubernetes.io/ingress.class: gce
    ingress.kubernetes.io/enable-cors: "true"
    kubernetes.io/ingress.allow-http: "false"
spec:
  backend:
    serviceName: devfleet-django-endpoint-service
    servicePort: 443

知道我做错了什么吗？
谢谢

解决方法

因此，为了将您的 API 部署暴露在 esp 容器和您的应用容器在同一个 Pod 中的入口上的 google 端点上，您需要创建具有特定运行状况检查的后端配置。

这是我所做的：

1. 添加了后端配置：

apiVersion: cloud.google.com/v1  
kind: BackendConfig  
metadata:  
   name: esp-config  
  namespace: my-namespace  
spec:  
  healthCheck:  
    checkIntervalSec: 80  
    port: the-node-port-on-which-your-service-is-exposed  
    type: HTTPS  
    requestPath: /healthz

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: devapi-config
  namespace: my-namespace
spec:
  healthCheck:
    checkIntervalSec: 80
    port: the-node-port-on-which-your-service-is-exposed
    requestPath: /

2. 更新我的服务：

apiVersion: v1
kind: Service
metadata:
  name: name
  namespace: my-namespace
  annotations:
    cloud.google.com/backend-config: '{"ports": {
    "443":"esp-config","8035":"devapi-config"
    }}'

spec:
  ports:
  - port: Exposed port
    targetPort: target-port-you-set-in-deployment
    protocol: TCP
    name: name
  - port: Exposed port
    targetPort: target-port-you-set-in-deployment
    protocol: TCP
    name: another-name
  selector:
    app: name
  type: NodePort

3. 更新我的入口

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: name
  namespace: my-namespace
  annotations:
    kubernetes.io/ingress.global-static-ip-name: static-ip-name
    ingress.gcp.kubernetes.io/pre-shared-cert: my-cert
    kubernetes.io/ingress.class: gce
    ingress.kubernetes.io/enable-cors: "true"

spec:
  backend:
    serviceName: my-service-name
    servicePort: The-exposed-port-of-my-application (not esp)