如何对 ingress-nginx 负载均衡器 IP 进行分段

编程问答 2022-05-18

问题描述

我有一个 kubespray 部署了 3 个 masters/3workers 集群。它还安装了 ingress-nginx：

kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: ingress-nginx-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      containers:
        - name: ingress-nginx-controller
          image: k8s.gcr.io/ingress-nginx/controller:v0.43.0
          args:
            - /nginx-ingress-controller
            - '--configmap=$(POD_NAMESPACE)/ingress-nginx'
            - '--tcp-services-configmap=$(POD_NAMESPACE)/tcp-services'
            - '--udp-services-configmap=$(POD_NAMESPACE)/udp-services'
            - '--annotations-prefix=nginx.ingress.kubernetes.io'
          ports:
            - name: http
              hostPort: 80
              containerPort: 80
              protocol: TCP
            - name: https
              hostPort: 443
              containerPort: 443
              protocol: TCP
            - name: metrics
              hostPort: 10254
              containerPort: 10254
              protocol: TCP
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          resources: {}
          livenessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            timeoutSeconds: 5
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            timeoutSeconds: 5
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            capabilities:
              add:
                - NET_BIND_SERVICE
              drop:
                - ALL
            runAsUser: 101
            allowPrivilegeEscalation: true
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      serviceAccount: ingress-nginx
      securityContext: {}
      schedulerName: default-scheduler
      priorityClassName: k8s-cluster-critical
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  revisionHistoryLimit: 10

我还让 MetalLB 分发一组预定义的 IP，我可以路由到 (10.1.8.150-199)。

我创建了一个入口点：

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller-internal
  namespace: ingress-nginx
spec:
  type: LoadBalancer
  loadBalancerIP: 10.1.8.150
  externalTrafficPolicy: Cluster
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx


apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller-admission-internal
  namespace: ingress-nginx
spec:
  type: ClusterIP
  ports:
    - name: https-webhook
      port: 443
      targetPort: webhook
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

将此入口点与测试 hello world 部署一起使用非常完美。

我为 hello world 创建了一个单独的命名空间，名为“hello-world-imranh”。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world-deployment
  namespace: hello-world-imranh
spec:
  selector:
    matchLabels:
      app: hello-world-debug
  replicas: 3
  template:
    metadata:
      labels:
        app: hello-world-debug
    spec:
      containers:
      - name: hello-world-debug
        image: "imranh2/hello-world-debug"
        env:
        - name: "PORT"
          value: "3000"

---
apiVersion: v1
kind: Service
metadata:
  name: hello-world-service
namespace: hello-world-imranh
spec:
  type: ClusterIP
  selector:
    app: hello-world-debug
  ports:
  - protocol: TCP
    port: 8008
    targetPort: 3000

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hello-world-ingress
  namespace: hello-world-imranh
  annotations: 
     nginx.ingress.kubernetes.io/rewrite-target: /
     nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
  - http:
      paths:
      - path: /abc
        backend:
          serviceName: hello-world-service
          servicePort: 8008

结果：

kubectl get svc -n hello-world-imranh

NAME                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
hello-world-service   ClusterIP   10.144.211.196   <none>        8008/TCP   102m

kubectl get svc -n ingress-nginx

NAME                                          TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller-admission-internal   ClusterIP      10.144.234.30   <none>        443/TCP                      102m
ingress-nginx-controller-internal             LoadBalancer   10.144.195.51   10.1.8.150    80:31334/TCP,443:31575/TCP   102m

我现在可以访问 https://10.1.8.150/abc（尽管在 3 次 diff SSL 异常点击后，如果在 GUI 中完成的很奇怪）并且 MetalLB 甚至似乎正在运行，正如不同容器响应每个 curl 所见证的那样：

[root@ARCHITECT-PC ~]# curl -k https://10.1.8.150/abc
Hello,world!

Container Hostname: hello-world-deployment-6d668b96d5-pngjv
Container Network Info: {"lo":[{"address":"127.0.0.1","netmask":"255.0.0.0","family":"IPv4","mac":"00:00:00:00:00:00","internal":true,"cidr":"127.0.0.1/8"},{"address":"::1","netmask":"ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff","family":"IPv6","scopeid":0,"cidr":"::1/128"}],"eth0":[{"address":"10.244.36.51","netmask":"255.255.255.255","mac":"2a:ff:e9:e3:61:fd","internal":false,"cidr":"10.244.36.51/32"},{"address":"fe80::28ff:e9ff:fee3:61fd","netmask":"ffff:ffff:ffff:ffff::","scopeid":4,"cidr":"fe80::28ff:e9ff:fee3:61fd/64"}]}
Container IP Guess: "10.244.36.51"

Client connection IP: 10.244.173.35
Client socket IP: 10.244.173.35
Client connection.socket.remoteAddress IP: connect.socket unset
x-forwarded-for: Not set
host: 10.1.8.150
[root@ARCHITECT-PC ~]# curl -k https://10.1.8.150/abc
Hello,world!

Container Hostname: hello-world-deployment-6d668b96d5-kzvsg
Container Network Info: {"lo":[{"address":"127.0.0.1","eth0":[{"address":"10.244.173.34","mac":"7e:5c:1e:52:21:f7","cidr":"10.244.173.34/32"},{"address":"fe80::7c5c:1eff:fe52:21f7","scopeid":3,"cidr":"fe80::7c5c:1eff:fe52:21f7/64"}]}
Container IP Guess: "10.244.173.34"

Client connection IP: 10.244.136.29
Client socket IP: 10.244.136.29
Client connection.socket.remoteAddress IP: connect.socket unset
x-forwarded-for: Not set
host: 10.1.8.150

我现在想做的是公开另一个 hello-world 路由，但是在另一个 IP 上。换句话说，我试图模仿有两个不同的入口 IP。

我尝试了两种不同的方法：

  1. 创建另一个 nginx-controller 服务定义以获取我所需范围内的另一个 IP（有效）。以及第二个入口定义，但路径如 /zyc。但我不确定如何确保我的 hello-world 入口只听它。我最终得到的两个 IP 都为我的所有路径服务。

  2. 解决 #1，我复制了 DaemonSet kubespray 给我并包含 --watch-namespace 和带注释的 ingress.class。这还包括带有这些注释的第二个 nginx 控制器和 service/ingress hello world 定义。虽然此 DaemonSet 部署成功，但一旦出现 - 我上面创建的测试路由只有 50% 的时间有效。好像现在 DaemonSet 正在共享？

这是第二个 DaemonSet 配置：

kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: ingress-nginx-controller-public
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    ingress-class: nginx-external
  annotations: 
     nginx.ingress.kubernetes.io/ingress.class: "nginx-public"
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      containers:
        - name: ingress-nginx-controller-public
          image: k8s.gcr.io/ingress-nginx/controller:v0.43.0
          args:
            - /nginx-ingress-controller
            - '--ingress-class=nginx-public'
            - '--watch-namespace=hello-world-imranh'
            - '--configmap=$(POD_NAMESPACE)/ingress-nginx'
            - '--tcp-services-configmap=$(POD_NAMESPACE)/tcp-services'
            - '--udp-services-configmap=$(POD_NAMESPACE)/udp-services'
            - '--annotations-prefix=nginx.ingress.kubernetes.io'
          ports:
            - name: http
              hostPort: 8080
              containerPort: 80
              protocol: TCP
            - name: https
              hostPort: 4433
              containerPort: 443
              protocol: TCP
            - name: metrics
              hostPort: 10255
              containerPort: 10254
              protocol: TCP
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          resources: {}
          livenessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            timeoutSeconds: 5
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            timeoutSeconds: 5
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            capabilities:
              add:
                - NET_BIND_SERVICE
              drop:
                - ALL
            runAsUser: 101
            allowPrivilegeEscalation: true
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      serviceAccount: ingress-nginx
      securityContext: {}
      schedulerName: default-scheduler
      priorityClassName: k8s-cluster-critical
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  revisionHistoryLimit: 10

我是 kubernetes n00b，恳请您的帮助。

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）

kuberneteskubernetes-ingresskubespraynginx-ingress

相关问答

matplotlib报错：AttributeError: module 'backend_interagg' has no attribute 'FigureCanvas'. Did you mean: 'FigureCanvasAgg'?
使用本地python环境可以成功执行 import pandas as pd impor...
gitlab登录失败，报错：This challenge page was accidentally cached by an intermediary and is no longer available.
设置时间 控制面板
后端开发常见错误
错误1：Request method ‘DELETE‘ not supported 错误还原：...
docker常见错误
错误1：启动docker镜像时报错：Error response from daemon:...
idea常见错误
错误1：private field ‘xxx‘ is never assigned 按Alt...
pip安装依赖失败
报错如下，通过源不能下载，最后警告pip需升级版本 Requirem...