Kubernetes：启用代理协议时Tomcat抛出异常

问题描述

我现在有点迷路。我建立了一个Kubernetes集群，部署了一个Spring Boot API和一个LoadBalancer，它们运行良好。现在，我想在LoadBalancer上启用代理协议以保留实际的客户端IP，但是一旦执行此操作，我的Spring Boot API总是返回400 Bad Request并抛出IllegalArgumentException。

这是简短的堆栈跟踪（我屏蔽了IP地址）：

2020-09-29 20:05:58.382  INFO 1 --- [nio-8080-exec-1] o.apache.coyote.http11.Http11Processor   : Error parsing HTTP request header
 Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.

java.lang.IllegalArgumentException: Invalid character found in the HTTP protocol [255.255.255.253 255.255.255.254]
        at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:560) ~[tomcat-embed-core-9.0.37.jar!/:9.0.37]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:260) ~[tomcat-embed-core-9.0.37.jar!/:9.0.37]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat-embed-core-9.0.37.jar!/:9.0.37]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) ~[tomcat-embed-core-9.0.37.jar!/:9.0.37]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589) ~[tomcat-embed-core-9.0.37.jar!/:9.0.37]
        at org.apache.tomcat.util.net.socketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat-embed-core-9.0.37.jar!/:9.0.37]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(UnkNown Source) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(UnkNown Source) ~[na:na]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-embed-core-9.0.37.jar!/:9.0.37]
        at java.base/java.lang.Thread.run(UnkNown Source) ~[na:na]

我正在使用Hetzner的hcloud-cloud-controller-manager。

这是我的负载均衡器：

apiVersion: v1
kind: Service
Metadata:
  labels:
    service: auth-service
  name: auth-service-service
  annotations:
    load-balancer.hetzner.cloud/name: "lb-backend"
    load-balancer.hetzner.cloud/health-check-port: "80"
    load-balancer.hetzner.cloud/uses-proxyprotocol: "true"
spec:
  ports:
    - name: http
      port: 80
      targetPort: 8080
  selector:
    service: auth-service
  externalTrafficPolicy: Local
  type: LoadBalancer

这是我的Spring Config：

spring:
  datasource:
    platform: postgres
    url: ${DATABASE_CS}
    username: ${DATABASE_USERNAME}
    password: ${DATABASE_PASSWORD}
    driver-class-name: org.postgresql.Driver
  flyway:
    schemas: authservice
  jpa:
    show-sql: false
    properties:
      hibernate:
        dialect: org.hibernate.dialect.PostgresqlDialect
        jdbc:
          lob:
            non_contextual_creation: true
    hibernate:
      ddl-auto: validate

security:
  jwt:
    secret-key: ${JWT_SECRET_KEY}
    expires: ${JWT_EXPIRES:300000}

mail:
  from: ${MAIL_FROM}
  fromName: ${MAIL_FROM_NAME}
  smtp:
    host: ${SMTP_HOST}
    username: ${SMTP_USERNAME}
    password: ${SMTP_PASSWORD}
    port: ${SMTP_PORT:25}
  mjml:
    app-id: ${MJML_APP_ID}
    app-secret: ${MJML_SECRET_KEY}
stripe:
  keys:
    secret: ${STRIPE_SECRET_KEY}
    public: ${STRIPE_PUBLIC_KEY}
server:
  forward-headers-strategy: native

您可能已经注意到，我已经尝试基于this issue启用转发标题。

感谢您的帮助！

解决方法

您启用了PROXY协议，该协议不是HTTP，而是另一种协议，用于将TCP连接隧穿到下游服务器，从而保留了尽可能多的信息。 https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt

我很确定您要禁用此功能 load-balancer.hetzner.cloud/uses-proxyprotocol: "true" 而是依靠远程地址的转发头为客户端提供正确的值。

说实话，我不知道tomcat支持PROXY协议。 （编辑：当前不是，请参见https://bz.apache.org/bugzilla/show_bug.cgi?id=57830）