无法在Cloud Foundry中创建共享域-504网关超时

编程问答 2022-06-05

问题描述

我无法在Cloud Foundry中创建共享域，任何推送的应用程序的运行状况检查连接均被拒绝。

我有一个基于OpenStack IaaS的Cloud Foundry环境。一切都按预期进行。我获取了部署文件，并在一段时间后将其成功部署到IaaS Vmware VSphere 7中。问题是，我推送的每个应用程序都存在运行状况检查问题：

   2020-10-29T16:55:01.43+0000 [CELL/0] OUT Cell 938b869c-5a68-40cc-9486-c5bc0d53a73a successfully destroyed container for instance 44e9c2a6-b54d-4fc4-4118-6d6d
   2020-10-29T16:55:36.55+0000 [CELL/0] OUT Cell 938b869c-5a68-40cc-9486-c5bc0d53a73a creating container for instance 17f161a2-9788-426d-414d-6c33
   2020-10-29T16:55:37.18+0000 [CELL/0] OUT Cell 938b869c-5a68-40cc-9486-c5bc0d53a73a successfully created container for instance 17f161a2-9788-426d-414d-6c33
   2020-10-29T16:55:37.47+0000 [CELL/0] OUT Downloading droplet...
   2020-10-29T16:55:37.75+0000 [CELL/0] OUT Downloaded droplet
   2020-10-29T16:55:37.75+0000 [CELL/0] OUT Starting health monitoring of container
   2020-10-29T16:56:38.45+0000 [HEALTH/0] ERR Failed to make TCP connection to port 8080: connection refused
   2020-10-29T16:56:38.45+0000 [CELL/0] ERR Timed out after 1m0s: health check never passed.
   2020-10-29T16:56:38.46+0000 [CELL/SSHD/0] OUT Exit status 0
   2020-10-29T16:56:38.48+0000 [APP/PROC/WEB/0] OUT Exit status 143

我也无法创建任何共享域：

bash-5.0# cf create-shared-domain tcp.cf.test-env.net --router-group default-tcp -v
REQUEST: [2020-10-29T17:03:33Z]
GET /v2/info HTTP/1.1
Host: api.cf.test-env.net
Accept: application/json
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)

RESPONSE: [2020-10-29T17:03:33Z]
HTTP/1.1 200 OK
Content-Length: 561
Content-Type: application/json;charset=utf-8
Date: Thu,29 Oct 2020 17:03:33 GMT
Server: Nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 4badb79b-2faf-4623-6c3c-ce5fa3223cd5::dc43d2c9-c902-4429-9d65-d9a0060983c5
{
  "api_version": "2.144.0","app_ssh_endpoint": "ssh.cf.test-env.net:2222","app_ssh_host_key_fingerprint": "ae:a3:ed:ad:37:d3:8a:7b:ed:b4:e5:d2:25:e5:8c:d0","app_ssh_oauth_client": "ssh-proxy","authorization_endpoint": "https://login.cf.test-env.net","build": "","description": "","doppler_logging_endpoint": "wss://doppler.cf.test-env.net:443","min_cli_version": null,"min_recommended_cli_version": null,"name": "","osbapi_version": "2.15","routing_endpoint": "https://api.cf.test-env.net/routing","support": "","token_endpoint": "https://uaa.cf.test-env.net","version": 0
}


REQUEST: [2020-10-29T17:03:33Z]
GET /login HTTP/1.1
Host: login.cf.test-env.net
Accept: application/json
Connection: close
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)

RESPONSE: [2020-10-29T17:03:34Z]
HTTP/1.1 200 OK
Cache-Control: no-store
Content-Language: en-US
Content-Length: 384
Content-Type: application/json;charset=UTF-8
Date: Thu,29 Oct 2020 17:03:34 GMT
Set-Cookie: X-Uaa-Csrf=NJlSPAjspn7m8oWuQdKsVD; Max-Age=86400; Expires=Fri,30-Oct-2020 17:03:34 GMT; Path=/; Secure; HttpOnly
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
x-frame-options: DENY
X-Vcap-Request-Id: 577d4d31-ec30-477e-6f44-c0dd9306270d
X-Xss-Protection: 1; mode=block
{
  "app": {
    "version": "74.12.0"
  },"commit_id": "7311e68","entityID": "login.cf.test-env.net","idpDeFinitions": {},"links": {
    "login": "https://login.cf.test-env.net","passwd": "/forgot_password","register": "/create_account","uaa": "https://uaa.cf.test-env.net"
  },"prompts": {
    "password": "[PRIVATE DATA HIDDEN]","username": [
      "text","Email"
    ]
  },"timestamp": "2019-12-02T22:53:03+0000","zone_name": "uaa"
}


Creating shared domain tcp.cf.test-env.net as admin...
REQUEST: [2020-10-29T17:03:34Z]
GET /routing/v1/router_groups?name=default-tcp HTTP/1.1
Host: api.cf.test-env.net
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/json
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)
[application/json Content Hidden]

RESPONSE: [2020-10-29T17:03:34Z]
HTTP/1.1 200 OK
Content-Length: 114
Content-Type: application/json
Date: Thu,29 Oct 2020 17:03:34 GMT
X-Vcap-Request-Id: 9459b068-0987-4f5e-7dee-1efdb5ca6fb8
[
  {
    "guid": "343ba1e8-88a7-4003-6db6-4feabedd072b","name": "default-tcp","reservable_ports": "1024-2048","type": "tcp"
  }
]


REQUEST: [2020-10-29T17:03:34Z]
POST /v2/shared_domains HTTP/1.1
Host: api.cf.test-env.net
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/json
User-Agent: cf/6.47.2+d526c2cb3.2019-11-05 (go1.12.12; amd64 linux)
{
  "internal": false,"name": "tcp.cf.test-env.net","router_group_guid": "343ba1e8-88a7-4003-6db6-4feabedd072b"
}


RESPONSE: [2020-10-29T17:04:04Z]
HTTP/1.0 504 Gateway Time-out
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>


Error unmarshalling the following into a cloud controller error: <html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>

Failed

我怀疑网络配置问题，该问题阻止了某些内部CF部件的连接。在VMware中找不到任何防火墙或任何规则。我还可以ping并在bosh创建的VM之间建立ssh连接。

任何想法，我还能做什么？

解决方法

问题出在VmWare NSX-T上的DNAT和SNAT规则上。如果任何内部VM询问dns名称“ api.cf.test-env.net”，它将获得远程（公用）IP地址作为答案。建立连接时，内部VM一直在通过公用IP地址询问api.cf.test-env.net，并在第二阶段进行TCP三向握手获取本地-这是TCP RST产生的原因。正确创建DNAT和SNAT规则后，一切正常。我仍然想知道为什么带有内部地址的bosh-dns无法回答“ api.cf.test-env.net”。有谁知道为什么这么做以及如何更改？

cf-boshcloudfoundryopenstackopenstackvmware