从Docker容器300秒后，Python HTTP GET引发异常

问题描述

我正在尝试使用Docker容器中的Python从外部服务获取数据。 我需要达到两个不同的端点，比如：

endpointA = "https://myhost:1234/resA" # Requests take about 1 minute to complete
endpointB = "https://myhost:1234/resB" # Requests take about 8 minutes to complete

我尝试使用两个不同的库Requests：

import requests
headers = {'Authorization': 'Basic XXXXXXXXXXXXXXXXXXXXXX','Content-Type': 'application/json'}
response = requests.get(endpointA,verify=False,headers=headers)

和Pycurl：

import pycurl

headers = ['Authorization: Basic XXXXXXXXXXXXXXXXXXXXXX','Content-Type: application/json']

buffer = BytesIO()
c = pycurl.Curl()
c.setopt(c.VERBOSE,True)
c.setopt(c.URL,endpointA)
c.setopt(c.WRITEDATA,buffer)
c.setopt(c.SSL_VERIFYPEER,False)
c.setopt(c.SSL_VERIFYHOST,False)
c.setopt(c.HTTPHEADER,headers)
c.perform()
c.close()

我的代码在本地运行时没有问题（端点A和端点B都返回）。

在Docker容器中运行时，对终结点计算机A的请求可以工作，但对终结点计算机B的请求会在300秒后发生异常：

对于请求：

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py",line 670,in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py",line 426,in _make_request
    six.raise_from(e,None)
  File "<string>",line 3,in raise_from
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py",line 421,in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.8/http/client.py",line 1347,in getresponse
    response.begin()
  File "/usr/local/lib/python3.8/http/client.py",line 307,in begin
    version,status,reason = self._read_status()
  File "/usr/local/lib/python3.8/http/client.py",line 276,in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception,another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py",line 439,in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py",line 726,in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py",line 403,in increment
    raise six.reraise(type(error),error,_stacktrace)
  File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py",line 734,in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py",in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.',RemoteDisconnected('Remote end closed connection without response'))

对于pycurl：

Traceback (most recent call last):
  File "test.py",line 230,in get_data
    c.perform()
pycurl.error: (52,'Empty reply from server')

这仅在Docker上发生，因为在我的控制台上一切正常。

我尝试了python：3.6和python：3.8 Docker镜像。

我还尝试从容器内部对两个端点进行cURL传递，并且服务器正确答复。

解决方法

您可以通过使用容器外部的nc -l和容器中的nc来验证docker是否正在终止连接，以连接到第一个nc，然后将它们两个放置10分钟，然后发送一些内容确认连接仍然正常。

要对pycurl进行故障排除，请参见http://pycurl.io/docs/latest/troubleshooting.html#transfer-related-issues。