socket.gaierror: [Errno -3] 名称解析暂时失败

5

我正在使用Python requests库请求一个带有这种代码的API:

api_request = requests.get(f"http://data.api.org/search?q=example&ontologies=BFO&roots_only=true",
                             headers={'Authorization': 'apikey token=' + 'be03c61f-2ab8'})

api_result = api_request.json()
collection = api_result["collection"]
...

这段代码在我不请求大量内容时运行良好,但否则会出现错误。奇怪的是,并非每次请求大量内容时都会出现错误。错误消息如下:

Traceback (most recent call last):
  File "/home/nobu/.local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/home/nobu/.local/lib/python3.6/site-packages/urllib3/util/connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nobu/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/nobu/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/nobu/.local/lib/python3.6/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/home/nobu/.local/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f4bdeca7080>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nobu/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/nobu/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/nobu/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='data.api.org', port=80): Max retries exceeded with url: /ontologies/NCIT/classes/http%3A%2F%2Fncicb.nci.nih.gov%2Fxml%2Fowl%2FEVS%2FThesaurus.owl%23C48481/descendants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4bdeca7080>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "new_format.py", line 181, in <module>
    ontology_api(extraction(90))
  File "new_format.py", line 142, in ontology_api
    concept_extraction(collection)
  File "new_format.py", line 100, in concept_extraction
    api_request_tree = requests.get(f"{leaf}", headers={'Authorization': 'apikey token=' + f'{api_key}'})
  File "/home/nobu/.local/lib/python3.6/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/home/nobu/.local/lib/python3.6/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/nobu/.local/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/nobu/.local/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/home/nobu/.local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='data.api.org', port=80): Max retries exceeded with url: /ontologies/NCIT/classes/http%3A%2F%2Fncicb.nci.nih.gov%2Fxml%2Fowl%2FEVS%2FThesaurus.owl%23C48481/descendants (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4bdeca7080>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

我不知道是否因为我过度请求API而导致错误,或者是由于其他原因。我无法在SO或其他地方找到解决我的问题的答案。

提前感谢您的时间和关注。


我也遇到了这个错误,虽然不经常,但发生的频率仍然太高以至于不能忽视。你有取得任何进展吗?你在哪个服务器/主机上遇到了这个问题? - tobltobs
3
是的 @tobltobs 我找到了一个解决方法。我使用了 requests 库的 Session() 方法来增加最大重试次数,如下所示:session = requests.Session() 然后 session.mount('http://', requests.adapters.HTTPAdapter(max_retries=100)) - Takamura
@Takamura,你有没有考虑把你的评论变成一个答案? - questionto42
@questionto42standswithUkraine 我所评论的已经在lam vu Nguyen的答案中提到了。 - Takamura
2个回答

2
with requests.Session() as s:
    s.get('http://google.com')

或者

with requests.get('http://httpbin.org/get', stream=True) as r:
    # Do something

这是另一种方法

Python-Requests关闭HTTP连接

但感谢 session.mount('http://', requests.adapters.HTTPAdapter(max_retries=100))


0

查询返回的空值或错误输出可能导致错误

当我使用正确的参数在一个容器中运行代码时,所涉及的错误消失了。之前,在其中一个参数中有一个不在我运行代码的数据集中的id。这意味着:一个查询尝试在数据集中查找一个id,没有找到任何东西,然后没有输出,然后这个旧容器报告:

/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/properties.py:1029: SAWarning: On Product.product_attribute,'passive_deletes' is normally configured on one-to-many, one-to-one, many-to-many relationships only. self._check_cascade_settings(self._cascade) Traceback (most recent call last): File "/usr/lib/python2.7/logging/handlers.py", line 556, in emit self.send(s) File "/usr/local/lib/python2.7/dist-packages/graypy/handler.py", line 37, in send DatagramHandler.send(self, s) File "/usr/lib/python2.7/logging/handlers.py", line 607, in send self.sock.sendto(s, (self.host, self.port)) gaierror: [Errno -3] Temporary failure in name resolution

并且在运行结束时进一步向下:

    raise e
AutoReconnect: connection closed

看起来 sqlalchemy 无法处理查询的 None 输出,这会关闭连接,并且它会一遍又一遍地尝试连接,直到在 x 次尝试后连接被关闭。

其他可能有用的调试步骤

我是一个初学者,不要完全相信我的答案。但我还是敢说一下。

"名称解析中的临时故障" 意味着您无法访问网络中的服务器,无论是您的主机、DNS、登录位置还是云端。

  • 第一步是 ping 您的代码尝试访问的每个服务器以及您网络的 DNS,以查看名称是否有效。
  • 您可能忘记了仍在运行的容器,它会更改您的网络 (docker ps) 或通过其网络流量干扰模块。

但如果它只在某些时间发生,并且在大部分相同的数据负载中开关式地切换,您可以尝试以简单的方式进行调试:

  • 如果在错误发生时能够访问所有服务器,请进行异常处理并记录。
  • 通过在代码中注释掉日志记录来关闭日志记录。测试代码更长的时间,看看是否会出现错误。日志记录会导致网络流量增加。对于API调用也是一样,但我猜问题可能是一个无法处理竞争条件的Python模块,在API调用峰值时会被记录下来。

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接