使用aiohttp进行代理请求

Question

我有以下代码。

#example.py

import asyncio
import aiohttp

async def foo():
    conn = aiohttp.ProxyConnector(proxy='http://222.222.251.185:9797')
    r = await aiohttp.get('https://www.python.org/', connector=conn)
    return await r.read()

async def main():
    tasks = []
    for i in range(500):
    tasks.append(foo())
    for r in asyncio.as_completed(tasks):
        print(await r)

parsed_data = asyncio.get_event_loop().run_until_complete(main())

如果我不使用代理，一切正常。但是当我使用代理时，程序会崩溃，并显示以下跟踪信息：

      Traceback (most recent call last):
  File "/home/vika/Work/example.py", line 19, in <module>
    parsed_data = asyncio.get_event_loop().run_until_complete(main())
  File "/usr/lib/python3.5/asyncio/base_events.py", line 337, in run_until_complete
    return future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/home/vika/Work/example.py", line 16, in main
    print(await r)
  File "/usr/lib/python3.5/asyncio/tasks.py", line 486, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
    result = coro.throw(exc)
  File "/home/vika/Work/example.py", line 7, in foo
    r = await aiohttp.get('https://www.python.org/', connector=conn)
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/client.py", line 552, in __await__
    return (yield from self._coro)
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/client.py", line 174, in _request
    conn = yield from self._connector.connect(req)
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/connector.py", line 307, in connect
    transport, proto = yield from self._create_connection(req)
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/connector.py", line 683, in _create_connection
    resp = yield from proxy_resp.start(conn, True)
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/client_reqrep.py", line 597, in start
    message = yield from httpstream.read()
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/streams.py", line 578, in read
    result = yield from super().read()
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/streams.py", line 433, in read
    yield from self._waiter
  File "/usr/lib/python3.5/asyncio/futures.py", line 358, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.5/asyncio/tasks.py", line 290, in _wakeup
    future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
aiohttp.errors.ServerDisconnectedError
Exception ignored in: <coroutine object foo at 0x7fcf51566518>
Traceback (most recent call last):
  File "/home/vika/Work/example.py", line 7, in foo
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/client.py", line 552, in __await__
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/client.py", line 174, in _request
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/connector.py", line 307, in connect
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/connector.py", line 686, in _create_connection
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/connector.py", line 84, in close
  File "/opt/python_custom/lib/python3.5/site-packages/aiohttp/connector.py", line 375, in _release
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 569, in close
  File "/usr/lib/python3.5/asyncio/base_events.py", line 447, in call_soon
  File "/usr/lib/python3.5/asyncio/base_events.py", line 456, in _call_soon
  File "/usr/lib/python3.5/asyncio/base_events.py", line 284, in _check_closed
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object foo at 0x7fcf515664c0>
...
Task was destroyed but it is pending!
task: <Task pending coro=<foo() done, defined at /home/vika/Work/example.py:5> wait_for=<Future pending cb=[Task._wakeup()]> cb=[as_completed.<locals>._on_completion() at /usr/lib/python3.5/asyncio/tasks.py:472]>
Task was destroyed but it is pending!
task: <Task pending coro=<foo() done, defined at /home/vika/Work/example.py:5> wait_for=<Future pending cb=[Task._wakeup()]> cb=[as_completed.<locals>._on_completion() at /usr/lib/python3.5/asyncio/tasks.py:472]>
Task was destroyed but it is pending!
task: <Task pending coro=<foo() done, defined at /home/vika/Work/example.py:5> wait_for=<Future pending cb=[Task._wakeup()]> cb=[as_completed.<locals>._on_completion() at /usr/lib/python3.5/asyncio/tasks.py:472]>
Task was destroyed but it is pending!
task: <Task pending coro=<foo() done, defined at /home/vika/Work/example.py:5> wait_for=<Future pending cb=[Task._wakeup()]> cb=[as_completed.<locals>._on_completion() at /usr/lib/python3.5/asyncio/tasks.py:472]>

可能出现了什么问题？我使用了不同的代理，因此问题必须在代码逻辑中。

- v18o

---

当连接到代理时，您会遇到 aiohttp.errors.ServerDisconnectedError 错误。您确定您有可用的代理吗？首先尝试在浏览器中设置代理以查看其是否可靠。 - JBernardo

@JBernardo 没错！谢谢。 - v18o

Answer 1

代理出了问题。不同的代理会导致不同的错误，因此很难找到一个好的代理。上面的代码是完全有效的（但请更改代理！）。

- v18o