Python,捕获流请求超时

4
我正在使用请求库读取XML事件,如下所示的代码。一旦请求开始,我该如何引发连接丢失错误?服务器模拟HTTP推送/长轮询-> http://en.wikipedia.org/wiki/Push_technology#Long_polling,默认情况下不会结束。如果10分钟后没有新消息,则应退出while循环。
import requests
from time import time


if __name__ == '__main__':
    #: Set a default content-length
    content_length = 512
    try:
        requests_stream = requests.get('http://agent.mtconnect.org:80/sample?interval=0', stream=True, timeout=2)
        while True:
            start_time = time()
            #: Read three lines to determine the content-length         
            for line in requests_stream.iter_lines(3, decode_unicode=None):
                if line.startswith('Content-length'):
                    content_length = int(''.join(x for x in line if x.isdigit()))
                    #: pause the generator
                    break

            #: Continue the generator and read the exact amount of the body.        
            for xml in requests_stream.iter_content(content_length):
                print "Received XML document with content length of %s in %s seconds" % (len(xml), time() - start_time)
                break

    except requests.exceptions.RequestException as e:
        print('error: ', e)

服务器推送可以通过命令行使用curl进行测试:
curl http://agent.mtconnect.org:80/sample\?interval\=0
1个回答

0

这可能不是最好的方法,但您可以使用多进程来在单独的进程中运行请求。 像这样的东西应该可以工作:

import multiprocessing
import requests
import time

class RequestClient(multiprocessing.Process):
    def run(self):
        # Write all your code to process the requests here
        content_length = 512
        try:
            requests_stream = requests.get('http://agent.mtconnect.org:80/sample?interval=0', stream=True, timeout=2)

            start_time = time.time()
            for line in requests_stream.iter_lines(3, decode_unicode=None):
                if line.startswith('Content-length'):
                    content_length = int(''.join(x for x in line if x.isdigit()))
                    break

            for xml in requests_stream.iter_content(content_length):
                print "Received XML document with content length of %s in %s seconds" % (len(xml), time.time() - start_time) 
                break
        except requests.exceptions.RequestException as e:
            print('error: ', e)


While True:
    childProcess = RequestClient()
    childProcess.start()

    # Wait for 10mins
    start_time = time.time()
    while time.time() - start_time <= 600:
        # Check if the process is still active
        if not childProcess.is_alive():
            # Request completed
            break
        time.sleep(5)    # Give the system some breathing time

    # Check if the process is still active after 10mins.
    if childProcess.is_alive():
        # Shutdown the process
        childProcess.terminate()
        raise RuntimeError("Connection Timed-out")

这不是解决您问题的完美代码,但您可以理解其中的思路。


哦,看起来它还可以运行。不过,我每5秒钟只收到一个XML消息。我需要尽可能快地获得它们 ;) - wiesson
5秒的睡眠实际上并不会挂起子进程,它只是让主线程休眠。XML消息应该在子进程接收到它后立即处理。很有可能是服务器或requests模块添加了这5秒钟的延迟。 - Sunny Nanda
如果它起作用了,你可以继续并接受这个答案 :) - Sunny Nanda
它确实起作用了。但是拥有更多的线程和进程并不是理想的解决方案。我认为在循环中有一个可以使用的方法/函数。 :) - wiesson
1
如果 requests_stream.iter_lines 是一个阻塞调用(很可能是这样),那么没有其他方法可以做到,因为在等待数据时,循环中的超时函数不会被调用。 - Sunny Nanda

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接