我正在使用urllib2通过POST方法将一个相当大的文件上传到服务器端脚本。我希望显示一个进度指示器,以显示当前的上传进度。是否有urllib2提供的钩子或回调函数可用于监视上传进度?我知道可以使用连接的read()方法连续调用来下载并实现此功能,但我没有看到write()方法,只是将数据添加到请求中。
可以做到,但需要执行以下几个步骤:
__len__
属性,使len(data)
返回正确的大小,用于填充Content-Length标头。read()
方法:当httplib调用read()
时,您的回调函数将被调用,从而允许您计算百分比并更新进度条。这可以使用任何类似文件的对象实现,但我已经包装了file
以显示如何将其应用于从磁盘流式传输的非常大的文件:
import os, urllib2
from cStringIO import StringIO
class Progress(object):
def __init__(self):
self._seen = 0.0
def update(self, total, size, name):
self._seen += size
pct = (self._seen / total) * 100.0
print '%s progress: %.2f' % (name, pct)
class file_with_callback(file):
def __init__(self, path, mode, callback, *args):
file.__init__(self, path, mode)
self.seek(0, os.SEEK_END)
self._total = self.tell()
self.seek(0)
self._callback = callback
self._args = args
def __len__(self):
return self._total
def read(self, size):
data = file.read(self, size)
self._callback(self._total, len(data), *self._args)
return data
path = 'large_file.txt'
progress = Progress()
stream = file_with_callback(path, 'rb', progress.update, path)
req = urllib2.Request(url, stream)
res = urllib2.urlopen(req)
输出:
large_file.txt progress: 0.68
large_file.txt progress: 1.36
large_file.txt progress: 2.04
large_file.txt progress: 2.72
large_file.txt progress: 3.40
...
large_file.txt progress: 99.20
large_file.txt progress: 99.87
large_file.txt progress: 100.00
我认为这是不可能的,但是pycurl确实有上传/下载进度回调函数可以使用。
poster 支持此操作
import json
import os
import sys
import urllib2
from poster.encode import multipart_encode
from poster.streaminghttp import register_openers
def _upload_progress(param, current, total):
sys.stdout.write(
"\r{} - {:.0f}% "
.format(param.name,
(float(current) / float(total)) * 100.0))
sys.stdout.flush()
def upload(request_resource, large_file_path):
register_openers()
with open(large_file_path, 'r') as large_file:
request_data, request_headers = multipart_encode(
[('file', largs_file)],
cb=_upload_progress)
request_headers.update({
'X-HockeyAppToken': 'we use this for hockeyapp upload'
})
upload_request = urllib2.Request(request_resource,
request_data,
request_headers)
upload_connection = urllib2.urlopen(upload_request)
upload_response = json.load(upload_connection)
print "Done"
urllib2
的AbstractHTTPHandler.do_request_()
和httplib HttpConnect._send_request()
中,其中调用len(<file obj>)
来设置Content-length
头。 - samplebias