使用cPickle将大数组写入网络驱动器时出现Python“IOError：[Errno 22]无效参数”错误

Question

使用cPickle将大数组写入网络驱动器时出现Python“IOError：[Errno 22]无效参数”错误

6

编辑：根据J.F.Sebastian的建议，我可以更简单地获得相同的错误：

Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]
Type "copyright", "credits" or "license" for more information.

IPython 0.10 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

  Welcome to pylab, a matplotlib-based Python environment.
  For more information, type 'help(pylab)'.

In [1]: open(r'c:\test.bin', 'wb').write('a'*67076095)

In [2]: open(r'c:\test.bin', 'wb').write('a'*67076096)

In [3]: open(r'z:\test.bin', 'wb').write('a'*67076095)

In [4]: open(r'z:\test.bin', 'wb').write('a'*67076096)
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)

C:\Documents and Settings\User\<ipython console> in <module>()

IOError: [Errno 22] Invalid argument

In [5]:

请注意，C：是本地驱动器，Z：是网络驱动器。

原始问题：

如果我使用cPickle将文件写入大于约67 MB的网络驱动器（ReadyNAS Pro Pioneer版），则Windows XP上的Python 2.6.4会崩溃。我希望能够pickle大型文件。这是已知的问题吗？是否有解决方法？

以下脚本会导致崩溃：

import cPickle, numpy

a = numpy.zeros(8385007)
print "Writing %i bytes..."%(a.nbytes)
cPickle.dump(a, open('test_a.pkl', 'wb'), protocol=2)
print "Successfully written."

b = numpy.zeros(8385008)
print "Writing %i bytes..."%(b.nbytes)
cPickle.dump(b, open('test_b.pkl', 'wb'), protocol=2) ##Crashes on a network drive
print "Successfully written." ##Doesn't crash on a non-network drive

以下是我在ipython提示符下制造崩溃的步骤：

Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]
Type "copyright", "credits" or "license" for more information.

IPython 0.10 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

  Welcome to pylab, a matplotlib-based Python environment.
  For more information, type 'help(pylab)'.

In [1]: pwd
Out[1]: 'C:\\Documents and Settings\\User'

In [2]: run test
Writing 67080056 bytes...
Successfully written.
Writing 67080064 bytes...
Successfully written.

In [3]: cd Z:
Z:\

In [4]: pwd
Out[4]: 'Z:\\'

In [5]: run 'C:\\Documents and Settings\\User\\test'
Writing 67080056 bytes...
Successfully written.
Writing 67080064 bytes...
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)

C:\Documents and Settings\User\test.py in <module>()
      8 b = numpy.zeros(8385008)
      9 print "Writing %i bytes..."%(b.nbytes)
---> 10 cPickle.dump(b, open('test_b.pkl', 'wb'), protocol=2)
     11 print "Successfully written."
     12

IOError: [Errno 22] Invalid argument
WARNING: Failure executing file: <C:\\Documents and Settings\\User\\test.py>

In [6]:

C: 是计算机上的本地硬盘。Z: 是我们连接在网络上的存储设备。

- Andrew

注意：numpy数组可以通过a.dump自行封存。 - jfs

1

open(r'z:\test.bin','wb').write('a'*67080064) 能够正常工作吗？ - jfs

非常好的问题！有一个更简单的方法来重现相同的错误。我会编辑这个问题。 - Andrew

2

这个错误已经在Visual C++ 2008中修复，因此在Python 2.7+中不存在该问题。 - Piotr Dobrogost

1个回答

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- Gerrat · Accepted Answer

我认为问题与以下相关：

http://support.microsoft.com/default.aspx?scid=kb;en-us;899149

因此，请尝试执行以下操作：

open(r'z:\test.bin','w+b').write('a'*67080064)

*请注意参数：'w+b'