在Jupyter中,我有一个包含400,000个对象的数据框,如果不遵循以下错误,就无法完全将其导出到JSON文件中。
只要将导出限制在前141,000个对象上,无论这些第一个对象的顺序如何,导出都很好地工作。
我应该注意处理大型JSON文件时是否存在任何大小限制吗? 谢谢。
只要将导出限制在前141,000个对象上,无论这些第一个对象的顺序如何,导出都很好地工作。
我应该注意处理大型JSON文件时是否存在任何大小限制吗? 谢谢。
OverflowError Traceback (most recent call last)
<ipython-input-254-b59373f1eeb2> in <module>
----> 1 df4.to_json('test.json', orient = 'records')
~/anaconda3/lib/python3.7/site-packages/pandas/core/generic.py in to_json(self, path_or_buf, orient, date_format, double_precision, force_ascii, date_unit, default_handler, lines, compression, index)
1889 default_handler=default_handler,
1890 lines=lines, compression=compression,
-> 1891 index=index)
1892
1893 def to_hdf(self, path_or_buf, key, **kwargs):
~/anaconda3/lib/python3.7/site-packages/pandas/io/json/json.py in to_json(path_or_buf, obj, orient, date_format, double_precision, force_ascii, date_unit, default_handler, lines, compression, index)
56 double_precision=double_precision, ensure_ascii=force_ascii,
57 date_unit=date_unit, default_handler=default_handler,
---> 58 index=index).write()
59
60 if lines:
~/anaconda3/lib/python3.7/site-packages/pandas/io/json/json.py in write(self)
99 return self._write(self.obj, self.orient, self.double_precision,
100 self.ensure_ascii, self.date_unit,
--> 101 self.date_format == 'iso', self.default_handler)
102
103 def _write(self, obj, orient, double_precision, ensure_ascii,
~/anaconda3/lib/python3.7/site-packages/pandas/io/json/json.py in _write(self, obj, orient, double_precision, ensure_ascii, date_unit, iso_dates, default_handler)
154 double_precision,
155 ensure_ascii, date_unit,
--> 156 iso_dates, default_handler)
157
158
~/anaconda3/lib/python3.7/site-packages/pandas/io/json/json.py in _write(self, obj, orient, double_precision, ensure_ascii, date_unit, iso_dates, default_handler)
110 date_unit=date_unit,
111 iso_dates=iso_dates,
--> 112 default_handler=default_handler
113 )
114
OverflowError: int too big to convert