我的问题类似于@liyuanhe211,我有一个本地的copy.py
文件,但是我有一个本地的org.py
文件,所以在我的情况下运行pyspark
时就会发生这种情况:
pyspark
在执行import copy
时,沿途进行了一些操作。
但是内置的copy.py
执行了from org.python.core import PyStringMap
因此,由于我有一个import pandas as pd
的org.py
,看起来沿途也调用了org.deepcopy
,所以它就变成了这个循环依赖关系。
但是如果没有我的org.py
,那么问题就解决了。
$ pyspark
Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStoreType=KeychainStore
Python 3.10.11 (main, Apr 7 2023, 07:33:46) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
File "/Users/hello/Downloads/2023-06-01-spark/spark-3.4.0-bin-hadoop3/python/pyspark/shell.py", line 31, in <module>
import pyspark
File "/Users/hello/Downloads/2023-06-01-spark/spark-3.4.0-bin-hadoop3/python/pyspark/__init__.py", line 59, in <module>
from pyspark.rdd import RDD, RDDBarrier
File "/Users/hello/Downloads/2023-06-01-spark/spark-3.4.0-bin-hadoop3/python/pyspark/rdd.py", line 18, in <module>
import copy
File "/usr/local/Cellar/python@3.10/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 60, in <module>
from org.python.core import PyStringMap
File "/Users/hello/Repos/blah/src/org.py", line 2, in <module>
import pandas as pd
File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pandas/__init__.py", line 50, in <module>
from pandas.core.api import (
File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pandas/core/api.py", line 29, in <module>
from pandas.core.arrays import Categorical
File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pandas/core/arrays/__init__.py", line 20, in <module>
from pandas.core.arrays.string_arrow import ArrowStringArray
File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pandas/core/arrays/string_arrow.py", line 65, in <module>
import pyarrow.compute as pc
File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/compute.py", line 331, in <module>
_make_global_functions()
File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/compute.py", line 328, in _make_global_functions
g[cpp_name] = g[name] = _wrap_function(name, func)
File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/compute.py", line 299, in _wrap_function
return _decorate_compute_function(wrapper, name, func, options_class)
File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/compute.py", line 165, in _decorate_compute_function
options_class_doc = _scrape_options_class_doc(options_class)
File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/compute.py", line 112, in _scrape_options_class_doc
doc = docscrape.NumpyDocString(options_class.__doc__)
File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/vendored/docscrape.py", line 151, in __init__
self._parsed_data = copy.deepcopy(self.sections)
AttributeError: partially initialized module 'copy' has no attribute 'deepcopy' (most likely due to a circular import)