shutil.rmtree：文件未找到错误：[Errno 2]没有这样的文件或目录：'._xxx'。

Question

代码和错误（缩短版，如果您想查看完整代码，请参见最后一节）：

import shutil
import os

filteredCleaned = 
 '/Volumes/Extreme_SSD/Raymond_Lab/Day_4_Rotarod_Videos_Rotated_if_Necessary_copy/filtered_cleaned_WT_cleaned_YAC128'

if os.path.exists(filteredCleaned):
    shutil.rmtree(filteredCleaned)

^ 显示隐藏文件（按下 shift+command+.）并显示存在的其他目录中的隐藏文件。

runfile('/Users/ksb7640/Documents/UBC_Academic/Raymond_Lab/448/rotarod/svm_all/data_filter.py', wdir='/Users/ksb7640/Documents/UBC_Academic/Raymond_Lab/448/rotarod/svm_all')
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3417, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-b9676f9bf96c>", line 1, in <module>
    runfile('/Users/ksb7640/Documents/UBC_Academic/Raymond_Lab/448/rotarod/svm_all/data_filter.py', wdir='/Users/ksb7640/Documents/UBC_Academic/Raymond_Lab/448/rotarod/svm_all')
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/ksb7640/Documents/UBC_Academic/Raymond_Lab/448/rotarod/svm_all/data_filter.py", line 21, in <module>
    shutil.rmtree(filteredCleaned)
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/shutil.py", line 715, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/shutil.py", line 672, in _rmtree_safe_fd
    onerror(os.unlink, fullname, sys.exc_info())
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/shutil.py", line 670, in _rmtree_safe_fd
    os.unlink(entry.name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: '._filtered_combined_Experiment2-190630_Day4_145m2_rotarod2_Cam2onRotarodDeepCut_resnet50_rotarod3Jul17shuffle1_1030000.csv'

问题：

'._filtered_combined_Experiment2-190630_Day4_145m2_rotarod2_Cam2onRotarodDeepCut_resnet50_rotarod3Jul17shuffle1_1030000.csv'来自哪里？

我确实使用了df.to_csv(csv, index=False)来创建filtered_combined_Experiment2-190630_Day4_145m2_rotarod2_Cam2onRotarodDeepCut_resnet50_rotarod3Jul17shuffle1_1030000.csv在filtered_cleaned_WT_cleaned_YAC128中，但我从未使用chmod来隐藏它们。

此外，正如您在屏幕截图中看到的那样，甚至没有隐藏的文件。这个错误是什么意思？

可能是冗余的完整代码（如果需要相关的、缩短的代码，请参见上面）

data_filter.py

import shutil

from export_df_to_csv import export_df_to_csv
from extract_parent_current import extract_parent_current
from import_df import *

prefix = 'filtered_'

bound = 0.9

# make a directory for output files
cleanedWT = '/Volumes/Extreme_SSD/Raymond_Lab/Day_4_Rotarod_Videos_Rotated_if_Necessary_copy/cleaned_WT'
cleanedYAC128 = '/Volumes/Extreme_SSD/Raymond_Lab/Day_4_Rotarod_Videos_Rotated_if_Necessary_copy/cleaned_YAC128'

cleanedWTParentDir, WTdir = extract_parent_current(cleanedWT)
cleanedYAC128ParentDir, YAC128dir = extract_parent_current(cleanedYAC128)

filteredCleaned = os.path.join(cleanedYAC128ParentDir, prefix + WTdir + '_' + YAC128dir)

if os.path.exists(filteredCleaned):
    shutil.rmtree(filteredCleaned)
os.mkdir(filteredCleaned)

csvs_labels_arr = import_csvs(cleanedWT, cleanedYAC128)
paths_dfs_labels_arr = csvs_to_paths_dfs_labels_arr(csvs_labels_arr)
for path_df_label in paths_dfs_labels_arr:
    _, dfFileName = extract_parent_current(path_df_label[0])
    df = path_df_label[1]

    # filter out rows below bound
    df = df[
        (df['Rightpaw likelihood'] > bound) & (df['Leftpaw likelihood'] > bound) & (df['Tail likelihood'] > bound)]

    export_df_to_csv(df, os.path.join(filteredCleaned, prefix + dfFileName))

export_df_to_csv.py

input_file = '/Volumes/Extreme SSD/Raymond Lab/Day_4_Rotarod_Videos_Rotated_if_Necessary copy/cleaned_WT/cleaned_Experiment2-190630_Day4_145m1_rotarod3_Cam2onRotarodDeepCut_resnet50_rotarod3Jul17shuffle1_1030000.csv'
output_file = '/Volumes/Extreme SSD/Raymond Lab/Day_4_Rotarod_Videos_Rotated_if_Necessary copy/cleaned_WT/cleaned_Experiment2-190630_Day4_145m1_rotarod3_Cam2onRotarodDeepCut_resnet50_rotarod3Jul17shuffle1_1030000.csv'


def export_df_to_csv(df, csv):
    df.to_csv(csv, index=False)

extract_parent_current.py

import os


def extract_parent_current(dir):
    if str.endswith(dir, '/'):
        dir = dir[:-1]
    return os.path.split(dir)

import_df.py

import os
import random
from copy import deepcopy
import pandas as pd


def import_csvs(WT_file_path, YAC_file_path):
    csv_paths_arr = []
    for root, dirs, files in os.walk(WT_file_path, topdown=False):
        for file in files:
            if not file.startswith('.'):
                csv_paths_arr.append([os.path.join(root, file), 0])

    for root, dirs, files in os.walk(YAC_file_path, topdown=False):
        for file in files:
            if not file.startswith('.'):
                csv_paths_arr.append([os.path.join(root, file), 1])

    return csv_paths_arr


def csvs_to_paths_dfs_labels_arr(csvpaths_labels_arr):
    paths_dfs_labels_arr = deepcopy(csvpaths_labels_arr)
    for i, csvpath_label_arr in enumerate(csvpaths_labels_arr):
        (paths_dfs_labels_arr[i])[0] = pd.read_csv(csvpath_label_arr[0], encoding='unicode_escape')
        paths_dfs_labels_arr[i].insert(0, csvpath_label_arr[0])
    return paths_dfs_labels_arr


def import_df(WT_file_path, YAC_file_path):
    csv_paths_arr = import_csvs(WT_file_path, YAC_file_path)
    dfs_labels = csvs_to_paths_dfs_labels_arr(csv_paths_arr)
    random.shuffle(dfs_labels)
    return dfs_labels

- underdisplayname

---

._filtered_combined_Experiment2-190630_Day4_145m2_rotarod2_Cam2onRotarodDeepCut_resnet50_rotarod3Jul17shuffle1_1030000.csv 是一个符号链接吗？ - Arkadiusz Drabczyk

我不知道符号链接是什么，只是简单地了解了一下。它似乎是一个指向文件的指针，就像在桌面上创建的快捷方式，它指向不同目录中的文件，实际上并不在桌面上。如果我没错的话，那么我不知道它是如何创建的，也不知道为什么要创建它。我的猜测是不需要创建。 - underdisplayname

你尝试过复制创建._somename.csv的过程吗？我的意思是，使用不同的文件名，调用df.to_csv，你是否会得到更多这样的“隐藏文件”？如果无法复制，则可能是在创建文件时出现了某种通信错误或其他问题。 - Niko Pasanen

你找到解决方案了吗？ - Jürgen K.

Answer 1

我已经为链接问题提供了答案Delete directory and all symlinks recursively，但决定在这里再次复制它。在那里，提供解决方案而不是实际问题的答案是错误的，而这个答案直接回答了._filtered_combined_Experiment2-190630_Day4_145m2_rotarod2_Cam2onRotarodDeepCut_resnet50_rotarod3Jul17shuffle1_1030000.csv的来源。 /Volumes/Extreme_SSD/ 路径表明您正在使用Mac OSX，并且您的目录至少部分位于非Mac文件系统上（即不是HFS +）。在这些情况下，Mac文件系统驱动程序会自动创建以._为前缀的二进制伴随文件，以记录所谓的扩展属性（在https://apple.stackexchange.com/questions/14980/why-are-dot-underscore-files-created-and-how-can-i-avoid-them中有所解释，但也在下面说明），例如使用默认的OSX查看器查看文件时进行的例行操作。

对于不支持os.scandir中的文件描述符的系统（如Mac OSX）的rmtree现在会不安全地创建一个条目列表，然后逐个取消链接它们（创建已知的竞争条件：https://github.com/python/cpython/blob/908fd691f96403a3c30d85c17dd74ed1f26a60fd/Lib/shutil.py#L592-L621）。不幸的是，两种不同的行为使得这种情况在每次都成立：

1. 原始文件总是在扩展属性文件之前列出，以及
2. 当删除原始文件(test.txt)时，元文件(._test.txt)同时被删除。

因此，在轮到扩展属性文件时，它将丢失并抛出您遇到的FileNotFoundError。

我认为通过cpython#14064最好解决这个错误，该解决方案旨在通常忽略rmtree中的FileNotFoundError。

缓解措施

与此同时，您可以通过onerror忽略那些元文件的取消链接错误：

def ignore_extended_attributes(func, filename, exc_info):
    is_meta_file = os.path.basename(filename).startswith("._")
    if not (func is os.unlink and is_meta_file):
        raise

shutil.rmtree(path_dir, onerror=ignore_extended_attributes)

Mac扩展属性的展示案例

为了说明，您可以创建一个小的ExFAT磁盘映像，并使用以下命令将其挂载到/Volumes/Untitled：

hdiutil create -size 5m -fs exfat test.dmg
hdiutil attach test.dmg            # mounts at /Volumes/Untitled
cd /Volumes/Untitled

mkdir test                         # create a directory to remove
cd test
touch test.txt
open test.txt                      # open the test.txt file in the standard editor 

在标准文本编辑器中打开文件会创建一个扩展属性文件._test.txt并记录最后访问时间：

/Volumes/Untitled/test $ ls -a
.          ..         ._test.txt test.txt
/Volumes/Untitled/test $ xattr test.txt
com.apple.lastuseddate#PS

问题在于自动取消原文件的链接也会取消附属文件的链接。

/Volumes/Untitled/test $ rm test.txt
/Volumes/Untitled/test $ ls -a
.          ..

- Jonas Hörsch