在Python中扁平化复杂的目录结构

Question

我想将文件从复杂的目录结构移动到一个地方。例如，我有这个深层次的层次结构:

foo/
    foo2/
        1.jpg
    2.jpg
    ...

我希望它变成：

1.jpg
2.jpg
...

我的当前解决方案：

def move(destination):
    for_removal = os.path.join(destination, '\\')
    is_in_parent = lambda x: x.find(for_removal) > -1
    with directory(destination):
        files_to_move = filter(is_in_parent,
                               glob_recursive(path='.'))
    for file in files_to_move:
        shutil.move(file, destination)

定义：directory和glob_recursive。请注意，我的代码仅将文件移动到它们的共同父目录，而不是任意目标。

如何简洁优雅地将复杂层次结构中的所有文件移动到一个单一位置？

- Mirzhan Irkegulov

Answer 1

我不喜欢测试即将移动到的文件名称以查看我们是否已经在目标目录中。相反，此解决方案仅扫描目标目录的子目录。

import os
import itertools
import shutil


def move(destination):
    all_files = []
    for root, _dirs, files in itertools.islice(os.walk(destination), 1, None):
        for filename in files:
            all_files.append(os.path.join(root, filename))
    for filename in all_files:
        shutil.move(filename, destination)

说明：os.walk以“自上而下”的方式递归遍历目标。通过os.path.join(root, filename)调用来构建完整的文件名。现在，为了防止扫描目标顶部的文件，我们只需要忽略os.walk的迭代的第一个元素即可。为此，我使用islice(iterator, 1, None)。另一种更明确的方法是：

def move(destination):
    all_files = []
    first_loop_pass = True
    for root, _dirs, files in os.walk(destination):
        if first_loop_pass:
            first_loop_pass = False
            continue
        for filename in files:
            all_files.append(os.path.join(root, filename))
    for filename in all_files:
        shutil.move(filename, destination)

- Scout

---

你没有处理移动的文件名与目标文件夹中的文件名相同的情况。例如，如果源子目录只包含名为hello.txt的文件，move()函数将抛出错误。 - 7stud

这个解决方案也会留下空的子目录。为了解决这个问题：在你将first_loop_pass设置为False的地方旁边存储_dirs，然后在文件移动完成后，只需循环遍历dirs并调用shutil.rmtree()即可。 - Ben

Answer 2

这个代码可以实现功能，如果文件名冲突还会重命名文件（我注释掉了实际的移动操作，用复制代替）：

import os
import sys
import string
import shutil

#Generate the file paths to traverse, or a single path if a file name was given
def getfiles(path):
    if os.path.isdir(path):
        for root, dirs, files in os.walk(path):
            for name in files:
                yield os.path.join(root, name)
    else:
        yield path

destination = "./newdir/"
fromdir = "./test/"
for f in getfiles(fromdir):
    filename = string.split(f, '/')[-1]
    if os.path.isfile(destination+filename):
        filename = f.replace(fromdir,"",1).replace("/","_")
    #os.rename(f, destination+filename)
    shutil.copy(f, destination+filename)

- DRC

---

目标路径+文件名可以是不需要的文件路径位置。最好使用os.path.join(destination，'filename')。 - pebox11

考虑将所有的【"/"】替换为【os.sep】。否则，在跨操作系统时会引起许多问题。 - Lancer.Yan

Answer 3

递归地遍历目录，移动文件并对目录执行move命令：

import shutil
import os

def move(destination, depth=None):
    if not depth:
        depth = []
    for file_or_dir in os.listdir(os.path.join([destination] + depth, os.sep)):
        if os.path.isfile(file_or_dir):
            shutil.move(file_or_dir, destination)
        else:
            move(destination, os.path.join(depth + [file_or_dir], os.sep))

- mishik

---

绝对路径版本import shutil import os def move(destination, depth=''): current_depth = os.path.join(destination, depth) for file_or_dir in os.listdir(current_depth): file_or_dir = os.path.join(current_depth, file_or_dir) if os.path.isfile(file_or_dir): if depth: shutil.move(file_or_dir, destination) else: move(destination, os.path.join(depth, file_or_dir)) move(r'F:\sw\PycharmProjects\yappy-dev\lectures') - Jerry

这段代码是无法正常工作的。os.path.join不接受列表作为参数，你需要使用splat将列表转换为单独的参数：os.path.join(*([destination] + depth)) - Max Uppenkamp

Answer 4

import os.path, shutil

def move(src, dest):
    not_in_dest = lambda x: os.path.samefile(x, dest)
    files_to_move = filter(not_in_dest,
                           glob_recursive(path=src))

    for f in files_to_move:
        shutil.move(f, dest)

源代码用于glob_recursive。如果它们冲突，不会更改文件名。

samefile是一种安全比较路径的方法。但它在Windows上无法使用，所以请查看如何在Windows和Python 2.7上模拟os.path.samefile行为？。

- Mirzhan Irkegulov

Answer 5

def splitPath(p):
    a,b = os.path.split(p)
    return (splitPath(a) if len(a) and len(b) else []) + [b]

def safeprint(s):
    try:
        print(s)
    except UnicodeEncodeError:
        if sys.version_info >= (3,):
            print(s.encode('utf8').decode(sys.stdout.encoding))
        else:
            print(s.encode('utf8'))

def flatten(root, doit):
    
    SEP  = "¦"
    REPL = "?"

    folderCount = 0
    fileCount = 0

    if not doit:
        print("Simulating:")

    for path, dirs, files in os.walk(root, topdown=False):

        if path != root:

            for f in files:

                sp = splitPath(path)

                np = ""

                for element in sp[1:]:
                    e2 = element.replace(SEP, REPL)
                    np += e2 + SEP

                f2 = f.replace(SEP, REPL)
                newName = np + f2

                safeprint("Moved:   "+ newName )
                if doit:
                    shutil.move(os.path.join(path, f), os.path.join(root, f))
                    # Uncomment, if you want filenames to be based on folder hierarchy.
                    #shutil.move(os.path.join(path, f), os.path.join(root, newName))
                fileCount += 1

            safeprint("Removed: "+ path)
            if doit:
                os.rmdir(path)
            folderCount += 1

    if doit:
        print("Done.")        
    else:
        print("Simulation complete.")


    print("Moved files:", fileCount)
    print("Removed folders:", folderCount)


directory_path = r"C:\Users\jd\Documents\myFtpData"
flatten(directory_path, True)

- Janzaib M Baloch

Answer 6

在其他答案的基础上，我相信我的答案将满足您的所有需求。当存在具有与上级目录相同文件名的子目录和文件时，其他答案会失败。

这个问题已经在这里得到了解决，另外请查看我的Github Repo for Structured File Copy and Flattened File Copy：

import os, fnmatch, shutil

PATTERN = '*.txt' # Regex Pattern to Match files
INPUT_FOLDER = "A" # os.getcwd()
INPUT_FOLDER = os.path.abspath(INPUT_FOLDER)
include_input_foldername = False
prepend = "_included" if include_input_foldername else ""
OUTPUT_FOLDER = f"Structured_Copy_{os.path.basename(INPUT_FOLDER)}{prepend}"

os.makedirs(OUTPUT_FOLDER, exist_ok=True)

def find(pattern, path):
    """Utility to find files wrt a regex search"""
    result = []
    for root, dirs, files in os.walk(path):
        for name in files:
            if fnmatch.fnmatch(name, pattern):
                result.append(os.path.join(root, name))
    return result

all_files = find(PATTERN, INPUT_FOLDER)

for each_path in all_files:
    relative_path = os.path.relpath(each_path, os.path.dirname(INPUT_FOLDER)) if include_input_foldername else os.path.relpath(each_path, INPUT_FOLDER) 
    flattened_relative_fullpath = os.path.join(OUTPUT_FOLDER, relative_path)
    os.makedirs(os.path.dirname(flattened_relative_fullpath), exist_ok=True)
    shutil.copy(each_path, flattened_relative_fullpath)
    print(f"Copied {each_path} to {flattened_relative_fullpath}")
    
print(f"Finished Copying {len(all_files)} Files from : {INPUT_FOLDER} to : {OUTPUT_FOLDER}")

- Farhan Hai Khan