我一直忙于编写我的第一份多进程代码,而它现在已经可行了,太好了。然而,现在我想要一些进度反馈,但我不确定最好的方法是什么。
我的代码(见下文)简述:
- 扫描目标目录中的mp4文件
- 每个文件由一个单独的进程分析,进程保存结果(一张图片)
我所寻找的内容可能包括:
- 简单
- 每当一个进程完成一个文件时,它会发送“完成”消息
- 主要代码会计算已完成的文件数
- 高级一些
Core 0 processing file 20 of 317 ||||||____ 60% completed
Core 1 processing file 21 of 317 |||||||||_ 90% completed
...
Core 7 processing file 18 of 317 ||________ 20% completed
我阅读了关于队列、池、tqdm等各种信息,但不确定应该选择哪种方法。请问有人能指出在这种情况下适用的方法吗?
提前感谢!
编辑:根据gsb22的建议更改了启动进程的代码。
我的代码:
# file operations
import os
import glob
# Multiprocessing
from multiprocessing import Process
# Motion detection
import cv2
# >>> Enter directory to scan as target directory
targetDirectory = "E:\Projects\Programming\Python\OpenCV\\videofiles"
def get_videofiles(target_directory):
# Find all video files in directory and subdirectories and put them in a list
videofiles = glob.glob(target_directory + '/**/*.mp4', recursive=True)
# Return the list
return videofiles
def process_file(videofile):
'''
What happens inside this function:
- The video is processed and analysed using openCV
- The result (an image) is saved to the results folder
- Once this function receives the videofile it completes
without the need to return anything to the main program
'''
# The processing code is more complex than this code below, this is just a test
cap = cv2.VideoCapture(videofile)
for i in range(10):
succes, frame = cap.read()
# cv2.imwrite('{}/_Results/{}_result{}.jpg'.format(targetDirectory, os.path.basename(videofile), i), frame)
if succes:
try:
cv2.imwrite('{}/_Results/{}_result_{}.jpg'.format(targetDirectory, os.path.basename(videofile), i), frame)
except:
print('something went wrong')
if __name__ == "__main__":
# Create directory to save results if it doesn't exist
if not os.path.exists(targetDirectory + '/_Results'):
os.makedirs(targetDirectory + '/_Results')
# Get a list of all video files in the target directory
all_files = get_videofiles(targetDirectory)
print(f'{len(all_files)} video files found')
# Create list of jobs (processes)
jobs = []
# Create and start processes
for file in all_files:
proc = Process(target=process_file, args=(file,))
jobs.append(proc)
for job in jobs:
job.start()
for job in jobs:
job.join()
# TODO: Print some form of progress feedback
print('Finished :)')
proc.start()
和proc.join()
是错误的。你正在启动进程,然后立即加入它,这实际上会阻塞你的for loop
,所以我认为你现在并没有进行多进程处理。你能否在process_file
方法中放置一个 10 秒的延迟,并查看最终结果是否仅延迟了 10 秒或 10 *(文件数)? - gsb22