我使用GCS Python SDK和Google API客户端的组合来循环访问启用版本控制的存储桶,并根据元数据下载特定对象。
from google.cloud import storage
from googleapiclient import discovery
from oauth2client.client import GoogleCredentials
def downloadepoch_objects():
request = service.objects().list(
bucket=bucket_name,
versions=True
)
response = request.execute()
for item in response['items']:
if item['metadata']['epoch'] == restore_epoch:
print(item['bucket'])
print(item['name'])
print(item['metadata']['epoch'])
print(item['updated'])
blob = source_bucket.blob(item['name'])
blob.download_to_filename(
'/Users/admin/git/data-processing/{}'.format(item))
downloadepoch_objects()
上述函数对于不在目录中的 blob(gs://bucketname/test1.txt)可以正常工作,因为传递的项只是 test1.txt。我遇到的问题是在尝试从复杂的目录树中下载文件时(gs://bucketname/nfs/media/docs/test1.txt),传递的项是 nfs/media/docs/test1.txt。是否可能让 .download_to_file() 方法在必要时创建目录?
prefix + response['prefixes'][0]
上再次调用 list()。 - Brandon Yarbroughresponse = request.execute() subdirectories = response['prefixes'] print(subdirectories)
- glux