如何使用Python 3上传文件到Google Cloud Storage?如果从Python 3无法实现,可以考虑使用Python 2。
我找了很多资料,但没有找到可行的解决方案。我尝试了boto,但是当我尝试通过gsutil config -e
生成必要的.boto文件时,它一直说我需要通过gcloud auth login
进行身份验证配置。然而,我已经多次执行后者,但并没有帮助。
如何使用Python 3上传文件到Google Cloud Storage?如果从Python 3无法实现,可以考虑使用Python 2。
我找了很多资料,但没有找到可行的解决方案。我尝试了boto,但是当我尝试通过gsutil config -e
生成必要的.boto文件时,它一直说我需要通过gcloud auth login
进行身份验证配置。然而,我已经多次执行后者,但并没有帮助。
使用标准的gcloud库,支持Python 2和Python 3。
from gcloud import storage
from oauth2client.service_account import ServiceAccountCredentials
import os
credentials_dict = {
'type': 'service_account',
'client_id': os.environ['BACKUP_CLIENT_ID'],
'client_email': os.environ['BACKUP_CLIENT_EMAIL'],
'private_key_id': os.environ['BACKUP_PRIVATE_KEY_ID'],
'private_key': os.environ['BACKUP_PRIVATE_KEY'],
}
credentials = ServiceAccountCredentials.from_json_keyfile_dict(
credentials_dict
)
client = storage.Client(credentials=credentials, project='myproject')
bucket = client.get_bucket('mybucket')
blob = bucket.blob('myfile')
blob.upload_from_filename('myfile')
oauth2client
已被弃用。https://github.com/google/oauth2client。此外,在较新版本的Google云Python软件包中,请使用`from google.cloud import storage而不是
from gcloud...`。 - Shiva.get_bucket
,则可以改用 .bucket
。 https://cloud.google.com/storage/docs/uploading-objects#storage-upload-object-code-sample - Phootipfrom google.cloud import storage
#pip install --upgrade google-cloud-storage.
def upload_to_bucket(blob_name, path_to_file, bucket_name):
""" Upload data to a bucket"""
# Explicitly use service account credentials by specifying the private key
# file.
storage_client = storage.Client.from_service_account_json(
'creds.json')
#print(buckets = list(storage_client.list_buckets())
bucket = storage_client.get_bucket(bucket_name)
blob = bucket.blob(blob_name)
blob.upload_from_filename(path_to_file)
#returns a public url
return blob.public_url
您可以使用此链接生成凭据文件:https://cloud.google.com/storage/docs/reference/libraries?authuser=1#client-libraries-install-python
异步示例:
import asyncio
import aiohttp
# pip install aiofile
from aiofile import AIOFile
# pip install gcloud-aio-storage
from gcloud.aio.storage import Storage
BUCKET_NAME = '<bucket_name>'
FILE_NAME = 'requirements.txt'
async def async_upload_to_bucket(blob_name, file_obj, folder='uploads'):
""" Upload csv files to bucket. """
async with aiohttp.ClientSession() as session:
storage = Storage(service_file='./creds.json', session=session)
status = await storage.upload(BUCKET_NAME, f'{folder}/{blob_name}', file_obj)
#info of the uploaded file
# print(status)
return status['selfLink']
async def main():
async with AIOFile(FILE_NAME, mode='r') as afp:
f = await afp.read()
url = await async_upload_to_bucket(FILE_NAME, f)
print(url)
# Python 3.6
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Python 3.7+
# asyncio.run(main())
导入谷歌云客户端库(需要凭证)
from google.cloud import storage
import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="C:/Users/siva/Downloads/My First Project-e2d95d910f92.json"
实例化一个客户端
storage_client = storage.Client()
buckets = list(storage_client.list_buckets())
bucket = storage_client.get_bucket("ad_documents") # your bucket name
blob = bucket.blob('chosen-path-to-object/{name-of-object}')
blob.upload_from_filename('D:/Download/02-06-53.pdf')
print(buckets)
安装 Google Cloud Storage API 时:
pip install google-cloud
可能会出现 ModuleNotFoundError
的错误:
from google.cloud import storage
ModuleNotFoundError: No module named 'google'
请按以下方式安装: Cloud Storage客户端库文档:
pip install --upgrade google-cloud-storage
这个官方仓库包含了几个片段,演示了将文件上传到存储桶的不同方式: https://github.com/googleapis/python-storage/tree/05e07f248fc010d7a1b24109025e9230cb2a7259/samples/snippets
upload_from_string()
upload_from_file()
upload_from_filename()