我想从本地设备的特定文件夹上传多个文件到AWS S3。但是我遇到了以下错误。
这是我的terraform代码。
resource "aws_s3_bucket" "testbucket" {
bucket = "test-terraform-pawan-1"
acl = "private"
tags = {
Name = "test-terraform"
Environment = "test"
}
}
resource "aws_s3_bucket_object" "uploadfile" {
bucket = "test-terraform-pawan-1"
key = "index.html"
source = "/home/pawan/Documents/Projects/"
}
我该如何解决这个问题？
从Terraform 0.12.8开始，您可以使用fileset函数获取给定路径和模式的文件列表。结合for_each，您应该能够将每个文件作为自己的aws_s3_bucket_object上传：
resource "aws_s3_bucket_object" "dist" {
for_each = fileset("/home/pawan/Documents/Projects/", "*")
bucket = "test-terraform-pawan-1"
key = each.value
source = "/home/pawan/Documents/Projects/${each.value}"
# etag makes the file update when it changes; see https://dev59.com/yFMI5IYBdhLWcg3wfbfN
etag = filemd5("/home/pawan/Documents/Projects/${each.value}")
}
请查看GitHub上terraform-providers/terraform-provider-aws: aws_s3_bucket_object: 支持目录上传#3020。
注意:这不会设置像content_type这样的元数据，据我所知，Terraform没有内置的方法来推断文件的内容类型。这些元数据对于诸如从浏览器访问HTTP等事情的正确工作非常重要。如果这对您很重要，则应考虑手动指定每个文件而不是尝试自动获取文件夹中的所有内容。
resource "null_resource" "remove_and_upload_to_s3" {
provisioner "local-exec" {
command = "aws s3 sync ${path.module}/s3Contents s3://${aws_s3_bucket.site.id}"
}
}
module "template_files" {
source = "hashicorp/dir/template"
base_dir = "${path.module}/src"
template_vars = {
# Pass in any values that you wish to use in your templates.
vpc_id = "vpc-abc123"
}
}
resource "aws_s3_bucket_object" "static_files" {
for_each = module.template_files.files
bucket = "example"
key = each.key
content_type = each.value.content_type
# The template_files module guarantees that only one of these two attributes
# will be set for each file, depending on whether it is an in-memory template
# rendering result or a static file on disk.
source = each.value.source_path
content = each.value.content
# Unless the bucket has encryption enabled, the ETag of each object is an
# MD5 hash of that object.
etag = each.value.digests.md5
}
JSON格式：
{
"resource": {
"aws_s3_bucket_object": {
"static_files": {
"for_each": "${module.template_files.files}"
#...
}}}}
#...
}
来源：https://registry.terraform.io/modules/hashicorp/dir/template/latest
- Directories
- Folder1
* test_file_1.txt
* test_file_2.txt
- Folder2
* test_file_3.txt
locals{
folder_files = flatten([for d in flatten(fileset("${path.module}/Directories/*", "*")) : trim( d, "../") ])
}
folder_files = [
"Folder1/test_file_1.txt",
"Folder1/test_file_2.txt",
"Folder2/test_file_3.txt",
]
resource "aws_s3_object" "this" {
for_each = { for idx, file in local.folder_files : idx => file }
bucket = aws_s3_bucket.this.bucket
key = "/Directories/${each.value}"
source = "${path.module}/Directories/${each.value}"
etag = "${path.module}/Directories/${each.value}"
}
Directory
- Folder1
- test_file_1.txt
- test_file_2.txt
- Folder2
- test_file_3.txt
resource "aws_s3_bucket_object" "files" {
bucket = aws_s3_bucket.buc.bucket
for_each = fileset("C:\\Users\\terraform-aws\\", "**")
key=each.key
source = "C:\\Users\\terraform-aws\\${each.value}"
content_type = each.value
etag = filemd5("C:\\Users\\terraform-aws\\${each.value}")
}
aws_s3_object，但发现对于我的用例来说并不是一个好的方法：构建前端文件并将其上传到位于cloudfront后面的S3。为了完成这个任务，我需要完成以下三件事情：
.env文件构建前端文件，该文件将包含api的端点 -- (1)我在这里没有使用自定义域，所以我使用的是弹性负载均衡器生成的DNS名称，而在ELB创建之前我是不知道它的。content-type，否则，(2)它将自动上传为八进制流或类似的内容，这将导致当您访问网站时，您将下载HTML文件而不是浏览器渲染它。aws_s3_object在这种情况下存在以下问题：
aws_s3_object没有指定正确的内容类型，导致了我上面提到的副作用，标记为2。要解决这个限制，你需要使用类似于terraform-template-dir的东西。terraform-template-dir需要在运行terraform apply之前确保文件已经存在！因为如果它们不存在，并且在terraform apply期间生成，terraform-template-dir将不会捕获任何内容，这意味着你必须在运行terraform apply之前运行yarn build，但是如果你的yarn build依赖于在terraform中生成的.env文件，你将不得不再次运行build。我认为最好的解决方案是这样的：
resource "null_resource" "frontend_files" {
triggers = {
always_run = timestamp()
}
// if we have a local .env file, we want to make sure we keep it before creating a new one with different contents
provisioner "local-exec" {
command = <<EOF
[[ ! -f ../apps/web/.env ]] || mv ../apps/web/.env ../apps/web/.env.backup
touch ../apps/web/.env
echo "VITE_TRPC_ENDPOINT=http://${aws_lb.ecs_lb.dns_name}/trpc" >> ../apps/web/.env
yarn --cwd ../apps/web build
aws s3 sync ../apps/web/dist s3://${aws_s3_bucket.frontend_s3_bucket.id}
aws cloudfront create-invalidation --distribution-id ${aws_cloudfront_distribution.frontend_cf.id} --paths "/*"
[[ ! -f ../apps/web/.env.backup ]] || mv ../apps/web/.env.backup ../apps/web/.env
EOF
}
}
aws cloudfront create-invalidation需要你的CloudFront ID，而你在运行terraform apply之前是不知道这个ID的。为了解决这个问题，你可以输出一个包含CloudFront ID的文件，并在Bash脚本中读取该文件的内容。resource "aws_s3_object" "files_upload" {
for_each = fileset("${path.root}/dir_upload", "**/*.*")
bucket = "bucket-name"
key = "key-prefix/${each.value}"
source = "${path.root}/dir_upload/${each.value}"
source_hash = filemd5("${path.root}/dir_upload/${each.value}")
}
content_type：https://registry.terraform.io/modules/hashicorp/dir/template/latest - Flaircontent_type。你能否引用S3示例代码和链接作为单独的答案，以便更加可见？ - meustrus.tf.json提供了一个更小的片段。 - Flairsource_hash比etag更好。 - sdgfsdh