使用cloudformation将Lambda函数启用到S3桶

Question

我们正在使用CloudFormation模板创建一个S3存储桶。 我想在S3存储桶中添加文件时关联（添加事件到S3存储桶）Lambda函数。

如何通过CloudFormation模板实现？需要在CloudFormation中使用哪些属性？

- shiv455

---

您想在创建存储桶或在存储桶中创建对象（PutObject）时触发Lambda吗？ - helloV

Answer 1

这是一个完整的、自包含的CloudFormation模板，演示了如何在文件添加到S3存储桶时触发Lambda函数：

Description: Upload an object to an S3 bucket, triggering a Lambda event, returning the object key as a Stack Output.
Parameters:
  Key:
    Description: S3 Object key
    Type: String
    Default: test
  Body:
    Description: S3 Object body content
    Type: String
    Default: TEST CONTENT
  BucketName:
    Description: S3 Bucket name
    Type: String
Resources:
  Bucket:
    Type: AWS::S3::Bucket
    DependsOn: BucketPermission
    Properties:
      BucketName: !Ref BucketName
      NotificationConfiguration:
        LambdaConfigurations:
        - Event: 's3:ObjectCreated:*'
          Function: !GetAtt BucketWatcher.Arn
  BucketPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: 'lambda:InvokeFunction'
      FunctionName: !Ref BucketWatcher
      Principal: s3.amazonaws.com
      SourceAccount: !Ref "AWS::AccountId"
      SourceArn: !Sub "arn:aws:s3:::${BucketName}"
  BucketWatcher:
    Type: AWS::Lambda::Function
    Properties:
      Description: Sends a Wait Condition signal to Handle when invoked
      Handler: index.handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          exports.handler = function(event, context) {
            console.log("Request received:\n", JSON.stringify(event));
            var responseBody = JSON.stringify({
              "Status" : "SUCCESS",
              "UniqueId" : "Key",
              "Data" : event.Records[0].s3.object.key,
              "Reason" : ""
            });
            var https = require("https");
            var url = require("url");
            var parsedUrl = url.parse('${Handle}');
            var options = {
                hostname: parsedUrl.hostname,
                port: 443,
                path: parsedUrl.path,
                method: "PUT",
                headers: {
                    "content-type": "",
                    "content-length": responseBody.length
                }
            };
            var request = https.request(options, function(response) {
                console.log("Status code: " + response.statusCode);
                console.log("Status message: " + response.statusMessage);
                context.done();
            });
            request.on("error", function(error) {
                console.log("send(..) failed executing https.request(..): " + error);
                context.done();
            });
            request.write(responseBody);
            request.end();
          };
      Timeout: 30
      Runtime: nodejs4.3
  Handle:
    Type: AWS::CloudFormation::WaitConditionHandle
  Wait:
    Type: AWS::CloudFormation::WaitCondition
    Properties:
      Handle: !Ref Handle
      Timeout: 300
  S3Object:
    Type: Custom::S3Object
    Properties:
      ServiceToken: !GetAtt S3ObjectFunction.Arn
      Bucket: !Ref Bucket
      Key: !Ref Key
      Body: !Ref Body
  S3ObjectFunction:
    Type: AWS::Lambda::Function
    Properties:
      Description: S3 Object Custom Resource
      Handler: index.handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          var response = require('cfn-response');
          var AWS = require('aws-sdk');
          var s3 = new AWS.S3();
          exports.handler = function(event, context) {
            console.log("Request received:\n", JSON.stringify(event));
            var responseData = {};
            if (event.RequestType == 'Create') {
              var params = {
                Bucket: event.ResourceProperties.Bucket,
                Key: event.ResourceProperties.Key,
                Body: event.ResourceProperties.Body
              };
              s3.putObject(params).promise().then(function(data) {
                response.send(event, context, response.SUCCESS, responseData);
              }).catch(function(err) {
                console.log(JSON.stringify(err));
                response.send(event, context, response.FAILED, responseData);
              });
            } else if (event.RequestType == 'Delete') {
              var deleteParams = {
                Bucket: event.ResourceProperties.Bucket,
                Key: event.ResourceProperties.Key
              };
              s3.deleteObject(deleteParams).promise().then(function(data) {
                response.send(event, context, response.SUCCESS, responseData);
              }).catch(function(err) {
                console.log(JSON.stringify(err));
                response.send(event, context, response.FAILED, responseData);
              });
            } else {
              response.send(event, context, response.SUCCESS, responseData);
            }
          };
      Timeout: 30
      Runtime: nodejs4.3
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal: {Service: [lambda.amazonaws.com]}
          Action: ['sts:AssumeRole']
      Path: /
      ManagedPolicyArns:
      - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
      Policies:
      - PolicyName: S3Policy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - 's3:PutObject'
                - 'S3:DeleteObject'
              Resource: !Sub "arn:aws:s3:::${BucketName}/${Key}"
Outputs:
  Result:
    Value: !GetAtt Wait.Data

- wjordan

---

非常感谢您提供的工作示例，它节省了很多时间！但是，您能告诉我为什么在那个模板中有 S3Object 吗？我没有看到它被引用在任何地方？ - Cherry

@Cherry S3Object和S3ObjectFunction会自动在S3存储桶中创建一个对象来触发Lambda函数，仅供测试使用。通常情况下，这将由您手动上传的对象触发，因此在您自己的实现中不是必需的。 - wjordan

我意识到，为了使其正常工作，文件实际上需要命名为“test”。这就是为什么我的“S3ObjectFunction”未被调用的原因。 - dan.m.kumar

2021年4月：当我运行这段代码片段时，它不会创建任何函数。有人知道如何让它工作吗？ - GlaceCelery

更新2021：将nodeJS版本从4更新到14.x以构建此堆栈。 - Xonshiz

Answer 2

在您的CloudFormation模板中，需要一个NotificationConfiguration属性。不幸的是，它似乎要求存储桶已经存在。为了解决这个问题，您可以创建一个初始堆栈，然后使用NotificationConfiguration进行更新。例如：

    // template1.json
    {
      "AWSTemplateFormatVersion": "2010-09-09",
      "Parameters": {
        "mylambda": {
          "Type": "String"
        }
      },
      "Resources": {
        "bucketperm": {
          "Type": "AWS::Lambda::Permission",
          "Properties" : {
            "Action": "lambda:InvokeFunction",
            "FunctionName": {"Ref": "mylambda"},
            "Principal": "s3.amazonaws.com",
            "SourceAccount": {"Ref": "AWS::AccountId"},
            "SourceArn": { "Fn::Join": [":", [
                "arn", "aws", "s3", "" , "", {"Ref" : "mybucket"}]]
            }
          }
        },
        "mybucket": {
          "Type": "AWS::S3::Bucket"
        }
      }
    }

    // template2.json -- adds the NotificationConfiguration
    {
      "AWSTemplateFormatVersion": "2010-09-09",
      "Parameters": {
        "mylambda": {
          "Type": "String"
        }
      },
      "Resources": {
        "bucketperm": {
          "Type": "AWS::Lambda::Permission",
          "Properties" : {
            "Action": "lambda:InvokeFunction",
            "FunctionName": {"Ref": "mylambda"},
            "Principal": "s3.amazonaws.com",
            "SourceAccount": {"Ref": "AWS::AccountId"},
            "SourceArn": { "Fn::Join": [":", [
                "arn", "aws", "s3", "" , "", {"Ref" : "mybucket"}]]
            }
          }
        },
        "mybucket": {
          "Type": "AWS::S3::Bucket",
          "Properties": {
            "NotificationConfiguration": {
              "LambdaConfigurations": [
                {
                  "Event" : "s3:ObjectCreated:*",
                  "Function" : {"Ref": "mylambda"}
                }
              ]
            }
          }
        }
      }
    }

您可以使用AWS CLI工具创建堆栈，方法如下：

    $ aws cloudformation create-stack --stack-name mystack --template-body file://template1.json --parameters ParameterKey=mylambda,ParameterValue=<lambda arn>
    # wait until stack is created
    $ aws cloudformation update-stack --stack-name mystack --template-body file://template2.json --parameters ParameterKey=mylambda,ParameterValue=<lambda arn>

- ataylor

---

问题是如何在创建存储桶时触发 Lambda，而不是在向存储桶添加新文件时触发。 - helloV

@ataylor，当我上传此模板以创建堆栈时，它会给我一个“无法验证模板中的配置”的错误。 - shiv455

@ataylor 我正在使用Lambda和嵌套堆栈来创建S3存储桶，嵌套堆栈还创建了SNS、负载均衡器和应用程序堆栈...所以现在当我创建没有NotificationConfiguration的S3存储桶后，我是否需要在嵌套堆栈中使用另一个资源（使用CFT）来添加NotificationConfiguration？ - shiv455

@ataylor，因为我在使用嵌套堆栈创建堆栈，所以无法在Lambda函数中更新堆栈。 - shiv455

@ataylor 我得到了以下错误信息：无法创建以下资源：[mybucket]。在执行第二个命令（使用第二个模板）时，用户请求回滚。 - florins

Answer 3

我已经在CloudFormation中添加了以下S3存储桶权限以及通知配置，这将用于创建S3存储桶。它已经生效！

"bucketperm": {
            "Type": "AWS::Lambda::Permission",
            "Properties": {
                "Action": "lambda:invokeFunction",
                "FunctionName": "<arnvalue>",
                "Principal": "s3.amazonaws.com"
            }
}

- shiv455

---

这似乎不起作用。这是必需的，但即使在https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-notificationconfig.html#cfn-s3-bucket-notificationconfig-lambdaconfig上的文档中也提到这是问题的一部分。 - digitalfoo

Answer 4

是的，通过Cloudformation可以实现，您需要配置以下内容：

1）AWS::S3::Bucket 资源

2）NotificationConfiguration 配置（在这种情况下使用LambdaConfigurations）适用于上述s3资源。

需要相关文档：

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket.html#cfn-s3-bucket-notification

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-notificationconfig.html

- Winters Huang

Answer 5

在AWS文档中明确指出，AWS::S3::Bucket用于创建资源。如果我们已经有一个存在的存储桶，我们无法修改它以添加NotificationConfiguration。 因此，为了使上述模板工作，S3存储桶必须不存在。 让CloudFormation创建所有资源，包括S3存储桶。

- shobhit bhatnagar

---

我使用NotificationConfiguration和Lambda Permission更新了一个现有的Bucket，并且操作成功。 - Russell Keane