Jul 6, 2022
11 mins read
Lambda@Edge can simplify your application architecture, decrease latency and improve response time. There are a number of appealing use-cases for Lambda@Edge including user authentication using JWT Tokens and signed cookies. In this post I am sharing ideas on how to pass parameters such as URLs, names and ARNs of AWS resources to a Lambda@Edge function when creating them from a CloudFormation or Serverless Application Model template. Like many developers I experienced this problem when I started working with Lambda@Edge.
There are certain restrictions on Lambda@Edge functions which you need to consider. In particular, environment variables and layers are not supported. Thus, if your function uses other AWS resources like S3 or DynamoDB you need to figure out a way to pass them into the function without using the environment variables. In large projects the problem is usually solved by using CI/CD systems. For relatively small applications it is often desirable to deploy them using the Serverless Application Model CLI. After all, this is what the tool was designed for.
One way of achieving this goal would be to inline your function in the Cloudformation/SAM template and use Fn::Sub to embed the actual parameters in your code. Unfortunately, this would only work if the function does not rely on external dependencies except those already packaged by AWS. Since layers are not supported, we are no longer able to move most of the function implementation into a layer and reduce the function code to a simple wrapper.
Alternatively, we can define custom headers in CloudFront which will be passed to the Lambda@Edge function handler. For most applications this is a viable, albeit somewhat ad hoc, approach. There are functions, however, that need to perform considerable amount of work during static initialization. Obviously, the custom headers are not available at that point and a better solution is required.
Lastly, there is AWS Secrets Manager that would happily store the required parameters and provide programmatic access to them from the Lambda Function. This approach is described in this post. You can use CloudFormation to create a secret and save all the required information in its SecretString as a JSON-formatted string. The value can be easily retrieved from the static initialization section of the Lambda function. All we need to know is the SecretID, a parameter that we need to pass from the template to the function code. And now we came to the same problem we started with: how do we pass this parameter?
Jokes aside, this is the most versatile solution. The SecretID is a unique string parameter that can be hard-coded in both the template and the function code. This is not ideal of cause but much better than other options.
To demonstrate the concept we are going to build a patently useless application that utilises Lambda@Edge functions to list the content of a private S3 bucket. We are going to use Python runtime, but implementations in other languages should look pretty similar. The complete SAM template is available on GitHub.
This is by far the easiest way to include resources in your code. Use the InlineCode property of your function and !Sub to replace all references:
LambdaEdgeFunctionInline:
Type: AWS::Serverless::Function
Properties:
Role: !GetAtt LambdaEdgeFunctionRole.Arn
Runtime: python3.9
Handler: index.handler
AutoPublishAlias: live
InlineCode: !Sub |
def handler(event, context):
return {
"status": "200",
"statusDescription": "OK",
"body": "bucket name is ${Bucket}; ARN ${Bucket.Arn}"
}
Note the AutoPublisAlias property. It is required for the CloudFront to be able to associate the Lambda@Edge function with one of the events it supports, in our case origin-request.
Ignore the Role property for now, we’ll return to roles and permissions later. With inline function you can include references to any resources and attributes, with the obvious exception of the CloudFront distribution to which your Lambda@Edge function is attached since this would create a circular dependency.
There are several issues with this approach. Firstly, it violates the separation of concerns principle as your code is now part of your infrastructure template. Then, the code can only be tested after it is deployed in the cloud. And finally, as mentioned before there is no way to include dependencies in your code since layers are not supported.
In the template inline Lambda@Edge function is used to generate HTML page with the name of the bucket and links to the URLs invoking two other functions.
CloudFront can add custom headers to the request it sends to the origin. Custom headers are configured in the Origins property of the DistributionConfig:
Origins:
- Id: cloudfront-default-origin
DomainName: aws.amazon.com
CustomOriginConfig:
HTTPPort: 80
OriginProtocolPolicy: match-viewer
OriginCustomHeaders:
- HeaderName: bucket
HeaderValue: !Ref Bucket
Here we defined a custom header with the name bucket which contains resource-accessed-from-edge-lambda the name of our S3 bucket.
The Lambda@Edge function can access the custom header from the event object. It is hidden inside request.origin.custom.customHeaders element of the cf object which is available as the first element of the event.Records list. The custom header has the following format:
{
"bucket": [{ "key": "bucket", "value": "resource-accessed-from-edge-lambda" }]
}
So, to get to this value we use:
bucket_name = event["Records"][0]["cf"]["request"]["origin"]["custom"]["customHeaders"]["bucket"][0]["value"]
I included some checks in the code to verify that the custom headers were defined. As a side note, debugging Lambda@Edge is a task most people would like to avoid so coding defensively is always a good idea.
The rest of the code is fairly straightforward. The functions creates a boto3 S3 resource and accesses the required bucket:
s3 = boto3.resource("s3")
bucket = s3.Bucket(bucket_name)
Next, we list the content of the bucket with bucket.objects.all(), iterate over the collection returned from this method and extract the object name (or key), its size and modification date into the stats list. The total_size variable is updated at each iteration. The function returns correct output for the empty bucket.
We could have optimised the function and store the bucket object in a global variable that would contain None on the first invocation of the handler function, but I did not want to further complicate this example.
To demonstrate the use of third-party dependencies the output is formatted as YAML with the help of PyYAML module. The dependencies are packaged automatically by SAM CLI, all you need to do is to create a pip dependency file in the same directory as your function source file.
The section of the template that defines this Lambda@Edge looks like this:
LambdaEdgeFunctionHeaders:
Type: AWS::Serverless::Function
Properties:
Role: !GetAtt LambdaEdgeFunctionRole.Arn
Runtime: python3.9
CodeUri: src/headers/
Handler: app.handler
AutoPublishAlias: live
Apart from optional Timeout and MemorySize properties this is all that is required to specify the Lambda@Edge resource.
CloudFront custom headers are quite versatile. You can define them per origin and re-use the same Lambda@Edge code to access different resources. Multiple headers can be added if required. The main drawback of the method is that the custom headers are not available at the static initialization stage. This issue can be partially overcome by storing the objects which depend on the resources passed in custom headers in global variables that are assigned during the first invocation of the handler function.
In the example, the custom header (bucket) was hard-coded in the function. This can be avoided by adding some “header discovery” code.
The Secrets Manager can store sensitive information like client IDs for third-party services that you don’t want to include in your code for security reasons. Since CloudFormation supports creating secrets from templates we can use Secrets Manager to store resources and other configuration information that can be easily retrieved by Lambda@Edge function.
The following CloudFormation resource creates a secret:
StoredSecrets:
Type: AWS::SecretsManager::Secret
Properties:
Description: Store resources for Lambda@Edge
Name: edge-lambda-resources
SecretString: !Sub '{"bucket": "${Bucket}"}'
Here bucket name is saved in a secret called edge-lambda-resources as a JSON-formatted string. You can use other formats as well, for example “
To access the secret from the Lambda@Edge function you can use the following code:
client = boto3.client(service_name="secretsmanager",
region_name="us-east-1")
response = client.get_secret_value(SecretId="edge-lambda-resources")
resources = json.loads(response["SecretString"])
bucket_name = resources["bucket"]
Be aware that get_secret_value throws an exception if it does not succeed. This could occur for a number of reasons including insufficient permissions or wrong Secret ID. It is recommended that you catch these exceptions and return an error code from your function to facilitate debugging instead of allowing it to crash.
The resource definition for this function is nearly identical to LambdaEdgeFunctionHeaders, the only difference is the source location CodeUri: /src/secrets/. The output is also formatted as YAML so requirements.txt includes this dependency for SAM to package.
It is important to remember that your Lambda@Edge function will be replicated across multiple regions and that secrets are specific to a region. Thus, specifying the region_name for your Secrets Manager client in boto3 is mandatory. Obviously, your function should have permissions to access the Secret Manage which brings us to the next section.
At this point we have defined three Lambda@Edge functions: an inline code that generates HTML content, the one that uses the custom headers and the variant that utilises the Secrets Manager. Apart from the standard execution role we need to define permissions for the latter two functions to access the S3 bucket. Additionally, read access to the Secrets Manager is also required for the LambdaEdgeFunctionSecrets to work.
In a real application you would need to define three execution roles, one for each function providing access only to the resources they use. For the purposes of this example only one role was created that combines all required permissions:
LambdaEdgeFunctionRole:
Type: "AWS::IAM::Role"
Properties:
RoleName: LambdaEdgeFunctionRoleWithAccessToBucketAndSecrets
Path: "/"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: "AllowLambdaServiceToAssumeRole"
Effect: "Allow"
Action:
- "sts:AssumeRole"
Principal:
Service:
- "lambda.amazonaws.com"
- "edgelambda.amazonaws.com"
Policies:
- PolicyName: AllowBucketAccess
PolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: "AccessBucketObjects"
Effect: "Allow"
Action:
- "s3:GetObject"
- "s3:ListBucket"
Resource:
- !Sub "arn:aws:s3:::${Bucket}/*"
- !Sub "arn:aws:s3:::${Bucket}"
- PolicyName: AllowAccessToSecret
PolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: "AllowReadingSecretValue"
Effect: "Allow"
Action: "secretsmanager:GetSecretValue"
Resource: !Ref StoredSecrets
This role definition adds two policies to the standard AWS Lambda@Edge execution role to provide permissions to list the bucket and read its content, and read-only access to the secret that was added to the Secrets Manager. The inline function needs neither of the permissions and the “headers” lambda only requires the S3 bucket permissions.
Most of the hard work done, we now need to configure the CloudFront to use the lambdas. Even though the distribution is not serving any content from any origin we still need to define one, so aws.amazon.com is used as a placeholder. The “inline” function is added to the DefaultCacheBehavior:
LambdaFunctionAssociations:
- EventType: origin-request
LambdaFunctionARN: !Ref LambdaEdgeFunctionInline.Version
The function is triggered on the origin-request event. It returns a response containing a simple HTML document which CloudFormation serves without accessing the origin.
Two other functions are added as CacheBehaviours elements for /headers/* and /secrets/* paths. So, apart from the custom headers discussed above, the setup is fairly standard.
The full template, source code for the functions and SAM configuration file is available on GitHub. The instructions for setting up the projects can be found in the README file.
We considered three common approaches to passing parameters into a Lambda@Edge function: (1) inlining the code in the template; (2) using custom headers; and (3) storing the information in the Secrets Manager. Other options are available to the developer including CI/CD and CloudFormation custom resources. We focused on solutions that rely on SAM CLI, do not require other tools or additional build and deployment steps.
Thank you for reading. To report a problem with this code please raise an issue on GitHub. If you found this information useful please star the repository, this way I know you read the post to the end. Happy coding and remember “Clouds are Fun”.