Returning customized content based on location with S3 Object Lambda

03 Aug 2023
7 min read
AWS
When we retrieve objects from S3, we can customize their content based on geographic location with S3 Object Lambda. This way, we might not need to upload multiple versions of the same object. Instead, the Object Lambda function will dynamically transform the response based on criteria that we define.

1. Problem statement

Bob has an application that should return the same object with some of its elements adjusted to the caller’s geographic location.

Say for the sake of this example that the object name is location.json, and Bob stores it in an S3 bucket.

location.json looks like this:

{
  "location": "Test country"
}

Users will retrieve this object from S3. The returned response should contain the caller’s country in the location and the currency they use in that country in the currency property. Something like this:

{
  "location": "Germany",
  "currency": "EUR"
}

One solution is to create multiple versions of the same object and store them all in the bucket. The application will then return the version corresponding to the user location.

Another solution is to use S3 Object Lambda that dynamically changes the content. This way, we will write slightly more code and provision more resources but eliminate the need to store tens of versions of the same object.

Let’s take a look at the second solution in more detail below.

2. Architecture

Creating an Object Lambda is slightly more complex than provisioning a simple resource because it involves several components. I will skip the details about creating an Object Lambda (references are at the bottom of this page). Instead, let’s review a simple but working architecture:

Sample architecture
Sample architecture

The key parts of the architecture are the following.

2.1. S3 bucket

First, we need an S3 bucket where we store the content. We can call this bucket test-bucket.

2.2. Access Point

We must then create an S3 Access Point called test-access-point.

It will be a supporting access point, and the Lambda function will use it to retrieve the object from S3 via a pre-signed URL. Lambda will receive the URL as an input, and the application needs to call the endpoint.

2.3. Lambda function

It comes as no surprise that we need a Lambda function too. Let’s call it object-lambda-function.

The function will receive a config object with the pre-signed URL in the input event. It downloads the file from the bucket, changes the response based on the relevant config object properties, and then passes the modified response to a GetObject operation. The S3 GetObject API call will then return the modified object. More details on the process (including some sample codes) will come below.

2.4. Object Lambda Access Point

The next resource we must create is an Object Lambda Access Point. Its alias will become the entry point to our application. In this case, it will serve as the origin of a CloudFront distribution.

If the name of the access point is test-object-lambda-access-point, its alias will look like this:

test-object-lambda-a-SOME_CHARACTERS--ol-s3

Other services (e.g., other Lambda functions, ECS containers) can also invoke the endpoint alias.

2.5. CloudFront distribution

Finally, a CloudFront distribution will receive the requests from end users. Let’s emphasize some key points here.

Origin

As discussed above, the Object Lambda Access Point will be the origin. We can’t specify the access point’s alias (see above) as is, but we should format it properly:

test-object-lambda-a-SOME_CHARACTERS--ol-s3.s3.eu-central-1.amazonaws.com

The origin must have the ALIAS.s3.REGION.amazonaws.com format. Otherwise, CloudFront will throw an error.

Origin Access Control

Object Lambda Access Points cannot be public, so we must add an origin access control to the origin. The origin type should be S3.

Cache policy

In this example, the Lambda function will modify the content based on the location information. For the logic to use the correct location, we should add some built-in CloudFront headers to the cache key. Let’s add the CloudFront-Viewer-Country and CloudFront-Viewer-Country-Name headers. There are many more headers available. I have attached a link at the end of this post where you can find more information about them.

CloudFront location headers
CloudFront location headers

We can create a custom cache policy called geographic-locations, and add the above headers to the Cache key settings. If we do so, CloudFront will store the responses for various viewer requests in the corresponding edge location cache.

3. Permissions

Everything on AWS is about permissions. Event one service cannot call another without proper permissions.

We must add some IAM statements to the elements of architecture.

3.1. Object Lambda Access Point

Our test-object-lambda-access-point is just a regular access point, so we can attach an access point policy to it. Because the access point is the origin of the CloudFront distribution, we must allow it to invoke the endpoint:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudfront.amazonaws.com"
      },
      "Action": "s3-object-lambda:Get*",
      "Resource": "arn:aws:s3-object-lambda:eu-central-1:123456789012:accesspoint/test-object-lambda-access-point",
      "Condition": {
        "StringEquals": {
          "aws:SourceArn": "arn:aws:cloudfront::123456789012:distribution/DISTRIBUTION_ID"
        }
      }
    }
  ]
}

It is a very similar resource-based policy to the one we have in an S3 bucket when the origin directly points to the bucket. The difference is the Action (s3-object-lambda) and the Resource, which is the access point in this case.

3.2. Supporting access point

The supporting access point called test-access-point should have a policy similar to this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudfront.amazonaws.com"
      },
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:eu-central-1:123456789012:accesspoint/test-access-point",
        "arn:aws:s3:eu-central-1:123456789012:accesspoint/test-access-point/object/*"
      ],
      "Condition": {
        "ForAnyValue:StringEquals": {
          "aws:CalledVia": "s3-object-lambda.amazonaws.com"
        }
      }
    }
  ]
}

This policy allows CloudFront to call S3 actions through the access point only if the request comes from S3 Object Lambda.

3.3. Bucket policy

test-buckets policy should allow every call from all access points (including test-access-point) for the given account:

{
  "Effect": "Allow",
  "Principal": {
    "AWS": "*"
  },
  "Action": "*",
  "Resource": [
    "arn:aws:s3:::test-bucket",
    "arn:aws:s3:::test-bucket/*"
  ],
  "Condition": {
    "StringEquals": {
      "s3:DataAccessPointAccount": "1232456789012"
    }
  }
}

3.4. Lambda function resource policy

We should allow CloudFront to invoke the Object Lambda function, which is a regular function, so it has a resource-based policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudfront.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:eu-central-1:123456789012:function:object-lambda-function"
    }
  ]
}

3.5. Lambda function execution role

As mentioned above, Object Lambda comes with a separate Action set. Because the function will write the modified content to the GetObject operation, we should Allow the s3-object-lambda:WriteGetObjectResponse action in the execution role.

4. Code

Finally, some sample Lambda function code!

The function will do the following:

  1. Get the object context and user location info from the event.
  2. Download the object from S3 using the pre-signed URL.
  3. Convert the content based on the required business logic.
  4. Send the modified content to the `GetObject` operation.

4.1. Object context and location info

When a user wants to retrieve the location.json object from S3 using our architecture, the event object will look like this:

getObjectContext: {
  outputRoute: ROUTE_ID,
  outputToken: TOKEN,
  inputS3Url: 'https://test-access-point-123456789012.s3-accesspoint.eu-central-1.amazonaws.com/location.json?SOME_INFO'
},
configuration: {
  accessPointArn: 'arn:aws:s3-object-lambda:eu-central-1:123456789012:accesspoint/test-object-lambda-access-point',
  supportingAccessPointArn: 'arn:aws:s3:eu-central-1:123456789012:accesspoint/test-access-point',
  payload: '{"accessPoint": "test"}'
},
userRequest: {
  url: 'test-object-lambda-a-SOME_CHARACTERS--ol-s3.s3.eu-central-1.amazonaws.com/location.json',
  headers: {
    'CloudFront-Viewer-Country': 'GB',
    'CloudFront-Viewer-Country-Name': 'United Kingdom',
    // some other headers
  }
},
// other properties

Let’s quickly go over the properties.

getObjectContext

The function will need outputRoute and outputToken when it sends the modified content. It will use the inputS3Url pre-signed URL to download the original location.json object from S3.

configuration

The payload property is an optional object we can set for the transformation at the given Object Access Point. In our case, the transformation is GetObject (because we want to retrieve a specific object from S3). We can specify other transformation types too, and flag them in payload. Or, we can add some custom fields here.

This example won’t use this option.

userRequest

This object contains information about the geographic location of the user. The function will extract these headers and use them to create custom content.

4.2. Downloading the original object from S3

The code uses axios to retrieve location.json from S3 via the pre-signed URL defined in getObjectContext.

4.3. Converting the original object

Similar to location.json, the business logic is rather basic. The currencyMap object contains the currencies for some countries. If the user is from a country not specified in the map, we will add the not specified value to the currency field.

We can also have a database call here instead of the map or any other logic that the business requires.

4.4. Sending the modified content

We can use the SDK v3 to send the modified object. The RequestRoute and RequestToken fields are required, and we can get them from getObjectContext.

4.5. Putting everything together

So the sample code can look like this:

import { S3Client, WriteGetObjectResponseCommand } from "@aws-sdk/client-s3";
import axios from 'axios';

const s3Client = new S3Client();

const currencyMap = {
  DE: 'EUR',
  US: 'USD',
  GB: 'GBP',
};

export const handler = async (event) => {
  const { getObjectContext: objectContext, userRequest } = event;
  const countryName = userRequest.headers['CloudFront-Viewer-Country-Name'];
  const country = userRequest.headers['CloudFront-Viewer-Country'];

  let data;
  try {
    const response = await axios.get(objectContext.inputS3Url);
    data = response.data;
  } catch (error) {
    console.error('Axios error: ', error.message);
    throw error;
  }


  const converted = {
    ...data,
    location: countryName,
    currency: currencyMap[country] ?? 'not specified',
  };
  const input = {
    RequestRoute: objectContext.outputRoute, // required
    RequestToken: objectContext.outputToken, // required
    Body: JSON.stringify(converted),
  };

  const command = new WriteGetObjectResponseCommand(input);
  try {
    await s3Client.send(command);
  } catch (error) {
    console.error('Object Lambda error: ', error.message);
    throw error;
  }

  return {
    statusCode: 200,
  };
};

Let’s now try to retrieve location.json via the distribution URL from a browser window, and we should get the location-specific response.

5. Summary

S3 Object Lambda can dynamically modify the content of the downloaded object based on some business logic. We can use the Object Lambda Access Point alias as a CloudFront origin and make our application available for users in different geographic locations.

We must add the relevant IAM permissions to each element of the architecture.

6. Further reading

Getting started with Amazon S3 - How to create a bucket in S3

Getting started with Lambda - How to create a Lambda function

Creating Object Lambda Access Points - No further info is needed here

Creating a distribution - This one is straighforward, too

Creating a new origin access control - Straight-to-the-point titles in the AWS documentation

Creating cache policies - What can I say

Headers for determining the viewer’s location - CloudFront headers we can use to identify the location of the user

Previous post
Next post