Dynamically routing user requests to multiple CloudFront origins

1. The scenario

Say we have a new website and want to test it with real users in a production environment. We allow a subset of users access to the new page while the majority should still go to the existing site. This testing type is known as A/B testing or split testing.

In this scenario, we host the website on S3. We have two buckets, one for the existing site and another for the new one, which we want to test. We have also set up a CloudFront distribution in front of the buckets for optimization and security reasons. The buckets are private, and only the distribution can access them through an origin access control. The website files are in the root of the buckets, so no v2 folder exists for the new variant. Both the existing and new sites should be available under the same URL.

It’s a requirement that 10% of the users should view the new site via the same custom domain name as the existing page. We need to perform the user selection and redirection inside the AWS infrastructure.

2. Solution overview

The CloudFront distribution has two origins, one for the existing bucket and another for the new one. It also has the default behavior with the * path pattern and the existing bucket defined as the origin.

CloudFront can deploy Lambda functions to the points of presence around the world and invoke them in various situations. We can use these functions to modify the requests and responses originated from both the users and the origin.

We can create two Lambda functions.

First, we decide if the user is lucky to be in the 10% who has the privilege to access the new page. We want to remember these users, so we add a cookie to their requests. We invoke this function on the viewer request.

The second function assigned to the origin request will check if the cookie is present. If so, the function will change the default origin (pointing to the existing website bucket) to the new one.

3. Settings and code

Let’s see how we can set up this solution.

3.1. CloudFront settings

We need a custom origin request policy that forwards our cookie called X-New-Page-Test (the name can be different) to the origin. We want the second (origin request) Lambda@Edge function to decide if it needs to redirect the request to the new origin.

3.2. Viewer request Lambda

We associate the first Lambda function to the viewer request (configurable on the Behavior).

An example code can look like this:

import { CloudFrontRequestEvent, CloudFrontRequest, CloudFrontResponse }
from 'aws-lambda';

const NEW_PAGE_COOKIE = 'X-New-Page-Test';

export const handler = async (event: CloudFrontRequestEvent):
  Promise<CloudFrontRequest | CloudFrontResponse> => {
  const request = event.Records[0].cf.request;
  const cookies = request.headers.cookie;

  // 1. Check if the cookie is present
  const hasNewPageCookie = cookies?.find((cookie) =>
    cookie.value.includes(NEW_PAGE_COOKIE));
  if (hasNewPageCookie) {
    // The cookie is present, and the function has nothing to do here.
    // The origin request function will handle it.
    return request;
  }

  // 2. If the cookie is not present, we decide if the user should access
  // the new page
  const shouldGoToNewPage = Math.random() >= 0.9;
  if (shouldGoToNewPage) {
    const response = {
      status: '302',
      statusDescription: 'Found',
      headers: {
        'cache-control': [
          {
            key: 'Cache-Control',
            value: 'no-store',
          },
        ],
        'set-cookie': [
          {
            key: 'Set-Cookie',
            value: `${NEW_PAGE_COOKIE}=true`,
          },
        ],
        location: [
          {
            key: 'Location',
            value: request.uri,
          },
        ],
      },
    };
    // 3. We send the request back to the browser with the cookie
    return response;
  }

  // 4. The user doesn't belong to the tester group, so no cookie is set.
  // The request proceeds as intended.
  return request;
};

The function does a couple of things.

It checks if the X-New-Page-Test cookie is in the request (1). If so, it means that we already set it up earlier. The request can proceed with the cookie, and the origin request Lambda@Edge function will re-route it to the new bucket.

If the cookie is not in the request, the function decides if it should add to it (2). The Math.random() function returns a random number between 0 and 1. If the number is greater than or equal to 0.9, the user will belong to the 10% who get access to the new version. In this case, we add the X-New-Page-Test cookie and respond with a 302 status code. It will indicate to the browser that it should redirect the request to the URL specified in Location. But the value of the Location header is the same as before, so the browser will call the same URL again. This time, the cookie will be part of the request (3).

If the user turns out to belong to the 90% based on the Math.random() function’s outcome, we don’t set the custom cookie. Everything will go as usual this time, and we return the original request object (4).

3.3. The origin request Lambda

The second Lambda@Edge function will redirect the user to the new site if the X-New-Page-Test cookie is in the request.

Its code can look like this:

import { CloudFrontRequestEvent, CloudFrontRequest } from 'aws-lambda';

const NEW_PAGE_COOKIE = 'X-New-Page-Test';

export const handler = async (event: CloudFrontRequestEvent):
  Promise<CloudFrontRequest> => {
  const request = event.Records[0].cf.request;
  const cookies = request.headers.cookie;

  // 1. Check if our cookie is in the request
  const hasNewPageCookie = cookies?.find((cookie) =>
    cookie.value.includes(NEW_PAGE_COOKIE));
  if (hasNewPageCookie) {
    // 2. Change the relevant request fields to the new value
    const newPageDomainName = 'NEW_BUCKET_NAME.s3.eu-central-1.amazonaws.com';
    request.headers.host = [{ key: 'Host', value: newPageDomainName }];
    request.origin = {
      s3: {
        domainName: newPageDomainName,
        authMethod: 'none',
        customHeaders: {},
        path: '',
        region: 'eu-central-1',
      },
    };

    return request;
  }

  // 3. The cookie is not in the request, so go to the original page
  return request;
};

It starts similarly to the first function. We check if the X-New-Page-Test cookie is part of the request (1).

If so, we change the host header and domainName field in the origin object to the domain name of the second bucket that contains the new version (2). We then return the modified request object. CloudFront will then know that the request should go to the second origin.

If the cookie is not in the request, we return the same request object we received from CloudFront. In this case, the request will proceed to the existing bucket as usual.

4. Considerations

I want to highlight some points regarding the solution.

4.1. Lambda permissions

CloudFront can only invoke the functions if we add the edgelambda.amazonaws.com service to the Principal element. So the trust policy of both Lambda functions’ execution role should look like this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "edgelambda.amazonaws.com",
          "lambda.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

The permission policy is a different discussion. In our case, the AWSLambdaBasicExecutionRole AWS-managed policy is sufficient as the functions don’t call any other AWS service.

4.2. Other Lambda requirements

For a Lambda function to become Lambda@Edge, we should be aware of some caveats:

• The functions must be deployed to us-east-1 regardless of the buckets' regions. Because CloudFront is a global service, it lives in Northern Virginia.
• The maximum allowed function timeout is 5 seconds.
• Lambda@Edge only supports the x86_64 architecture. We can't use Graviton-based processors at the moment for Lambda@Edge.
• We must specify a qualified function ARN when we configure Lambda@Edge in the CloudFront behavior. It means that we must create a numbered version of the function and can't use $LATEST.
• We can't use environment variables in the functions.

4.3. Logs in the closest region

Lambda@Edge functions send their logs to a log group in the region closest to the user. It’s eu-central-1 for me, even if I have deployed the functions to us-east-1.

So, if your first thought is that CloudFront doesn’t trigger the function, check CloudWatch Logs in the region closest to you before starting debugging.

We can always aggregate the logs into a single log group in a dedicated region, but it’s beyond the scope of this article.

4.4. Not the only way to solve the problem

As always, the solution described above is not the only way to do A/B testing with CloudFront. Requirements are always different, and there might be a solution that suits a specific use case better.

That said, you might find an easier way to perform a similar logic. A different solution may be a better fit for your scenario, which is it’s OK. We can solve one problem in multiple ways. I added some links to the end that provide some alternative solutions.

5. Summary

We can use Lambda@Edge functions to perform live A/B testing with CloudFront. We want to send a group of users to the new version of our website by adding a custom cookie to the request. A Lambda function can inspect the request, and if the cookie is present, it can dynamically point the request to the new origin.

6. Further reading

Content-based dynamic origin selection - examples - Dynamically selecting origin for different scenarios

Example: A/B testing - Path-based A/B testing in CloudFront

Performing A/B tests on static websites using Cloudfront and Lambda@Edge - Great article on the same problem with a slightly different solution

Getting started with Lambda - How to create a Lambda function

Creating a distribution - How to create a CloudFront distribution