Protecting APIs with custom headers in CloudFront

1. The problem

Alice is a cloud engineer and works for Example Corp. They built one of their microservices a few years ago, and although Alice recently wanted to redesign the architecture, deploying new features was always a priority.

The service was behind a legacy Classic Load Balancer, and it had worked well until the service received a DDOS attack one day.

The service was down for hours, and the company lost tens of thousands of dollars.

2. Chosen solution

Suddenly it became urgent to redesign the service.

Alice and her coworkers decided to put the service behind an Amazon API Gateway and protect the API with CloudFront.

2.1. Regional endpoint with self-managed distribution

When we create a new REST API with API Gateway, we can choose if the endpoint is edge-optimized or regionally deployed.

Edge-optimized endpoints come with an AWS-managed CloudFront distribution, which we can’t access, so we can’t make any changes to its settings.

But we can manually create and deploy a self-managed CloudFront distribution and put it before the regional endpoint. In this case we need to select the API URL (which has the https://{restapi_id}.execute-api.{region}.amazonaws.com/{stage_name}/ format) as a custom origin for the distribution.

2.2. Restricting access with origin headers

We haven’t finished it yet because the API is public, and anyone can invoke it directly.

We can set up a custom origin header in CloudFront, which will protect our API. When we send a request to the endpoint via CloudFront (either by invoking the distribution’s URL or setting up a custom domain), CloudFront will add the custom header to the request made to API Gateway (origin request). This header won’t be in the client request to CloudFront (viewer request) because the client is not aware of the existence of the custom header.

The custom header is called x-api-key, and we add a random string as a value. The longer the string is the better it is for our purpose (although there might be some unexpected limitations on the length, see this article, but 32 characters work well from my experience).

We then set up this string as an API key in API Gateway. It means that API Gateway will deny any requests that don’t have the specified API key. This way, direct invocations to the endpoint will not receive a successful response because the caller won’t know the API key. But when we invoke the endpoint via CloudFront, the request will succeed because the distribution will add the necessary key to the request.

3. Protection at the edge

CloudFront provides effective protection against DDOS attacks at Layer 3/4 (network and transport layers) and Layer 7 (application level).

It integrates with Shield Standard and WAF, and it won’t let malicious traffic reach the origin because it captures it at the adge.

4. Main steps

I’m not writing about how to create an API or set up a CloudFront distribution (links to the relevant documentation in the References section). Instead, let’s discuss the core steps that make the architecture work.

4.1. Forwarding all headers to the origin

We have to set up the behavior to forward all headers to the origin, in this case, API Gateway. We can do it by selecting the CachingDisabled cache policy.

This setting will prevent CloudFront from storing the headers in the cache key.

4.2. Adding the custom header

We also have to add the x-api-key header with the value of the API key in the origin settings:

CloudFront will add this header to all requests made to API Gateway.

4.3 Creating an API Key in API Gateway

We must let API Gateway know of the custom header CloudFront will send. API Gateway will convert the value of the x-api-key header to an API key value. Here we choose HEADER as the API Key source.

After creating the API key, we can add it to the endpoints we want to protect.

4.4. Setting up a usage plan

The API key alone won’t work because it should belong to a usage plan. We have to set up a usage plan in API Gateway, attach it to the API stage, and add the newly created API key.

If we don’t create a usage plan, we’ll receive a 403 error and will see the following log message:

API Key  not authorized because method 'GET /...' requires API Key and API Key is not associated with a Usage Plan for API Stage STAGE NAME: API Stage is not in a usage plan

4.5. Ready to go!

This part of the architecture is complete, we can make requests to the API via CloudFront, and we should receive the response payload.

If we investigate the API Gateway logs in Cloudwatch, we’ll see the User-Agent=Amazon CloudFront header.

But if we try to invoke the API directly, we will receive a 403 Forbidden error.

5. Considerations

5.1. Other solutions are possible

It’s not the only solution Alice can chosen. She can set up an Application Load Balancer if she wants to keep the original architecture. CloudFront can protect ALBs in a similar way.

In this case, Alice has to create a rule in the ALB, which watches the custom header CloudFront sents. If the custom header is in the request, ALB will forward it to the target. If not, it will deny the request.

Alice can also choose different solutions with CloudFront and API Gateway, which I will explore in a separate post.

5.2 Securing the API key

Malicious actors are also aware that the x-api-key header can exist, so they might try to invoke the API directly. Many APIs have an easily guessable custom domain like api.example.com. In this case, Alice should be very thoughtful when protecting the endpoint from unwanted access.

It’s important to find a strong and secure API key and apply rate limits in API Gateway.

Alice can also create a rule in WAF to block the IP addresses from which the number of requests exceeds the allowed limit.

6. Summary

Defense against DDOS attacks has multiple components. One of them is to protect the API endpoints and hide them behind a CloudFront distribution.

If we set up custom origin headers in CloudFront, we will ensure that the API will only accept requests originating from the distribution. Any direct requests that don’t have the API key will result in a 403 Forbidden error.

7. References and further reading

Creating a REST API in Amazon API Gateway - It does exactly what the title of the article says

Creating a distribution - How to create a CloudFront distribution

Custom origin headers - How to add custom headers to the CloudFront distribution

Set up API keys using the API Gateway console - No surprise here either

API Gateway API usage plans - Creating a usage plan, adding an API key to it and adding it to a stage