Encrypting objects with dual-layer server-side encryption using Lambda functions

1. Dual-layer server-side encryption (DSSE-KMS)

Some companies must meet strict compliance rules requiring that they must encrypt top-secret data with two layers.

One solution to this problem was to encrypt the object on the client side, and then encrypt it again using an existing S3 server-side encryption (SSE) option.

S3 now supports dual-layer server-side encryption (DSSE-KMS), which makes this process easier to apply. It provides two independent layers of encryption at the object level where we have control over the permission and rotation of the key. This way, we won’t need the extra process of encrypting the object before we upload it to the bucket. S3 will manage the whole process after we have specified the encryption key.

DSSE-KMS is similar to other server-side encryption methods meaning that S3 performs the encryption with the key we specify. It supports both bucket and object-level encryptions. We can use the default S3 KMS key (aws/s3), or we can create and control a customer-managed key. Either way, the key must be in the region where we have provisioned the bucket.

It’s important to note that dual-layer encryption doesn’t support bucket keys.

Dual-layer encryption comes with an additional cost. In addition to the usual fee for KMS API calls, we pay for the second layer of encryption and decryption per gigabyte of data.

2. How it works

In this scenario, S3 performs the encryption, and KMS manages the key.

S3 will first request a data key from KMS, which provides both a plaintext and an encrypted version of the data key. The data key encryption will occur using the KMS key we specify. S3 will then encrypt the object with the plaintext data key, removes it from memory, and stores the encrypted data key as object metadata.

Then, S3 will repeat the process and encrypt the encrypted data object for the second time.

When we want to get the object from S3, the GetObject API call will automatically return the decrypted object. S3 sends the encrypted data keys to KMS and requests their decryption. KMS will respond with the plaintext data keys. S3 will then use the plaintext data keys to unencrypt the object and removes them from memory afterward.

3. Let’s see it in the code

Let’s say that we have a Lambda function that uploads objects to an S3 bucket with dual-layer encryption.

We’ll also have another function that downloads the object from S3 and performs some logic with it.

This blog doesn’t explain how to create buckets in S3 or functions in Lambda. I’ve attached some links below that show these operations in detail.

3.1. Uploading objects to S3

Let’s create a Lambda function that uploads an object to S3. A sample code can look like this:

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';

// The object can come from anywhere
const testObject = {
  test: 'object'
}

const { KMS_KEY_ID, BUCKET_NAME } = process.env;

const s3Client = new S3Client();

export const handler = async (event) => {
  const params = {
    Bucket: BUCKET_NAME,
    Body: JSON.stringify(testObject),
    Key: 'test-object.json',
    SSEKMSKeyId: KMS_KEY_ID, // specify the KMS key
    ServerSideEncryption: 'aws:kms:dsse', // request dual-layer encryption
  };
  const command = new PutObjectCommand(params);

  try {
    const response = await s3Client.send(command);
    return response;
  } catch (error) {
    console.error('ERROR: ', error)
    throw error;
  }
}

The first key point is the SSEKMSKeyId. The value of this property is the key ID that S3 will use the encrypt the object. The ServerSideEncryption field indicates that we want it to be dual-layer encrypted. We get the bucket name and KMS key ID from environment variables.

The Lambda execution role must have kms:GenerateDataKey permission on the corresponding KMS key:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
          "kms:GenerateDataKey"
      ],
      "Resource": "arn:aws:kms:eu-central-1:123456789012:key/KEY_ID"
    }
  ]
}

At first, it might be strange that we don’t need the kms:Encrypt permission. It is because, as pointed out above, S3 requests data keys from KMS and encrypts the object using the plaintext data keys.

We would need the kms:Encrypt permission only if we wanted to encrypt something small (max. 4 KB of data) with the KMS key, but this is not the case. We use KMS keys to encrypt data keys or small data, so the size limit is intentional.

Of course, we should add the s3:PutObject permission to the role too, or the function will throw an error on the object upload.

3.2. Getting objects from S3

Getting dual-layer encrypted objects from S3 is even easier than uploading them. The code can look like this:

import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3';

const { BUCKET_NAME } = process.env;

const s3Client = new S3Client();

export const handler = async (event) => {
  const params = {
    Bucket: BUCKET_NAME,
    Key: event.key,
  };
  const command = new GetObjectCommand(params);

  try {
    const response = await s3Client.send(command);
    return response.Body.transformToString();
  } catch (error) {
    console.error('ERROR: ', error);
    throw error;
  }
};

We add the object’s key we want to download from the bucket as part of the event object. The only other mandatory input field is Bucket, which we get from an environment variable.

The execution role must have kms:Decrypt permission or the function will throw an AccessDenied error. S3 will need the plaintext version of the encrypted data keys, and KMS will require valid permission before it decrypts them. Both data encryption and decryption will happen using plaintext data keys.

4. Summary

S3 has recently introduced dual-layer server-side encryption, which meets some strict regulatory requirements. DSSE-KMS applies two layers of encryption and is a suitable replacement for the more complicated client-side plus server-side encryption combination.

We should specify the KMS key we want S3 to use. We should also provide kms:GenerateDataKey permission for uploading and kms:Decrypt permission for downloading objects.

5. Further reading

Amazon S3 pricing - Dual-layer encryption and other S3-related pricing info (Hint: The pricing page URLs follow the same pattern. Replace s3 with something else, like ec2 to get pricing information for EC2.)

Getting started with Amazon S3 - How to create a bucket in S3

Getting started with Lambda - How to create a Lambda function

Using dual-layer server-side encryption with AWS KMS keys (DSSE-KMS) - Some more info on DSSE-KMS