Importing VPC IDs into a stack with CDK

1. The problem

I wanted to add a resource to an existing VPC using CDK in TypeScript the other day. The VPC was in another stack, and I exported its ID as an Output there:

import { CfnOutput } from 'aws-cdk-lib';

// ...
// inside the construct
new CfnOutput(this, 'VpcIdOutput', {
  exportName: `my-vpc-id-${stage}`,
  value: this.vpc.vpcId,
})

In the other stack, where I wanted to use the VPC, I tried the following:

import { Vpc } from 'aws-cdk-lib/aws-ec2';

// ...
// inside the construct
const vpc = Vpc.fromLookup(this, 'Vpc', {
  vpcId: Fn.importValue(`my-vpc-id-${stage}`),
});

When I wanted to create a CloudFormation template from the CDK code with cdk synth, I received the following error:

Error: All arguments to Vpc.fromLookup() must be concrete (no Tokens)

It was not the optimal outcome, and I had to find another way to import the VPC ID.

2. The reason

We have a couple of things going on here.

2.1. Tokens

Tokens are placeholders for the resources whose values are only available at a later phase of the deployment process.

We will see a string containing a sequence of random characters if we log the tokens. At the beginning of the deployment, CDK builds the template from the code. At this time, CDK will pass placeholders on to the relevant part of the template code when needed. An example is when we use the imported VPC ID to get the whole VPC object.

At a later stage, when the token’s value (for example, a VPC ID) is available, CDK will replace it with the actual value.

2.2. Vpc.fromLookup

In this case, the Vpc.fromLookup() method doesn’t allow tokens as arguments, only concrete values. It indicates that we can synthesize the infrastructure code if we replace the import value with the real hard-coded VPC ID. It is because it’s a defined value and not a placeholder that the process will resolve later.

The AWS documentation states it:

Calling this method will lead to a lookup when the CDK CLI is executed.
You can therefore not use any values that will only be available at
CloudFormation execution time (i.e., Tokens).

vpc.fromLookup() didn’t work, so we have to find another way to get the VPC.

2.3. Synthesis

The cdk synth CLI command executes the synthesis phase.

When CDK synthesizes the constructs, it will create some necessary artifacts for the deployment.

One of these artifacts is the CloudFormation template, which CDK creates and writes in JSON to the cdk.out folder. It also shows the template in YAML format in the terminal.

The generated template is now ready for deployment.

The error message and the documentation indicate that imported VPC IDs are not available at the time when the Vpc.fromLookup() method runs. It will only be available at CloudFormation execution time. It’s the last stage when CloudFormation creates or updates the resources that we have defined in the CDK app code. The process will resolve most placeholder (token) values at this phase.

3. A solution

But we can replace the Vpc.fromLookup method with Vpc.fromVpcAttributes:

import { Vpc } from 'aws-cdk-lib/aws-ec2';

const vpc = Vpc.fromVpcAttributes(this, 'Vpc', {
  availabilityZones: ['us-west-2a', 'us-west-2b'],
  privateSubnetIds: [
    Fn.importValue(`private-subnet-id-1-${stage}`),
    Fn.importValue(`private-subnet-id-2-${stage}`)
  ],
  vpcId: Fn.importValue(`my-vpc-id-${stage}`),
});

In this solution, we will need the subnet ids where we want to create the additional resources. So we will have to export them from the stack where they live.

CDK offers lots of options here. We can import not only private but also public and isolated subnets. I wanted to add the resources to the private subnets in this case.

This solution involves more work in the current and other stacks, but it is a working solution.

4. Considerations

A word of caution about Vpc.fromVpcAttributes. The documentation has the following warning:

NOTE: using `fromVpcAttributes()` with deploy-time parameters
(like a `Fn.importValue()` or `CfnParameter` to represent a list
of subnet IDs) sometimes accidentally works. It happens to work
for constructs that need a list of subnets (like `AutoScalingGroup`
and `eks.Cluster`) but it does not work for constructs that need
individual subnets (like `Instance`).
See https://github.com/aws/aws-cdk/issues/4118 for more information.

Prefer to use `Vpc.fromLookup()` instead.

The sometimes accidentally works part sounds a bit scary as AWS won’t guarantee that the method will work in every scenario. So we should thoroughly test our solution with cdk synth and deploy it to our feature stage before we merge the code.

But the above solution has worked for me. It can be a good workaround in this scenario!

I’ve seen solutions that use AWS SDK to get all CloudFormation exports, filter out the one with the correct VPC ID and pass it down to the construct. The script can run when the CDK app starts.

I didn’t want to implement this solution because it seemed to require more work and would have resulted in harder-to-read code. Also, I wouldn’t mix up CDK with SDK. I believe CDK is about infrastructure, and SDK is about application code. But this is just my opinion, so feel free to challenge me and use SDK in your CDK code.

If nothing else works, we can always hardcode the value of the VPC ID. It’s not elegant or dynamic, but it will work with Vpc.fromLookup(). In our world of “deploy as many features as you can in the shortest time possible”, hardcoding can sound good temporary solution. (We all know what it means. It will probably remain there forever.)

5. Summary

Imported values from another stack are available at a later phase of the deployment process. Some CDK methods won’t accept placeholders called tokens for these values.

There are different ways to work around this issue. We can use another CDK method if it exists, create custom solutions to get the value from all exports, or hardcode it. Each has pros and cons, and we should use the solution that fits our use case and circumstances.

6. Further reading

Tokens - AWS documentation about placeholder tokens in CDK

App lifecycle - Brief description of the CDK app lifecycle stages

class Vpc (construct) - The VPC construct API documentation for CDK

Subnets for your VPC - Subnet basics, private and public subnets