Simplifying CloudFormation stack management with the new refactoring feature

13 Feb 2025
6 min read
AWS
The new CloudFormation Stack Refactoring feature simplifies modifying existing stacks. With minimal coding and a few CLI commands, we can rename and move resources between stacks.

1. Problem statement

If you’ve worked with CloudFormation stacks, regardless of the infrastructure as code method, you’ve likely encountered the need to change a stack.

Either the logical name didn’t accurately reflect the resource, or you wanted to move one or more resources to a different stack.

1.1. Changing the logical name

The logical name (or ID) is unique within the stack, and other resources reference it to use the given resource. Changing the logical name requires updating every reference in the template file.

AWSTemplateFormatVersion: '2010-09-09'

Resources:
  MyBucket: # <-- This is the logical name (ID).
    Type: AWS::S3::Bucket
    Properties:
      BucketName: mybucket-1234 # This is the physical name of the bucket,
        # which is usually different from the logical ID. It's what we see
        # in the S3 console.

When you update a stack after changing a resource’s ID, CloudFormation creates a new resource with the new logical ID and, by default, attempts to delete the old one. This behaviour can be problematic, especially if the resource is part of a heavily used application. For example, changing the logical name of an S3 bucket requires migrating all objects from the old bucket to the new one. This can be inconvenient.

1.2. Refactoring a stack

Another scenario is stack refactoring. It’s common to work with multiple stacks, even for smaller applications. These stacks often depend on each other, using resources from one stack in another by exporting and importing.

As the application grows, the stacks become heavily interdependent. In such cases, you might want to place some resources in a centralized, shared stack, aiming for a hub-and-spoke style architecture.

Reducing stack dependencies
Reducing stack dependencies

Migrating resources from one stack to another almost always requires taking down some stacks, leading to service disruptions.

2. Stack refactoring

AWS announced the CloudFormation Stack Refactoring feature a few days ago. This addition can solve most of the problems described above.

As a new feature, it currently has some limitations, and as of writing this, it only works in the CLI and the SDK.

In this post, I’ll explore some scenarios where this feature can be a good solution. I’ll use native CloudFormation yaml templates and CLI commands throughout.

This post assumes that you already have one or more CloudFormation stacks running.

3. How stack refactoring works

During stack refactoring, CloudFormation moves resources by exporting them from the current stack and importing them into the new stack (or the same one in the case of a rename) in the background.

Exporting resources
Exporting resources

Using the new feature involves several steps.

3.1. Create the new templates

First, we must create the infrastructure templates that reflect the new stack states. If we move a resource from StackA to StackB, the new template for StackB should contain the resources already existing in StackB plus the ones we move. Also, StackA’s new template should not include the resource we want to remove.

Let’s say that StackA contains an S3 bucket and an SQS queue. This is StackA’s current template:

AWSTemplateFormatVersion: '2010-09-09'

Resources:
  MyBucket:
    Type: AWS::S3::Bucket
  MyQueue:
    Type: AWS::SQS::Queue

We want to move the queue to StackB that already has an SNS topic. StackB’s current template can look like this:

AWSTemplateFormatVersion: '2010-09-09'

Resources:
  MyTopic:
    Type: AWS::SNS::Topic

We will create two new templates that will look as follows.

StackA’s new template will contain the S3 bucket but won’t have the SQS queue. Let’s call the template file S3Only.yaml:

AWSTemplateFormatVersion: '2010-09-09'

Resources:
  MyBucket:
    Type: AWS::S3::Bucket

Then, we have to add the SQS queue to StackBs new template called SnsSqs.yaml that already contains the SNS topic:

AWSTemplateFormatVersion: '2010-09-09'

Resources:
  MyTopic:
    Type: AWS::SNS::Topic
  MyQueue: # This is the same queue as in the original template
    Type: AWS::SQS::Queue

3.2. Create a refactor mappings file (optional)

If we want to rename some resources, we must create a JSON file that looks like this:

[
  {
    "Source": {
      "StackName": "S3Stack",
      "LogicalResourceId": "MyBucket"
    },
    "Destination": {
      "StackName": "S3Stack",
      "LogicalResourceId": "MyNewBucket"
    }
  }
]

Inside the array, we can have multiple Source-Destination pairs. This example leaves the resource in the same stack (the StackName values in the Source and Destination objects are the same) but we change the logical ID of the bucket.

3.3. Create stack refactor

Next, we execute the create-stack-refactor CLI command:

aws cloudformation create-stack-refactor --stack-definitions
 StackName=StackA,TemplateBody@=file://S3Only.yaml StackName=StackB,
 TemplateBody@=file://SnsSqs.yaml

This command specifies that StackA should be built from the S3Only.yaml template file, and StackB from the SnsSqs.yaml template file.

The response will be similar to this:

{
  "StackRefactorId": "REFACTOR_ID"
}

We will need the refactor ID in later steps.

At this stage, CloudFormation has not yet made any changes to the stacks.

Moving to a new stack

If we want to move some resources to a new stack, we should add the --enable-stack-creation flag.

Renaming resources

If we want to rename some resources, we need to add the --resource-mappings file://refactor-mappings.json option, where refactor-mappings.json is the name of the mappings file.

3.4. Check if the refactor is successful

It’s worth checking that the request is in CREATE_COMPLETE status since this indicates that we can execute the refactor. We can use the following command to check if the refactor has been successful:

aws cloudformation describe-stack-refactor --stack-refactor-id REFACTOR_ID

If everything goes well, we will see a response similar to this:

{
  "StackRefactorId": "REFACTOR_ID",
  "StackIds": [
    "arn:aws:cloudformation:eu-central-1:ACCOUNT_ID:stack/STACK_NAME/STACK_ID"
  ],
  "ExecutionStatus": "AVAILABLE",
  "Status": "CREATE_COMPLETE"
}

If we see this, we can move on to the next step.

Create failed

It sometimes happens that the refactor request goes into the CREATE_FAILED status. This usually indicates a syntax error in the command or the refactor mappings file.

3.5. Execute the refactor

We can now execute the refactor:

aws cloudformation execute-stack-refactor --stack-refactor-id REFACTOR_ID

This is when CloudFormation will update existing stacks and create new stacks if applicable.

4. Some refactoring scenarios

Let’s investigate some stack refactoring scenarios.

4.1. Change the logical ID of the bucket

Stack Refactoring will change the logical ID of the bucket without us needing to touch the objects in the bucket! This can save us from a lot of stress.

Since we rename resources in the template, we must create a mappings file. In this example, we change the logical ID of the bucket but leave it in the same stack:

[
  {
    "Source": {
      "StackName": "S3Stack",
      "LogicalResourceId": "MyBucket"
    },
    "Destination": {
      "StackName": "S3Stack",
      "LogicalResourceId": "MyNewBucket"
    }
  }
]

We must also add the --resource-mappings option to the create-stack-refactor command.

4.2. Rename and move at the same time

This example is more complex because we change the logical ID of a resource and also move it to a new stack at the same time.

We need the mappings file called refactor-mappings.json here too:

[
  {
    "Source": {
      "StackName": "S3VpcStack",
      "LogicalResourceId": "MyBucket"
    },
    "Destination": {
      "StackName": "NewS3Stack",
      "LogicalResourceId": "MyNewBucket"
    }
  },
  {
    "Source": {
      "StackName": "S3VpcStack",
      "LogicalResourceId": "MyVPC"
    },
    "Destination": {
      "StackName": "S3VpcStack",
      "LogicalResourceId": "RenamedVPC"
    }
  }
]

In this case, the S3VpcStack contains an S3 bucket, a VPC, an internet gateway, and an attachment. We:

  1. Move the S3 bucket to a brand new stack called NewS3Stack.
  2. Change the logical ID of the bucket while moving it to the new stack.
  3. Change the logical ID of the VPC, which remains in the original S3VpcStack.

We run the following command:

aws cloudformation create-stack-refactor --stack-definitions
 StackName=NewS3Stack,TemplateBody@=file://newS3.yaml StackName=S3VpcStack,
 TemplateBody@=file://vpcLeft.yaml --enable-stack-creation
 --resource-mappings file://refactor-mappings.json

The template file for the new stack (NewS3Stack) is called newS3.yaml, and it contains the S3 bucket we want to move. The new content of the original stack (S3VpcStack) has been written to the vpcLeft.yaml file, which contains the VPC, the internet gateway, and the internet gateway attachment.

After executing the create-stack-refactor command, CloudFormation will create the new stack (NewS3Stack). The stack will be in REVIEW_IN_PROGRESS status until we confirm and execute the refactor. Since a new stack is created, we must add the enable-stack-creation flag to the command.

After we run execute-stack-refactor in the CLI, the new stack with the S3 bucket will be created, and the original stack will be updated. CloudFormation will move the S3 bucket with the objects in the bucket to the new stack. The bucket’s physical name will remain the same.

4.3. Moving resources between two existing stacks

It’s also possible to move resources between two existing stacks. Please see the before/after template files under 3.1. above.

5. Considerations

The feature is very new and, as such, it has some limitations.

One of them is that not every resource can be refactored. You can find the current list of such resources in the documentation.

Also, we can’t add resources that haven’t existed yet, delete existing ones permanently, or change their configurations. That is, stack refactoring is a lift-and-shift operation.

Lastly, we can’t leave a stack empty. There has to be at least one resource type left in the stack.

6. Summary

CloudFormation Stack Refactoring allows us to move resources between stacks and rename resource logical IDs. The new feature addresses some pain points and makes stack refactoring easier, saving developers time and making the migration less complex.

As of writing this, the feature only works in the CLI and the SDK.

7. Further reading

Intrinsic Function Reference - CloudFormation built-in functions.

Create a Stack from the CloudFormation Console - How to create a stack

Previous post