Continue Rolling Back an Update for AWS CloudFormation stacks in the UPDATE_ROLLBACK_FAILED state

AWS CloudFormation allows developers and systems administrators to create and manage a collection of related AWS resources (called a stack) by provisioning and updating them in an orderly and predictable way.

In this blog post, we will describe a new feature in AWS CloudFormation that will allow continue rolling back an update in a self-service manner for stacks in UPDATE_ROLLBACK_FAILED state.

A stack’s state is set to UPDATE_ROLLBACK_FAILED when CloudFormation cannot roll back all changes during an update. For example, if you manually deleted a resource outside of CloudFormation, the service’s attempt to roll back to the original state will fail and the stack’s state will be set to UPDATE_ROLLBACK_FAILED. Because CloudFormation does not know a change (in this case, the deletion of a resource) was made, it assumes the resource still exists and attempts to roll back to it, causing the update rollback to fail. There are other reasons an update rollback might fail, including an AWS service limit violation or because a dependent resource did not stabilize. For more information about common errors, see the AWS CloudFormation User Guide.

Until recently, after a stack was in the UPDATE_ROLLBACK_FAILED state, you had only two options: delete the stack or contact AWS Support to return the stack to a working state. In many cases (for example, if it is running production workloads), deleting the stack is not an acceptable option.

AWS CloudFormation now offers a third option: continue update rollback, which you can initiate from the AWS CloudFormation console or with the continue-update-rollback command in the AWS Command Line Interface (CLI). This functionality is enabled for all the stacks in the UPDATE_ROLLBACK_FAILED state.

For example, if the stack is in the UPDATE_ROLLBACK_FAILED state due to a service limit violation, you can request a limit increase or delete resources to stay within the limit and then use the continue update rollback functionality to reinitiate the rollback and bring the stack to the UPDATE_ROLLBACK_COMPLETE state.

In the AWS CloudFormation console, select the stack, choose Actions, and then choose Continue Update Rollback.

The use of the continue update rollback functionality means you do not need to contact AWS Support. In situations where you can find and fix the error, the continue update rollback process will let you recover the stack to the UPDATE_ROLLBACK_COMPLETE state so you can run updates again. You can however still contact the support if this option does not recover the stack or if you need any further assistance.  

For more information about update rollback failures, see the AWS CloudFormation User Guide.