Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/cli-reference/remove.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

The `osls remove` command will remove the deployed service, defined in your current working directory, from the provider.

If the stack has deletion protection enabled (see [`provider.deletionProtection`](../guides/deploying.md#deletion-protection)), the command fails before deleting anything.

```bash
osls remove
```
Expand Down
33 changes: 33 additions & 0 deletions docs/guides/deploying.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,39 @@ provider:
deploymentMethod: direct
```

### Deletion protection

Set `provider.deletionProtection` to have osls manage [CloudFormation termination protection](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-protect-stacks.html) for the service stack:

```yaml
provider:
name: aws
deletionProtection: true
```

To protect only some stages, list them:

```yaml
provider:
name: aws
deletionProtection:
stages:
- prod
```

After every successful `osls deploy`, including deploys that are skipped because nothing changed, osls sets the stack's termination protection to match the configuration: enabled when the value is `true` or the current stage is listed in `stages`, disabled otherwise. With the `stages` form, deploying an unlisted stage therefore actively disables protection on that stage's stack. Third-party termination protection plugins typically only ever enable protection, so check the `stages` list when migrating from one. `osls deploy function` and `osls rollback` never change the setting, and removing `provider.deletionProtection` from `serverless.yml` does not disable protection on an existing stack; it only stops osls from managing it.

While a stack is protected, `osls remove` fails early with `AWS_CLOUDFORMATION_DELETION_PROTECTION_ENABLED`, before any deployment artifacts are deleted, and deleting the stack in the AWS console or CLI is rejected by CloudFormation. To remove the service, set `provider.deletionProtection` to `false` (or drop the stage from `stages`), deploy, then remove. If deploying is not possible, for example because the stack is stuck in a failed state, disable protection directly with `aws cloudformation update-termination-protection --no-enable-termination-protection --stack-name <stack-name>`.

Keep in mind:

- The deploying identity needs `cloudformation:UpdateTerminationProtection` on the stack. `osls remove` uses `cloudformation:DescribeStacks` to check the flag; if that call is denied, osls logs a warning and continues, and CloudFormation still refuses to delete a protected stack.
- Protection is applied after the stack has been created or updated, so a brand-new stack is unprotected until its first deployment completes.
- An invalid value fails the deploy with `INVALID_DELETION_PROTECTION_CONFIG` before anything is uploaded. Configuration validation already rejects most invalid shapes; this also covers `configValidationMode: warn` and `off`.
- `osls deploy --package` uses the value saved by `osls package`; re-run `osls package` after changing it.
- Nested stacks inherit the root stack's setting.
- Termination protection prevents accidents, not malicious deletion: anyone allowed to call `UpdateTerminationProtection` can turn it off, and deploying with `deletionProtection: false` does exactly that.

### Tips

- Use this in your CI/CD systems, as it is the safest method of deployment.
Expand Down
7 changes: 7 additions & 0 deletions docs/guides/serverless.yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,13 @@ provider:
key: value
# Method used for CloudFormation deployments: 'changesets' or 'direct' (default: changesets)
deploymentMethod: direct
# Manage CloudFormation termination protection for the stack after each deploy (not managed by default).
# `true`/`false` applies to every stage; the `stages` form enables protection for the listed
# stages and disables it for all others.
deletionProtection: true
# deletionProtection:
# stages:
# - prod
# List of existing Amazon SNS topics in the same region where notifications about stack events are sent.
notificationArns:
- 'arn:aws:sns:us-east-1:XXXXXX:mytopic'
Expand Down
93 changes: 93 additions & 0 deletions lib/plugins/aws/deletion-protection.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
'use strict';

const {
CloudFormationClient,
UpdateTerminationProtectionCommand,
} = require('@aws-sdk/client-cloudformation');
const ServerlessError = require('../../serverless-error');
const { log } = require('../../utils/serverless-utils/log');
const { getAwsErrorMessage } = require('../../aws/aws-sdk-v3-error');

class AwsDeletionProtection {
constructor(serverless, options) {
this.serverless = serverless;
this.options = options || {};
this.provider = this.serverless.getProvider('aws');

this.hooks = {
'before:deploy:deploy': async () => this.validateConfiguration(),
'after:deploy:deploy': async () => this.postDeploy(),
};
}

resolveDeletionProtection() {
const setting = this.serverless.service.provider.deletionProtection;
if (setting == null) return null;
if (typeof setting === 'boolean') return { enabled: setting };
if (
typeof setting === 'object' &&
Array.isArray(setting.stages) &&
setting.stages.length > 0 &&
setting.stages.every((stage) => typeof stage === 'string')
) {
return { enabled: setting.stages.includes(this.provider.getStage()) };
}
throw new ServerlessError(
'provider.deletionProtection must be a boolean or an object with a non-empty stages list',
'INVALID_DELETION_PROTECTION_CONFIG'
);
}

validateConfiguration() {
this.resolveDeletionProtection();
}

async postDeploy() {
const config = this.resolveDeletionProtection();
if (!config) return;

// A change set based first deploy that ends with an empty change set does not create the
// stack: it is left in REVIEW_IN_PROGRESS and the next deploy asks to run `remove` (see
// `createFallback` in lib/plugins/aws/lib/update-stack.js). Protecting it would block that
// `remove`, so leave it alone until the stack exists
if (
this.provider.didCreateService &&
this.serverless.service.provider.deploymentWithEmptyChangeSet
) {
log.info('Skipping deletion protection update as the stack has not been created');
return;
}

const stackName = this.provider.naming.getStackName();
const cloudFormation = await this.getCloudFormationClient();
try {
await cloudFormation.send(
new UpdateTerminationProtectionCommand({
StackName: stackName,
EnableTerminationProtection: config.enabled,
})
);
} catch (error) {
throw new ServerlessError(
`Could not ${config.enabled ? 'enable' : 'disable'} deletion protection for stack ` +
`"${stackName}" (the stack itself was deployed): ${
getAwsErrorMessage(error) || String(error)
}`,
'AWS_CLOUDFORMATION_DELETION_PROTECTION_UPDATE_FAILED',
{ cause: error }
);
}
log.info(
`${config.enabled ? 'Enabled' : 'Disabled'} deletion protection for stack "${stackName}"`
);
}

async getCloudFormationClient() {
this.cloudFormationClientPromise ||= this.provider
.getAwsSdkV3Config()
.then((config) => new CloudFormationClient(config));
return this.cloudFormationClientPromise;
}
}

module.exports = AwsDeletionProtection;
18 changes: 18 additions & 0 deletions lib/plugins/aws/provider.js
Original file line number Diff line number Diff line change
Expand Up @@ -594,6 +594,23 @@ class AwsProvider {
awsLambdaTimeout: { type: 'integer', minimum: 1, maximum: 900 },
awsLambdaTracing: { anyOf: [{ enum: ['Active', 'PassThrough'] }, { type: 'boolean' }] },
awsLambdaVersioning: { type: 'boolean' },
awsDeletionProtection: {
anyOf: [
{ type: 'boolean' },
{
type: 'object',
properties: {
stages: {
type: 'array',
minItems: 1,
items: { $ref: '#/definitions/stage' },
},
},
required: ['stages'],
additionalProperties: false,
},
],
},
awsPruneFunctionVersions: {
anyOf: [
{ type: 'boolean' },
Expand Down Expand Up @@ -925,6 +942,7 @@ class AwsProvider {
},
additionalProperties: false,
},
deletionProtection: { $ref: '#/definitions/awsDeletionProtection' },
deploymentBucket: {
anyOf: [
{ $ref: '#/definitions/awsS3BucketName' },
Expand Down
6 changes: 5 additions & 1 deletion lib/plugins/aws/remove/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,12 @@ class AwsRemove {
}
},
'remove:remove': async () => {
const doesEcrRepositoryExistPromise = this.checkIfEcrRepositoryExists();
await this.validate();
const doesEcrRepositoryExistPromise = this.checkIfEcrRepositoryExists();
// The probe is only awaited once the stack is gone; do not let a failure of the steps
// in between (e.g. the deletion protection guard) turn its rejection into an unhandled one
doesEcrRepositoryExistPromise.catch(() => {});
await this.ensureStackIsNotDeletionProtected();
mainProgress.notice('Removing objects from S3 bucket', { isMainEvent: true });
await this.emptyS3Bucket();
mainProgress.notice('Removing CloudFormation stack', { isMainEvent: true });
Expand Down
43 changes: 42 additions & 1 deletion lib/plugins/aws/remove/lib/stack.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
'use strict';

const { CloudFormationClient, DeleteStackCommand } = require('@aws-sdk/client-cloudformation');
const {
CloudFormationClient,
DeleteStackCommand,
DescribeStacksCommand,
} = require('@aws-sdk/client-cloudformation');
const ServerlessError = require('../../../../serverless-error');
const { log } = require('../../../../utils/serverless-utils/log');
const {
getAwsErrorMessage,
isCloudFormationMissingStackError,
} = require('../../../../aws/aws-sdk-v3-error');

function getCloudFormationClient(context) {
context.cloudFormationClientPromise ||= context.provider
Expand All @@ -10,6 +20,37 @@ function getCloudFormationClient(context) {
}

module.exports = {
async ensureStackIsNotDeletionProtected() {
const stackName = this.provider.naming.getStackName();
const cloudFormation = await getCloudFormationClient(this);
let stack;

try {
const result = await cloudFormation.send(new DescribeStacksCommand({ StackName: stackName }));
stack = result.Stacks && result.Stacks[0];
} catch (error) {
if (isCloudFormationMissingStackError(error)) return;
// Removal must not depend on cloudformation:DescribeStacks being granted: fall back to
// the previous behaviour, where CloudFormation itself rejects deleting a protected stack
log.warning(
`Could not check whether stack "${stackName}" has deletion protection enabled ` +
`(${getAwsErrorMessage(error) || String(error)}). Continuing with removal.`
);
return;
}

if (!stack || !stack.EnableTerminationProtection) return;

throw new ServerlessError(
`Cannot remove stack "${stackName}" because deletion protection is enabled. ` +
'Set provider.deletionProtection to false (or remove this stage from ' +
'provider.deletionProtection.stages) and deploy the service before removing it. ' +
'To turn it off without deploying, run: aws cloudformation ' +
`update-termination-protection --no-enable-termination-protection --stack-name ${stackName}`,
'AWS_CLOUDFORMATION_DELETION_PROTECTION_ENABLED'
);
},

async remove() {
const stackName = this.provider.naming.getStackName();
const params = {
Expand Down
1 change: 1 addition & 0 deletions lib/plugins/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ module.exports = [
require('./aws/remove/index.js'),
require('./aws/rollback.js'),
require('./aws/rollback-function.js'),
require('./aws/deletion-protection.js'),
require('./aws/prune.js'),
require('./aws/package/compile/layers.js'),
require('./aws/package/compile/functions.js'),
Expand Down
1 change: 1 addition & 0 deletions test/lib/configure-aws-sdk-v3-stub.js
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ const serviceDefinitions = {
deleteChangeSet: 'DeleteChangeSetCommand',
getTemplate: 'GetTemplateCommand',
setStackPolicy: 'SetStackPolicyCommand',
updateTerminationProtection: 'UpdateTerminationProtectionCommand',
describeStackEvents: 'DescribeStackEventsCommand',
},
},
Expand Down
Loading