Setting up Pulumi with S3

The hosted Pulumi service is great, but for the time being we are happy with using S3 as we can open up managing infrastructure to our entire team while our Pulumi adoption is still in it’s early stages.

There are a few catches to managing the state yourself. You have to manage secrets yourself and the docs around stack referencing don’t cover the naming differences.

S3 State Bucket

We have a single state bucket for all our projects, this means that all our deployments need to be run from the account which the bucket exists in.

I will write another post soon on how we manage multiple AWS accounts.

I then configure my Pulumi project to always use that state bucket.

backend:
    url: s3://pulumi-state.mydomain.com

Stack naming

With the S3 provider the paths in S3 only include the stack name, they are not fully qualified with the project name like the hosted service.

This means we include the project in the stack name. ie my-project.dev. This allows us to use cross project stack referencing and not have stacks clash across projects.

Secrets Management

To make it easy to manage secrets in AWS, create a Symmetric KMS key through the console with an alias of pulumi. Then when creating stacks simply pass

--secrets-provider awskms://alias/pulumi?region=ap-southeast-2