When I cut this site’s deploy role down to size, the line that bothered me most was that its old permissions reached every Terraform state file in the account. It’s worth spelling out why that’s alarming, because a lot of teams treat state as an implementation detail: a JSON file Terraform needs, stored in a bucket, never looked at.

State is a complete, plaintext record of every attribute of every resource Terraform manages. If a secret passes through Terraform, it’s in there.

What actually ends up in state

Some of it is obvious once you think about it, and some of it surprises people:

  • Anything you pass as an argument. A database password set on aws_db_instance, a token in a provider-configured resource, an API key in a Lambda environment variable. All stored as given.
  • Anything Terraform generates. random_password exists to create secrets, and its result lives in state. So does the private key from tls_private_key.
  • Anything a data source reads. Using aws_secretsmanager_secret_version to look up a secret “so it isn’t hardcoded” copies the secret value into state on every refresh.
  • Saved plans. A plan file from terraform plan -out contains the values too. If your CI uploads plan files as build artifacts, those artifacts are secrets.

And sensitive = true doesn’t change any of this. It redacts values from CLI and plan output, which keeps them out of your terminal scrollback and CI logs. That’s genuinely useful. But it does nothing to the state file, which still has the plaintext. Anyone who can run this gets everything:

terraform state pull | grep -i -E 'password|secret|private_key'

So the practical definition is simple: whoever can read your state can read your secrets. Every access decision about the state bucket is an access decision about those secrets.

Protect state like the secret store it is

The backend for my dashboard site’s stack looks like this:

backend "s3" {
  bucket       = "my-terraform-state"
  key          = "hub/hub-dev.tfstate"
  region       = "us-east-1"
  encrypt      = true
  use_lockfile = true
}

encrypt = true turns on server-side encryption for the state object. For stronger separation, point kms_key_id at a customer-managed KMS key, so reading state requires kms:Decrypt on that key in addition to s3:GetObject on the object. That’s a second, independent permission to get wrong before anything leaks, and the key policy becomes a short, reviewable list of who can actually read state.

use_lockfile = true uses S3’s native locking, available since Terraform 1.10, instead of a separate DynamoDB table. Locking isn’t a confidentiality control, but it protects integrity: two concurrent applies writing the same state is how state gets corrupted.

Beyond the backend block:

  • Block Public Access on the bucket, always. State should never be one bad bucket policy away from the internet.
  • Turn on versioning, so a corrupted or deleted state can be recovered. Know the trade-off: old versions keep old secrets. When you rotate a secret that was ever in state, assume every previous version of the file still has the old value.
  • One key per stack, and ideally separate buckets or accounts per environment. A CI role that plans the dev stack shouldn’t be able to read production state.
  • Audit who reads it. S3 data events in CloudTrail can log every GetObject against the state bucket. It’s worth the cost on a bucket this sensitive.

Better: keep secrets out of state entirely

Protecting state well is the floor. The better move is designing so the secret never touches Terraform at all. There are a few good patterns.

Write the secret out of band. My dashboard site signs session cookies with a secret that has to exist in two places: a CloudFront KeyValueStore for the edge function and an SSM SecureString for the sign-in Lambda. Terraform creates the key store and outputs its ARN and the parameter name, but the value comes from a short script:

SECRET="$(openssl rand -hex 32)"
aws ssm put-parameter --name "$PARAM" --type SecureString --value "$SECRET" --overwrite

It then writes the same value into the key store. Terraform manages the container; the script manages the contents. Rotation is just running the script again, and the secret has never been in a state file, a plan, or a log.

The user accounts follow the same rule. Terraform creates the Cognito users from an allowlist, but sets no password. Cognito emails a temporary one, and the first sign-in replaces it. Nothing credential-like ever passes through Terraform.

Let the service own the secret. Some resources can generate and store their own credentials. For RDS, manage_master_user_password = true has RDS create the master password and keep it in Secrets Manager. Terraform only ever sees the secret’s ARN, and rotation happens without a Terraform run.

Use ephemeral values and write-only arguments. Newer Terraform versions address this directly. Terraform 1.10 added ephemeral resources, which can fetch a value for use during a run without persisting it. Terraform 1.11 added write-only arguments, which send a value to the provider but never store it. The AWS provider supports them on resources like aws_db_instance (password_wo), paired with a version argument you bump when you want to rotate. If you’re on a current version, this is the cleanest fix for “Terraform has to set a password.”

A quick audit

If you want to know how exposed your state is today:

  1. Pull each stack’s state and search it for password, secret, token, and private_key. Make a list of what you find.
  2. For each item, decide which pattern removes it: out-of-band write, service-managed secret, or write-only argument.
  3. List every principal that can read the state bucket, including CI roles and anything with a broad managed policy like AmazonS3FullAccess. That list is the real access list for every secret on your first list.
  4. Check whether your pipelines save plan files as artifacts, and how long those artifacts are kept.
  5. Rotate anything that was exposed more widely than you intended. Old state versions mean “was” counts.

None of this requires abandoning Terraform or adding a new tool. It requires accepting what state is. Once you see it as a secret store, the controls are the same ones you’d put on any other secret store: least privilege, encryption with a key you control, audit logging, and keeping the secrets that don’t need to be there out of it.