Blog

Tracking Infrastructure with SSM and Terraform

Terraform is the most popular cross cloud, infrastructure as code tool. Not only does Terraform specify what is to be deployed, it tracks what was deployed. This is really useful when all of your infrastructure is managed centrally.

It’s not always practical or desirable to track all resources in a single account in the same project. For example low level infrastructure, such as VPCs or SSO configuration, change infrequently. On the other hand applications might be redeployed dozens of time per day.

Both applications and low level infrastructure can be managed by Terraform. There are also cases when the low level infrastructure is managed with Terraform while other services are managed using other tooling such as CDK, CloudFormation or Pulumi.

Teams need a way of sharing configuration, without resorting to hard coded strings. When using AWS, SSM Parameter Store (aka SSM Params) helps support an eclectic mix of environment management tooling. Values such as ARNs or other identifiers can be stored in SSM. Other tools and applications can pull in these parameters and use them. It’s even possible to use the parameters across accounts, which is useful with shared resources.

Your environment might contain multiple subnets. While it is possible to lookup the subnets using tags, these conventions can vary from team to team. Instead the subnets can be stored as SSM Params. This partial snippet shows how to store these values:

variable "tags" {
  description = "Tags to apply to all resources."
  type        = map(string)

  default = {
    todo = "Add real values here"
  }
}

variable "vpc_cidr" {
  description = "CIDR block used by the VPC. Must be a /16 for the dynamic subnet creation to work."
  type        = string
  default     = "10.0.0.0/16"
}

data "aws_availability_zones" "all" {}

locals {
  az_names = toset(data.aws_availability_zones.all.names)
}

resource "aws_vpc" "application" {
  cidr_block           = var.vpc_cidr
  enable_dns_hostnames = true

  tags = merge(
    var.tags,
    {
      Name = "application"
    }
  )
}

resource "aws_subnet" "private" {
  for_each = local.az_names

  vpc_id            = aws_vpc.application.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 4, index(local.az_names, each.value) + 8)
  availability_zone = each.value

  tags = merge(
    var.tags,
    {
      "Name" = "application-private-${each.value}"
    }
  )
}

resource "aws_subnet" "public" {
  for_each = local.az_names

  vpc_id            = aws_vpc.application.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 4, index(local.az_names, each.value))
  availability_zone = each.value

  tags = merge(
    var.tags,
    {
      "Name" = "application-public-${each.value}"
    }
  )
}

resource "aws_ssm_parameter" "subnets" {
  name = "/vpc/application/subnets"
  type = "String"
  value = jsonencode({
    "private" : [for subnet in aws_subnet.private : subnet.id],
    "public" : [for subnet in aws_subnet.public : subnet.id],
  })

  tags = var.tags
}

In an application pipeline these values can be retrieved and validated using code like this:

# Get the value from SSM
data "aws_ssm_parameter" "subnets" {
  name = "/vpc/application/subnets"
}

# Use a local to avoid decoding the string multiple times
locals {
  subnets = jsondecode(data.aws_ssm_parameter.subnets.insecure_value)
}

# Validate the subnets, this also makes it easy to access
# the VPC ID and other properties associated with the subnet
data "aws_subnet" "public" {
  for_each = toset(local.subnets.public)

  id = each.key
}

This can be used with other resources. For example a GitHub Action might need to publish web content to a S3 bucket. The target bucket can be fetched from SSM, rather than the target being tracked using GitHub secrets. This ensures the permissions are also configured properly for pushing changes to the bucket.

resource "aws_s3_bucket" "website" {
  bucket = "website-example.com"

  tags = var.tags
}

# Rest of S3 config goes here

resource "aws_ssm_parameter" "website_bucket" {
  name  = "/resources/website_bucket_arn"
  type  = "String"
  value = aws_s3_bucket.website.arn

  tags = var.tags
}

Once you have OIDC properly configured for AWS, in the build you can fetch the bucket ARN by running the command: aws ssm get-parameter --name /resources/website_bucket_arn --output text --query Parameter.Value

This pattern can be used for storing the security group IDs for VPC endpoints. This works well when you allow all traffic from the VPC to access the endpoints, the use the egress controls on the security groups used on other resources such as Lambdas and ECS instances. The values used on Lambda security groups. This snippet can be used in your application Terraform configuration.

# Fetch the VPC endpoint security group ID
data "aws_ssm_parameter" "sec_group_vpce_sqs" {
  name = "/vpc/application/sec_groups/vpce_sqs"
}

# If the VPCe is the same account we can validate the ID
data "aws_security_group" "vpce_sqs" {
  id = data.aws_ssm_parameter.sec_group_vpce_sqs
}

resource "aws_security_group" "my_lambda" {
  name        = "my-lambda"
  description = "Controls for my-lambda"

  tags = merge(
    var.tags,
    {
      "Name" = "my-lambda"
    }
  )
}

resource "aws_security_group_rule" "my_lambda_to_sqs" {
  type      = "egress"
  from_port = 443
  to_port   = 443
  protocol  = "tcp"

  security_group_id = aws_security_group.my_lambda.id

  source_security_group_id = data.aws_security_group.vpce_sqs.id
}

There are many other ways to leverage this pattern. Try it out and see where your teams can benefit from it.

Tracking Infrastructure with SSM and Terraform

Tags

Share this post