Lean Terraform, Part 1: Modules Are Logic, tfvars Are Input

The rule

In this post we cover the one rule I apply to Terraform, and the patterns that follow from it. In the second post we discuss state and environments. In the third we cover the operational layer that has to exist around the code. In the fourth we look at two real repositories, one of which had 1,400 Terraform files, and where these rules came from.

The rule is this: modules are pure logic, and tfvars are the only thing that changes.

A module should not know that production exists. It should not know your VNet is 10.18.0.0/16, that your SSO container needs session affinity, or that QA has one peering while production has two. It takes typed input and renders resources. The facts live in a tfvars file, which is data, and a person can read that file top to bottom and know what is deployed.

If you do not write Terraform this way, three things happen over a year or two. Environments drift apart, because the only way to express a difference is to edit code, and the edit lands inside a conditional. Modules accumulate special cases, because each new environment adds one more branch. And you lose the ability to answer what is deployed, because the answer is now spread across several files and a conditional you have to evaluate by hand. None of these are visible on the day they start.

The shape

Terraform is closer to a data pipeline than to a program. There is a dataset, a function, and a rendered result. The layout follows from that:

  DATA                       WIRING            LOGIC              CLOUD
  (changes per env)          (same)            (same)             (rendered)

  qa-containers.tfvars   \
  prod-containers.tfvars  +--->  main.tf  --->  modules/  --->  resources
  rbac.tfvars            /       for_each       for_each

  |______________________|      |_________________________|
    you edit this                you almost never edit this

A new environment or a new service shows up on the left. The right side is a function you wrote once. One repo I run has about 2,100 lines in the root and 14 modules, and it renders two full environments, two Front Doors, two Postgres servers and around forty container apps. Neither environment has a line of code of its own.

A module is a resource with the boring parts filled in

The most common mistake I see is modules that try to be frameworks. A module is one resource, or one tightly bound cluster of resources, with the naming and tagging already decided. It is not an abstraction layer over your cloud provider.

Here is a complete NSG module, not an excerpt:

resource "azurerm_network_security_group" "nsg" {
  name                = var.name
  location            = var.location
  resource_group_name = var.resource_group_name

  tags = {
    Env = var.environment
  }
}

resource "azurerm_network_security_rule" "security_rule" {
  for_each = var.rules

  resource_group_name         = var.resource_group_name
  network_security_group_name = azurerm_network_security_group.nsg.name

  name                        = each.key
  priority                    = each.value.priority
  direction                   = each.value.direction
  access                      = each.value.access
  protocol                    = each.value.protocol
  source_port_range           = each.value.source_port_range
  destination_port_range      = each.value.destination_port_range
  source_address_prefix       = each.value.source_address_prefix
  destination_address_prefix  = each.value.destination_address_prefix
}

Twenty five lines, and it has outlived every environment we built with it. It has no idea what a firewall rule is for, it cannot tell production from QA, and it has no defaults worth arguing about. When someone asks where we allow Redis in QA, the answer is in a tfvars file.

A line like if var.environment == "prod" inside a module means the module knows something it should have been told. It should be a variable with a default.

Type the input

If modules are the function, the variable block is the signature. With map(object({...})) and optional(), a wrong tfvars fails at plan time and names the field you got wrong. Left as any, you find out from a half-created environment twenty minutes into an apply.

variable "subnets" {
  type = map(object({
    address_prefix          = string
    create_nat_gateway      = optional(bool)
    create_public_ip        = optional(bool)
    allocation_method       = optional(string)
    sku                     = optional(string)
    nat_gateway_sku_name    = optional(string)
    idle_timeout_in_minutes = optional(number)
    network_security_group  = optional(string)
    route_table             = optional(string)
    service_endpoints       = optional(list(string))
    delegation              = optional(object({
      name = string
      service_delegation = object({
        name    = string
        actions = list(string)
      })
    }))
    flow_logs_subnets       = optional(bool)
  }))
}

Required fields are the ones with no sane default. Everything optional is a knob most environments never touch. The tfvars then reads as a description of the network rather than a list of arguments:

subnets = {
  "qa-containers-subnet-aca" = {
    address_prefix         = "10.17.2.0/24"
    flow_logs_subnets      = true
    service_endpoints      = ["Microsoft.KeyVault"]
    network_security_group = "sg-qa-containers-aca-weu"

    delegation = {
      name = "dlg-Microsoft.App-environments"
      service_delegation = {
        name    = "Microsoft.App/environments"
        actions = ["Microsoft.Network/virtualNetworks/subnets/join/action"]
      }
    }
  }
}

The map key is the resource name. This is worth doing everywhere, because the plan then reads module.vnet.azurerm_subnet.subnet["qa-containers-subnet-aca"] instead of subnet[2], and reordering the file destroys nothing.

Names in tfvars, IDs in code

A resource ID is a fact about a deployed cloud. If it appears in tfvars you have put output back into input, and the file can no longer be copied to a new environment without a scavenger hunt. The person writing tfvars should only ever write names. The root config resolves names to IDs, since that is wiring, and wiring is code.

# in tfvars: a human says which subnets, by name
network_acls = {
  bypass                  = "AzureServices"
  default_action          = "Deny"
  virtual_network_subnets = ["qa-containers-subnet-aca"]
}

# in main.tf: the wiring turns names into IDs
network_acls = {
  bypass         = each.value.network_acls.bypass
  default_action = each.value.network_acls.default_action
  ip_rules       = each.value.network_acls.ip_rules
  virtual_network_subnet_ids = [
    for s in each.value.network_acls.virtual_network_subnets : module.vnet.subnet_ids[s]
  ]
}

The Key Vault module never learns what a subnet is, it receives a list of IDs. The tfvars author never touches an ID. The root is the only place that knows both, which is what a root config is for. The same pattern applies to NSGs and route tables one level down, inside the VNet module:

resource "azurerm_subnet_network_security_group_association" "nsg" {
  for_each = { for k, v in var.subnets : k => v if v.network_security_group != null }

  subnet_id                 = azurerm_subnet.subnet[each.key].id
  network_security_group_id = var.nsg_ids[each.value.network_security_group]
}

Name a subnet's NSG and you get an association. Say nothing and you get none.

for_each over count, and filter instead of flagging

count gives you indexes. Delete the second subnet from a list of five and everything after it shifts down by one, so Terraform proposes to destroy and recreate three subnets you did not touch. With for_each over a map, resources are keyed by name and this cannot happen.

For conditional creation, do not add an enable_nat_gateway boolean to the module. Filter the map:

resource "azurerm_nat_gateway" "nat_gateway" {
  for_each = {
    for name, sub in var.subnets : name => sub
    if coalesce(sub.create_nat_gateway, false)
  }

  name                    = "${each.key}-nat-gateway"
  location                = var.location
  resource_group_name     = var.resource_group_name
  sku_name                = coalesce(each.value.nat_gateway_sku_name, var.default_natgw_sku_name)
  idle_timeout_in_minutes = coalesce(each.value.idle_timeout_in_minutes, var.default_natgw_idle_timeout_in_minutes)
}

The set of NAT gateways is a query over the data. There is no second list to keep in sync with the subnet list, so the two cannot disagree. An environment that wants no NAT gateway does not set a flag to false, it simply does not ask for one and the query returns nothing.

The coalesce calls do the other half of the work. A default lives in the module, and any single item can override it without every other item having to say anything. Defaults belong to the logic, exceptions belong to the data.

Access rules are data too

The test of whether you believe any of this is what happens when the company hires thirty people.

Terraform should describe rules, not users. Groups get scopes, scopes get roles:

# Pattern: define RULES (group -> scopes -> roles), not USERS.
# Adding 100 developers = add them to the group in the directory.
# No Terraform changes needed.

group_role_mappings = {
  "SG-Platform-DevOps" = {
    scopes = {
      "/subscriptions//resourceGroups/rg-qa-containers-weu"   = "Contributor"
      "/subscriptions//resourceGroups/rg-prod-containers-weu" = "Contributor"
      "/subscriptions//resourceGroups/rg-qa-weu"              = "Reader"

      # Key Vault data plane. Contributor alone does not grant this.
      "/subscriptions//resourceGroups/rg-qa-containers-weu/providers/Microsoft.KeyVault/vaults/kv-qa-weu" = "Key Vault Secrets Officer"
    }
  }
}

Onboarding a hundred developers becomes a group membership. Terraform does not run and nobody opens a PR. Meanwhile the access model is a file you can hand to an auditor, and it is the same file that is deployed, which is the part auditors have learned to be suspicious about.

The comment about Key Vault is the kind of thing worth writing where the decision lives. Contributor on a vault lets you manage the vault without reading a single secret out of it. This surprises people at 2am.

What it costs

• Typed maps are verbose. A deep map(object) with nested optionals is not pretty, and you are buying plan-time errors with typing effort. The exchange rate is good, but you do pay it.
• Nesting past two levels turns into soup. When a variable needs a third level of nested objects, that is usually the data telling you it wanted to be its own module.
• The module boundary is the actual skill. Too many small modules and the root becomes wiring soup, too few and the special cases come back. I aim for one module per thing I would name out loud in a meeting.
• Clean structure does not protect you from provider behaviour. Terraform will not tell you that a managed identity needs vault access at creation time, so apps land in a Failed state, silently, if the vault firewall is shut during the apply. That belongs in a runbook, and part 3 is about that layer.

Two questions worth asking your repo

When reviewing infrastructure code, mine included, I ask two things.

Can you add an environment by copying one file? If a new environment means editing modules, the modules were never logic. They are accumulated special cases in a module's clothing.

Can you read one file and know what is deployed? If the answer requires opening modules and evaluating conditionals by hand, the property that made this worth doing is gone. Not because the code is ugly, but because in six months that question gets asked during an incident, by someone tired, and the answer needs to be a file.

I once spent twelve hours provisioning an environment that should have taken forty five minutes. The cause was a shell script with hardcoded IPs and hostnames from the environment it was written for, sitting in a repo where everything else was parameterised.

Part 2 covers state: one state file per environment, no workspaces, why the state key is the single input you cannot afford to get wrong, and what to do when a cancelled CI job leaves a lock behind.