Infrastructure as code

Deep dive

Tool map

Decision table

Code

Hetzner server + firewall + Cloudflare DNS in OpenTofu/Terraform, with remote locked state:

terraform {
  required_providers {
    hcloud     = { source = "hetznercloud/hcloud", version = "~> 1.48" }
    cloudflare = { source = "cloudflare/cloudflare", version = "~> 4.40" }
  }
  backend "s3" {                         # any S3-compatible store; lock file needs a recent tool version
    bucket       = "tfstate-prod"
    key          = "edge/terraform.tfstate"
    region       = "eu-central-1"
    encrypt      = true
    use_lockfile = true
  }
}

variable "ssh_key_id" { type = string }
variable "zone_id"    { type = string }

resource "hcloud_firewall" "web" {
  name = "web"
  rule {
    direction  = "in"
    protocol   = "tcp"
    port       = "443"
    source_ips = ["0.0.0.0/0", "::/0"]
  }
}

resource "hcloud_server" "api" {
  count        = 2
  name         = "api-${count.index}"
  server_type  = "cpx31"
  image        = "debian-12"
  location     = count.index == 0 ? "fsn1" : "nbg1"   # spread over locations
  ssh_keys     = [var.ssh_key_id]
  firewall_ids = [hcloud_firewall.web.id]
  user_data    = file("${path.module}/cloud-init.yaml")
  labels       = { service = "api", env = "prod" }
  lifecycle { prevent_destroy = true }
}

resource "cloudflare_record" "api" {
  count   = 2
  zone_id = var.zone_id
  name    = "api"
  type    = "A"
  content = hcloud_server.api[count.index].ipv4_address
  proxied = true
}

CI gate: plan on pull request, apply only from main with the reviewed plan:

tofu init -input=false
tofu plan -input=false -out=plan.bin -lock-timeout=60s
tofu show -no-color plan.bin > plan.txt          # post as PR comment
# on merge, apply exactly the reviewed plan
tofu apply -input=false plan.bin

Efficiency

• Cost: a plan refreshes every resource through provider API calls and then diffs it against the configuration, so plan time grows with resource count times API latency, and large stacks hit provider rate limits. OpenTofu walks the graph with 10 concurrent operations by default (-parallelism, OpenTofu docs). State storage is negligible. Bound: provider API latency and rate limits.
• Levers:
  • split stacks by layer and environment — smaller graphs, shorter plans, smaller blast radius (this page) — large estates
  • -refresh=false — fewer remote API requests, but external changes are ignored (OpenTofu plan docs) — local iteration only, never the CI gate
  • -parallelism=n — more concurrent operations than the default 10 (OpenTofu plan docs) — until rate limits answer
  • apply the saved plan file — no second plan between review and apply (Code above; OpenTofu docs) — CI pipelines
• Naive → standard → state of the art:
  • console clicks · minutes per change, no review, not repeatable
  • one stack per environment, plan on pull request, apply the saved plan · one refresh pass per stack
  • layered stacks, scheduled drift plans, continuous reconciliation (Crossplane, Argo CD) · only the touched layer is refreshed on a change

Implementation notes

• Split state per environment and per layer (network, data, apps); a bad apply should not be able to touch the database layer from the app stack.
• prevent_destroy on stateful resources; deletion protection on databases at the provider level too.
• Pin provider versions; upgrade deliberately with a plan diff.
• Import existing hand-made resources (import blocks) instead of recreating them.
• Run drift detection on a schedule (plan -detailed-exitcode, exit 2 = drift) and alert.
• CI credentials: short-lived OIDC federation from the CI system to the cloud, least privilege (Least privilege and isolation).

Pitfalls

• Local state files on laptops: lost, conflicting, containing secrets.
• count index shifts: removing item 0 recreates every later resource; use for_each with stable keys.
• Renaming a resource destroys and recreates it unless you add a moved block.
• Console hot-fixes create drift that the next apply silently reverts.
• One giant state for everything: slow plans, huge blast radius, lock contention.

Implementations

Sources

• Kief Morris, Infrastructure as Code, 2nd ed., O'Reilly, 2020.
• Yevgeniy Brikman, Terraform: Up & Running, 3rd ed., O'Reilly, 2022.
• OpenTofu, Documentation. docs · OpenTofu, tofu plan. docs · HashiCorp, Terraform documentation. docs