Running Terraform in Azure DevOps Pipelines

Terraform is happy running from a laptop, and that is exactly the problem. State drifts, two people apply at once, and nobody can say for certain what is actually deployed. Moving Terraform into an Azure DevOps pipeline fixes the boring but important things: one place runs apply, state lives somewhere safe and locked, and every change leaves a record. This is the setup I reach for.

Prerequisites

• An Azure DevOps project and an Azure subscription.
• An ARM service connection in the project. Workload identity federation is the option to pick, since it lets the pipeline authenticate to Azure without a stored secret.
• A storage account for remote state. Keep it in its own resource group, away from the workloads it tracks.
• The Terraform extension from the Azure DevOps Marketplace (published by Microsoft DevLabs), which gives you the TerraformTask used below.

1. Put state in Azure Storage, not in the repo

Local state is fine until a second person runs Terraform. Remote state in an Azure Storage account gives you a single source of truth and, just as importantly, state locking. The azurerm backend takes a blob lease while it works, so a second apply waits instead of corrupting state.

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.0"
    }
  }

  backend "azurerm" {
    resource_group_name  = "rg-tfstate"
    storage_account_name = "sttfstateprod"
    container_name       = "tfstate"
    key                  = "app.prod.tfstate"
  }
}

provider "azurerm" {
  features {}
}

Create that storage account once, by hand or with a small bootstrap template, and turn on versioning and soft delete on the blob so you can recover a corrupted or deleted state file.

2. Split the pipeline into plan and apply

The most useful thing you can do is to separate the plan from the apply, with a human in between. Plan runs on every change and produces a saved plan file. Apply runs only that exact plan, and only after approval. That way the thing you reviewed is the thing that runs.

trigger:
  branches:
    include: [ main ]

pool:
  vmImage: ubuntu-latest

variables:
  serviceConnection: sc-azure-prod
  backendRg: rg-tfstate
  backendSa: sttfstateprod

stages:
  - stage: plan
    jobs:
      - job: terraform_plan
        steps:
          - task: TerraformTaskV4@4
            displayName: terraform init
            inputs:
              provider: azurerm
              command: init
              backendServiceArm: $(serviceConnection)
              backendAzureRmResourceGroupName: $(backendRg)
              backendAzureRmStorageAccountName: $(backendSa)
              backendAzureRmContainerName: tfstate
              backendAzureRmKey: app.prod.tfstate

          - task: TerraformTaskV4@4
            displayName: terraform validate
            inputs:
              provider: azurerm
              command: validate

          - task: TerraformTaskV4@4
            displayName: terraform plan
            inputs:
              provider: azurerm
              command: plan
              environmentServiceNameAzureRM: $(serviceConnection)
              commandOptions: '-out=$(Build.ArtifactStagingDirectory)/tfplan'

          - publish: $(Build.ArtifactStagingDirectory)/tfplan
            artifact: plan

The saved plan goes to the pipeline artifacts so the apply stage runs precisely what you reviewed, not a fresh plan that might differ.

3. Apply behind an approval

The apply stage downloads the plan artifact and runs it as an Azure DevOps deployment job targeting a prod environment. Approvals and checks attach to that environment, so apply waits for a human.

  - stage: apply
    dependsOn: plan
    jobs:
      - deployment: terraform_apply
        environment: prod
        strategy:
          runOnce:
            deploy:
              steps:
                - download: current
                  artifact: plan
                - task: TerraformTaskV4@4
                  displayName: terraform apply
                  inputs:
                    provider: azurerm
                    command: apply
                    environmentServiceNameAzureRM: $(serviceConnection)
                    commandOptions: '$(Pipeline.Workspace)/plan/tfplan'

Set the approval under Pipelines, Environments, prod, Approvals and checks. Now a noncompliant or surprising plan stops at a person before it changes anything.

4. Authentication without secrets

If you wire the service connection up with workload identity federation, the pipeline gets a short lived token from Azure DevOps and exchanges it with Entra ID at run time. There is no client secret sitting in a variable group waiting to leak or expire. If you cannot use federation yet, store the service principal credentials in a variable group backed by Azure Key Vault rather than as plain pipeline variables, and rotate them on a schedule.

5. Keep the working directory clean

A couple of small things save real pain later:

• Pin the provider and Terraform versions. A pipeline that silently upgrades the azurerm provider can produce a plan full of changes you did not make.
• Run terraform fmt -check and validate in the plan stage. Formatting and syntax problems should fail fast, before the plan.
• One state file per environment. Separate keys for dev, test, and prod. Never let a dev apply reach into prod state.

Closing thoughts

None of this makes Terraform do anything new. What it does is remove the failure modes that have nothing to do with infrastructure: lost state, concurrent applies, and changes nobody reviewed. Plan on every commit, apply only what was reviewed, keep state locked in Azure Storage, and authenticate without a stored secret. The piece this pipeline does not yet have is a check that the plan actually meets your security and compliance rules before it reaches apply. That is worth its own article, and it is where Azure Policy comes in.