How to Run Terraform in Bitbucket Pipelines

Automating your processes is a key best practice when working with Terraform, OpenTofu, and similar tools for infrastructure as code. This gives you a consistent workflow for all changes to the infrastructure you manage using these tools. Two common approaches are:

1. Use a dedicated infrastructure automation platform such as Spacelift or HCP Terraform
2. Use a generalized automation tool where you can build your own automation workflows.

The second option often involves using features available on a version control and collaboration platform such as GitHub, GitLab, or Bitbucket. In this blog post, we will cover an option in the second category of automation tools: how to run Terraform in Bitbucket Pipelines.

Note that this blog post covers how to run Terraform in Bitbucket Pipelines, but the steps are also applicable to OpenTofu with minor changes (e.g., use the OpenTofu binary instead of the Terraform binary).

To run Terraform in Bitbucket Pipelines, you define your pipelines in a single bitbucket-pipelines.yml file at the root of your repository. Bitbucket Pipelines is Bitbucket Cloud’s built-in CI/CD service, and every step runs in a fresh Docker container, so you choose an image such as hashicorp/terraform and script the Terraform commands yourself.

What are Bitbucket Pipelines?

Bitbucket Cloud, or simply Bitbucket, is a software-as-a-service (SaaS) collaboration platform for source code. Bitbucket is part of the Atlassian suite of products and integrates well with Jira and Confluence, which many organizations use for project management and documentation.

Bitbucket was previously available as a single-instance, self-hosted product called Bitbucket Server, but Atlassian discontinued support for it in February 2024. If you want to host Bitbucket yourself, an alternative product called Bitbucket Data Center continues to be supported. Bitbucket Data Center is designed for high availability and is intended for larger deployments.

One of Bitbucket’s features is the built-in continuous integration and continuous delivery (CI/CD) service, Bitbucket Pipelines.

With Bitbucket Pipelines, you can define pipelines that should run when you push code to branches, when you open or update a pull-request, on a schedule, manually, and more. Your pipelines can include arbitrary scripts that perform steps such as building an application, migrating a database, scaling up a virtual machine, provisioning infrastructure with Terraform, and much more.

Pipelines are defined as code in a file named bitbucket-pipelines.yml, and each pipeline step is executed in a fresh Docker container. You can use an existing Docker image from a container registry or use your own custom images.

Bitbucket Pipelines is a natural choice for automation if your source code is hosted on Bitbucket.

How to run Terraform in Bitbucket Pipelines

In this section, we will describe how to enable and run Terraform in Bitbucket Pipelines for an existing Bitbucket repository. The prerequisites for following this walkthrough are:

• Access to a Bitbucket Cloud workspace
• Access to at least one repository where you can configure a pipeline and run Terraform
• Administrator access to an AWS account and an S3 bucket for state storage

In the following sections, we will build a few basic pipelines for running Terraform, with the aim of achieving the following:

• Provision infrastructure on AWS.
• Use an AWS S3 state backend.
• Use OIDC workload identity to authenticate to AWS for state management and provisioning.
• Run init, fmt, validate, and plan commands for every push to a feature branch and every pull request targeting the main branch.
• Run init, plan and apply for every commit to the main branch.

Enable Bitbucket Pipelines for a repository

First, we need to enable Bitbucket Pipelines. Bitbucket Pipelines are disabled by default. You enable pipelines by manually committing a file named bitbucket-pipelines.yml or using the wizard in the UI.

Your first pipeline run must be triggered manually. After committing the bitbucket-pipelines.yml file to the repository, go to the “Pipelines” section of your repository and you will see the following view. Click on “Run initial pipeline” to get started. The following runs will be triggered automatically based on the trigger events you have defined in your pipelines.

Define your pipelines in bitbucket-pipelines.yml

You can configure one or more pipelines for your Bitbucket repository using YAML configuration in a file named bitbucket-pipelines.yml. The file must have this exact name, and it must be placed in the root of your repository. You can only have one pipeline file per repository.

We begin by specifying which Docker image we want to use for the steps of our pipelines.

image: hashicorp/terraform:1.15.7

Next, we define reusable steps for fmt, validate, plan and apply that we can include in the pipelines we will define later. We add these steps under definitions.steps using YAML anchors.

Configuring the S3 Terraform backend

To configure the AWS S3 state backend, add the following code under the definitions section of your pipeline:

definitions:
  scripts:
    - &tf-init >
      terraform init -input=false
        -backend-config="bucket=${TF_STATE_BUCKET}"
        -backend-config="key=${TF_STATE_KEY}"
        -backend-config="region=${AWS_DEFAULT_REGION}"

Go to your repository settings and click on “Repository variables”, then configure TF_STATE_BUCKET, TF_STATE_KEY, and AWS_DEFAULT_REGION with values corresponding to your bucket.

Configuring AWS OIDC authentication

We want to use the OIDC workload identity federation feature to achieve this. The benefit of this authentication mechanism is that we do not have to deal with long-lived credentials.

The first step is to go to your repository settings and click on “OpenID Connect”. Copy the values for the Identity provider URL, Audience, Workspace UUID, and Repository UUID.

In the bitbucket-pipelines.yml file, add another reusable script definition for OIDC:

definitions:
  scripts:
    - &oidc-setup >
      echo "$BITBUCKET_STEP_OIDC_TOKEN" > /tmp/web-identity-token &&
      export AWS_ROLE_ARN="$AWS_ROLE_ARN" &&
      export AWS_WEB_IDENTITY_TOKEN_FILE=/tmp/web-identity-token

Best practices for running Terraform in Bitbucket Pipelines

1. Use OIDC authentication if your Terraform provider supports it

If your target platform supports authentication with OIDC workload identity federation, you should always use this method of authentication to avoid managing long-lived credentials.

2. Use the built-in secrets management feature

Manage secrets by marking a variable as sensitive in your repository variables. Sensitive variables will not be exposed in the pipeline logs.

3. Define reusable steps and scripts

YAML anchors allow you to easily build reusable steps and scripts for composing your pipelines, ensuring a consistent experience.

4. Add concurrency control

If you have one or more steps that should not run at the same time, you can add them to a concurrency-group to avoid locked Terraform state file issues.

5. Use pipeline deployments for multiple environments

You can configure each deployment with unique variable values, allowing you to use different Terraform state backend configurations and authentication details for each environment.

Frequently asked questions

• Why use Bitbucket Pipelines for Terraform?

  Bitbucket Pipelines runs Terraform builds directly inside Bitbucket Cloud, so teams already using Jira and Confluence avoid adopting a separate CI/CD tool. It supports Docker-based runners, deployment environments for scoped secrets, and manual approval steps.

• How do I authenticate to AWS from Bitbucket Pipelines?

  Use OpenID Connect (OIDC): register Bitbucket as an identity provider in AWS IAM, create a role with a trust policy scoped to your workspace or repository, then set oidc: true on the step so STS exchanges BITBUCKET_STEP_OIDC_TOKEN for short-lived credentials.

• What's the difference between Bitbucket Pipelines and GitHub Actions for Terraform?

  Bitbucket Pipelines is Atlassian’s built-in CI/CD tied to Bitbucket Cloud, while GitHub Actions is a marketplace-driven workflow engine native to GitHub.

• Is Bitbucket Pipelines free for Terraform workflows?

  A free tier is available with 50 build minutes per workspace each month, enough for light experimentation.