Skip to main content
← Back to workflows

How to integrate CI/CD workflows with Terraform in GitHub Actions?

diggerhq/digger -
GitHub Action
v0.4.22
2,883
Contributors
Contributor - veziakContributor - motatoes
Categories
CICUBE ANALYTICS INSIGHTS
Engineering Velocity: 25% Team Time Lost to CI Issues
View Platform →
3.5h
Time Saved/Dev/Week
40%
Faster Releases
Click for next insight
Usage
- name: run-digger
uses: diggerhq/[email protected]

digger logo

digger

Digger is an open source IaC orchestration tool. Digger allows you to run IaC in your existing CI pipeline ⚡️


We've been diving into integrating CI/CD workflows with Terraform and stumbled upon a tool named Digger. It's pretty neat for streamlining Terraform operations within our existing CI/CD setup.

Digger eliminates the need for a separate CI system just for Terraform by leveraging our existing CI infrastructure. This approach is not only secure—since it doesn't require sharing cloud access secrets with third-party systems—but it's also cost-effective because it utilizes the compute resources we already pay for.

Key Features:

  • Direct execution of Terraform plans and apply operations within pull request comments.
  • Utilizes the same CI runners, hence no need for separate ones.
  • Supports Open Policy Agent (OPA) for role-based access control.
  • Implements PR-level locks to prevent conflicts across multiple pull requests, which is a neat addition to the native state locks provided by Terraform.
  • Compatibility with multiple tools and services like Terragrunt, various Terraform versions, and static analysis tools like Checkov.
  • It also offers drift detection to keep track of unwanted changes.

From my understanding, Digger seems particularly useful for teams looking to maintain a lean CI/CD stack while ensuring secure, scalable, and efficient Terraform management. It reduces overhead and simplifies management by integrating directly with existing CI tools, potentially speeding up deployment cycles and reducing security risks.

How to Apply on Merge in Terraform?

By default, Digger avoids running the apply command automatically when a merge occurs. Instead, it promotes running apply manually while the PR is open and only proceeding with the merge once it has successfully executed across all target environments. This approach prioritizes safety by ensuring that changes are thoroughly vetted before integration into the main branch. However, this method can cause the main branch to lag behind the actual state of our environments. F rom a strict CI/CD standpoint, this isn't ideal because the theory holds that the plan should be treated as an immutable artifact, and apply should merely deploy that artifact without discrepancies.

For teams preferring automation, Digger allows configuration adjustments to automate the apply on merge. Here’s how you can set it up in the digger.yml:

projects:
...
workflows:
dev:
workflow_configuration:
on_pull_request_pushed: [digger plan]
on_pull_request_closed: [digger unlock]
on_commit_to_default: [digger apply]

With this setup, digger plan is triggered when a pull request is pushed, ensuring that plans are generated and reviewed. Once the pull request is closed, Digger releases any locks with digger unlock. Most critically, when commits are made to the default branch (typically after a merge), digger apply is executed automatically, aligning the main branch with the current state of the environments swiftly.

This configuration could streamline our workflow by automating parts of the process, but it’s important to balance the benefits of automation against the risks of reduced oversight on each change. It’s definitely a setup worth considering if we're confident in our testing and review processes.

How to Auto-merge with Digger Terraform?

Digger allows for an auto_merge option, which can be extremely handy. When set to true, this option will automatically merge the pull request as soon as all checks pass, including the successful application of the Terraform changes. This is particularly useful if we want to streamline our workflow and reduce the manual steps required during the integration process.

Here's how you can configure it in the digger.yml file:

projects:
- name: development
dir: dev
- name: production
dir: prod
auto_merge: true

It's important to note that the auto_merge setting is applied at the top level of the configuration, not at the individual project level. This means it affects all projects defined in the YAML file.

Implementing auto_merge can significantly speed up our development cycle by ensuring that merges are only delayed by necessary checks and not by manual processes. However, it's vital to ensure our testing and review systems are robust to prevent any unanticipated issues from slipping through into production.

How to use Mutliple Aws accounts with Digger Terraform?

We've been getting up to speed with setting up Digger for managing Terraform in multiple AWS accounts. Here's a simplified guide on how to configure Digger to operate separately across development and production environments, each with their dedicated AWS accounts.

  • Configuration File:

    • Create a digger.yml file and place it at the root of your repository. This file will specify paths to the Terraform directories for development and production:
      projects:
      - name: develop
      dir: dev
      workflow_file: digger-run-dev.yml

      - name: production
      dir: prod
      workflow_file: digger-run-prod.yml
  • GitHub Environments Setup:

    • Navigate to your repository settings in GitHub, then to Environments.
    • Create two environments: development and production.
    • For each environment, configure two secrets: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
  • GitHub Actions Workflows:

    • Create two GitHub Actions workflow files:
      • .github/workflows/digger-run-dev.yml for the development environment.
      • .github/workflows/digger-run-prod.yml for the production environment.
    • Adjust each workflow to point to the correct environment and modify steps to rename and use the appropriate digger.yml configuration.
    name: Digger Dev
    on:
    pull_request:
    branches: [ "main" ]
    types: [ opened, synchronize ]
    issue_comment:
    types: [created]
    workflow_dispatch:

    jobs:
    plan:
    runs-on: ubuntu-latest
    environment: development
    permissions:
    contents: write
    id-token: write
    pull-requests: write
    statuses: write

    steps:
    - uses: actions/checkout@v4
    - name: Rename
    run: |
    mv digger.dev.yml digger.yml
    - name: digger run
    uses: diggerhq/[email protected]
    with:
    setup-aws: true
    aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
    aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    aws-region: us-east-1
    env:
    GITHUB_CONTEXT: ${{ toJson(github) }}
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}