- name: run-digger
uses: diggerhq/[email protected]
digger
Digger is an open source IaC orchestration tool. Digger allows you to run IaC in your existing CI pipeline ⚡️
We've been diving into integrating CI/CD workflows with Terraform and stumbled upon a tool named Digger. It's pretty neat for streamlining Terraform operations within our existing CI/CD setup.
Digger eliminates the need for a separate CI system just for Terraform by leveraging our existing CI infrastructure. This approach is not only secure—since it doesn't require sharing cloud access secrets with third-party systems—but it's also cost-effective because it utilizes the compute resources we already pay for.
Key Features:
- Direct execution of Terraform plans and apply operations within pull request comments.
- Utilizes the same CI runners, hence no need for separate ones.
- Supports Open Policy Agent (OPA) for role-based access control.
- Implements PR-level locks to prevent conflicts across multiple pull requests, which is a neat addition to the native state locks provided by Terraform.
- Compatibility with multiple tools and services like Terragrunt, various Terraform versions, and static analysis tools like Checkov.
- It also offers drift detection to keep track of unwanted changes.
From my understanding, Digger seems particularly useful for teams looking to maintain a lean CI/CD stack while ensuring secure, scalable, and efficient Terraform management. It reduces overhead and simplifies management by integrating directly with existing CI tools, potentially speeding up deployment cycles and reducing security risks.
How to Apply on Merge in Terraform?
By default, Digger avoids running the apply
command automatically when a merge occurs. Instead, it promotes running apply
manually while the PR is open and only proceeding with the merge once it has successfully executed across all target environments. This approach prioritizes safety by ensuring that changes are thoroughly vetted before integration into the main branch. However, this method can cause the main branch to lag behind the actual state of our environments. F
rom a strict CI/CD standpoint, this isn't ideal because the theory holds that the plan
should be treated as an immutable artifact, and apply
should merely deploy that artifact without discrepancies.
For teams preferring automation, Digger allows configuration adjustments to automate the apply on merge. Here’s how you can set it up in the digger.yml
:
projects:
...
workflows:
dev:
workflow_configuration:
on_pull_request_pushed: [digger plan]
on_pull_request_closed: [digger unlock]
on_commit_to_default: [digger apply]
With this setup, digger plan
is triggered when a pull request is pushed, ensuring that plans are generated and reviewed. Once the pull request is closed, Digger releases any locks with digger unlock
. Most critically, when commits are made to the default branch (typically after a merge), digger apply
is executed automatically, aligning the main branch with the current state of the environments swiftly.
This configuration could streamline our workflow by automating parts of the process, but it’s important to balance the benefits of automation against the risks of reduced oversight on each change. It’s definitely a setup worth considering if we're confident in our testing and review processes.
How to Auto-merge with Digger Terraform?
Digger allows for an auto_merge
option, which can be extremely handy. When set to true
, this option will automatically merge the pull request as soon as all checks pass, including the successful application of the Terraform changes. This is particularly useful if we want to streamline our workflow and reduce the manual steps required during the integration process.
Here's how you can configure it in the digger.yml
file:
projects:
- name: development
dir: dev
- name: production
dir: prod
auto_merge: true
It's important to note that the auto_merge
setting is applied at the top level of the configuration, not at the individual project level. This means it affects all projects defined in the YAML file.
Implementing auto_merge
can significantly speed up our development cycle by ensuring that merges are only delayed by necessary checks and not by manual processes. However, it's vital to ensure our testing and review systems are robust to prevent any unanticipated issues from slipping through into production.
How to use Mutliple Aws accounts with Digger Terraform?
We've been getting up to speed with setting up Digger for managing Terraform in multiple AWS accounts. Here's a simplified guide on how to configure Digger to operate separately across development and production environments, each with their dedicated AWS accounts.
-
Configuration File:
- Create a
digger.yml
file and place it at the root of your repository. This file will specify paths to the Terraform directories for development and production:projects:
- name: develop
dir: dev
workflow_file: digger-run-dev.yml
- name: production
dir: prod
workflow_file: digger-run-prod.yml
- Create a
-
GitHub Environments Setup:
- Navigate to your repository settings in GitHub, then to Environments.
- Create two environments:
development
andproduction
. - For each environment, configure two secrets:
AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
.
-
GitHub Actions Workflows:
- Create two GitHub Actions workflow files:
.github/workflows/digger-run-dev.yml
for the development environment..github/workflows/digger-run-prod.yml
for the production environment.
- Adjust each workflow to point to the correct environment and modify steps to rename and use the appropriate
digger.yml
configuration.
name: Digger Dev
on:
pull_request:
branches: [ "main" ]
types: [ opened, synchronize ]
issue_comment:
types: [created]
workflow_dispatch:
jobs:
plan:
runs-on: ubuntu-latest
environment: development
permissions:
contents: write
id-token: write
pull-requests: write
statuses: write
steps:
- uses: actions/checkout@v4
- name: Rename
run: |
mv digger.dev.yml digger.yml
- name: digger run
uses: diggerhq/[email protected]
with:
setup-aws: true
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
env:
GITHUB_CONTEXT: ${{ toJson(github) }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - Create two GitHub Actions workflow files: