Introduction

I will try to provide some insights on how parallel running jobs using GitHub Actions can be helpful in optimizing our CI/CD pipelines. Their parallel running jobs thus allow running independent jobs, which may save much precious time in our workflows. This is very helpful for larger projects because the overall build time will be reduced and debugging can get very easy since the jobs are separated themselves.

Stop talking, show me the code!

name: (Compiler) Rust

on:
  push:
    branches: ["main"]

jobs:
  test: # Job 'test' starts same time as Job 'lint'
    name: Rust Test (${{ matrix.target.os }})
    strategy:
      matrix: # Parallelize jobs across different OS environments
        target:
          - target: ubuntu-latest
            os: ubuntu-latest
          - target: macos-latest
            os: macos-latest
          - target: windows-latest
    runs-on: ${{ matrix.target.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: Swatinem/rust-cache@v2 # Cache Rust dependencies
      - name: cargo test
        run: cargo test

  lint: # Job 'lint' starts same time as Job 'test'
    name: Rust Lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions-rs/toolchain@v1
        with:
          toolchain: nightly-2023-08-01
          override: true
          components: rustfmt, clippy
      - uses: Swatinem/rust-cache@v2
      - name: rustfmt
        run: grep -r --include "*.rs" --files-without-match "@generated" crates | xargs rustup run nightly-2023-08-01 rustfmt --check --config="skip_children=true"

What’s going on here?

• Matrix Strategy: This is where things get interesting. The matrix section allows us to run the same job on different platforms or configurations, which in our case is just Ubuntu.
• Parallel Jobs - Note above that the test and the lint jobs run in parallel, reducing overall job runtime.
• Caching: This is done by leveraging the Rust cache, which increases build times in subsequent compilations by skipping superfluous downloads and recompilations.

Depandent GitHub Actions Workflows

By using the dependencies between jobs, we don’t waste time and resources on work that doesn’t need to be done on the off-chance that earlier jobs may fail. In such a way, it makes sure that more efficiently run jobs will reduce debugging time when failures are caught earlier and later jobs are correctly skipped.

name: (Compiler) Rust

on:
  push:
    branches: ["main"]

jobs:
  test:
    name: Rust Test (${{ matrix.target.os }})
    strategy:
      matrix:
        target:
          - target: ubuntu-latest
            os: ubuntu-latest
          - target: macos-latest
            os: macos-latest
          - target: windows-latest
    runs-on: ${{ matrix.target.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: Swatinem/rust-cache@v2
      - name: cargo test
        run: cargo test

  lint:
    name: Rust Lint
    runs-on: ubuntu-latest
    needs: [test] # Job 'lint' depends on Job 'test'
    steps:
      - uses: actions/checkout@v4
      - uses: actions-rs/toolchain@v1
        with:
          toolchain: nightly-2023-08-01
          override: true
          components: rustfmt, clippy
      - uses: Swatinem/rust-cache@v2 # Reuse cache from Job #1
      - name: rustfmt
        run: grep -r --include "*.rs" --files-without-match "@generated" crates | xargs rustup run nightly-2023-08-01 rustfmt --check --config="skip_children=true"

The important thing here is the use of setting jobs.<job-id>.needs, which defines dependences between jobs. In this way, we could be assured about the order of job execution, and also avoid spending resources on running useless jobs in case something in any critical task fails.

Conclusion

This post elaborated on parallel jobs within GitHub Actions, which resulted in a significant improvement to the efficiency of the CI/CD workflow by reducing build times and making debugging easier. Further performance optimization is possible with a matrix strategy: running jobs on different platforms or configurations is achievable. Additionally, implementing dependencies between jobs using the jobs.<job-id>.needs configuration ensures that jobs only run when previous, critical jobs are successful, saving resources and avoiding unnecessary executions.

Monitoring Duration of GitHub Actions Workflows

Would you like to see how much time your GitHub Actions are saving after adding parallelization? Remember, you can’t optimize metrics you don’t measure. You can start using cicube.io for free and get GitHub Actions workflows's insights right away.