Introduction
In a complex CI/CD workflow, various data would often have to be exchanged across jobs. Anything from generated environment variables down to important build artifacts would be fair game. GitHub Actions supports this via what are called job outputs, which can make your workflows much more modular and flexible.
Be it API credentials, a database ID, something dynamically generated, job outputs provide a structured way to carry information forward between dependent jobs.
Steps we will cover in this article:
- Definition of Job Outputs
- Using Outputs in Matrix Jobs
- Passing Sensitive Data Between Jobs
- Passing Database Credentials -Real-World Use Case
- Best Practices for Job Outputs
Definition of Job Outputs
Outputs of a job in GitHub Actions are defined inside of the job using the outputs field. These will map to the results of steps in the same job and can be referenced from other jobs utilizing the dependsOn keyword. Outputs will always be strings, and will be evaluated at the closing of the job execution. Outputs are particularly useful for when you want to be able to pass dynamic information such as tokens, IDs, or version numbers across your workflow.
jobs:
job1:
runs-on: ubuntu-latest
outputs:
output1: ${{ steps.step1.outputs.result }}
steps:
- id: step1
run: echo "result=hello" >> "$GITHUB_OUTPUT"
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo "Job 1 output: ${{ needs.job1.outputs.output1 }}"
In the example above, job1 defines an output named output1 that is set to the result of the first step. Then in job2 you can use the value of that output using the needs context in order to pass the value between jobs seamlessly.
Using Outputs in Matrix Jobs
Things get somewhat more interesting if you are employing a matrix approach. Matrix jobs create one output for each instance of the job. Best to make sure, though, that the names for these outputs are unique, especially if you are using a combination of multiple versions or configurations since the order of execution for matrix jobs is not guaranteed.
jobs:
job1:
runs-on: ubuntu-latest
strategy:
matrix:
version: [1, 2, 3]
outputs:
output_${{ matrix.version }}: ${{ steps.generate_output.outputs.version }}
steps:
- id: generate_output
run: echo "version=${{ matrix.version }}" >> "$GITHUB_OUTPUT"
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo "Matrix outputs: ${{ toJSON(needs.job1.outputs) }}"
Here we define a three-version matrix. Each of the matrix jobs outputs a job displaying its version number. job2 can then utilize all matrix outputs en bloc due to the needs context.
Passing Sensitive Data Between Jobs
Sometimes, you want to pass some secret information between jobs, be it API keys, passwords, or tokens. GH Actions creates functionality for handling secrets, such as their redaction in logs, out of the box. You can also explicitly mask sensitive information that you pass as job outputs using the add-mask command.
jobs:
job1:
runs-on: ubuntu-latest
outputs:
api_key: ${{ steps.generate_key.outputs.key }}
steps:
- id: generate_key
run: |
key="my-sensitive-api-key"
echo "::add-mask::$key"
echo "key=$key" >> "$GITHUB_OUTPUT"
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo "Using API key: ${{ needs.job1.outputs.api_key }}"
Here, in the above example, API key created in job1 is masked before passing to job2 so that it won't show up in the logs.
Passing Database Credentials -Real-World Use Case
Consider an applied view-a real-world example-only integrating a database in a CI pipeline. You would create, in a job, the database and then pass the credential or the connection string to another job for the testing purpose.
jobs:
create-db:
runs-on: ubuntu-latest
outputs:
db_id: ${{ steps.db_setup.outputs.db_id }}
db_password: ${{ steps.db_setup.outputs.db_password }}
steps:
- id: db_setup
run: |
db_id=$(uuidgen)
db_password=$(openssl rand -base64 32)
echo "db_id=$db_id" >> "$GITHUB_OUTPUT"
echo "db_password=$db_password" >> "$GITHUB_OUTPUT"
test-db:
runs-on: ubuntu-latest
needs: create-db
steps:
- run: |
echo "Testing database with ID: ${{ needs.create-db.outputs.db_id }}"
echo "Using password: ${{ needs.create-db.outputs.db_password }}"
Here, create-db outputs a database id and password. These are then given as arguments to test-db, which can use them in conducting integration tests.
Best Practices for Job Outputs
- Keep Outputs Simple: Outputs should be simple strings: for example, IDs, paths or tokens. If your use requires more complex data to be passed, encode it first (for example Base64) before passing as an output.
- Mask Sensitive Data: Generally, the outputs must not be sensitive; by example, passwords, API keys must not appear in logs using
add-mask. - Using Unique Output Names in Matrices: If output names are used within a matrix, use unique names to avoid name conflicts when jobs run on matrix.
- Limitation in Size of Output: The outputs that could be provided to a single action are not more than 1 MB maximum and also cannot exceed a total of 50MB for all the outputs in one workflow run. Larger data is to be considered to be passed using artifacts instead.
Conclusion
Passing information between jobs in GitHub Actions can really clean up and streamline your CI/CD workflows. Whether you are passing API keys, dynamic values, or build artifacts, job outputs give you flexible and secure ways to handle data across jobs. By following best practices, add-mask features, and matrix strategies, you'll be in a position to create workflows that are secure, maintainable, and efficient.