- name: dbt-checkpoint
uses: dbt-checkpoint/dbt-[email protected]
dbt-checkpoint
🎣 List of pre-commit
hooks to ensure the quality of your dbt
projects.
dbt-checkpoint
GitHub Action is use to ensure the quality of our dbt projects.
The content informs readers about how to use pre-commit hooks, made with dbt-checkpoint, to automatically check their validity in dbt projects, keeping the quality of the code up and the manual reviewing effort down.
Using dbt-checkpoint in our workflow, one can enforce a homogeneous criterion very straightforwardly across all our dbt projects. Automating checks and validations relieves review burdens and prevents as many errors as possible.
Using dbt-checkpoint
makes dbt project validations automated enhances the code review process and makes the release process easier. making sure every model, source, or script meets our quality standards.
How to install dbt-checkpoint?
Installation:
pip install pre-commit
Install pre-commit
to manage pre-commit hooks which upon installation, will include dbt-checkpoint
hooks for validating dbt projects.
How to configure dbt-checkpoint in a project?
Setup
Create a .pre-commit-config.yaml
file in your project root:
repos:
- repo: https://github.com/dbt-checkpoint/dbt-checkpoint
rev: v1.2.1
hooks:
- id: dbt-parse
- id: dbt-docs-generate
args: ["--cmd-flags", "++no-compile"]
- id: check-script-semicolon
- id: check-script-has-no-table-name
- id: check-model-has-all-columns
name: Check columns - core
files: ^models/core
- id: check-model-has-all-columns
name: Check columns - mart
files: ^models/mart
- id: check-model-columns-have-desc
files: ^models/mart
This setup would enforce code quality by checking model and script properties.
Checking if debt-checkpoint is configured properly:
Append the below content in a file .dbt-checkpoint.yaml
with values
version: 1
disable-tracking: true
dbt-project-dir: my_dbt_project
Custom parameters/ configurations like turning off telemetry or setting a specific directory of the best project that could easily configure the hook.
How can this dbt-checkpoint be run manually?
Manual Run:
pre-commit run --all-files
You should run your hooks manually to cover more files, not just the staged ones. This can be useful at first setup or when performing a thorough audit across the whole project.
How to run debt-checkpoint in CI/CD?
Add a development
profile to your profiles.yml
file in the project root directory. This file is used by dbt to map configuration information about the connections to different databases used in your project.
# profiles.yml
- name: development
target: dev
outputs:
dev:
type: snowflake
threads: 8
client_session_keep_alive: true
account: "{{ env_var('ACCOUNT') }}"
database: "{{ env_var('DATABASE') }}"
schema: "{{ env_var('SCHEMA') }}"
user: "{{ env_var('USER') }}"
password: "{{ env_var('PASSWORD') }}"
role: "{{ env_var('ROLE') }}"
warehouse: "{{ env_var('WAREHOUSE') }}"
Create a GitHub actions workflow file to be triggered in response to various "push" and "pull_request" events. This example sets a trigger for changes to the main branch:
name: dbt-checkpoint
on:
push:
pull_request:
branches:
- main
jobs:
dbt-checkpoint
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Setup Python
uses: actions/setup-python@v2
- id: Get file changes
uses: trilom/file-changes-[email protected]
- name: Run dbt checkpoint
uses: dbt-checkpoint/[email protected]
with:
extra_args: --files ${{ steps.
How to set up pre-commit hooks in GitHub CI?
repos:
- repo: https://github.com/dbt-checkpoint/dbt-checkpoint
rev: v1.2.1
hooks:
- id: debt-parse
- id: check-model-has-tests
args: ["--test-cnt", "2", "--"]
It runs all pre-commit hooks in CI to ensure that each committed piece of code meets the desired quality standards, thus stopping problems from going into production.