Skip to main content
← Back to workflows

How to use dbt-checkpoint to ensure the quality of your dbt projects?

dbt-checkpoint/dbt-checkpoint -
GitHub Action
v2.0.1
573
Contributors
Contributor - BAntonelliniContributor - ssassiContributor - tomsej
Categories
CICUBE ANALYTICS INSIGHTS
Engineering Velocity: 25% Team Time Lost to CI Issues
View Platform →
3.5h
Time Saved/Dev/Week
40%
Faster Releases
Click for next insight
Usage
- name: dbt-checkpoint
uses: dbt-checkpoint/dbt-[email protected]

dbt-checkpoint logo

dbt-checkpoint

🎣 List of pre-commit hooks to ensure the quality of your dbt projects.


dbt-checkpoint GitHub Action is use to ensure the quality of our dbt projects.

The content informs readers about how to use pre-commit hooks, made with dbt-checkpoint, to automatically check their validity in dbt projects, keeping the quality of the code up and the manual reviewing effort down.

Using dbt-checkpoint in our workflow, one can enforce a homogeneous criterion very straightforwardly across all our dbt projects. Automating checks and validations relieves review burdens and prevents as many errors as possible.

Using dbt-checkpoint makes dbt project validations automated enhances the code review process and makes the release process easier. making sure every model, source, or script meets our quality standards.

How to install dbt-checkpoint?

Installation:

pip install pre-commit

Install pre-commit to manage pre-commit hooks which upon installation, will include dbt-checkpoint hooks for validating dbt projects.

How to configure dbt-checkpoint in a project?

Setup Create a .pre-commit-config.yaml file in your project root:

repos:
- repo: https://github.com/dbt-checkpoint/dbt-checkpoint
rev: v1.2.1
hooks:
- id: dbt-parse
- id: dbt-docs-generate
args: ["--cmd-flags", "++no-compile"]
- id: check-script-semicolon
- id: check-script-has-no-table-name
- id: check-model-has-all-columns
name: Check columns - core
files: ^models/core
- id: check-model-has-all-columns
name: Check columns - mart
files: ^models/mart
- id: check-model-columns-have-desc
files: ^models/mart

This setup would enforce code quality by checking model and script properties.

Checking if debt-checkpoint is configured properly:

Append the below content in a file .dbt-checkpoint.yaml with values

version: 1
disable-tracking: true
dbt-project-dir: my_dbt_project

Custom parameters/ configurations like turning off telemetry or setting a specific directory of the best project that could easily configure the hook.

How can this dbt-checkpoint be run manually?

Manual Run:

pre-commit run --all-files

You should run your hooks manually to cover more files, not just the staged ones. This can be useful at first setup or when performing a thorough audit across the whole project.

How to run debt-checkpoint in CI/CD?

Add a development profile to your profiles.yml file in the project root directory. This file is used by dbt to map configuration information about the connections to different databases used in your project.

# profiles.yml
- name: development
target: dev
outputs:
dev:
type: snowflake
threads: 8
client_session_keep_alive: true
account: "{{ env_var('ACCOUNT') }}"
database: "{{ env_var('DATABASE') }}"
schema: "{{ env_var('SCHEMA') }}"
user: "{{ env_var('USER') }}"
password: "{{ env_var('PASSWORD') }}"
role: "{{ env_var('ROLE') }}"
warehouse: "{{ env_var('WAREHOUSE') }}"

Create a GitHub actions workflow file to be triggered in response to various "push" and "pull_request" events. This example sets a trigger for changes to the main branch:

name: dbt-checkpoint
on:
push:
pull_request:
branches:
- main
jobs:
dbt-checkpoint
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Setup Python
uses: actions/setup-python@v2
- id: Get file changes
uses: trilom/file-changes-[email protected]
- name: Run dbt checkpoint
uses: dbt-checkpoint/[email protected]
with:
extra_args: --files ${{ steps.

How to set up pre-commit hooks in GitHub CI?

repos:
- repo: https://github.com/dbt-checkpoint/dbt-checkpoint
rev: v1.2.1
hooks:
- id: debt-parse
- id: check-model-has-tests
args: ["--test-cnt", "2", "--"]

It runs all pre-commit hooks in CI to ensure that each committed piece of code meets the desired quality standards, thus stopping problems from going into production.