How to use dbt-checkpoint to ensure the quality of your dbt projects?

- name: dbt-checkpoint
  uses: dbt-checkpoint/dbt-[email protected]    

dbt-checkpoint

🎣 List of pre-commit hooks to ensure the quality of your dbt projects.

dbt-checkpoint GitHub Action is use to ensure the quality of our dbt projects.

The content informs readers about how to use pre-commit hooks, made with dbt-checkpoint, to automatically check their validity in dbt projects, keeping the quality of the code up and the manual reviewing effort down.

Using dbt-checkpoint in our workflow, one can enforce a homogeneous criterion very straightforwardly across all our dbt projects. Automating checks and validations relieves review burdens and prevents as many errors as possible.

Using dbt-checkpoint makes dbt project validations automated enhances the code review process and makes the release process easier. making sure every model, source, or script meets our quality standards.

How to install dbt-checkpoint?​

Installation:

pip install pre-commit

Install pre-commit to manage pre-commit hooks which upon installation, will include dbt-checkpoint hooks for validating dbt projects.

How to configure dbt-checkpoint in a project?​

Setup Create a .pre-commit-config.yaml file in your project root:

repos:
- repo: https://github.com/dbt-checkpoint/dbt-checkpoint
  rev: v1.2.1
  hooks:
  - id: dbt-parse
  - id: dbt-docs-generate
    args: ["--cmd-flags", "++no-compile"]
  - id: check-script-semicolon
  - id: check-script-has-no-table-name
  - id: check-model-has-all-columns
    name: Check columns - core
    files: ^models/core
  - id: check-model-has-all-columns
    name: Check columns - mart
    files: ^models/mart
  - id: check-model-columns-have-desc
    files: ^models/mart

This setup would enforce code quality by checking model and script properties.

Checking if debt-checkpoint is configured properly:

Append the below content in a file .dbt-checkpoint.yaml with values

version: 1
disable-tracking: true
dbt-project-dir: my_dbt_project

Custom parameters/ configurations like turning off telemetry or setting a specific directory of the best project that could easily configure the hook.

How can this dbt-checkpoint be run manually?​

Manual Run:

pre-commit run --all-files

You should run your hooks manually to cover more files, not just the staged ones. This can be useful at first setup or when performing a thorough audit across the whole project.

How to run debt-checkpoint in CI/CD?​

Add a development profile to your profiles.yml file in the project root directory. This file is used by dbt to map configuration information about the connections to different databases used in your project.

# profiles.yml
- name: development
  target: dev
  outputs:
    dev:
      type: snowflake
      threads: 8
      client_session_keep_alive: true
      account: "{{ env_var('ACCOUNT') }}"
      database: "{{ env_var('DATABASE') }}"
      schema: "{{ env_var('SCHEMA') }}"
      user: "{{ env_var('USER') }}"
      password: "{{ env_var('PASSWORD') }}"
      role: "{{ env_var('ROLE') }}"
      warehouse: "{{ env_var('WAREHOUSE') }}"

Create a GitHub actions workflow file to be triggered in response to various "push" and "pull_request" events. This example sets a trigger for changes to the main branch:

name: dbt-checkpoint
on:
  push:
  pull_request:
    branches:
      - main
jobs:
  dbt-checkpoint
    runs-on: ubuntu-latest
    steps:
    - name: Checkout code
      uses: actions/checkout@v2
    - name: Setup Python
      uses: actions/setup-python@v2
    - id: Get file changes
      uses: trilom/file-changes-[email protected]
    - name: Run dbt checkpoint
      uses: dbt-checkpoint/[email protected]
      with:
        extra_args: --files ${{ steps.

How to set up pre-commit hooks in GitHub CI?​

repos:
- repo: https://github.com/dbt-checkpoint/dbt-checkpoint
  rev: v1.2.1
  hooks:
  - id: debt-parse
  - id: check-model-has-tests
    args: ["--test-cnt", "2", "--"]

It runs all pre-commit hooks in CI to ensure that each committed piece of code meets the desired quality standards, thus stopping problems from going into production.