DEVOURED
Building My Own Self-Hosted dbt Cloud
Building a custom wrapper around dbt Core using React, FastAPI, and Prefect provides a cheaper, more customizable alternative to managed dbt Cloud.
What: The author built a self-hosted orchestration layer that uses Prefect for job management and FastAPI to bridge the gap between dbt's CLI and a web-based UI.
Why it matters: Moving away from proprietary vendor dashboards allows teams to gain deeper insight into log history and job metadata, provided they rely on API-level integration instead of screen-scraping the CLI output.
Takeaway: If building your own dbt management layer, interface directly with the dbt metadata artifacts (like run_results.json) rather than attempting to parse standard output logs.
Decoder
- dbt (data build tool): A framework that enables data analysts and engineers to transform data in their warehouse using SQL.
Original article
A self-hosted dbt Cloud-style app can deliver much of the developer experience by combining dbt Core with a React/FastAPI interface and Prefect for orchestration. The biggest lesson is to use APIs, not CLI scraping, for reliable job management, logs, deployments, and real-time run status.
Explore all 71 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi