how to orchestrate a dbt project using Airflow | Airflow | dbt | ETL

Streamline your data pipeline by orchestrating dbt with Airflow! 🎛️

With Airflow, you can schedule and monitor dbt transformations, ensuring seamless data workflows from start to finish. Automate ETL processes, gain visibility into your data jobs, and maintain full control over dependencies. It's the perfect combo for scaling your data infrastructure! In the upcoming vidoe we will cover dbt integration with Airlfow.

Benefits of using Airflow for DBT Orchestration:

✨ Simplify your data workflows: Manage DBT projects and dependencies with ease
✨ Increase productivity: Automate repetitive tasks and focus on high-value work
✨ Improve collaboration: Share workflows, monitor progress, and collaborate with team members

Link to GitHub repo (Airflow): https://github.com/hnawaz007/dbt-dw/t...
Link to dbt series: https://hnawaz007.github.io/mds.html
Link to Airflow series: https://hnawaz007.github.io/airflow.html
Link to dbt dagster orchestration:    • Orchestrate Airbyte & dbt with Dagster | O...  
Link to dbt GitHub project: https://github.com/hnawaz007/dbt-dw

Command to build Docker Airflow image: docker build -t airflow-sql -f Dockerfile . --no-cache
Command to build & start containers: docker-compose up -d

#dbt #airflow #etl

💥Subscribe to our channel:
   / haqnawaz  

📌 Links
-----------------------------------------
#️⃣ Follow me on social media! #️⃣

🔗 GitHub: https://github.com/hnawaz007
📸 Instagram:   / bi_insights_inc  
📝 LinkedIn:   / haq-nawaz  
🔗   / hnawaz100  

-----------------------------------------


Topics covered in this video:
==================================
0:00 - Introduction to dbt & Airflow
0:55 - dbt project
1:04 - Build Airflow Docker image
1:35 - Review & test dbt project on Docker
2:30 - Airflow DAG
3:27 - Review the DAG in UI
3:50 - Execute the DAG
4:17 - Review DAGs Outupt