Build an End-to-End ETL Pipeline Using open source stack; MinIO, Airbyte, dbt, and Postgres

In this video we are building a complete end to end ETL pipeline with open source data stack. No managed services or vendor lock-in. We will use MinIO as our data source. It’s an S3 compatible service. We will ingest data from MinIO with the help of Airbyte and load it into a Postgres database. Using dbt we will transform this data into a final reporting layer as dimensions and facts. We will use Airflow to orchestrate the whole ETL pipeline.
By the end, you'll have a fully functional ETL pipeline running locally.

GitHub repo:
https://github.com/hnawaz007/modern-d...

MinIO Install on docker video:
   • How to build on-premise Data Lake? | Build...  

Airbyte Install video:
   • how to install Airbyte via command line | ...  

Airflow Installation video:
   • Airflow Installation & Configurations | Em...  

Postgres install video:
   • How to install PostgreSQL  on windows | cr...  

Python Install video:
   • Install Jupyter Notebook on Windows | Pyth...  

#etl #dataengineering #moderndatastack

💥Subscribe to our channel:
   / haqnawaz  

📌 Links
-----------------------------------------
#️⃣ Follow me on social media! #️⃣

🔗 GitHub: https://github.com/hnawaz007
📸 Instagram:   / bi_insights_inc  
📝 LinkedIn:   / haq-nawaz  
🔗   / hnawaz100