Real-Time Forex ETL Pipeline | PySpark + AWS Glue + S3 + Redshift + Tableau

Welcome to my walkthrough of a complete real-time Forex exchange rate data pipeline built using modern data engineering tools and AWS cloud services. In this project, I fetch live Forex data using an API, process it with AWS Glue & PySpark, store it in Amazon Redshift, and visualize trends using Tableau Public.This project showcases how to build a scalable, production-ready ETL pipeline—ideal for financial data analytics and real-world big data systems.

What you'll learn in this video :

How to fetch real-time Forex data using Alpha Vantage API
Store raw and processed data in Amazon S3
Transform data using PySpark in AWS Glue
Load data efficiently into Amazon Redshift using COPY
Monitor performance using CloudWatch & SNS Alerts
Create interactive dashboards in Tableau Public

Tech Stack Used :

Python (for API ingestion)
PySpark & AWS Glue (for ETL)
Amazon S3 (storage)
Amazon Redshift (data warehouse)
AWS CloudWatch + SNS (monitoring & alerting)
Tableau Public (visualization)

Git Repo - https://github.com/Ved2310?tab=repositories