Module 4: Data Pipelines

🎯 Learning Objectives

• Design and implement data ingestion pipelines with Apache Airflow
• Build feature engineering systems at scale with feature stores
• Implement data quality validation with Great Expectations

📚 Module Content

Chapter 1: Data Ingestion Pipelines

ETL for ML, streaming vs batch processing, Apache Airflow basics.

Chapter 2: Feature Engineering at Scale

Feature stores (Feast, Tecton), feature computation, online vs offline features.

Chapter 3: Data Quality and Validation

Great Expectations, data contracts, drift detection, schema validation.

🚀 Let's Begin!

Start with Chapter 1: Data Ingestion Pipelines.