adls-gen2

End-to-end Azure data engineering pipeline ingesting real-time earthquake data from the USGS API. Implements a Bronze–Silver–Gold lakehouse using Azure Data Factory, Databricks, ADLS Gen2, and Synapse Analytics, with both manual execution and fully automated daily-triggered workflows.

End-to-end Azure Data Engineering pipeline using ADF, Databricks (PySpark), ADLS Gen2, Azure SQL, and Power BI for COVID-19 analytics

Master Azure Data Engineering with this Basic to Advance guide! Covers SQL, PySpark, Kafka, Databricks, Snowflake & Airflow. Build 15+ industrial projects using Azure (ADF, Synapse, Event Hubs), GCP & modern table formats (Delta Lake, Iceberg, Hudi). Learn real-time streaming, Medallion architecture, and cloud data warehousing with hands-on labs.

End-to-end Metadata-Driven Data Engineering framework built on Azure. Features dynamic SQL/REST API ingestion with range pagination, automated schema mapping, and event-driven orchestration. Implements robust CI/CD via GitHub Actions/YAML and automated failure alerting with Logic Apps. Optimized for scalability and DE best practices.

End-to-end Azure Data Engineering project using ADF for incremental ingestion, Databricks (DLT) for Medallion Architecture, and Delta Lake for CDC (SCD Type 1). Managed via Databricks Asset Bundles (DABs) for professional CI/CD. Focuses on real-time streaming, scalability, and Star Schema modeling.

Designed a production-grade Azure Data Engineering project centered on Azure Data Factory. Built dynamic, metadata-driven pipelines to ingest data from on-prem systems, REST APIs, and Azure SQL into ADLS Gen2 using Medallion Architecture, incremental loading, and enterprise-scale orchestration patterns.

AZURE - Repositório de estudos

Kafka Connect Sink for ADLS Gen2 with SAS auth, Avro support, GZIP compression, and partition-based file batching.

End-to-end Azure Data Engineering project using Medallion Architecture, Databricks, Synapse, and Power BI.

ETL/ELT of Grocery Sales data by Azure Data Factory (ADF)

End-to-end Azure Data Engineering project using ADF, Databricks, PySpark, Delta Lake, SQL, and Power BI to build Bronze–Silver–Gold healthcare analytics pipelines from public Australian health expenditure data.

End-to-end Azure Data Lakehouse with Medallion architecture (Bronze/Silver/Gold), Delta Lake, Unity Catalog and Power BI dashboard. Final project of DMC Azure Data Engineer Specialization.

End-to-end Azure Data Engineering pipeline using Databricks, Delta Lake and ADLS Gen2. Medallion architecture (Bronze/Silver/Gold) with automated scheduling, monitoring and cost-optimized orchestration.

ShopVista Data Modernization Platform – An end-to-end Azure Lakehouse Architecture project using Azure Databricks, Delta Lake, Unity Catalog, Auto Loader, Structured Streaming, Databricks Workflows, and Power BI, following the Medallion Architecture.

GitOps-driven Azure Data Factory pipeline that ingests multi-source data (GitHub + ADLS) into ADLS Bronze using dynamic, parameterized ETL workflows.

Production-style Azure Data Factory platform featuring incremental ingestion, SCD Type 2, Azure SQL, Key Vault, monitoring, Bicep IaC, and Azure DevOps CI/CD.

End-to-end Azure data engineering pipeline processing 38M+ NYC taxi trips using ADF, Databricks, Delta Lake and PySpark

Production-style Azure analytics engineering project for adult social care using ADF, ADLS, Synapse and Power BI.

Production-oriented Azure data pipeline implementing Medallion Architecture with Event Hub, Functions, and Data Lake Gen2.

Enterprise HR analytics pipeline | 500K+ daily records · 3,000+ stores · 99.9% reliability | Azure Databricks · Delta Lake · PySpark