Designed and implemented an enterprise-grade upsert framework in Azure Data Factory to support incremental data loads across multiple source systems.

The solution leveraged Copy Activity for high-performance staging into landing tables, followed by a parameterized Stored Procedure–driven merge strategy to efficiently handle inserts and updates in the target layer.

Key Highlights:

- Built scalable ingestion pipelines using ADF Copy Activity.

- Implemented controlled upsert logic via SQL Stored Procedures (MERGE pattern)

- Ensured data integrity with staging-layer validation

- Optimized performance for large datasets through indexing and batch processing

- Designed reusable pipeline templates to standardize data onboarding

- Integrated error handling and logging mechanisms for operational reliability

- This solution was implemented in a structured Dev/Test/Prod environment, supporting enterprise data governance and minimizing downstream reporting impact.