Job Summary
The Senior Data Engineer will design, build, and optimize scalable data solutions leveraging SQL, Snowflake, Databricks, and Azure Data Factory (ADF).
In addition to data engineering, this role includes responsibilities in Quality Engineering and Business Process Analysis to ensure accurate data delivery, optimized workflows, and adherence to enterprise governance.
The ideal candidate possesses a blend of technical expertise, analytical thinking, and cross-functional collaboration skills.
Key Responsibilities
Data Engineering
- Design and implement scalable and secure data architectures and data models.
- Build, maintain, and optimize ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks.
- Enhance data storage and retrieval efficiency across Snowflake and relational databases.
- Ensure adherence to data governance, security, and compliance standards.
- Collaborate with analytics, product, and business teams to support data integration and advanced reporting.
Quality Engineering
- Perform data validation, reconciliation, and regression testing for ETL/ELT pipelines.
- Develop automated QA frameworks for continuous testing of data processes.
- Enforce data quality standards and ensure compliance with enterprise and regulatory requirements.
Business Process Analysis
- Analyze and document existing business processes to identify optimization opportunities.
- Develop standard operating procedures (SOPs) for data workflows and operations.
- Work with stakeholders to implement automation initiatives and process efficiency improvements.
Required Qualifications
Technical Expertise
- Advanced proficiency in SQL, including complex queries and performance tuning.
- Hands-on experience with Snowflake, Databricks, and Azure Data Factory (ADF).
Additional Skills
- Strong understanding of data governance, data quality frameworks, and QA processes.
- Experience conducting business process analysis and documentation.
- Excellent analytical thinking and communication skills.
Preferred Qualifications
- Experience in the healthcare domain.
- Familiarity with cloud-native architectures and Agile methodologies.
- Knowledge of Python and data visualization tools (Power BI or Tableau).
Certifications
- Relevant certifications such as Microsoft DP-203 or Azure Databricks (preferred).
Primary Skills