Job description

Skills:
Apache Spark, Databricks, AWS Glue, SQL, Python, ETL Pipelines, Kubernetes, Kafka,

We are looking for a Lead Data Engineer to drive the migration of our existing Azure data and Power BI artifacts to the new Microsoft Fabric environment. This role will involve designing and implementing a scalable data architecture, optimizing data pipelines, and ensuring seamless integration with Microsoft Fabrics ecosystem.

Key Responsibilities

Lead the design, development, and deployment of data pipelines in Microsoft Fabric.Assess the existing Azure data ecosystem (Azure Data Factory, Azure Synapse, Data Lake, etc.) and plan migration to Fabric.Develop and maintain scalable ETL/ELT processes for data ingestion, transformation, and processing in OneLake.Work closely with Microsoft Fabric Administrator, Power BI Developers, and Business Analysts to ensure smooth transition and performance optimization.Implement Data Governance, Security, and Compliance policies within the new environment.Optimize data storage, retrieval, and processing for performance and cost-efficiency in Fabric.

Provide technical leadership, mentoring, and best practices to data engineering teams.

Required Qualifications & Skills

8+ years of experience in Data Engineering with expertise in cloud-based data platforms.

Strong knowledge of Microsoft Fabric and its core services (OneLake, Lakehouse, Dataflows Gen2, Pipelines, etc.).

Expertise in Azure Data Services (Azure Synapse, Data Factory, Data Lake, Databricks).

Strong SQL and Python skills for data transformation and processing.

Experience with Power BI integration and migration of reports/datasets.

Knowledge of Data Governance best practices is a plus.

Hands-on experience in orchestrating large-scale data migration projects.

Excellent problem-solving and stakeholder management skills.

Certification

Microsoft Certified: Azure Data Engineer Associate or equivalent.

Required profile