Offer summary

Qualifications:

8+ years of experience in Data Engineering with at least 5 years of hands-on experience in Pyspark., Strong knowledge of Azure Data Bricks, Azure Data Lake, and related Azure services., Proficiency in writing complex SQL queries and understanding DWH concepts., Experience with Python or Scala and familiarity with DevOps and Agile methodologies..

Key responsibilities:

Develop and implement data transformations using Pyspark and Databricks.

Coordinate with business stakeholders to gather and understand business requirements.

Utilize Azure Data Factory and other Azure services for data management and processing.

Write and optimize complex SQL queries and manage data storage solutions.

Job description

Job Description :

Data Engineer with Pyspark, Databricks skillset

8+ years of experience Data Engineering with min 5+ years hands on experience in Pyspark for data transformations.
Sound Knowledge on - Azure Data Bricks, Azure Data Lake. Delta Lake, Azure SQL and Azure blob storage, Azure logic apps, Azure Functions and Azure Synapse, Azure Purview.
Extensive knowledge on big data concepts like Hive, and spark framework.
Should be able to write complex SQL queries.
Sound understanding of DWH concepts.
Strong hands-on experience on Python or Scala.
Hands on experience on SSMS tool related concepts such as Index, store procedure and triggers.
Hands on experience on Azure Data Factory.
Should be able to coordinate independently with business stake holders and understand the business requirements
Knowledge on DevOps and Agile methodologies-based projects.
Knowledge of version control tool such as Git/Bitbucket.
Should have basic understanding on Batch Account configuration, various control and monitoring options