Match score not available

Data Engineer

Remote: 
Full Remote
Contract: 
Work from: 

Offer summary

Qualifications:

5 to 8 years of experience in data engineering roles., Strong programming skills in Python, Scala, or SQL, and Pyspark., Experience with data pipeline architecture and implementation using Apache Spark., Familiarity with AWS EMR, Redshift, Azure Synapse, and data integration tools like Azure Data Factory..

Key responsabilities:

  • Architect and implement data pipelines for efficient data processing and transformation.
  • Develop and maintain automated data workflows using Apache Airflow.
  • Monitor and troubleshoot data pipelines to ensure data availability and integrity.
  • Implement data security best practices and ensure compliance with data privacy regulations.

Digi Digital Placement logo
Digi Digital Placement Scaleup https://digidigitalplacement.com/
51 - 200 Employees
See all jobs

Job description

JD:
• Role: Sr / Lead Data Engineer
• Experience: 5 to 8 years
• Job Location : Hyderabad, Work from Office
• Mode of employment: Permanent
• Notice period: Immediate to 15 days ONLY.
 Skills and Responsibilities.
• Strong programming skills in Python, Scala, or SQL, as well as Pyspark.
• Architect and implement data pipelines using Apache Spark for efficient data processing and transformation.
• Develop and maintain data workflows using Apache Airflow to automate data pipeline orchestration and scheduling.
• Create and optimize SQL queries for data extraction, transformation, and loading (ETL) processes.
• Harness the power of Python for data manipulation, scripting, and automation.
• Manage and optimize data storage and processing on AWS EMR, Redshift, and Azure Synapse.
• Collaborate with teams to design and maintain Azure Data Factory (ADF) pipelines for data integration.
• Leverage DBT for data modelling and version control, ensuring data accuracy and consistency.
• Integrate data from diverse sources using Fivetran, ensuring seamless data ingestion.
• Continuously improve data quality, performance, and reliability across the data ecosystem.
• Monitor and troubleshoot data pipelines, ensuring data availability and integrity.
• Implement data security best practices and ensure compliance with data privacy regulations.

Required profile

Experience

Spoken language(s):
English
Check out the description to know which languages are mandatory.

Other Skills

  • Collaboration
  • Problem Solving

Data Engineer Related jobs