Offer summary

Qualifications:

Expertise in Python and PySpark, Experience with AWS EMR and Lambda, Familiarity with DynamoDB, Athena, OpenSearch, Background in API development, Agile environment experience with JIRA.

Key responsibilities:

Update and enhance existing PySpark scripts

Develop and optimize Lambda functions

Create new PySpark scripts for data processing

Support business insights through data analysis

Collaborate on Tableau dashboards and engage in Agile processes

Job description

Data Engineer

We are seeking an experienced Data Engineer to join the Digital Memory (DIMSUM) pod within OTA. The ideal candidate is proficient in Python, PySpark, and AWS tooling, specifically EMR, Lambda functions, DynamoDB, Athena, OpenSearch, and API development. Familiarity with Sagemaker and Elasticache is a plus.

Responsibilities:

Update and enhance existing PySpark scripts running on EMR on EKS.
Develop and optimize Lambda functions (Python).
Create new PySpark scripts for data processing.
Support the development of business insights through data analysis.
Collaborate on Tableau dashboards for data quality monitoring.
Contribute to iterative development for Customer Frustration Score and Predictive Intent models.
Engage in Agile processes, working closely with a 4-member team, using JIRA for project management.

Qualifications:

Expertise in Python, PySpark, AWS EMR, Lambda, DynamoDB, Athena, OpenSearch.
Experience with API development and optimization.
Familiarity with data-driven solutions, Sagemaker, and Elasticache.
Experience working in an Agile environment with team communication tools like JIRA.
Exposure to ML models and algorithms is a plus, though not mandatory.

This is an exciting opportunity to work on innovative digital memory initiatives in a collaborative and agile team environment!

Required profile