Full Stack Azure Data Engineer-11910-Remote at Shuvel Digital

Purpose

In Digital, we develop innovative member and team member experiences that leverage the latest technologies in open source and the Cloud. Digital Information Management (DIM) is a team of engineers committed to championing a data- driven decision-making culture and meets the business demand for timely insight-focused analytics and information delivery.

You will be working with all levels of technology from backend data processing technologies (Databricks/Apache Spark) to other Cloud computing technologies / Azure Data Platform. You should be a strong analytical thinker, detail-oriented and love working with data with a strong background in data engineering and application development. Must be a hand-on technologist passionate about learning new technologies and help improve the ways we can better leverage Advanced Analytics and Machine Learning.

Responsibilities

• Build end-to-end direct capabilities.
• Create and maintain optimal data pipeline architecture.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
• Use analytics for capitalizing on the data for making decisions and achieving better outcomes for the business.
• Derive insights to differentiate member and team member experiences.
• Collaborate with cross-functional teams.
• Analyze and define with product teams the data migration and data integration strategies.
• Apply experience in analytics, data visualization and modeling to find solutions for a variety of business and technical problems.
• Querying and analyzing small and large data sets to discover patterns and deliver meaningful insights.
• Integrate source systems with information management solutions and target systems for automated migration processes.
• Create proof-of-concepts to demonstrate viability of solutions under consideration.

Qualifications

• Strong hands-on experience leading design thinking as well as the ability to translate ideas to clearly articulate technical solutions.
• Bachelor’s degree in computer science, information systems, or other technology-related field or equivalent number of years of experience.
• Advanced hands-on experience implementing and supporting large scale data processing pipelines and migrations using technologies (eg. Azure Services, Python programming)
• Significant hands-on experience guiding technical teams as a mentor while leading collaboration with multiple teams across the organization.
• Significant hands-on experience with Azure services such as Azure Data Factory, Azure Databricks, Azure Data Lake Storage (ADLS Gen2), Azure SQL, and other data sources.
• Significant hands-on experience designing and implementing reusable frameworks using Apache Spark (PySpark preferred or Java/Scala)
• Solid foundation in data structures, algorithms, design patterns and strong analytical and problem-solving skills.
• Experience with any of the following Analytics and Information Management competencies: Data Management and Architecture, Performance Management, Information Delivery and Advanced Analytics Posting Description – Digital Data Engineer

Desired

Proficiency in collaborative coding practices, such as pair programming, and ability to thrive in a team-oriented environment

The following certifications:

• Microsoft Certified Azure Data Engineer
• Microsoft Certified Azure Solutions Architect
• Databricks Certified Associate Developer for Apache 2.4/3.0