Job description

This is a remote position.

Develop & Maintain Data Pipelines: Design and build high-performance ETL/ELT pipelines for AI/ML workloads.
AI Data Infrastructure: Architect and optimize data lakes, data warehouses, and vector databases (FAISS, Pinecone, Weaviate) to support AI-driven analytics.
Big Data Processing: Handle large-scale structured and unstructured data using Apache Spark, Kafka, AWS Glue.
AI Model Integration: Ensure efficient data ingestion and transformation for AI model training, including text-to-video, speech-to-text, and generative AI.
Cloud & Infrastructure: Deploy and manage data pipelines on AWS (Lambda, S3, Redshift, DynamoDB, Bedrock, SageMaker). Experience with GCP/Azure is a plus.
Data Governance & Security: Implement data quality checks, access control, and compliance using AWS IAM, AWS Lake Formation, and AWS Glue Data Catalog.
Automation & Performance Tuning: Optimize data workflows, implement CI/CD pipelines, and ensure high availability.
Collaboration: Work closely with AI/ML engineers, software developers, and DevOps teams to integrate AI-driven data solutions into production

Requirements

8+ years of experience in Data Engineering, with a strong focus on AI-driven data solutions.
Strong expertise in Python for data processing, automation, and AI/ML integration.
Hands-on experience with AWS services (Lambda, S3, Redshift, DynamoDB, API Gateway, SageMaker, Bedrock).
Experience with big data technologies like Apache Spark, Kafka, and AWS Glue.
Proficiency in ETL/ELT processes and data warehousing solutions (AWS Redshift, Snowflake).
Deep knowledge of vector databases (FAISS, Pinecone, Weaviate) for AI-powered search and retrieval.
Experience in real-time AI data processing and event-driven architectures (AWS Kinesis, SNS, SQS).
Strong database expertise in SQL (PostgreSQL, MySQL) and NoSQL (DynamoDB, MongoDB).
Experience implementing CI/CD pipelines for data workflows using Terraform, Docker, and GitHub Actions.
Ability to work in a UK time zone.

Preferred Qualifications:

Experience in AI-powered media processing (text-to-video, speech-to-text, generative AI for media).
Knowledge of MLOps, AI model deployment, and automated AI data pipelines.
Experience with real-time data streaming frameworks (Flink, Apache Beam).
Familiarity with AWS services for AI acceleration (Inferentia, Neuron SDK, SageMaker Pipelines).

Benefits

At eligarf Technologies, we transcend the boundaries of conventional IT services, crafting tailor-made solutions that catalyse your business ambitions. With a global footprint, we seamlessly blend innovation and expertise to architect a digital landscape that amplifies your growth trajectory.

Our core ethos revolves around sculpting bespoke services that resonate with your unique business DNA. We don’t offer mere solutions; we engineer strategies that become an intrinsic part of your success story. From Agile software development and meticulously curated software development to cutting-edge AI & ML and seamless cloud integration, our offerings are a symphony of technology and strategy.

We want enthusiastic people to be part of our family and have an open mind-frame and are willing to experiment with different options and multi-task in different aspects of business. Let us assure you that work will not be easy, but it will be worth it. We are an established company with a start-up mind-set and are looking for members to skip lunches over meetings, brainstorm on ideas on different aspects of business and roll up their sleeves and get their hands dirty. We want members who want to do something new and different and make a mark in whatever they do. We are not looking for time watching members. We are looking for a member who stays with us through the highs and lows of a dynamic and fast growing organization. You will have enough challenges to grow and enjoy your work. We have an interesting ABCDEF philosophy, which we believe will help you an individual. If you would like to work in this dynamic environment, know more about ABCDEF and contribute to your and the company’s growth.

If this job description is of your interest, please revert ASAP to move ahead.

For any further queries, feel free to contact.

Salary:

as per std

Required profile