Data Engineer
Job Reference: 161721
Industry: Information and Communications Technology
brand-id: R1656741
Brand Name: 02C3423
We are seeking a highly skilled and detail-oriented Data & Analytics Team Lead to design, develop, and lead enterprise data engineering, analytics, and AI-powered solutions. In this role, you will be responsible for building scalable data platforms, integrating enterprise systems, developing reporting solutions, and leveraging Generative AI to deliver intelligent business insights. The ideal candidate will possess good technical expertise in data engineering, cloud technologies, analytics, and AI, along with excellent leadership and stakeholder management skills.
Responsibilities
- Design, develop, and maintain scalable data pipelines, ETL/ELT workflows, and cloud-based data platforms.
- Collect, analyse, and interpret data from various sources to identify trends, patterns, and actionable business insights.
- Develop and maintain Tableau dashboards, SAP BusinessObjects reports, and other analytical solutions.
- Design and implement data warehousing solutions using Snowflake and modern data lakehouse architectures.
- Develop and integrate REST APIs and enterprise data services to support business applications and data exchange.
- Build and optimize data ingestion, transformation, and orchestration workflows using Python and Apache Airflow.
- Design and develop Generative AI solutions, including AI agents, Retrieval-Augmented Generation (RAG) workflows, and enterprise knowledge retrieval systems.
- Build, deploy, and maintain Model Context Protocol (MCP) servers to securely integrate AI applications with enterprise platforms and data sources.
- Collaborate with cross-functional teams to gather business requirements, ensure data integrity, and deliver scalable data and AI solutions.
- Identify opportunities for process automation, operational efficiency, and data quality improvements.
- Monitor data quality, perform data validation, and resolve data discrepancies.
- Lead a team of developers, provide technical guidance, conduct code reviews, and ensure timely project delivery.
Requirements
- Proven experience in Data Engineering, Analytics, Reporting, and Business Intelligence.
- Good hands-on experience with Python, SQL, Snowflake, and cloud-based data platforms.
- Experience in building scalable ETL/ELT pipelines and data orchestration using Apache Airflow or similar tools.
- Experience with REST API development and enterprise system integration.
- Hands-on experience in designing and developing Generative AI applications, AI agents, RAG workflows, and LLM integrations.
- Experience building and deploying Model Context Protocol (MCP) servers and integrating AI applications with enterprise systems.
- Experience managing and mentoring development teams while ensuring high-quality and timely delivery.
- Good analytical and problem-solving skills with the ability to work on complex data platforms.
- Excellent communication skills with the ability to engage both technical and business stakeholders.
- Familiarity with data governance, data modeling, and modern data warehousing concepts.
- Ability to work independently and collaboratively in a fast-paced environment.
- Good organizational and time management skills with the ability to manage multiple concurrent projects.
Nice to Have
- Experience with SAP BusinessObjects reporting.
- Experience with PySpark, Databricks, Kafka, or streaming data platforms.
- Experience with AWS services such as S3, Lambda, Glue, API Gateway, or MSK.
- Knowledge of vector databases, prompt engineering, AI orchestration frameworks, and enterprise AI security.
- Experience with CI/CD pipelines, GitHub Actions, Docker, Kubernetes, and Infrastructure as Code.
