Databricks Data Engineer l Contract

Job Responsibilities

• Design, develop, and maintain scalable data solutions to improve data accessibility, usability, and reliability across the organization.
• Build, optimize, and support end-to-end data pipelines and ETL/ELT processes for data ingestion, transformation, and integration.
• Collaborate with business stakeholders and cross-functional teams to understand data requirements and deliver effective data-driven solutions.
• Query, cleanse, transform, and validate large datasets using SQL, Python, Databricks, and PySpark.
• Monitor, troubleshoot, and optimize data pipelines to ensure high performance, scalability, and operational reliability.
• Conduct regular data quality assessments and implement controls to ensure data accuracy, consistency, and integrity.
• Ensure compliance with data governance standards and relevant government requirements, including IM8 standards where applicable.
• Develop, enhance, and maintain dashboards and reports for operational monitoring, performance tracking, and user behavior analytics.
• Translate business requirements into meaningful visualizations and actionable insights using tools such as Power BI and Tableau.
• Create and maintain comprehensive technical documentation, including data workflows, process documentation, dashboard user guides, and data dictionaries.
• Maintain and update knowledge repositories (e.g., Confluence) to support knowledge sharing and operational continuity.
• Stay current with emerging technologies, industry trends, and best practices in data engineering and analytics, driving continuous process improvements and innovation.

Job Qualification

• Bachelor's Degree in Computer Science, Information Systems, Data Engineering, Data Science, or a related discipline.
• Minimum 5 years of hands-on experience in Data Engineering, including data pipeline development, ETL/ELT processes, and data integration projects.
• Proficiency in SQL and Python for data processing, transformation, automation, and analytics.
• Hands-on experience with Databricks, PySpark, or similar big data and analytics platforms.
• Experience designing, developing, and maintaining scalable and reliable data pipelines.
• Proficiency in dashboard development and data visualization tools such as Power BI, Tableau, or equivalent platforms.
• Good understanding of data quality management, data governance, and data validation practices.
• Experience producing high-quality technical documentation, user guides, and process documentation.
• Familiarity with system testing methodologies, including SIT, UAT, and QA testing.
• Knowledge of Apache Spark and distributed data processing frameworks.
• Working knowledge of Git or other version control systems.
• Demonstrated ability to work independently while collaborating effectively within cross-functional teams.
• Highly organized, detail-oriented, and committed to delivering high-quality solutions in a fast-paced environment.