Big Data Engineer | Spark

Location Singapore
Discipline Information & Communications Technology
Job Reference BBBH115149_1655530456
Salary S$6500 - S$9000 per month
Consultant Name Cristina Malabuyoc Malijan
Consultant Email cristina.malijan@manpower.com.sg
Consultant Contact No. 62328835
EA License No. 02C3423
Consultant Registration No. R1111547


Job scopes

  • Enhance the efficiency of existing Data pipeline Job/framework and setup a best practice for Big Data projects planned on the BI & Analytics platform.
  • Review and tune the code for the Spark framework to increase the throughput.
  • Establish best practices and guidelines to be followed by tech teams while writing codes in Spark.
  • Create a framework to continuously monitor the performance and identify the resource intensive jobs.


Job requirements

  • Bachelor's Degree in Computer Science or equivalent
  • At least 7 years of working experience in big data
  • Able to write code, services and components in Java, Apache Spark and Hadoop.
  • Expert knowledge of technologies e.g. Hadoop, Hive, Presto, Spark, Java, RDBMS like Teradata and File storage like S3
  • Responsible for systems analysis Design, Coding, Unit testing, CICD and other SDLC activities.
  • Proven experience working with Apache Spark streaming and batch framework.
  • Proven experience in performance tuning Java, Spark based applications is a must.
  • Knowledge of working with different file formats like Parquet, JSON and AVRO.
  • Data Warehouse experience working with RDBMS and Hadoop
  • Well versed with concepts of change data capture and SCD implementation.
  • Hands-on experience in writing codes to efficiently handle files on Hadoop.
  • Ability to work in projects following Micro Services based architecture.
  • Knowledge of S3 will be a plus.
  • Ability to work proactively, independently and with global teams.
  • Excellent communication skills should be able to communicate effectively with the stakeholders and present the outcome of analysis.
  • Working experience in projects following Agile methodology


Cristina Malabuyoc Malijan EA License No. 02C3423 Personnel Registration No. R1111547

Please note that your response to this advertisement and communications with us pursuant to this advertisement will constitute informed consent to the collection, use and/or disclosure of personal data by ManpowerGroup Singapore for the purpose of carrying out its business, in compliance with the relevant provisions of the Personal Data Protection Act 2012. To learn more about ManpowerGroup's Global Privacy Policy, please visit https://www.manpower.com.sg/privacy-policy