Apache Spark Specialist

Location Singapore
Discipline
Job Reference AppSpark-IBM_1564987694
Salary Negotiable
Consultant Email james.andalan@manpower.com.sg
EA License No. 02C3423

Job Description

The Client's Analytics team is looking for a strong technologist to help implement Spark/Scala jobs. The developer must not only know the Spark API well: They must also know about the pitfalls of distributed storage, how to structure a data processing pipeline that has to handle the 5V of Big Data-volume, velocity, variety, veracity, and value-and how to turn that into maintainable code.

Responsibilities:

  • Help write analytics code, services and components in Java, Apache Spark, Kafka, Storm, Redis and related technologies such as Hadoop, Zookeeper
  • Responsible for systems analysis - Design, Coding, Unit Testing and other SDLC activities
  • Requirement gathering and understanding, Analyse and convert functional requirements into concrete technical tasks and able to provide reasonable effort estimates
  • Spark query tuning and performance optimisation
  • Work proactively, independently and with global teams to address project requirements, and articulate issues/challenges with enough lead time to address project delivery risks
  • Providing expertise in technical analysis and solving technical issues during project delivery
  • Code reviews, test case reviews and ensure code developed meets the requirements

Qualifications:

  • Experience with Java essential. Python, and R highly desired.
  • Experience with Apache Spark streaming and batch framework
  • Experience with Kafka, Storm, Zookeeper
  • Knowledge of Redis, Hadoop
  • Agile/Scrum methodology experience is required.
  • Service oriented architecture, and data standards like JSON, Avro, Protobuf
  • Strong communication skills and should be able to communicate effectively with business and other stake holders
  • Demonstrate ownership and initiative taking

Preferred Qualifications:

  • 5+ years of experience in application development with Java/J2EE, Spring, Spring Boot, Web Services, Scala, AWS
  • 2+ years of experience with Docker containerization
  • 3+ year of experience working with Big Data Technologies (Apache Storm, Apache Spark and Kafka)
  • 4+ years of experience in Unix, No SQL

Rvin James Murillo Andalan EA License No. 02C3423 Personnel Registration No. R1331697

Please note that your response to this advertisement and communications with us pursuant to this advertisement will constitute informed consent to the collection, use and/or disclosure of personal data by ManpowerGroup Singapore for the purpose of carrying out its business, in compliance with the relevant provisions of the Personal Data Protection Act 2012. To learn more about ManpowerGroup's Global Privacy Policy, please click here.