About the role
Responsibilities
- Analyze, organize and process raw data using Bigdata technologies and Scala
- Perform data validation, cleaning and transformation using Big data technologies, Spark and Scala
- Ingest and manage data in HDFS/Hive and analyze data using Spark and produce meaningful insights for the business team
- Combine raw information from different sources, Explore ways to enhance data quality and reliability
- Develop code using Software Craftmanship best practices such as Continuous delivery and Clean coding principles
- Help business operations team with data samples to create proof of concepts
- Ensure the new data sets are feeding the Business Intelligence setup i.e Dashboards on PowerBI
- Liaise with BA to ensure subjects are getting prioritized, Communicate to PO about progress and potential issues
- 5 to 9 years of expertise and hands on experience in Spark with Scala and Big data technologies
- Good working experience in Scala and object oriented concepts
- Good working experience in HDFS, Spark, Hive and Oozie
- Technical expertise with data models, data mining, and partitioning techniques
- Hands-on experience with SQL database
- Have good understand on CI/CD tools (Maven, Git, Jenkins) and SONAR
- Good to have knowledge on Kafka and ELK stack. Knowledge on Data visualization tool like PowerBI will be an added advantage.
- Strong communication and coordination skills with multiple stake holders. Assess existing situation, propose and follow up improvement’s actions
- Good team spirit, ability to work in international/intercultural environments (lots of interactions with onsite stake holders)
- Professional attitude: Self-motivated, fast learner, team player, independent