About the role
Responsibilities
- Analyze, organize and process raw data using Bigdata technologies and Scala
- Perform data validation, cleaning and transformation using Big data technologies, Spark and Scala
- Ingest and manage data in HDFS/Hive and analyze data using Spark and produce meaningful insights for the business team
- Combine raw information from different sources, Explore ways to enhance data quality and reliability
- Develop code using Software Craftmanship best practices such as Continuous delivery and Clean coding principles
- Help business operations team with data samples to create proof of concepts
- Ensure the new data sets are feeding the Business Intelligence setup i.e Dashboards on PowerBI
- Liaise with BA to ensure subjects are getting prioritized, Communicate to PO about progress and potential issues