About the role
Job Summary:
We are seeking an experienced Pyspark Developer to join our team. As a Pyspark Developer, you will be responsible for designing, developing, and implementing data processing and analytics solutions using Pyspark, Big Data, and AWS technologies.
Responsibilities:
Develop and maintain data processing and analytics solutions using Pyspark, Spark, and Big Data technologies
Design and implement data pipelines using AWS EMR, S3, IAM, Lambda, SNS, and SQS
Collaborate with cross-functional teams to understand business requirements and deliver customized solutions
Optimize and troubleshoot data processing and analytics solutions
Develop and maintain technical documentation for data solutions
Requirements:
Good work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQL
Experience working on Data Engineering projects
Good understanding of SQL
Good understanding of Unix and HDFS commands
Experience working on Data Analytics and Pyspark