About the role
Lead Software Engineer- Pyspark,AWS
Location- Noida / Indore / Pune / Gurugram
Job Description
· 8-10 years of good hands on exposure with Big Data technologies – pySpark (Data frame and SparkSQL), Hadoop, and Hive.
· Good hands on experience of python, Bash Scripts.
· Good understanding of SQL and data warehouse concepts
· Strong analytical, problem-solving, data analysis and research skills
· Demonstrable ability to think outside of the box and not be dependent on readily available tools
· Excellent communication, presentation and interpersonal skills are a must
Cloud Skills:
· Hands-on experience with using Cloud Platform provided Big Data technologies (i.e. IAM, Glue, EMR, RedShift, S3, Kinesis)
· Orchestration with Airflow and Any job scheduler experience
· Experience in migrating workload from on-premise to cloud and cloud to cloud migrations
Roles & Responsibilities
· Develop efficient ETL pipelines as per business requirements, following the development standards and best practices.
· Perform integration testing of different created pipeline in AWS env.
· Provide estimates for development, testing & deployments on different env.
· Participate in code peer reviews to ensure our applications comply with best practices.
· Create cost effective AWS pipeline with required AWS services i.e S3,IAM, Glue, EMR, Redshift etc.
Mandatory Skills-Pyspark, AWS
Years Of Experience-8 to 10 Years
Education/Qualification
BE / B.Tech / MCA / M.Tech