About the role
Experience: 4 - 7 Years
Location: Bangalore
Mandatory Skills: Python, Scala, Spark, Hadoop, Hive, HDFS, Mapreduce, Yarn and Hadoop Architect.
Job Summary:
We are seeking a highly skilled Big Data Engineer with strong expertise in Spark & Scala, along with solid experience in Java 8, Python (PySpark), and Hadoop architecture. The ideal candidate should possess advanced problem-solving abilities and a strong foundation in building scalable data processing systems.
Key Responsibilities
● Design, develop, and optimise large-scale data processing pipelines using Spark and Scala.
● Work with distributed systems and components of the Hadoop ecosystem to
handle big data workloads.
● Write efficient, reusable, and scalable code in Java 8 and Python (PySpark).
● Apply advanced logic-building skills to solve complex data engineering problems.
● Collaborate with cross-functional teams, including data scientists, analysts, and product teams.
● Ensure data quality, performance tuning, and reliability of data pipelines.
● Participate in code reviews, testing, and deployment processes.
Required Skills
● Strong hands-on experience with Apache Spark (Scala & PySpark)
● Proficiency in Scala programming
● Experience with Java 8 ( good to have )
● Strong advanced logic-building and problem-solving skills
● Expertise in Python (PySpark)
● In-depth understanding of Hadoop Architecture and ecosystem (HDFS, YARN, MapReduce