NextHire Try for free

Zorba Consulting India

Posted 9 Aug 2026

DE-Spark,Scala,kafka

Apply with NextHire Auto Apply fills in the application for you.
Employment
Full time
Team
Information Technology (IT)
Posted
9 Aug 2026

About the role

5–10 years Data Engineering

  • Strong Scala programming

  • Strong Apache Spark

  • Strong Apache Kafka

  • Spark Structured Streaming

  • Advanced SQL

  • ETL/ELT & Data Pipelines

  • Distributed Systems

  • Cloud/Databricks exposure

  • Required Technical Skills1. Apache Spark – Mandatory

    Strong hands-on experience with:

    • Apache Spark
    • Spark Core
    • Spark SQL
    • Spark DataFrame
    • Spark Dataset
    • Spark Structured Streaming
    • Spark transformations and actions
    • RDD concepts
    • Joins and aggregations
    • Window functions
    • Partitioning and repartitioning
    • Caching and persistence
    • Broadcast joins
    • Handling data skew
    • Spark job optimization
    • Spark cluster execution and troubleshooting
    2. Scala – Mandatory
    • Strong programming experience in Scala.
    • Functional programming concepts.
    • Collections and higher-order functions.
    • Case classes, traits, objects, pattern matching.
    • Exception handling and reusable code development.
    • Development of Spark applications using Scala.
    • Ability to write clean, modular, scalable, and maintainable Scala code.
    3. Apache Kafka – Mandatory

    Strong practical experience with:

    • Kafka architecture
    • Kafka brokers
    • Topics and partitions
    • Producers and consumers
    • Consumer groups
    • Offsets and offset management
    • Replication factor
    • Partition strategy
    • Message retention
    • Kafka Producer/Consumer APIs
    • Kafka Streams / Kafka Connect
    • Schema Registry
    • Avro / JSON serialization
    • Consumer lag monitoring
    • Kafka troubleshooting
    • Kafka performance tuning
    • Integration of Kafka with Spark Structured Streaming
    4. SQL – Mandatory

    Strong SQL skills including:

    • Complex SQL queries
    • Joins
    • Subqueries
    • CTEs
    • Window functions
    • Aggregations
    • Query optimization
    • Data validation and reconciliation
    • Stored procedures/functions where applicable
    • Relational database concepts
    5. Data Engineering

    Strong understanding of:

    • ETL / ELT
    • Batch processing
    • Real-time/streaming processing
    • Data pipelines
    • Data lakes
    • Data warehouses
    • Data modeling
    • Dimensional modeling
    • Distributed systems
    • Data partitioning
    • Data quality
    • Data governance
    • Large-scale data processing
    Cloud / Big Data Experience

    Experience with at least one cloud platform is preferred:

  • Required Technical Skills1. Apache Spark – Mandatory

    Strong hands-on experience with:

    • Apache Spark
    • Spark Core
    • Spark SQL
    • Spark DataFrame
    • Spark Dataset
    • Spark Structured Streaming
    • Spark transformations and actions
    • RDD concepts
    • Joins and aggregations
    • Window functions
    • Partitioning and repartitioning
    • Caching and persistence
    • Broadcast joins
    • Handling data skew
    • Spark job optimization
    • Spark cluster execution and troubleshooting
    2. Scala – Mandatory
    • Strong programming experience in Scala.
    • Functional programming concepts.
    • Collections and higher-order functions.
    • Case classes, traits, objects, pattern matching.
    • Exception handling and reusable code development.
    • Development of Spark applications using Scala.
    • Ability to write clean, modular, scalable, and maintainable Scala code.
    3. Apache Kafka – Mandatory

    Strong practical experience with:

    • Kafka architecture
    • Kafka brokers
    • Topics and partitions
    • Producers and consumers
    • Consumer groups
    • Offsets and offset management
    • Replication factor
    • Partition strategy
    • Message retention
    • Kafka Producer/Consumer APIs
    • Kafka Streams / Kafka Connect
    • Schema Registry
    • Avro / JSON serialization
    • Consumer lag monitoring
    • Kafka troubleshooting
    • Kafka performance tuning
    • Integration of Kafka with Spark Structured Streaming
    4. SQL – Mandatory

    Strong SQL skills including:

    • Complex SQL queries
    • Joins
    • Subqueries
    • CTEs
    • Window functions
    • Aggregations
    • Query optimization
    • Data validation and reconciliation
    • Stored procedures/functions where applicable
    • Relational database concepts
    5. Data Engineering

    Strong understanding of:

    • ETL / ELT
    • Batch processing
    • Real-time/streaming processing
    • Data pipelines
    • Data lakes
    • Data warehouses
    • Data modeling
    • Dimensional modeling
    • Distributed systems
    • Data partitioning
    • Data quality
    • Data governance
    • Large-scale data processing
    Cloud / Big Data Experience

    Experience with at least one cloud platform is preferred:

  • Required Technical Skills1. Apache Spark – Mandatory

    Strong hands-on experience with:

    • Apache Spark
    • Spark Core
    • Spark SQL
    • Spark DataFrame
    • Spark Dataset
    • Spark Structured Streaming
    • Spark transformations and actions
    • RDD concepts
    • Joins and aggregations
    • Window functions
    • Partitioning and repartitioning
    • Caching and persistence
    • Broadcast joins
    • Handling data skew
    • Spark job optimization
    • Spark cluster execution and troubleshooting
    2. Scala – Mandatory
    • Strong programming experience in Scala.
    • Functional programming concepts.
    • Collections and higher-order functions.
    • Case classes, traits, objects, pattern matching.
    • Exception handling and reusable code development.
    • Development of Spark applications using Scala.
    • Ability to write clean, modular, scalable, and maintainable Scala code.
    3. Apache Kafka – Mandatory

    Strong practical experience with:

    • Kafka architecture
    • Kafka brokers
    • Topics and partitions
    • Producers and consumers
    • Consumer groups
    • Offsets and offset management
    • Replication factor
    • Partition strategy
    • Message retention
    • Kafka Producer/Consumer APIs
    • Kafka Streams / Kafka Connect
    • Schema Registry
    • Avro / JSON serialization
    • Consumer lag monitoring
    • Kafka troubleshooting
    • Kafka performance tuning
    • Integration of Kafka with Spark Structured Streaming
    4. SQL – Mandatory

    Strong SQL skills including:

    • Complex SQL queries
    • Joins
    • Subqueries
    • CTEs
    • Window functions
    • Aggregations
    • Query optimization
    • Data validation and reconciliation
    • Stored procedures/functions where applicable
    • Relational database concepts
    5. Data Engineering

    Strong understanding of:

    • ETL / ELT
    • Batch processing
    • Real-time/streaming processing
    • Data pipelines
    • Data lakes
    • Data warehouses
    • Data modeling
    • Dimensional modeling
    • Distributed systems
    • Data partitioning
    • Data quality
    • Data governance
    • Large-scale data processing
    Cloud / Big Data Experience

  • Required Technical Skills1. Apache Spark – Mandatory

    Strong hands-on experience with:

    • Apache Spark
    • Spark Core
    • Spark SQL
    • Spark DataFrame
    • Spark Dataset
    • Spark Structured Streaming
    • Spark transformations and actions
    • RDD concepts
    • Joins and aggregations
    • Window functions
    • Partitioning and repartitioning
    • Caching and persistence
    • Broadcast joins
    • Handling data skew
    • Spark job optimization
    • Spark cluster execution and troubleshooting

    2. Scala – Mandatory

    • Strong programming experience in Scala.
    • Functional programming concepts.
    • Collections and higher-order functions.
    • Case classes, traits, objects, pattern matching.
    • Exception handling and reusable code development.
    • Development of Spark applications using Scala.
    • Ability to write clean, modular, scalable, and maintainable Scala code.

    3. Apache Kafka – Mandatory

    Strong practical experience with:

    • Kafka architecture
    • Kafka brokers
    • Topics and partitions
    • Producers and consumers
    • Consumer groups
    • Offsets and offset management
    • Replication factor
    • Partition strategy
    • Message retention
    • Kafka Producer/Consumer APIs
    • Kafka Streams / Kafka Connect
    • Schema Registry
    • Avro / JSON serialization
    • Consumer lag monitoring
    • Kafka troubleshooting
    • Kafka performance tuning
    • Integration of Kafka with Spark Structured Streaming

    4. SQL – Mandatory

    Strong SQL skills including:

    • Complex SQL queries
    • Joins
    • Subqueries
    • CTEs
    • Window functions
    • Aggregations
    • Query optimization
    • Data validation and reconciliation
    • Stored procedures/functions where applicable
    • Relational database concepts

    5. Data Engineering

    Strong understanding of:

    • ETL / ELT
    • Batch processing
    • Real-time/streaming processing
    • Data pipelines
    • Data lakes
    • Data warehouses
    • Data modeling
    • Dimensional modeling
    • Distributed systems
    • Data partitioning
    • Data quality
    • Data governance
    • Large-scale data processing

    Cloud / Big Data Experience

    Experience with at least one cloud platform is preferred:

    Experience with at least one cloud platform is preferred:

  • Required Technical Skills1. Apache Spark – Mandatory

    Strong hands-on experience with:

    • Apache Spark
    • Spark Core
    • Spark SQL
    • Spark DataFrame
    • Spark Dataset
    • Spark Structured Streaming
    • Spark transformations and actions
    • RDD concepts
    • Joins and aggregations
    • Window functions
    • Partitioning and repartitioning
    • Caching and persistence
    • Broadcast joins
    • Handling data skew
    • Spark job optimization
    • Spark cluster execution and troubleshooting
    2. Scala – Mandatory
    • Strong programming experience in Scala.
    • Functional programming concepts.
    • Collections and higher-order functions.
    • Case classes, traits, objects, pattern matching.
    • Exception handling and reusable code development.
    • Development of Spark applications using Scala.
    • Ability to write clean, modular, scalable, and maintainable Scala code.
    3. Apache Kafka – Mandatory

    Strong practical experience with:

    • Kafka architecture
    • Kafka brokers
    • Topics and partitions
    • Producers and consumers
    • Consumer groups
    • Offsets and offset management
    • Replication factor
    • Partition strategy
    • Message retention
    • Kafka Producer/Consumer APIs
    • Kafka Streams / Kafka Connect
    • Schema Registry
    • Avro / JSON serialization
    • Consumer lag monitoring
    • Kafka troubleshooting
    • Kafka performance tuning
    • Integration of Kafka with Spark Structured Streaming
    4. SQL – Mandatory

    Strong SQL skills including:

    • Complex SQL queries
    • Joins
    • Subqueries
    • CTEs
    • Window functions
    • Aggregations
    • Query optimization
    • Data validation and reconciliation
    • Stored procedures/functions where applicable
    • Relational database concepts
    5. Data Engineering

    Strong understanding of:

    • ETL / ELT
    • Batch processing
    • Real-time/streaming processing
    • Data pipelines
    • Data lakes
    • Data warehouses
    • Data modeling
    • Dimensional modeling
    • Distributed systems
    • Data partitioning
    • Data quality
    • Data governance
    • Large-scale data processing
    Cloud / Big Data Experience

    Experience with at least one cloud platform is preferred:

  • Required Technical Skills1. Apache Spark – Mandatory

    Strong hands-on experience with:

    • Apache Spark
    • Spark Core
    • Spark SQL
    • Spark DataFrame
    • Spark Dataset
    • Spark Structured Streaming
    • Spark transformations and actions
    • RDD concepts
    • Joins and aggregations
    • Window functions
    • Partitioning and repartitioning
    • Caching and persistence
    • Broadcast joins
    • Handling data skew
    • Spark job optimization
    • Spark cluster execution and troubleshooting
    2. Scala – Mandatory
    • Strong programming experience in Scala.
    • Functional programming concepts.
    • Collections and higher-order functions.
    • Case classes, traits, objects, pattern matching.
    • Exception handling and reusable code development.
    • Development of Spark applications using Scala.
    • Ability to write clean, modular, scalable, and maintainable Scala code.
    3. Apache Kafka – Mandatory

    Strong practical experience with:

    • Kafka architecture
    • Kafka brokers
    • Topics and partitions
    • Producers and consumers
    • Consumer groups
    • Offsets and offset management
    • Replication factor
    • Partition strategy
    • Message retention
    • Kafka Producer/Consumer APIs
    • Kafka Streams / Kafka Connect
    • Schema Registry
    • Avro / JSON serialization
    • Consumer lag monitoring
    • Kafka troubleshooting
    • Kafka performance tuning
    • Integration of Kafka with Spark Structured Streaming
    4. SQL – Mandatory

    Strong SQL skills including:

    • Complex SQL queries
    • Joins
    • Subqueries
    • CTEs
    • Window functions
    • Aggregations
    • Query optimization
    • Data validation and reconciliation
    • Stored procedures/functions where applicable
    • Relational database concepts
    5. Data Engineering

    Strong understanding of:

    • ETL / ELT
    • Batch processing
    • Real-time/streaming processing
    • Data pipelines
    • Data lakes
    • Data warehouses
    • Data modeling
    • Dimensional modeling
    • Distributed systems
    • Data partitioning
    • Data quality
    • Data governance
    • Large-scale data processing
    Cloud / Big Data Experience

    Experience with at least one cloud platform is preferred:

85% of jobs are never posted

The AI Outreach Agent writes to recruiters and hiring managers at the companies you pick, from your own Gmail.

Try AI Outreach