Data Software Engineer

Data Software Engineer

Data Software Engineer - Spark, Python, (Azure Databricks )

Job Description:

1. 5-12 Years of in Big Data & Data related technology experience

2. Expert level understanding of distributed computing principles

3. Expert level knowledge and experience in Apache Spark  

4. Hands on programming with Python

5. Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop

6. Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming 

7. Experience with messaging systems, such as Kafka or RabbitMQ

8. Good understanding of Big Data querying tools, such as Hive, and Impala

9. Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files

10. Good understanding of SQL queries, joins, stored procedures, relational schemas

11. Experience with NoSQL databases, such as HBase, Cassandra, MongoDB

12. Knowledge of ETL techniques and frameworks

13. Performance tuning of Spark Jobs

14. Experience with native Cloud data services AZURE Databricks

15. Ability to lead a team efficiently