Lead Data Engineer - Graph

IDFC is seeking a highly motivated and experienced Data Engineer with a strong focus on graph databases to join the growing data team. In this role, you will be responsible for designing, building, and maintaining robust and scalable graph-based data solutions. You will work closely with data scientists, analysts, and application developers to leverage the power of graph databases for complex data analysis, knowledge representation, and real-time applications.

Responsibilities:

• Design and implement efficient and scalable graph data models using appropriate graph database technologies (e. g., Neo4j, Amazon Neptune, TigerGraph, etc. ).
• Translate business requirements into effective graph schemas and data structures.
• Optimize graph models for performance and query efficiency.
• Develop and maintain graph database instances, including installation, configuration, and performance tuning.
• Implement data ingestion and transformation pipelines to populate graph databases from various data sources.
• Develop and optimize Cypher, Gremlin, or other graph query languages for complex data retrieval and analysis.
• Implement data security and access control mechanisms for graph databases.
• Monitor and maintain the health of graph databases.
• Design and implement ETL/ELT processes to integrate data from diverse sources into graph databases.
• Develop and maintain data pipelines using tools like Apache Kafka, Apache Airflow, or similar technologies.
• Ensure data quality and consistency throughout the data integration process.
• Collaborate with data scientists and analysts to develop graph-based analytics solutions.
• Develop and implement APIs and applications that leverage graph database capabilities.
• Develop and implement algorithms for graph traversals, community detection, and other graph analytics tasks.

Secondary Responsibilities:

• Identify and resolve performance bottlenecks in graph database systems.
• Monitor and optimize query performance.
• Troubleshoot and resolve data-related issues.
• Document graph data models, data pipelines, and database configurations.
• Establish and enforce best practices for graph database development and management.
• Participate in code reviews.

Requirements:

• 5 to 10 years of experience in data engineering and proficiency in Python, PySpark, and Unix.
• Experience with GraphDB, Amazon Neptune, TigreGraph, Dgraph, etc., and graph query languages like Cypher, Gremlin, etc.
• Experience with Apache Airflow.
• Excellent communication and problem-solving abilities.
• Ability to work independently and manage tasks efficiently.
• Bachelor's or master's degree in a relevant field (B. Sc., B. Tech., BCA, M. Sc., M. Tech., MCA).