About the role
You Will:
- Data Architecture and Design: Designing and overseeing the architecture of scalable and reliable data platforms, including data pipelines, storage solutions, and processing systems
- Data Modelling and Management:Developing and implementing data models, ensuring data quality, and establishing data governance policies
- Data Pipeline Development: Building and optimising data pipelines for ingesting, processing, and transforming large datasets from various sources
- Performance Optimisation: Identifying and resolving performance bottlenecks in data pipelines and systems, ensuring efficient data retrieval and processing
- Technology Evaluation and Innovation: Staying abreast of emerging data technologies and exploring opportunities for innovation to improve the organisation’s data infrastructure
- Troubleshooting and Problem Solving: Diagnosing and resolving complex data-related issues, ensuring the stability and reliability of the data platform
- Data Security and Compliance: Implementing data security measures, ensuring compliance with data governance policies, and protecting sensitive data
- Perform other duties as assigned
You Have:
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- 10+ years of experience in data engineering or a similar role.
- Enterprise SaaS software solutions with high availability and scalability
- Solution handling large scale structured and unstructured data from varied data sources
- Experience in building and maintaining data platform systems such as distributed compute, data orchestration, distributed storage, streaming infrastructure ensuring scalability, reliability, efficiency and security
- Working with Product engineering team to influence designs with data, AI and analytics use cases in mind
- In depth experience in System design involving large Petabytes of data with Databricks Lakehouse
- Experience in modern AI/Data infrastructure patterns, Semantics layer Organizing data for AI agents (metadata, context)
- AWS, GCP, Snowflake and Data pipeline frameworks like Airbyte/Airflow
- Programming languages like Python, SQL, and potentially Java or Scala
- Modern software engineering practices like Kubernetes, CI/CD, IAC tools (Preferably Terraform), Observability, monitoring and alerting
- Solution Cost Optimisations and design to cost
- Driving engineering excellence initiative
- Legally eligible to work in India on an ongoing basis
- Fluency in English is required