About the role
Required Skills & Expertise
- Hands-on experience with Databricks
- Strong proficiency in PySpark, Apache Spark, and SQL
- Experience building and managing ETL/ELT pipelines
- Good knowledge of cloud platforms such as Azure, AWS, or GCP
- Experience with Delta Lake, Databricks Workflows, and Unity Catalog
- Understanding of data modeling and data warehousing concepts
- Familiarity with Git and CI/CD practices
- Excellent communication and stakeholder management skills
- Design, develop, and maintain scalable data pipelines and integration solutions.
- Build and optimize ETL/ELT processes for large-scale data environments.
- Collaborate with business and technical stakeholders to deliver effective data solutions.
- Provide support and enhancements for existing data platforms based on evolving business needs.
- Improve platform performance, scalability, and reliability.
- Ensure compliance with data quality, governance, and security standards.
- Experience with Azure Data Factory (ADF) and Azure Synapse
- Exposure to Kafka or Spark Streaming
- Databricks Certification is an added advantage
- Experience working in Agile environments