About the role
We are looking for a Senior Data Engineer with strong expertise in Databricks, PySpark, and SQL to join our growing team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines, supporting serverless migration initiatives, and managing CI/CD processes.
Responsibilities:
- Design, develop, and maintain data pipelines using Databricks and PySpark.
- Support migration and optimization of workloads in serverless environments.
- Build and manage CI/CD pipelines for data engineering workflows.
- Monitor, troubleshoot, and optimize data processing systems.
- Review code, manage repositories, and ensure best engineering practices.
- Collaborate with stakeholders to understand business requirements and deliver scalable solutions.
Requirements:
- 8+ years of experience in Data Engineering.
- Strong hands-on experience with Databricks, PySpark, and SQL.
- Experience with CI/CD pipelines and GitHub Actions.
- Knowledge of serverless architectures.
- Strong problem-solving and analytical skills.
- Ability to work independently and take ownership of deliverables.
- Experience with pricing, financial, or analytics data.
- Familiarity with enterprise support environments.