About the role
Responsibilities:
- Build and maintain API integrations, SFTP pipelines, and automated file ingestion.
- Develop data pipelines using Python, PySpark, and Databricks.
- Implement error handling, notifications, monitoring, and alerts.
- Manage low-complexity ingestions such as file drops and partner data shares.
- Maintain GitHub repositories and CI/CD workflows using GitHub Actions.
- Use Terraform for infrastructure management.
- Test pipelines, troubleshoot issues, and document data flows.
- Work in Agile environments using Jira.
Requirements:
- Experience: 5+ Years.
- Required Skills: Databricks, AWS, Python, PySpark, GitHub Actions, Terraform, Data Engineering, Jira, Agile.
- Only FTE candidates.