About the role
To strategically lead and support the API and Data Engineering function, enabling secure, scalable, and high-performance integration and data processing capabilities across the organisation. This role is responsible for hands-on design, development, and deployment of APIs and data pipelines using modern technologies such as FastAPI, REST APIs, Databricks, and SQL, while leveraging containerization and cloud-native platforms including Docker, Azure Container Apps (ACA), and AKS.
The role ensures seamless data movement and transformation through tools like Azure Data Factory (ADF) or equivalent, aligns API-led integration and data engineering initiatives with business and digital priorities, and provides expert guidance on microservices architecture, data platforms, and best practices within a scalable Lakehouse ecosystem.
Responsibilities:
- Design, develop, and maintain API-led integration solutions and data engineering pipelines to support business applications.
- Build and enhance scalable REST APIs using frameworks such as FastAPI, ensuring high performance and reliability.
- Develop and optimise data processing workflows using Databricks (PySpark, SQL) for efficient data transformation and analytics.
- Collaborate with business and technical teams to understand requirements and translate them into robust API and data solutions.
- Implement containerised applications using Docker and deploy/manage services on cloud platforms such as Azure Container Apps (ACA) and AKS.
- Develop and manage data integration workflows using Azure Data Factory (ADF) or equivalent to enable seamless data movement across systems.
- Ensure adherence to best practices in API design, microservices architecture, security, and performance optimisation.
- Troubleshoot, debug, and resolve issues related to APIs, data pipelines, and cloud deployments.
- Stay updated with emerging technologies and continuously improve technical skills in API engineering, cloud-native development, and data platforms.
Decision Making Authority:
- Decisions made alone - Management/maintenance/bug fixes/ stakeholders management.
- All items requiring key architectural calls, investments, or additional expense should be consulted with the manager and above.
- List of internal and external stakeholders the role is expected to interact with to execute duties effectively.
- Internal Stakeholders: Business Team Members - Wheels, Collections, emerging business and Non- Wheels.
- External Stakeholders: FSS Subsidiary members and Data vendors.
Key Challenges:
- Ensuring high performance, scalability, and reliability of REST APIs under varying workload conditions.
- Managing secure API integrations and data exchange across multiple systems with proper authentication and governance.
- Deploying, monitoring, and optimising containerised applications on Docker, ACA, and AKS environments.
- Handling large-scale data processing and optimisation in Databricks (PySpark, SQL) while maintaining efficiency.
- Troubleshooting and maintaining stability across APIs, data pipelines, and cloud-native infrastructure.
Requirements:
- Develop scalable and high-performance REST APIs using FastAPI.
- Strong hands-on experience with Databricks (PySpark, SQL) for data processing.
- Experience in building and managing data pipelines and integrations.
- Proficiency in SQL for data querying, transformation, and optimisation.
- Hands-on experience with Docker and deployment on ACA/AKS environments.
- Ability to develop and manage workflows using Azure Data Factory (ADF) or equivalent.
- Good understanding of database design and data architecture.
- Strong debugging, troubleshooting, and performance tuning skills.
- Python programming for API development and data engineering use cases.