About the role
- Design, develop, and maintain ETL/ELT data pipelines using Python.
- Build scalable and production-ready REST APIs using FastAPI.
- Develop efficient and optimized PostgreSQL queries, views, functions, and stored procedures.
- Build data ingestion pipelines from databases, APIs, files, and third-party systems.
- Perform data extraction, cleansing, transformation, validation, and loading.
- Develop reusable Python modules and automation scripts for data processing.
- Integrate internal and external systems through REST APIs.
- Implement API authentication, validation, error handling, logging, and monitoring.
- Monitor data pipelines and APIs and troubleshoot production issues.
- Implement data quality checks and reconciliation mechanisms.
- Optimize PostgreSQL queries, indexes, and database performance.
- Handle large datasets efficiently and ensure reliable data processing.
- Work with application developers, BI teams, analysts, and business stakeholders.
- Maintain technical documentation for APIs, pipelines, databases, and integrations.
- Follow coding standards, Git-based version control, testing, and deployment best practices.
- Strong hands-on experience with Python 3.x.
- Good understanding of OOP, functions, modules, exception handling, and package management.
- Experience with Pandas, NumPy, Requests or similar libraries.
- Ability to write clean, reusable, scalable, and production-ready Python code.
- Good understanding of asynchronous programming is preferred.
- Hands-on experience developing REST APIs using FastAPI.
- Understanding of:
- API routing
- Request/response models
- Pydantic
- Dependency injection
- Middleware
- Authentication and authorization
- Exception handling
- API validation
- Async endpoints
- API documentation using Swagger/OpenAPI
- Experience connecting FastAPI applications with PostgreSQL.
- Ability to design scalable and maintainable API architectures.
- Experience with API performance optimization and debugging.
- Strong hands-on experience with PostgreSQL.
- Excellent SQL skills including:
- Complex joins
- CTEs
- Window functions
- Subqueries
- Aggregations
- Views
- Stored procedures/functions
- Indexing
- Transactions
- Understanding of database design and normalization.
- Experience with query optimization and performance tuning.
- Ability to troubleshoot slow queries and database performance issues.
- Hands-on experience building ETL/ELT pipelines.
- Experience with data extraction, transformation, validation, and loading.
- Understanding of batch and incremental data processing.
- Experience handling JSON, CSV, Excel, API responses, and database data.
- Understanding of data modelling and data warehousing concepts.
- Experience implementing data quality and reconciliation checks.
- Strong experience consuming and developing REST APIs.
- Understanding of JSON, HTTP methods, authentication, pagination, rate limits, and error handling.
- Experience integrating multiple internal and external systems.
- Experience working with API-based data ingestion pipelines.
- Good knowledge of Git/GitHub/GitLab.
- Familiarity with Linux/Unix environments.
- Understanding of logging, monitoring, debugging, and production support.
- Basic knowledge of Docker is preferred.
- Exposure to AWS or Azure is an advantage.
- Experience with Apache Airflow, Prefect, Dagster, or similar orchestration tools.
- Experience with Docker and containerized FastAPI applications.
- Knowledge of Kafka, Redis, or messaging systems.
- Exposure to AWS/Azure data services.
- Experience with Snowflake, BigQuery, Redshift, or Azure Synapse.
- Knowledge of Power BI or other BI tools.
- Experience handling high-volume transactional/customer data.
- Understanding of PII/data security and access controls.
- Experience in healthcare, CRM, ERP, or other enterprise applications is an advantage.
- 2+ years of professional experience in Data Engineering, Backend Engineering, or a related role.
- Strong hands-on experience in Python + FastAPI + PostgreSQL + SQL.
- Experience developing production-grade APIs and data pipelines.
- Ability to independently troubleshoot application, API, database, and data pipeline issues.
- Have strong programming and problem-solving skills.
- Be comfortable working across API, database, and data engineering layers.
- Be capable of independently developing and deploying production solutions.
- Write clean, maintainable, and well-tested code.
- Understand data accuracy, reliability, and performance.
- Take ownership of production issues and deliver solutions within timelines.
- Work effectively with technical and business teams.
- Python programming
- FastAPI development
- REST API design
- PostgreSQL and advanced SQL
- ETL/ELT pipeline design
- Data modelling
- API and database integration
- Performance optimization
- Debugging and production troubleshooting
- Real-world data engineering/system design problems