About the role
Responsibilities:
- Design, develop, test, and maintain production-grade services and data pipelines using Python and Go.
- Build and manage Apache Airflow/MWAA DAGs, custom operators, sensors, integrations, and workflow orchestration.
- Develop Databricks and PySpark jobs for large-scale data processing, transformation, validation, and analytics.
- Design reliable ETL/ELT pipelines for customer, order, product, campaign, event, and marketing data.
- Build REST APIs, backend services, microservices, and platform capabilities using appropriate frameworks and design patterns.
- Develop and maintain AWS-based solutions using services such as S3 Lambda, MWAA, Glue, Redshift, SQS, Secrets Manager, ECS, and related services.
- Build event-driven systems using Kafka, SQS, Redis, and other messaging or caching technologies.
- Design data models and work with PostgreSQL, MySQL, Redshift, Redis, and other relational or non-relational data stores.
- Integrate internal systems and external marketing platforms such as Facebook, Google, CleverTap, Mixpanel, AppsFlyer, Google Analytics, and Netcore.
- Write clean, maintainable, secure, and scalable code with appropriate error handling, retries, idempotency, and observability.
- Improve the performance, reliability, availability, and cost efficiency of existing services and pipelines.
- Implement automated testing, code reviews, monitoring, logging, tracing, and production support practices.
- Participate in system design discussions and contribute to technical decisions and architecture.
- Work with DevOps and platform teams to improve CI/CD, deployment automation, and infrastructure reliability.
- Collaborate with cross-functional teams to understand requirements and deliver pragmatic technical solutions.
- Contribute to engineering standards, documentation, reusable components, and mentoring of junior engineers.
Requirements:
- 4+ years of professional software engineering experience.
- Strong proficiency in Python and/or Go, with experience delivering production systems.
- Good understanding of data structures, algorithms, object-oriented design, concurrency, and operating-system fundamentals.
- Hands-on experience building data pipelines, backend services, APIs, or distributed systems.
- Practical experience with Apache Airflow, workflow orchestration, or equivalent scheduling platforms.
- Experience with Databricks, PySpark, Spark, Delta Lake, or large-scale data processing.
- Strong SQL skills and experience with PostgreSQL, MySQL, Redshift, or similar databases.
- Experience with AWS services, especially S3 Lambda, MWAA, Glue, Redshift, SQS, Secrets Manager, ECS, or related services.
- Experience with Kafka, Redis, or other streaming, messaging, and caching systems.
- Understanding of REST APIs, microservices, asynchronous processing, and event-driven architecture.
- Experience with Git and collaborative development practices such as pull requests and code reviews.
- Experience writing unit, integration, and end-to-end tests.
- Knowledge of CI/CD tools and deployment practices using Jenkins, GitHub Actions, Docker, or similar technologies.
- Understanding of application and data security, access control, secrets management, privacy, and data protection.
- Strong analytical and problem-solving skills.
- Ability to independently own features or services from design through production deployment.
- Strong communication skills and the ability to collaborate effectively across teams.