About the role
Key ResponsibilitiesCore Duties
- Understand business requirements, architecture documents, and translate them into scalable technical solutions.
- Lead end-to-end development of Azure Data Factory (ADF) pipelines and enterprise data workflows.
- Design and develop scalable ELT/ETL frameworks using Azure services and Snowflake.
- Build and optimize analytical layers, dimensional models, data marts, and semantic data structures in Snowflake.
- Create reusable, maintainable, and performance-optimized SQL and Python-based data transformation frameworks.
- Drive data quality, validation, reconciliation, and governance processes across data platforms.
- Conduct detailed code reviews, design reviews, and ensure engineering best practices are followed.
- Troubleshoot complex production issues, optimize pipeline performance, and reduce data processing latency.
- Participate actively in solution architecture discussions and provide technical recommendations.
- Lead technical demos, walkthroughs, sprint reviews, and stakeholder presentations.
- Mentor junior and mid-level engineers and guide them in technical problem-solving.
- Contribute to framework development, automation initiatives, and process standardization.
- Support release management, CI/CD deployment processes, and environment migrations.
- Collaborate with cross-functional teams including business analysts, architects, QA teams, DevOps, and product owners.
Technical Skills or Knowledge AreasMandatory Skills
- Advanced SQL (complex joins, CTEs, window functions, query optimization, stored procedures)
- Python for data engineering and automation
- Azure Data Engineering Services:
- Azure Data Factory (ADF)
- Azure Data Bricks
- Azure Data Lake
- Azure Functions
- Azure Synapse Analytics
- Logic Apps
- Key Vault
- Snowflake Data Warehouse Development
- Data Modeling (Star Schema, Snowflake Schema, Dimensional Modeling)
- Performance tuning and optimization
- CI/CD and DevOps for data engineering
- Git/version control systems
- DBT (Data Build Tool)
- Apache Spark / PySpark
- Airflow
- Terraform or Infrastructure as Code
- Data governance and metadata management
- API integration and real-time data ingestion
- Monitoring and observability frameworks
Collaboration & Leadership
- Strong experience working in offshore-onshore delivery models.
- Ability to lead technical discussions and coordinate across multiple teams.
- Familiarity with Agile/Scrum methodologies.
- Hands-on experience with:
- Slack
- JIRA/Kanban
- Confluence
- Azure DevOps
- Ability to communicate effectively with technical and non-technical stakeholders.
- Provide mentorship and technical guidance to junior team members.
- Drive collaboration and foster a positive engineering culture.
Process Improvement
- Identify opportunities for automation, optimization, and engineering standardization.
- Improve reusable framework components and reduce operational overhead.
- Participate in architecture reviews and suggest scalable design improvements.
- Handle change requests, impact analysis, and production support activities efficiently.
- Contribute to engineering documentation standards and knowledge-sharing initiatives.
Compliance & Documentation
- Create and maintain High-Level Design (HLD) and Low-Level Design (LLD) documents.
- Prepare technical architecture diagrams and data flow documentation.
- Create operational runbooks, support documentation, and stakeholder training materials.
- Ensure compliance with enterprise security, governance, and audit requirements.
- Maintain documentation for data lineage, transformations, and business rules.
Required QualificationsEducation & Certifications
- BE / BTech / MCA / MTech in Computer Science, Information Technology, or related field.
- Azure Data Engineering certifications preferred:
- Microsoft Certified: Azure Data Engineer Associate
- Snowflake Certifications are a plus.
Experience
- 6 to 9 years of experience in Data Engineering.
- Strong enterprise experience in:
- SQL
- Python
- Azure Data Engineering ecosystem
- Snowflake Data Warehousing
- Experience handling large-scale enterprise data platforms and complex transformation pipelines.
Performance Metrics
- Timely and high-quality delivery of user stories and technical deliverables.
- Ability to independently own medium to high-complexity technical implementations.
- Ability to lead solutioning discussions and provide scalable recommendations.
- Strong troubleshooting and problem-solving capabilities.
- Effective stakeholder communication and status reporting.
- Contribution towards process optimization and engineering improvements.
- Mentoring effectiveness and team collaboration.
- Ability to adapt quickly in fast-changing technical environments.
- Ensure high reliability, scalability, and performance of data solutions.
Additional Expectations for Senior Role
- Ability to act as technical lead for modules/projects.
- Drive engineering best practices and coding standards.
- Participate in hiring and technical interviews.
- Guide architecture decisions and technology evaluations.
- Ownership mindset with proactive risk identification and mitigation.
- Strong business understanding and ability to align engineering solutions with business goals.