About the role

Data Engineer – Job DescriptionRole SummaryThe Data Engineer is responsible for designing, building, and maintaining scalable data pipelines, data platforms, and integration solutions across cloud environments. This role focuses on transforming raw data into reliable, high-quality datasets to support analytics, reporting, and AI/ML use cases.Key ResponsibilitiesData Engineering & Pipeline Development

Design, develop, and maintain ETL/ELT pipelines for data ingestion, transformation, and loading
Build scalable data workflows using batch and real-time processing frameworks
Develop and optimize data pipelines for performance, reliability, and scalability
Handle structured and unstructured data across multiple sources

Data Platform & Cloud Implementation

Work with cloud platforms (Azure / AWS / GCP) to build and manage data solutions
Utilize cloud-native services such as:
- Data lakes, warehouses, and lakehouse platforms
- Distributed compute (e.g., Spark, Databricks, Synapse)
Support deployment and management of data infrastructure and storage systems

Data Integration & Transformation

Integrate data from multiple systems including APIs, databases, applications, and streaming sources
Implement transformation logic using SQL, PySpark, or other data processing tools
Ensure consistency and accuracy across data pipelines

Data Quality, Governance & Security

Implement data validation, cleansing, and quality checks
Ensure compliance with data governance, privacy, and security policies (PII/PHI handling)
Maintain data lineage, metadata, and documentation

Monitoring, Optimization & Reliability

Monitor data pipelines and workflows for failures and performance issues
Implement logging, alerting, and troubleshooting mechanisms
Optimize pipelines for cost, speed, and resource utilization

Collaboration & Support

Work closely with data architects, analysts, and business stakeholders to understand requirements
Support analytics, BI, and AI teams with clean and reliable datasets
Participate in code reviews, testing, and deployment processes

Documentation & Best Practices

Document data flows, pipeline logic, and technical designs
Follow best practices for data modeling, schema design, and version control
Maintain reusable components and frameworks

Required Experience

3–8 years of experience in Data Engineering or related roles
Strong experience with:
- ETL/ELT tools and frameworks
- SQL and data modeling concepts
- Python / PySpark / Scala (at least one)
Hands-on experience with:
- Cloud platforms (Azure / AWS / GCP)
- Big data tools (Spark, Databricks, Synapse, etc.)
Experience with data streaming tools (Kafka/Event Hubs) is a plus
Understanding of CI/CD and DevOps practices

Engineer III - Data Engg & AI

About the role

Similar jobs

Engineer III - Data Engg & AI

85% of jobs are never posted