About the role
Hyderabad, Telangana, India (Hybrid/Office)
🧑💻 Job TitleData Engineer / Databricks Developer – PySpark Focus
📅 Experience3 to 5 Years in Data Engineering or similar role
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and PySpark.
- Build efficient data ingestion, transformation and processing workflows for large datasets.
- Write clean, optimized code using PySpark, Spark SQL, and Python for data transformation tasks.
- Work with Azure Data Lake (ADLS) or similar cloud storage to manage and process data.
- Collaborate with Data Analysts, Architects, and other Engineering teams to translate business requirements into technical design.
- Optimize data pipelines for performance and cost efficiency; implement error handling and monitoring.
- Write and tune complex SQL queries for data manipulation, reporting, and analytics support.
- Assist in data governance, documentation, and standardization practices.
- Support deployments, version control workflows (Git), and automation best practices. (CI/CD)
Mandatory Skills
- Hands-on experience with Databricks, PySpark, Python.
- Strong SQL and data manipulation skills.
- Experience building ETL/ELT pipelines on Azure Databricks.
- Good understanding of data lakes and cloud data platforms (Azure).
Nice to Have
- Experience with Azure Data Factory (ADF) or orchestration tools.
- Understanding of batch & streaming data processing.
- Familiarity with Delta Lake, Unity Catalog, or Delta Tables.
- Knowledge of data governance, CICD pipelines, and monitoring tools.
- Bachelor’s degree in Computer Science, IT, Engineering, or related field.
- Problem-solving mindset with ability to work independently and collaboratively.
- Good communication skills for stakeholder interaction.
- Ability to debug/optimize large data jobs and workflows.
- Comfortable working in Agile teams.