About the role
Key Responsibilities:
- Assist in developing and maintaining data pipelines using Databricks.
- Work with Python, SQL, and PySpark for data processing and transformation.
- Support data ingestion, cleansing, validation, and integration activities.
- Learn and work with Apache Spark concepts for distributed data processing.
- Assist in implementing ETL/ELT workflows to process structured and unstructured data.
- Work with Delta Lake and understand its role in reliable data storage and processing.
- Perform data quality checks and troubleshoot basic data processing issues.
- Support the development, testing, and documentation of data engineering solutions.
- Collaborate with team members to optimize data processing workflows.
- Learn cloud-based data engineering concepts and follow established coding standards and best practices.