About the role
TCS is inviting applications for Pyspark Developer/ Designer
Experience Range: 4+ Years
Location: Pune,
Mumbai, Bangalore, Chennai, Hyderabad
Job Description -
- ·Implementing data ingestion pipelines from different types of data sources i.e Databases, S3, Files etc..
- ·Experience in building ETL/ Data Warehouse transformation process.
- ·Experience working with structured and unstructured data.
- ·Developing Big Data and non-Big Data cloud-based enterprise solutions in PySpark and SparkSQL and related frameworks/libraries,
- ·Developing scalable and re-usable, self-service frameworks for data ingestion and processing,
- ·Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of data,
- ·Processing performance analysis and optimization,
- ·Bringing best practices in following areas: Design & Analysis, Automation (Pipelining, IaC), Testing, Monitoring, Documentation.