Pyspark Data Engineer

Job Description

About the Unit:

TCS Data & Analytics is powering the AI revolution by transforming data into a strategic enterprise asset. Built on the philosophy of Smart Data for Smarter AI, it enables organisations to create intelligent, AI-ready ecosystems that accelerate innovation, unlock new value, and drive business transformation at scale. As AI reshapes industries, TCS is helping businesses turn data into their most powerful competitive advantage.

Title Description for Opportunity :

Role Name: Pyspark Data Engineer

Responsibilities:

•  Design, develop, and optimize large-scale data processing solutions using PySpark, Apache Spark, and Python, ensuring high-performance batch and real-time data processing.

•  Build and maintain scalable data ingestion, transformation, and storage pipelines using Spark SQL, DataFrames, Kafka, and cloud-based data platforms such as AWS, Azure, or GCP.

•  Develop and support distributed computing frameworks and streaming applications, leveraging Spark Streaming, multithreading, and big data technologies to process high-volume datasets efficiently.

•  Collaborate with cross-functional teams to design, implement, and deploy data engineering and machine learning solutions, utilizing Python, Scala, Java, TensorFlow, PyTorch, and Scikit-learn.

•  Drive best practices for code quality, performance tuning, monitoring, troubleshooting, and optimization of Spark applications, while ensuring reliable and scalable data platforms for enterprise analytics and business needs.

Must Have Skills:

• PySpark: In-depth knowledge of PySpark, Apache Spark, and Python.
• Data Processing: Strong understanding of data processing concepts, including data ingestion, data transformation, and data storage.
• Cloud Experience: Experience with cloud-based data platforms, including AWS, Azure, or Google Cloud.
• Expertise in DataFrames & Spark SQL, Spark Streaming with Apache Kafka real-time data ingestion pipelines
• Very good conceptual understanding of Multithreading, distributed computing concepts of Pyspark
• Programming: Proficiency in programming languages, including Python, Java, or Scala.
• Communication: Excellent communication and collaboration skills.
• Preferred Skills:
• Apache Spark Certifications: Relevant certifications, such as Apache Spark Certification or Cloudera Certified Spark Developer.
• Big Data: Experience with big data technologies, including Hadoop, HBase, or Cassandra.
• Machine Learning: Experience with machine learning frameworks, including TensorFlow, PyTorch, or Scikit-learn.
• Data Science: Experience with data science tools, including Jupyter Notebook, Pandas, or NumPy

Walk-in Drive Locations - Bengaluru, Chennai, Hyderabad, Kolkata and Pune.

• Experience Range: 4 to 10 years
• Qualifications: Any Technical Degree(BE/B,Tech/M.tech/MCA/BCA/BSC/Diploma/Masters)

 Eligibility:

• 15 years of full-time education.
• Good communication and interpersonal skills
• Strong verbal and written communication skills
• Willingness to work in a collaborative and dynamic e nvironment
• Relevant experience as per the job requirements (if applicable)

      Documents required for interview:

• TCS Online Application Form
• Resume
• Govt. Photo ID Proof (Passport / Driving License / Aadhar Card / PAN Card)
• PAN Card
• Marksheet
• Work experience letters
• Last 3 months pay slips .

      Unit level assets:

     Solution- 

TCS Data Modernization Service on AWS

    Research insights

• Modernise ETL systems for a successful AI-first operating model

       Case studies-

• TCS accelerates Alstom’s digital transformation with a future-ready Data Integration Backbone

      Leadership Video- 

• The Future of AI is Built on Data. And It Starts Here- Satish Byravan, Global Head of Data & Analyt…

    Big Bet hiring page link:

• TCS Data and Analytics Careers