About the role
Cloudera platforms (CDP/CDH/HDP) including HDFS, Hive, Spark, Kafka, and Ranger.
• Design and architect end-to-end Big Data solutions using Cloudera CDP/CDH, Hadoop, Spark, Hive, Impala,
HBase, Kafka, and related components.
• Define data ingestion, processing, storage, and distribution frameworks aligned with enterprise standards.
• Develop high-level and low-level architecture documents, data flow diagrams, and integration blueprints.
• Lead modernization initiatives including migration from legacy systems to Cloudera Data Platform (CDP).
• Guide development teams on best practices for Hadoop, Spark, and distributed data processing.
• Conduct architecture reviews, performance tuning sessions, and code optimizations.
• Define governance, security, and cluster management standards including Kerberos, Ranger, Sentry, Atlas, and
Knox.
• Architect and build Spark-based ETL/ELT pipelines for batch and real-time processing.
• Optimize large-scale jobs for performance, reliability, and cost efficiency.
• Implement data quality, metadata management, and lineage tracking frameworks.
• Oversee capacity planning, resource management, and cluster performance optimization.
• Collaborate with DevOps to define CI/CD pipelines for big data workloads.
• Troubleshoot complex production issues within Hadoop/Spark jobs or Cloudera clusters.
• Work closely with business, data engineering, BI, and product teams to translate requirements into scalable
architectures.
• Present architectural recommendations to senior leadership and participate in solution walkthroughs.
Qualifications: BACHELOR OF ENGINEERING