About the role
Responsibilities:
- Should be able to deploy a Hadoop cluster (preferably CDP) and add and remove nodes.
- Keep track of jobs, monitor critical parts of the cluster, configure name-node HA, and schedule and configure backups. General operational expertise such as good troubleshooting skills, understanding a system's capacity, bottlenecks, and basics of memory, CPU, OS, storage and networks.
- Hadoop stacks like HIVE, Impala, HBase, Nifi, etc.
- Good knowledge of Linux, as Hadoop runs on Linux.
- Familiarity with open-source configuration and deployment tools such as Jenkins, Airflow, Puppet/Chef and Linux scripting.
- Responsible for implementation and ongoing administration of Hadoop infrastructure.
- Aligning with different engineering teams to propose and deploy hardware and software environments required for Hadoop and to expand existing environments.
- Working with data delivery teams to set up new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing the services for new users.
- Cluster maintenance using Cloudera Manager.
- Performance tuning for Hadoop clusters and Hadoop MapReduce routines.
- Monitor Hadoop cluster connectivity and security.
- Log and storage management.
- Diligently teaming with infrastructure, network, database, application and BI teams to guarantee high data availability.
Secondary Responsibilities:
- Software installation and configuration.
- Database backup and recovery.
- Database connectivity and security.
- Performance monitoring and tuning.
- Disc space management.
Requirements:
- Graduation: Graduation (B. E. / B. Tech).
- Experience: 2 to 5 years of relevant experience as a Hadoop admin.