About the role
Responsibilities:
- Lead the development of next-generation features, positioning Cloud Dataproc as the preferred platform for Spark, Flink, Trino, and emerging cloud technologies.
- Strategically define the roadmap for enhancing open-source technologies like Spark, Hive, Trino, Iceberg, Hudi, and Delta into Dataproc.
- Establish benchmarks to identify and resolve performance bottlenecks, ensuring large-scale Spark job certification.
- Enhance the performance and efficiency of open-source technologies within the platform.
- Drive the design and implementation of cutting-edge Data Lakes and Lake Houses, including Apache Iceberg and Hudi.
- Develop and implement software solutions leveraging Google technologies for accelerated cluster setup, streamlined operations, and comprehensive monitoring.
- Collaborate with GCP infrastructure teams to bring the latest GCP features to Dataproc users.
- Position Cloud Dataproc as the optimal choice for data science, machine learning, and artificial intelligence workloads.
- Demonstrate the ability to work effectively across diverse teams in a distributed environment.