Director of Engineering - Data Diversity

We are seeking an experienced Director of Engineering - Data Diversity to lead the strategy, architecture, and execution of enterprise-scale data engineering ecosystems supporting diverse and large-scale data environments across AI/ML and cloud platforms. This role requires deep expertise in Python, PySpark, Apache Airflow, GCP, AI/ML data engineering, distributed systems, and cloud-native architectures. The ideal candidate should have strong experience managing complex data ecosystems involving structured, unstructured, real-time, and AI-driven datasets across enterprise platforms. The candidate will work closely with Product, AI/ML, Data Science, Platform Engineering, and Executive Leadership teams to build scalable, intelligent, and high-performance data systems enabling enterprise AI and analytics initiatives.

The core responsibilities for the job include the following:

Engineering Leadership:

• Lead and scale large engineering teams across Data Engineering, Cloud Engineering, and AI/ML infrastructure domains.
• Define engineering strategy, platform architecture, execution roadmaps, and operational excellence frameworks.
• Mentor Engineering Managers, Architects, Principal Engineers, and senior technical leaders.
• Establish engineering best practices, governance models, scalability standards, and delivery excellence processes.

Data Diversity Engineering:

• Architect and build scalable systems handling diverse enterprise data ecosystems, including: Structured and unstructured datasets, Real-time and batch processing systems, AI/ML training and inference datasets, Enterprise analytics and reporting pipelines, Large-scale cloud-native data workflows.
• Drive development of: Data ingestion and transformation pipelines, Distributed data processing systems, and Workflow orchestration platforms.
• Enterprise data governance and observability frameworks.
• Ensure scalability, reliability, security, performance optimization, and operational readiness across enterprise data systems.

Requirements:

• 15-20 years of experience in Software Engineering / Data Engineering / Cloud Engineering.
• Strong leadership experience managing enterprise-scale engineering organizations.
• Proven expertise in Python, PySpark, Apache Airflow, Google Cloud Platform (GCP), AI/ML data engineering, Distributed systems, and Cloud-native architectures.
• Strong experience managing large-scale enterprise data ecosystems and AI/ML infrastructure.
• Excellent leadership, stakeholder management, and executive communication skills.

Technical Expertise:

• Strong understanding of Python and PySpark ecosystems, Apache Airflow orchestration, Google Cloud Platform (GCP), AI/ML data engineering platforms, Distributed systems and cloud-native architectures, Big data processing frameworks, Real-time and batch processing systems, Scalable APIs and backend integrations.
• Collaborate closely with AI/ML, Data Science, and Platform teams on enterprise-scale AI and analytics initiatives.
• Drive modernization of cloud-native data engineering ecosystems and enterprise transformation programs.

Cloud and AI/ML Ecosystems:

• Lead cloud-first engineering initiatives leveraging GCP services such as BigQuery, Dataflow, Pub/Sub, Kubernetes, and Spark ecosystems.
• Enable scalable AI/ML and analytics workflows supporting intelligent enterprise platforms.
• Drive adoption of modern data engineering, observability, governance, and platform reliability practices.

Cross-Functional Leadership:

• Partner closely with Product, AI/ML, Analytics, Security, and Business stakeholders.
• Align engineering execution with business priorities and enterprise transformation goals.
• Drive roadmap planning, stakeholder communication, budgeting, prioritization, and execution governance.