About the role
Responsibilities
- Define end‑to‑end architecture for enterprise data platforms across Databricks, Snowflake, and cloud ecosystems, ensuring scalability, performance, governance, and security.
- Design and implement ingestion, transformation, and compute optimization frameworks using Spark, SQL, Delta Lake, and platform‑native capabilities, balancing cost and performance.
- Develop standards for data modeling, data quality, medallion architecture, and workload orchestration, collaborating closely with data engineering, analytics, and ML teams.
- Partner with pre‑sales and solutioning teams to qualify use cases, shape solution architecture, estimate workload sizing, and build compelling POCs showcasing technical feasibility.
- Evaluate new capabilities in Databricks, Snowflake, and cloud services; drive platform modernization, automation, observability, and best‑practice adoption across teams.
Knowledge & Skills
- Expert-level knowledge of Databricks (Spark, Delta Live Tables, Unity Catalog) and/or Snowflake (Warehouses, Query Optimization, Snowpark), with strong architectural depth.
- Strong grounding in data modeling, data warehousing, data governance, lakehouse patterns, and distributed systems concepts.
- Understanding of cloud ecosystems (AWS/Azure/GCP), including storage systems, IAM, networking, orchestration, and CI/CD for data pipelines.
- Ability to articulate complex architectural concepts to technical and business stakeholders, connecting platform capabilities to business outcomes.
- Advanced SQL, Python, and Spark programming skills with ability to optimize distributed workloads, troubleshoot cluster performance, and tune compute resources.
Mandatory Experience
- 5–8 years of experience designing and delivering large‑scale data platform architectures, including at least 3 years hands‑on in Databricks and/or Snowflake.
- Demonstrated experience building POCs, reference implementations, and end‑to‑end solutions as an individual contributor with deep technical ownership.
- Strong background in performance tuning, cost optimization, cluster sizing, query optimization, and workload management on modern data platforms.
- Experience working with cross‑functional teams including data engineering, analytics, ML, and pre‑sales/sales solutioning functions.
- Hands‑on experience with cloud data services (e.g., S3/ADLS/GCS, Lambda/Functions, Glue/Synapse/Data Factory, Kubernetes, CI/CD).