Senior Azure Cloud & Databricks Administrator

The core responsibilities for the job include the following:

Azure Cloud Administration:

• Manage Azure subscriptions, resource groups, storage accounts, networking, and security controls.
• Administer Azure Storage Accounts, ADLS Gen2 and Azure SQL Databases.
• Implement and maintain RBAC, managed identities, service principals, and governance controls.
• Monitor platform availability, performance, security, and cost optimisation initiatives.
• Support cloud migration and modernisation activities.

Azure Databricks Administration:

• Administer Azure Databricks workspaces across Development, Test, and Production environments.
• Manage clusters, compute resources, SQL warehouses, jobs, workflows, and user access.
• Perform Databricks Runtime upgrades and compatibility assessments.
• Support Unity Catalogue enablement, governance, and access management.
• Configure and manage external locations, storage credentials, catalogues, schemas, and volumes.
• Troubleshoot storage integration, permission, authentication, and connectivity issues.
• Monitor cluster utilisation, job failures, and platform health.
• Support migration of workloads from legacy environments to Unity Catalogue-enabled environments.
• Creating Databricks-backed secret and scopes and providing necessary access to users
• Supporting users on Databricks apps creation and implementation

Data Platform Support:

• Support Azure Data Factory (ADF) pipelines and related integrations.
• Work closely with data engineering teams to troubleshoot pipeline and data processing issues.
• Assist with schema management, SQL database connectivity, and data access requirements.
• Support enterprise data governance and platform standards.

Monitor:

• Monitor Azure Databricks Clusters, Databricks Jobs and Workflows, Azure Data Factory Pipelines, Azure SQL Databases, and Storage Accounts.
• Perform root cause analysis and incident resolution.
• Participate in production support and platform maintenance activities.
• Manage service requests, access requests, and operational escalations.
• Implement security best practices across Azure and Databricks platforms.
• Manage Unity Catalogue permissions and data access governance.
• Conduct periodic access reviews and compliance checks.
• Ensure platform adherence to enterprise security standards.

Requirements:

• Required Skills: Azure, Azure Administration, Azure Storage Accounts, ADLS Gen2 Azure SQL Database, Azure Monitor, RBAC and Identity Management, Databricks, Workspace Administration, Cluster Administration, Unity Catalogue, SQL Warehouses, Databricks Workflows, External Locations, Storage Credentials, Runtime Upgrades, Platform Monitoring and Troubleshooting, Data Services, Azure Data Factory, Data Platform Operations, Databricks Apps, Secret and Scope Creation, and User Access Management.
• Good to Have: Spark/PySpark/Python Knowledge. FinOps / Cloud Cost Optimisation.