About the role
Job Title MLOps Specialist Databricks AWS
Key Responsibilities
EndtoEnd MLOps Lifecycle Management
Design implement and manage the full machine learning lifecycle using MLflow on Databricks
Automate model training validation deployment and monitoring pipelines
Model Deployment Serving
Deploy ML models using Databricks Model Serving and integrate with AWS services eg S3 Lambda SageMaker EKS
Ensure scalable and secure model serving with realtime and batch inference capabilities
CICD for ML Pipelines
Build and maintain CICD pipelines for ML workflows using tools like Bitbuket Jenkins or Databricks Repos
Implement automated testing versioning and rollback strategies for models
Monitoring Governance
Set up monitoring for model performance drift detection and logging using MLflow and Databricks features
Ensure compliance with data governance and model audit requirements
Collaboration Documentation
Work closely with data scientists data engineers and DevOps teams to streamline ML operations
Document architecture workflows and best practices for reproducibility and scalability
ML Model Development Support
Assist in building and optimizing ML models using PySpark scikitlearn Pytorch Keras within Databricks
Provide guidance on feature engineering hyperparameter tuning and model evaluation
Cloud Infrastructure Management
Manage and optimize cloud resources on AWS for ML workloads
Implement costeffective and secure infrastructure solutions for data and model storage
GenAI Frameworks Good to Have
Exposure to Generative AI frameworks like Databricks MosaicML LangChain or OpenAI APIs
Ability to integrate GenAI models into enterprise workflows and applications
Security Compliance
Implement rolebased access control RBAC encryption and secure data handling practices
Ensure compliance with industry standards eg GDPR HIPAA SOC2
Continuous Learning Innovation
Stay updated with the latest trends in MLOps cloud computing and GenAI
Propose and implement innovative solutions to improve ML lifecycle efficiency