About the job
At Dev. Pro, we are dedicated to projects that influence millions globally, recognizing that our people are the key to our success. We cherish individuality and are committed to fostering an inclusive, supportive, and friendly workplace.
About This Opportunity
We are seeking a talented Databricks Platform Administrator to join our dynamic Data Platform team. In this pivotal role, you will take charge of the daily administration, optimization, and advancement of our Databricks environment. Your efforts will ensure our data scientists and engineers have access to a dependable, well-governed, and high-performance platform tailored for analytics, machine learning (ML) experimentation, and production workloads.
What You Will Gain
- Lead the integration and growth of our Databricks platform within the wider data ecosystem.
- Acquire hands-on experience with the latest data and ML technologies (Databricks, Spark, Delta Live Tables, MLflow, AI/LLM tools, Unity Catalog).
- Collaborate with leading data scientists and ML specialists to tackle complex data challenges, making a tangible impact across the organization.
Key Responsibilities and Contributions
In this role, you will manage and optimize our Databricks platform, ensuring secure, reliable, and efficient operations for our data and ML teams.
- Oversee the health, security, and governance of the Databricks environment.
- Manage, scale, and optimize computing clusters for data science and ML workloads.
- Configure workspaces, access controls, and network settings for optimal performance.
- Guarantee high availability and reliability of analytics workloads.
- Monitor platform performance and create dashboards/alerts for cluster efficiency and job success.
- Enhance data ingestion, transformation, and model execution workflows.
- Develop self-service tools for provisioning notebooks, clusters, and environments; implement automation to minimize repetitive tasks.
- Lead the rollout/adoption of Unity Catalog for data discovery, lineage, and access governance.
- Review workspace configurations, cluster policies, and user permissions to uphold security.
- Support audit and compliance for data governance across the Databricks platform.
- Collaborate with data and ML teams to optimize the platform experience and facilitate workflows.
- Coordinate with Cloud DevOps on AWS infrastructure, networking, and IAM dependencies.
Are You the One?
- 5+ years of experience in Databricks administration or engineering, or a closely related data platform role.
- Proficient in Databricks workspace administration, including cluster policies, permissions, Unity Catalog, Jobs/Workflows, and networking.
- Knowledge of AWS services that support Databricks deployments (S3, IAM, VPC, EC2).
- Experience with data pipelines and orchestration tools.
