About the job
- Join us as a Cloud Architect specializing in Azure Databricks, where you will design and manage innovative data platforms on Azure. Your focus will be on leveraging Databricks for a seamless integration of data science, engineering, and analytics, with an emphasis on the two-plane architecture (Control & Data Planes), data governance (Unity Catalog), security (Entra ID, Key Vault), cost optimization, and the development of reliable Lakehouse solutions guided by Microsoft’s Well-Architected Framework principles for data pipelines (Bronze, Silver, Gold layers). You will also ensure effective integration with other Azure services like Synapse, Power BI, and Azure Monitor.
- Architecture Design: You will be responsible for designing comprehensive end-to-end data solutions, covering every aspect from data ingestion to final consumption utilizing the Databricks Lakehouse Platform on Azure.
- Control Plane vs. Data Plane: You will gain a deep understanding of the separation between the Control Plane, managed by Databricks, and the Data Plane, owned by the customer within their Azure subscription, ensuring data residency and robust security measures are in place.
- Data Governance: You will implement Unity Catalog for centralized access control, lineage tracking, and security across various data assets.
- Security & Identity: Integrating with Microsoft Entra ID (Azure AD) for single sign-on, utilizing Azure Key Vault for secrets management, and employing Microsoft Defender for Cloud to ensure comprehensive protection.
- Performance & Cost: Your role will involve optimizing cluster configurations (classic vs. serverless), storage solutions (ADLS Gen2), and resource utilization to enhance cost efficiency.

