About the job
Job Title: Principal Databricks Consultant (Enterprise Solution Architect)
Location: Remote
Employment Type: Contract
Experience Level: 10+ Years
About the Opportunity
Join our team as a Principal Databricks Consultant, where you will leverage your extensive expertise in Solution Architecture to spearhead transformative digital initiatives for our esteemed enterprise clients. In this pivotal role, you will not only design robust data pipelines but also shape the future of Data Lakehouse ecosystems, effectively aligning high-level business strategies with technical execution.
Your Key Responsibilities
- Architect at Scale: Design and implement global-scale Lakehouse architectures utilizing the Databricks Data Intelligence Platform.
- Strategic Advisory: Serve as the primary technical contact for C-suite executives, translating business objectives into actionable architectural plans.
- Govern & Secure: Establish enterprise-level security and governance frameworks through Unity Catalog.
- Optimize Performance: Conduct audits and optimize complex Spark clusters and SQL warehouses to enhance ROI and reduce cloud expenditures.
- Lead Innovation: Facilitate the integration of Generative AI and MLOps workflows utilizing MLflow and Mosaic AI.
Your Technical Expertise
- The Core: Advanced proficiency in Apache Spark, Delta Lake, and PySpark/Scala.
- Cloud Proficiency: Thorough experience in architecting solutions on AWS, Azure (ADLS/Synapse), or GCP.
- DevOps/IaC: Demonstrated ability to automate environments using Terraform, Jenkins, or GitHub Actions.
- Data Modeling: In-depth knowledge of Medallion architecture (Bronze/Silver/Gold) and Data Vault or Star Schema modeling.
Qualifications
- Experience: Over 10 years in Data Engineering/Architecture, with at least 4 years dedicated to enterprise-level Databricks solutions.
- Education: Bachelor's or Master's degree in Computer Science, Data Science, or a related field.
- Certifications: Databricks Solutions Architect Professional or Databricks Champion status is highly desirable.
- Soft Skills: Outstanding presentation abilities; adept at conveying complex technical concepts to non-technical stakeholders.
