About the job
- Leverage the Azure ecosystem and the Databricks platform to architect, design, and manage advanced data engineering solutions.
- Develop scalable and efficient data pipelines, execute data transformation and integration processes, and facilitate sophisticated analytics and machine learning workflows while collaborating with cross-functional teams including data scientists, analysts, and software engineers.
- Apply your expertise in Databricks and Azure services to enhance business intelligence initiatives and provide actionable data-driven insights.
- On the Azure platform, design, implement, and deploy comprehensive end-to-end data engineering solutions using Databricks.
- Work closely with stakeholders to gather and interpret business requirements, translating them into robust technical data engineering solutions.
- Conduct data ingestion processes, which involve aggregating data from various sources, processing it to meet business specifications, and integrating it into data lakes or data warehouses.
- Utilizing Azure Data Factory, Databricks, and other relevant tools, create and maintain scalable data pipelines and workflows.
- Design and optimize data models and schemas for effective data storage and retrieval.
- Implement security protocols to ensure adherence to all legal and regulatory standards regarding data protection.

