About the job
As a Data Engineer specializing in Azure, you will be responsible for developing robust data pipelines that facilitate the ingestion of data from diverse sources, including databases, APIs, and streaming platforms. Your role will involve integrating and transforming data to ensure alignment with target data models and formats.
You will design and enhance data storage architectures, encompassing data lakes, data warehouses, and distributed file systems. This includes implementing optimization techniques such as partitioning, compression, and indexing to improve data storage efficiency and retrieval speed. You will also identify and resolve bottlenecks, fine-tune queries, and apply caching strategies to boost overall system performance.
Your responsibilities will also include designing and implementing data models that promote efficient data storage, retrieval, and analysis. You will collaborate closely with data scientists and analysts to understand their data needs and deliver well-structured, optimized datasets for their analysis and modeling tasks.
Collaboration is key; you will work alongside cross-functional teams, including data scientists, analysts, and business stakeholders, to grasp their requirements and provide effective technical solutions. Your ability to communicate complex technical concepts clearly to non-technical stakeholders will be essential.
We expect you to take ownership of your projects and deliver solutions independently. Experience in Agile and Scrum methodologies is preferred.
Additionally, you will have the opportunity to mentor and train junior data engineers, sharing your knowledge and expertise to foster their growth.
Your role will also involve designing and implementing data processing systems on the Azure platform, where you will write efficient and scalable code for processing, transforming, and cleaning large volumes of structured and unstructured data.
