Qualifications
Alignment with Our Core Values: If reading this resonates with you, we can't wait to meet you! If not, we prefer to save both our time and yours.Consulting Skills: You excel in communication, simplifying complex concepts for others. You engage stakeholders in envisioning the future and actively participating in its creation.Proactive Doers: In our small team, individual contributions matter significantly; we need you to not only initiate projects but also see them through to completion. We're looking for a level 4+ problem solver.Technical Proficiency: You possess strong skills in Scala and/or Python, including a deep understanding of language internals, profiling, and testing methods. Experience with ZIO is a plus, as DataChef contributes to this project .AWS Experience: You have 2+ years of hands-on experience working with AWS services like Lambda functions, EMR, ElasticSearch, Lake Formation, and Glue.Asynchronous Systems Knowledge: You have a solid grasp of building pub-sub and asynchronous systems using Apache Kafka or other messaging APIs like SQS, Kinesis, Celery, RabbitMQ, and ActiveMQ.Defensive Design: You design and code with real-world challenges in mind, recognizing that missing, late, or low-quality raw data is common and that pipeline failures and reprocessing are part of the job.Data Ingestion Skills: You can ingest new data sources through REST APIs, file sharing, etc., and adapt to evolving schemas.Algorithm Analysis: You can analyze algorithm complexity and understand data structures beyond simple ones like Lists and Stacks, knowing the pros and cons of each for various problems.Collaboration Experience: You have been using Linux/macOS, Docker, and Git in collaborative environments for over 3 years.Quick Learners: Our culture is about moving fast. While mistakes may happen, we value learning and adapting quickly. We champion agility and continuous improvement.
About the job
Join Our Dynamic Team!
We are on the lookout for a seasoned and pragmatic Data Engineer to be part of our innovative team at DataChef. Your role will involve designing and constructing a cutting-edge data mesh platform on AWS from the ground up.
Your mission: Establish a scalable data foundation for one of the world's largest distribution companies. We’re searching for top-tier professionals who are eager to embrace the challenge of creating a unique backend for stream processing utilizing AWS services (including Lambda, Glue, Lake Formation, and more), Spark (Scala/Python), and Kafka.
This is a consulting position, and we seek a well-rounded candidate, not just a stellar engineer.
We value the following qualities: