About the job
The Opportunity
Join Enigma at a pivotal moment as we prepare to unveil our next generation of small business intelligence products, informed by years of industry experience. We are in search of a seasoned Senior Data Engineer who will play a crucial role in shaping the future of these innovative products. Your contributions will significantly enhance the accuracy, scalability, and predictive analytics of our data platform, directly impacting our clients' key business strategies.
The Role
As a Senior Data Engineer, your primary responsibility will be to design, construct, and manage our foundational small business data product. You'll work closely with cross-functional teams to develop new product functionalities, starting from raw data sources, through scalable data pipelines and models, to the customer interface. Our culture emphasizes customer focus and complete ownership, measuring your success by the value delivered and the technical excellence of our product.
What You’ll Do
- Identify and address the unmet needs of our customers through thorough data exploration, prototyping, and evaluation, culminating in the development of robust and maintainable data pipelines.
- Construct scalable and maintainable data systems that narrate the stories of millions of U.S. small businesses, utilizing tools like Databricks SQL, PySpark, and MLFlow.
- Leverage our pioneering data infrastructure designed for iterative development and experimentation, utilizing Dagster for orchestration and LakeFS for data version control.
- Employ a data-driven, first-principles approach to problem-solving, favoring straightforward and elegant solutions while being capable of deploying advanced models and techniques when necessary.
- Motivate your colleagues to excel while nurturing a collaborative and supportive team environment.
What Makes This Role Exciting?
- Ownership and Impact: Collaborate directly with customers and internal teams to develop product features in a high-trust environment.
- Technical Challenge: Solve complex data and engineering challenges, balancing innovation speed with scalability and reliability.
- Learning: Engage with unique, impactful data challenges that demand creative solutions and cutting-edge methodologies.