About the job
We are looking for a skilled Senior Data Engineer to architect, create, and sustain our data infrastructure that underpins essential energy operations. This role sits at the convergence of renewable energy and data analytics, where you will construct data pipelines that process a range of information from real-time asset performance metrics to intricate trading and risk analysis. This hybrid position allows you to directly influence clean energy initiatives while leveraging a state-of-the-art data stack that includes Snowflake, Dagster, dbt, Modal, and GitLab.
Key Responsibilities
Analytics Infrastructure & Data Warehouse Management
- Design, implement, and manage scalable data infrastructure to cater to enterprise analytics and reporting requirements.
- Oversee Snowflake instances, focusing on performance enhancement, security settings, and capacity forecasting for increasing data volumes.
- Maximize query performance and resource allocation to manage costs and enhance processing speed.
Data Pipeline Development & Orchestration
- Develop and manage intricate ETL/ELT workflows utilizing Dagster to secure dependable, automated data processing for asset management and energy trading.
- Create resilient data pipelines that manage high-volume, time-sensitive energy market data alongside asset generation and performance metrics.
- Implement workflow automation and dependency management to facilitate critical business operations.
Data Transformation & Analytics Support
- Build and maintain dbt models to convert raw data into business-ready analytical datasets and dimensional models.
- Execute efficient SQL-based transformations for complex energy market calculations and asset performance metrics.
- Facilitate advanced analytics efforts through adequate data preparation and feature engineering.
Data Quality & Governance
- Establish comprehensive frameworks for data validation, testing, and monitoring to ensure accuracy and consistency across all energy and financial data assets.
- Create data lineage tracking and privacy controls to comply with regulatory requirements in the energy industry.
- Develop alerting and monitoring systems for data pipelines, including error handling, SLA monitoring, and incident response mechanisms.
