About the job
Join Octopus Digital, a subsidiary of Avanceon Limited, as a Data Engineer and play a pivotal role in designing and implementing sophisticated data warehouse solutions across our organization. We are seeking skilled professionals who are adept at managing vast volumes of structured and unstructured data using cutting-edge technologies.
Job Responsibilities:
- Apply hands-on expertise in Azure or AWS platforms.
- Utilize Spark and Python for data processing tasks.
- Design and create efficient codes, scripts, and data pipelines to handle both structured and unstructured datasets.
- Oversee the management of data ingestion pipelines and stream processing.
- Develop and optimize complex SQL queries and shell scripts.
- Conduct Big Data querying to derive insights.
- Work with NoSQL databases to support data needs.
- Experience with Hadoop distributions, particularly Azure DataLake HDFS.
- Certification or training in Data Lake HDFS is an added advantage.
- Demonstrate the ability to meet deadlines, troubleshoot issues, and provide resolutions with minimal supervision.
