Required Expertise:Proficient in Spark with ScalaExperienced with CI/CD tools (e.g., GitLab, Jenkins)Familiar with HDFS and structured databases (SQL)Knowledgeable in Apache AirflowExperience with S3 storage/COS and parquet (and ORC) formatsStrong understanding of SQL and NoSQL databasesHands-on experience with HadoopComprehensive Understanding of:OozieShell ScriptingSoftware Development Lifecycle (SDLC)Agile principles and ceremoniesDesigning effective prompts to leverage Generative AI tools across IT domains during development phases.Familiarity with:Kubernetes containerizationDremio for data virtualizationPreferred Qualifications:Experience with Elasticsearch and KibanaKnowledge of streaming processes (e.g., Kafka, event streams)Familiarity with HVault
About the job
Join Us in a Hybrid Work Environment in Madrid! This position may require occasional travel within Italy.
Your Key Responsibilities:
Migrate existing Hadoop infrastructure to cloud solutions utilizing Kubernetes Engine, COS, Spark as a Service, and Airflow as a Service.
Implement data transformations and ensure data quality for consistency and accuracy using programming languages like Scala and SQL, along with tools such as Spark.
Establish CI/CD pipelines to automate deployments, unit testing, and developmental management.
Conduct unit and validation testing to affirm the accuracy and integrity of developed code.
Automate data pipelines and optimize data ingestion processes primarily through Airflow as a Service.
Create and maintain technical documentation (specifications, operational documents) to facilitate knowledge sharing.
Work collaboratively with cross-functional teams to ascertain data requirements and deliver effective solutions.
Encourage a culture of continuous learning and enhancement within the team.
About Inetum
Join Inetum – We are Hiring a Data Engineer! At Inetum, a prominent international digital consultancy, we empower over 27,000 professionals across 27 countries to forge their careers, drive innovation, and maintain a work-life balance. Proudly recognized as a Top Employer Europe 2024, we are dedicated to crafting impactful digital solutions.
This job posting is no longer active and is not accepting applications.
Required Expertise:Proficient in Spark with ScalaExperienced with CI/CD tools (e.g., GitLab, Jenkins)Familiar with HDFS and structured databases (SQL)Knowledgeable in Apache AirflowExperience with S3 storage/COS and parquet (and ORC) formatsStrong understanding of SQL and NoSQL databasesHands-on experience with HadoopComprehensive Understanding of:OozieShell ScriptingSoftware Development Lifecycle (SDLC)Agile principles and ceremoniesDesigning effective prompts to leverage Generative AI tools across IT domains during development phases.Familiarity with:Kubernetes containerizationDremio for data virtualizationPreferred Qualifications:Experience with Elasticsearch and KibanaKnowledge of streaming processes (e.g., Kafka, event streams)Familiarity with HVault
About the job
Join Us in a Hybrid Work Environment in Madrid! This position may require occasional travel within Italy.
Your Key Responsibilities:
Migrate existing Hadoop infrastructure to cloud solutions utilizing Kubernetes Engine, COS, Spark as a Service, and Airflow as a Service.
Implement data transformations and ensure data quality for consistency and accuracy using programming languages like Scala and SQL, along with tools such as Spark.
Establish CI/CD pipelines to automate deployments, unit testing, and developmental management.
Conduct unit and validation testing to affirm the accuracy and integrity of developed code.
Automate data pipelines and optimize data ingestion processes primarily through Airflow as a Service.
Create and maintain technical documentation (specifications, operational documents) to facilitate knowledge sharing.
Work collaboratively with cross-functional teams to ascertain data requirements and deliver effective solutions.
Encourage a culture of continuous learning and enhancement within the team.
About Inetum
Join Inetum – We are Hiring a Data Engineer! At Inetum, a prominent international digital consultancy, we empower over 27,000 professionals across 27 countries to forge their careers, drive innovation, and maintain a work-life balance. Proudly recognized as a Top Employer Europe 2024, we are dedicated to crafting impactful digital solutions.