Senior Software Engineer - Data Pipelines

Join Sword Group, a premier provider of innovative business technology solutions across the Energy, Public, and Finance sectors. Our mission is to catalyze transformational change for our clients, leveraging cutting-edge technology, specialized teams, and deep domain expertise to establish robust technical foundations across platforms, data, and business applications. We are passionate about utilizing technology to address complex business challenges and collaborate closely with our clients to help them achieve their objectives.About the Role:We are seeking a dynamic and experienced Data Engineer Lead to spearhead our expanding Data & AI business unit. In this strategic leadership role, you will report to the Data & AI Technical Director and significantly influence our engineering capabilities, standards, and technical strategies. You will guide a talented team of Senior Data Engineers, providing hands-on mentorship and ensuring the delivery of high-quality, secure, and scalable data solutions across various client projects.In this position, you'll have the opportunity to work with state-of-the-art Microsoft technologies, including Microsoft Fabric, Power BI, Purview, and Microsoft Foundry. You will play a crucial role in shaping solution design, architectural decisions, and maintaining delivery excellence. Engage with clients from diverse industries, facilitating discovery, design, and delivery processes, while also participating in pre-sales activities by contributing to client discussions, shaping proposals, and advising on solution strategies.This role perfectly balances leadership, technical expertise, innovation, and client engagement, offering significant scope to make a lasting impact on our clients and the evolution of our Data Engineering services.Key Responsibilities:Lead, mentor, and develop Senior Data Engineers, fostering best practices and ensuring high-quality engineering outputs.Provide technical direction, oversight, and guidance across all data engineering projects.Deliver technical assurance through thorough reviews of solution designs, data pipelines, architectures, and implementation strategies.Oversee the design and implementation of scalable, cloud-native data solutions utilizing Azure technologies.Ensure adherence to best practices for ETL/ELT, data modeling, data lakes, governance, and optimization across engineering teams.Support the Data & AI Technical Director in shaping the engineering strategy, standards, and capability roadmap.Act as a senior technical advisor to clients, translating business needs into modern data engineering solutions.Facilitate discovery sessions, workshops, and technical design reviews with stakeholders.Assist the sales team by providing technical expertise during client discussions and contributing to solution design.

Join Debenhams Group, a vibrant collection of five leading brands: boohoo, boohooMAN, PrettyLittleThing, Karen Millen, and Debenhams. We also operate a dynamic marketplace model and a burgeoning financial services division, making every day exciting!We are at the forefront of online fashion, constantly evolving and redefining industry standards. With ambitious goals and a high-energy environment, our mission is to deliver unparalleled propositions to our customers in the most engaging, innovative, and efficient ways.We embrace change, innovation, and speed. Each challenge is an opportunity to excel, and every day provides the chance to push the limits. If you are someone who thrives in a fast-paced, high-growth setting and is eager to shape the future of retail, you will thrive here. We seek visionaries, disruptors, and doers—individuals who bring fresh perspectives, boundless energy, and a strong desire to make a difference.Be bold. Be innovative. Be part of our community. THE ROLEAs the Data Insights Lead, you will be a storyteller, transforming intricate digital marketing data into actionable insights that drive revenue and maximize marketing ROI. You will supervise performance analytics and reporting utilizing tools like Google Analytics, Tableau, and Power BI while identifying market trends to enhance customer journeys.WHAT YOU’LL BE DOINGDevelop and manage the comprehensive marketing measurement strategy across all brands.Collaborate closely with key brand stakeholders to align on KPIs, performance frameworks, and measurement methodologies.Establish success metrics for campaigns, channels, and customer journeys, ensuring consistency in reporting standards.Lead the integration of AI-driven tools and models to enhance marketing performance, improve attribution accuracy, and provide predictive insights.Manage relationships with third-party vendors.Identify areas where AI can streamline processes, automate insights, or enhance targeting effectiveness.Act as the custodian of first-party marketing data, ensuring frameworks are in place for accurate and compliant data collection and usage.Oversee tracking and reporting of marketing expenditures across App, Web, CRM, and all digital channels.Collaborate with Product, BI, and Data Engineering teams to maintain robust data pipelines and unified data sources.Convert complex datasets into clear, commercially relevant narratives.Present insights to senior stakeholders, influencing decisions on budget allocation, growth opportunities, customer behavior, and channel performance.Facilitate discussions that challenge assumptions and promote data-driven recommendations.Manage, mentor, and develop one direct report, fostering a high-performance culture focused on continuous learning and improvement.

AJ Bell is looking for a Lead Analytics Engineer to join the team in Manchester on a hybrid basis for a 6-month fixed-term contract. This role centers on building and delivering advanced analytics solutions that support key decisions across the business. Role overview The Lead Analytics Engineer will collaborate with colleagues from various departments to turn complex data into clear, actionable insights. The work directly supports AJ Bell’s ongoing commitment to data-informed strategies. What you will do Develop and implement analytics solutions tailored to business needs Work with cross-functional teams to understand requirements and deliver insights Translate complex datasets into information that supports decision-making Contract details 6-month fixed-term contract Based in Manchester with hybrid working options

Salary: £28,300Assessment Centre Dates: JuneStart Date: Monday 7th September 2026 Role overview This Trainee Data Engineer Apprenticeship is based in Manchester and centers on supporting the operations, maintenance, and improvement of interactive investor's Data Platform. Apprentices work alongside experienced Data Engineers, gaining practical experience with data ingestion, transformation, and structuring from sources such as customer activity, transactions, marketing, web analytics, and market feeds. These efforts help drive analytics, Data Science, and machine learning projects across the business. The role offers hands-on development in Python and SQL, and involves working with data pipelines managed by Dagster. Apprentices deliver data to the Snowflake Data Warehouse and follow established standards for data quality, observability, security, and governance. There are opportunities to present data using Streamlit and Power BI, and to support Intelligent Automation projects that bring AI-assisted steps into business workflows. Key responsibilities Support daily Data Platform operations under supervision, including monitoring scheduled jobs, troubleshooting failures using runbooks, and escalating issues when necessary. Contribute to the development and maintenance of ELT/ETL pipelines with Dagster, manage dependencies and alerts, and suggest improvements during code reviews. Help build and maintain data warehouse structures such as star schemas, snapshot tables, and slowly changing dimensions with guidance from the team. Prepare, test, and document reusable SQL queries and data extracts for Data Analysts and business users, ensuring peer review where required. Assist with Data Request tickets by gathering requirements, drafting queries, validating results with reviewers, and documenting outcomes.

About UBDSAt UBDS Group, we are dedicated to enabling organizations to undergo exceptional digital transformations through pioneering technology solutions. Our collaboration spans both public and private sectors as we design and develop products, platforms, and services that truly enhance user experience.We are in search of a Senior Data Engineer who possesses a blend of advanced technical skills and a consulting-oriented mindset. This role is ideal for someone who thrives in uncertain environments, prioritizes user needs, and can transform complex business challenges into scalable, production-ready data solutions.The RoleAs a Senior Data Engineer at UBDS, you will collaborate with clients, designers, analysts, and engineers to conceptualize and implement innovative data solutions from start to finish. You will take the lead on technical strategy, make informed design decisions, and aid teams in delivering high-quality solutions that perform effectively in real-world scenarios.This is a hands-on position that grants you significant influence: you will be building, mentoring others, and assisting clients in recognizing exemplary practices.What You’ll DoTranslate ambiguous business challenges into precise data-driven analytical and technical solutions.Design, construct, and manage production-quality data pipelines, models, and products utilizing SQL, Python, and contemporary data engineering frameworks.Architect across data warehouses, lakehouses, and event-driven architectures, incorporating robust semantic models for analytics and visualization.Deliver solutions on cloud data and AI platforms (AWS/Azure), leveraging solid software engineering, testing, and CI/CD methodologies.Integrate analytics, experimentation, AI/ML, data governance, security, and data quality best practices as standard procedure.Produce clear data visualizations and narratives that empower clients to comprehend and act on insights.Engage closely with users and stakeholders in Agile, client-centric environments, deriving insights from live services and real-world feedback.Exhibit strong critical thinking, meticulous attention to detail, and adherence to industry and governmental standards to create solutions that are straightforward to transition, maintain, and evolve.What We’re Looking ForCore Data & Technical SkillsProficient in converting vague business problems into explicit data, analytical, and technical specifications.Strong expertise in SQL and Python for data modeling, transformation, automation, and analysis.Experience in designing data models and architectures across data warehouses, lakehouses, and event-driven systems, or strong semantic modeling proficiency in Power BI or other visualization tools.Hands-on experience with modern data engineering frameworks, including but not limited to Spark/PySpark, Pandas, and orchestration tools.Familiarity with cloud data and AI platforms on AWS or Azure (e.g., Databricks, Redshift, Bedrock, Microsoft Fabric).Solid foundation in software engineering principles, including Git, testing, and CI/CD practices.

Join Anaplan, a trailblazer in business decision-making optimization, where we harness the power of our AI-enhanced scenario planning and analysis platform. Our mission is to empower our customers to surpass their competition and thrive in the market.At Anaplan, we pride ourselves on our unwavering dedication to customer success and our vibrant Winning Culture.Our clientele includes industry giants like Coca-Cola, LinkedIn, Adobe, LVMH, and Bayer among over 2,400 global partners, all of whom depend on our top-tier platform.Our Winning Culture fuels our team of innovators. We celebrate diverse thoughts and ideas, embrace leadership at all levels, strive for ambitious objectives, and revel in our achievements—big and small.Guided by principles that emphasize strategy, values, and disciplined execution, you'll find inspiration, connection, development, and rewards here. We welcome your uniqueness—come join us and help shape the future together!Team OverviewThe Data Engineering team is pivotal in creating and sustaining the company’s data ingestion and processing pipelines. This role will collaborate closely with data scientists, software engineers, business analysts, and technical leads to establish a dependable data infrastructure that fosters informed decision-making for apparel brands and retailers. The position reports to the Technical Lead within the Data Engineering department.Your ImpactEnhance and develop data ingestion and processing pipelines.Collaborate with product managers, data scientists, and engineers to maintain accurate and robust data infrastructure for managing both first- and second-party data.Partner with technical leads and senior engineers to make informed data architecture decisions and implementation strategies.Support the team in delivering scalable solutions while expanding your expertise.Participate in a collaborative environment that values creativity and continuous improvement.Contribute to the construction and maintenance of data pipelines that support AI and decision-making tools.

Join UBDS Group as a Senior Data Engineer, where you will be at the forefront of designing, developing, and enhancing scalable data platforms within intricate transformation initiatives. This pivotal role demands extensive knowledge in cloud data engineering, architecture, and DevOps methodologies, with proficiency in both AWS and Azure environments.Your contributions will be crucial in driving high-quality data integration, facilitating advanced analytics applications, and delivering solid, reusable data solutions that align with client objectives.Key Responsibilities Architect and implement scalable, automated data pipelines utilizing cloud-native technologies such as Spark, while integrating with on-premise data sources as necessary. Develop and manage event-driven data architectures, maximizing pipeline efficiency, parallel processing, and proactive risk identification (e.g., fraud detection scenarios). Construct and oversee data access layers and ETL frameworks, allowing for both self-service and ad-hoc data asset onboarding. Innovate reusable data engineering patterns and tools to enhance productivity, scalability, and uniformity across projects. Employ DevOps and software engineering best practices, including Infrastructure as Code (IaC) and Continuous Integration/Continuous Deployment (CI/CD) pipelines. Collaborate with multidisciplinary teams to ensure high standards of data integration, governance, and performance. Operate across hybrid cloud frameworks (AWS and Azure), ensuring secure, resilient, and scalable solutions. Assist in the deployment and optimization of data solutions leveraging services such as: AWS: EC2, EMR, S3, Lambda Azure: Blob Storage and associated services Align efforts with client data strategies, governance frameworks, and security requirements. Foster knowledge sharing and skill enhancement within both UBDS and client teams.

Join PA Consulting as a Data Engineer and leverage your expertise in data processing, analysis, and visualization to drive strategic decision-making. In this role, you will collaborate with cross-functional teams to create innovative data solutions that enhance business performance.

Join Our Team as an Azure Data EngineerAt UBDS Group, we empower organizations to achieve impactful digital transformations using cutting-edge technology solutions. Collaborating with clients from both the public and private sectors, we design and implement products, platforms, and services that significantly enhance user experiences.We are on the lookout for a passionate Data Engineer who thrives on working with data and technology to tackle real-world challenges. If you're curious about how things work, enjoy collaborating within delivery teams, and are eager to craft robust and well-architected data solutions, we want to hear from you!Your RoleAs a Data Engineer at UBDS, you will be an integral part of multidisciplinary teams, working alongside clients, designers, analysts, and engineers to deliver state-of-the-art data solutions. Your contributions will include designing and building data pipelines, models, and analytical products while learning from seasoned professionals and adhering to industry best practices.This hands-on role offers the opportunity to develop your technical expertise, collaborate closely with colleagues, and gain valuable experience in deploying solutions that function seamlessly in real-world contexts.Key ResponsibilitiesTranslate business challenges into actionable data, analytical, and technical tasks with guidance from senior colleagues.Design, construct, and maintain data pipelines, models, and products utilizing SQL, Python, and contemporary data engineering tools.Engage with data warehouses, lakehouse platforms, and event-driven architectures, including the development of semantic models for analytics and visualization.Contribute to solutions built on cloud data and AI platforms (AWS/Azure), adhering to sound software engineering, testing, and CI/CD practices.Implement analytics, experimentation, AI/ML integration, governance, security, and data quality measures as part of your team’s delivery.Create clear and insightful data visualizations that empower users to understand and act on findings.Collaborate closely with users and stakeholders in Agile, client-focused settings, leveraging live services and user feedback for continuous improvement.Ensure compliance with relevant industry and government standards, contributing to well-documented, maintainable, and easily transferable solutions.

Role Overview AECOM is seeking a Lead Design Engineer with experience in the water sector to join the Manchester office. This position focuses on leading design projects that support vital water infrastructure. The work aims to improve community resilience and support responsible environmental practices. What You Will Do Lead design efforts for water sector projects, ensuring technical quality and project delivery. Work closely with multidisciplinary teams to develop practical and sustainable engineering solutions. Contribute engineering expertise to address infrastructure challenges and support community needs.

UBDS Group is hiring a Data Engineer with a focus on AWS technologies, based in Manchester, England. This role suits professionals experienced in building and optimizing data platforms, especially those with a background in cloud data pipelines and distributed data processing. The team values engineers who deliver scalable, reliable solutions and who take pride in maintaining high data quality. What You’ll Do Design, build, and maintain data pipelines and transformation workflows using modern data engineering tools. Develop and manage both batch and real-time data processing solutions. Create curated, reusable data assets to support analytics and reporting. Ensure data quality, integrity, and consistency across multiple sources. Work with data from APIs, file-based ingestion (such as SFTP), and event streams (including Kafka). Tune pipelines for performance, scalability, and cost efficiency. Support the evolution of data platform components and reusable engineering patterns. Follow established architecture and engineering standards. Collaborate with DevOps engineers, testers, and architects to deliver end-to-end solutions. Help onboard new data sources and integrate them into existing platforms. Share knowledge and contribute to continuous improvement within engineering teams. Key Technologies Core Stack AWS (main platform): S3, EMR, Glue Kafka for event-driven processing Python and PySpark for engineering and transformation tasks Terraform for infrastructure as code GitHub for version control and CI/CD Additional / Nice-to-Have Experience Azure data platforms Databricks or Microsoft Fabric Multi-cloud architectures and related tooling This position offers the chance to work alongside a multidisciplinary team, developing data assets that support both operational and analytical needs. UBDS Group values engineers who contribute to knowledge sharing and ongoing improvement across the team.

At Turnitin, we recognize that AI and data science are fundamental to our achievements and ambitious product strategy. As a Senior AI Data Engineer, you will join a dynamic global team of proactive and independent professionals dedicated to crafting sophisticated, well-structured AI and data systems. You will be at the forefront of developing our next-generation data and AI pipelines, significantly scaling our team's impact. You'll collaborate across various teams within Turnitin to integrate AI and data science into a diverse range of products aimed at enhancing learning, teaching, and academic integrity.Key Responsibilities:AI Data Infrastructure & Pipeline Management: Design, build, and operate scalable real-time data pipelines that facilitate ongoing Applied AI model training. Implement and maintain robust data infrastructure utilizing AI techniques and engineering best practices to ensure continuous model improvement.Data Collection: Lead efforts to collect, normalize, and store data from various sources, including external LLM providers.Collaboration: Work closely with AI R&D, Applied AI, and Data Platform teams to ensure smooth data flow and adherence to quality standards. Collaborate with stakeholders to curate and catalog high-quality datasets that support Applied AI retraining workflows and business goals.Support for AI R&D: Contribute to AI Research & Development initiatives by leveraging advanced data warehousing and engineering technologies. Engage in exploratory data projects to extract insights from Turnitin's extensive datasets.Communication: Foster clear communication across teams, aligning with the company vision while sharing insights on data infrastructure requirements and potential innovations.Technology Evolution: Stay updated with emerging tools and methodologies in AI data engineering, providing recommendations to enhance our AI data infrastructure and capabilities.

Join AECOM as a Principal Mechanical Engineer specializing in Data Centres. In this pivotal role, you will spearhead the design and implementation of cutting-edge mechanical systems within our data centre projects. You will collaborate with cross-functional teams to ensure the highest standards of efficiency and sustainability are met.Your expertise will be crucial in leading project initiatives, optimizing mechanical designs, and ensuring compliance with industry standards. We are seeking a proactive leader with a passion for innovation and a commitment to delivering exceptional results.

Mark43 is an approved employer in Canada, the UK, and 36 U.S. states, including Alabama, Arizona, California (excluding San Francisco), Colorado, Connecticut, Washington D.C., Florida, Georgia, Iowa, Idaho, Illinois, Indiana, Kansas, Massachusetts, Maryland, Maine, Michigan, Minnesota, Missouri, North Carolina, Nebraska, New Hampshire, New Jersey, New Mexico, New York, Ohio, Oklahoma, Oregon, Pennsylvania, South Carolina, Tennessee, Texas, Utah, Virginia, Vermont, Washington, Wisconsin, and West Virginia. Before applying for a remote role, please ensure that you can perform the position in one of the states listed above. State locations and specifics are subject to change as our hiring requirements shift. Applicants must be authorized to work for any employer in the country in which the role is being hired. We are unable to sponsor or take over sponsorship of an employment visa at this time.At Mark43, our mission is to empower communities and their governments through innovative technologies that enhance safety and quality of life for everyone. We develop robust, scalable, and sophisticated software that sets a new benchmark for the tools relied upon by first responders. Our team is diverse, and we are dedicated to fostering inclusivity and a variety of perspectives.We are currently seeking a Data Migration Engineer (ETL Developer) to join our dynamic team. In this pivotal role, you will play a crucial part in the customer transition experience, designing and implementing complex data migrations that allow agencies to confidently adopt Mark43’s systems. You will collaborate with internal teams and external partners to ensure the seamless delivery of clean, meaningful, and high-quality data into Mark43 products.This position is ideal for individuals who enjoy solving intricate challenges, are passionate about working with public safety data, and thrive in a role that balances technical expertise with customer interaction.What You Can Expect to Work On:If you were part of our team last week, you would have:Designed and executed a SQL-based ETL process to migrate decades of public safety data from a legacy system into the Mark43 platform.Collaborated with our data migration contractors, reviewing technical deliverables and providing guidance to help them succeed.

bet365 is looking for a Data Engineer with experience in Google Cloud Platform (GCP) to join the team in Manchester. This position centers on building and maintaining data pipelines that support analytics projects across the company. Role overview The Data Engineer will design, develop, and manage data pipelines on GCP. The work involves ensuring data flows efficiently and remains accessible for analytics teams. A strong background in cloud technologies is essential for optimizing the company's data infrastructure. What you will do Design and construct data pipelines using Google Cloud Platform tools Maintain and improve existing data infrastructure Support seamless integration and data accessibility for analytics teams Requirements Experience with Google Cloud Platform in a data engineering context Background in designing, building, and maintaining data pipelines Ability to optimize data infrastructure for analytics needs

Join the innovative team at UBDS Group as a Data Engineer specializing in Multi-Cloud environments. In this pivotal role, you will play a crucial part in designing, developing, and optimizing data platforms that drive complex transformation programs. We are looking for an engineer with practical experience in data pipelines, cloud platforms, and ETL processes who is eager to enhance their skills in a collaborative and results-oriented environment.Your contributions will be vital in building dependable, scalable data solutions that facilitate analytics, reporting, and operational use cases.Key Responsibilities Design and maintain data pipelines and ETL processes leveraging cloud-native technologies (e.g., Spark). Assist in integrating cloud and on-premise data sources effectively. Support the implementation of event-driven data pipelines to enhance performance and scalability. Help develop data access layers and reusable ETL components. Facilitate the onboarding of new data sources, focusing on the preparation and transformation of datasets. Collaborate with senior engineers to uphold data engineering best practices and standards. Contribute to CI/CD pipelines and Infrastructure as Code (IaC) initiatives. Operate within AWS and Azure environments, assisting with the deployment and maintenance of data solutions. Engage with services such as: AWS: EC2, EMR, S3, Lambda Azure: Blob Storage and associated services Work alongside cross-functional teams to ensure data quality, reliability, and performance. Participate in knowledge-sharing and continuous improvement activities.Skills & ExperienceEssential Proven experience in data engineering or ETL development. Familiarity with data pipelines and data processing frameworks (e.g., Spark or similar). Proficiency in cloud platforms (AWS and/or Azure; experience with both is highly desirable). Solid understanding of ETL processes and data integration concepts. Basic experience with Infrastructure as Code (e.g., Terraform) and/or CI/CD tools (e.g., GitLab). Knowledge of key services such as: AWS: EC2, S3 (with EMR/Lambda experience being beneficial) Azure: Blob Storage Understanding of version control and collaborative development practices. Awareness of data security and governance considerations.Desirable Experience with event-driven architectures or streaming technologies. Background in data access layers or data platform design. Experience in public sector or regulated environments. Knowledge of cloud networking fundamentals. Familiarity with fraud analytics or large-scale data use cases.Security Requirements This position requires Security Clearance (SC). Candidates must be eligible and willing to undergo the clearance process.

Join Ramboll as the Director of Structural Engineering for our Data Centre Sector. This pivotal role requires a visionary leader with extensive experience in structural design and engineering. You will spearhead projects in the rapidly evolving data centre industry, ensuring innovative solutions that meet our clients' needs.Your leadership will guide a talented team, driving excellence in project delivery while fostering a culture of collaboration and continuous improvement.

UBDS Group is hiring a Test Engineer focused on automation and data validation to support ongoing data and cloud transformation projects. This role is based in Manchester, England. Role Overview This position suits an engineer with hands-on experience in ETL testing, test automation, and cloud data platforms. The main focus is to design, build, and improve automated testing solutions that protect data quality and ensure reliable outcomes for clients. Main Responsibilities Create and maintain automated tests for data pipelines, ETL workflows, and cloud-based systems. Perform data validation to confirm accuracy, completeness, and integrity of data sets. Manage test automation using Python and pytest, applying TDD and BDD (Behave) practices. Generate and organize test data for ETL and data warehouse validation tasks. Upgrade automation frameworks to boost efficiency and promote reusability. Work closely with engineering teams to introduce shift-left testing approaches. Participate in peer code reviews and contribute to CI/CD workflows. Support cloud-based solutions on platforms such as AWS and Azure, and work with technologies like Kafka. Spot and act on opportunities to standardize and reuse testing strategies. Align testing methods with client objectives and project timelines. Share knowledge within teams to encourage ongoing skill development.

The Junior SQL Support Engineer role in the Regulatory Data team at bet365 centers on maintaining data accuracy and supporting compliance requirements. This entry-level position is based in Manchester and offers hands-on experience with SQL and regulatory reporting processes. Role overview This role focuses on supporting SQL queries and assisting with daily data management tasks. The position involves working closely with team members to improve regulatory data processes and ensure accurate reporting. What you will do Support and troubleshoot SQL queries as part of the regulatory data workflow. Assist with routine data management and integrity checks. Collaborate with colleagues across teams to strengthen regulatory reporting capabilities. Who this role suits This position is well suited for candidates starting their careers who want to develop SQL skills and gain experience in regulatory data management. An eagerness to learn and contribute to team goals is important.