Senior Python Data Scraping Engineer

Role Overview Jobgether is hiring a Senior Python Data Scraping Engineer for a freelance position based in Germany. This role focuses on designing, building, and maintaining web data extraction systems that serve both AI-powered and human-driven workflows. The position is fully remote within Germany and values contributors who work independently and pay close attention to detail. What You Will Do Create and improve scalable Python scraping solutions for dynamic websites and large datasets. Produce structured, accurate data ready for advanced analytics and AI use cases. Adjust scraping methods to keep up with changing web technologies and site structures. Collaborate with AI agents, maintaining high standards through validation and quality checks. Combine hands-on coding with analytical problem-solving to tackle complex extraction issues. Requirements Advanced Python skills, especially in web scraping and automation. Background in extracting data from dynamic websites and managing large-scale projects. Ability to adapt techniques as web environments evolve. Comfort working independently and taking responsibility for project outcomes. Keen attention to detail and a strong focus on data quality. Work Arrangement Freelance contract Remote work within Germany Flexible schedule with emphasis on autonomy

Mindrift is looking for a Freelance Python Data Scraping Engineer to join its remote team, supporting projects from Stuttgart, Baden-Württemberg, Germany. This part-time, freelance position centers on the Tendem project, where engineers contribute as "AI Pilots" to streamline and improve data workflows. Role overview This role focuses on extracting, processing, and validating data from complex and dynamic web sources. The engineer collaborates with Tendem Agents, applying analytical skills and domain expertise to ensure data is accurate and relevant. Work is fully remote and designed for professionals with a strong background in web scraping and data processing. What you will do Lead data extraction projects on complex websites, delivering structured datasets that meet quality standards. Use internal tools such as Apify and OpenRouter, along with custom workflows, to collect, verify, and process data according to project needs. Develop and refine scraping methods for dynamic and interactive sites, including those with JavaScript-rendered content or frequent changes. Maintain high data quality through validation checks, consistency controls, formatting compliance, and systematic verification. Optimize large-scale scraping using batching or parallel processing, and build resilience against minor site structure changes. Requirements Minimum 3 years of hands-on experience in data engineering, web scraping, and automation. Compensation Contributors can earn up to $32 per hour, depending on project complexity, performance, and expertise. Actual rates may vary between projects. How to apply Submit an application through the provided link. Qualified candidates are matched with projects that suit their technical strengths and can set their own schedules. Work includes programming, automation, and refining AI outputs, supporting the practical use of AI in real-world scenarios.

Are you ready to take your career to the next level? Join dev2 as a Senior Data Engineer specializing in Python and play a pivotal role in our data-driven projects. You will work closely with cross-functional teams to design and implement scalable data solutions that drive innovation and efficiency.In this role, you will:Design, develop, and maintain data pipelines and ETL processesCollaborate with data scientists and analysts to optimize data storage and retrievalEnsure data quality and integrity through rigorous testing and validationContribute to architectural discussions and guide best practices

About ClickHouseListed among the top innovators on the 2025 Forbes Cloud 100 list, ClickHouse is a leading, rapidly expanding private cloud company. With over 3,000 customers and an astonishing annual recurring revenue growth exceeding 250%, ClickHouse is at the forefront of real-time analytics, data warehousing, observability, and AI workloads.Our momentum was recently reinforced by a successful $400 million Series D funding round. In the past three months, renowned clients such as Capital One, Lovable, Decagon, Polymarket, and Airwallex have adopted or expanded their use of our platform. These new clients join a prestigious roster of AI innovators and global brands including Meta, Cursor, Sony, and Tesla.Join us in our mission to revolutionize the way companies leverage data!The Connectors team serves as the vital link between ClickHouse and the vast data ecosystem. We develop and maintain integrations that make ClickHouse accessible to millions of developers, data practitioners, and AI agents worldwide—ranging from high-level data visualization plugins (like Tableau, PowerBI, Superset, Metabase) to connectors for data frameworks (Apache Spark, Flink, Kafka Connect, Fivetran), orchestration platforms, and AI tools.Our work is pivotal in shaping how organizations process massive datasets—enabling real-time analytics platforms to ingest millions of events per second, observability systems to monitor global infrastructure, and increasingly, AI-driven data applications that redefine team collaboration with data. We work closely with the open-source community, our internal teams, and enterprise users to ensure that ClickHouse integrations lead the way in performance, reliability, and developer experience.About the RoleAs a Senior Software Engineer specializing in Python and the Data Ecosystem, you will be a key contributor, responsible for owning and advancing essential components of ClickHouse's data engineering ecosystem. This role exists at the crossroads of high-performance database engineering and enhancing developer experience. You will create tools that empower Data Engineers and Data Scientists to fully leverage ClickHouse's speed and scale within the frameworks they are already familiar with.We are seeking an individual who has direct experience as a Data Engineer or Data Scientist. The landscape for data practitioners is evolving rapidly; databases have progressed beyond mere query targets—they are now integral components of AI-powered workflows, serving as vector stores for RAG pipelines, backends for LLM-powered agents, and real-time feature stores for ML inference. You comprehend these workflows not from an outsider's perspective, but from personal experience within them. Your role is not just to build integrations; you will contribute product-level insights that enhance user experience.

Data Engineer (Python) Company Overview Orcrist Technologies is at the forefront of innovation with the Orcrist Intelligence Platform (OIP), a cutting-edge data intelligence system built on Kubernetes. Our platform is available as a SaaS solution or can be deployed on-premises, including air-gapped setups. We manage both streaming and batch data pipelines that empower search functionalities, machine learning enrichment, and investigative workflows for our mission-critical clientele. Role Summary As a Data Engineer, you will play a pivotal role in quickly validating new data initiatives from inception to deployment, ensuring they are adoptable and scalable. In this innovative environment, you will prototype effective connectors and pipelines, generate performance assessments, and create handoff packages for productization by our Foundation or delivery team. Key Responsibilities Prototype ingestion and connector patterns (batch and streaming) utilizing NiFi, Kafka, Kafka Connect/Streams, and Change Data Capture approaches. Design schemas and data models that are both prototype-grade and easily adoptable, ensuring semantic clarity and a disciplined approach to evolution. Develop incremental lakehouse datasets using Hudi, Iceberg, and Delta patterns, producing outputs for real-world latency and throughput evaluations. Implement data quality and provenance considerations early in the process, incorporating checks, metadata hooks, and operational basics. Containerize and deploy prototypes on Kubernetes, providing minimal runbooks and configurations for seamless adoption. Create adoption artifacts including schemas, reference implementations, technical design notes, and a backlog for integration. Qualifications Minimum of 3 years of experience in data engineering with a proven track record of delivering real-world data pipelines beyond ad-hoc scripts. Proficient in Python and SQL, skilled in building transformations, validation tools, and pipeline integration code. Solid understanding of streaming and Change Data Capture fundamentals, along with experience in the Kafka ecosystem. Familiar with lakehouse architectures and query layers (e.g., Hudi, Iceberg, Delta, Trino, Hive, Postgres) and their role in making datasets accessible. Comfortable working in Kubernetes and container environments and adept at documenting technical decisions clearly. Must be eligible to work in Germany; EU/NATO citizenship is preferred, and export-control screening will apply. Preferred Qualifications Experience with data quality tools such as Great Expectations or metadata/lineage platforms (OpenMetadata, DataHub, Atlas). Experience with on-premises or air-gapped deployments and awareness of governance and policy for regulated environments. Proficiency in German (B1+) and familiarity with OSINT, GEOINT, or multi-INT data structures. What We Offer A modern data stack with real-world constraints: Kafka, NiFi, and more.

We are seeking a highly skilled Senior Data Engineer to join our dynamic team at repriskag in Berlin. In this pivotal role, you will be responsible for designing, building, and maintaining scalable data pipelines that support our data-driven decision-making processes. You will work closely with data scientists, analysts, and other stakeholders to ensure data quality and accessibility.

Delivery Hero is looking for a Senior Python AI Engineer to focus on logistics, data, and machine learning projects in Berlin. This position centers on developing AI-powered tools that support and improve delivery operations. Role overview This role involves designing and building applications using Python, with a strong emphasis on artificial intelligence and data science. The main goal is to create solutions that make logistics processes more efficient and help raise customer satisfaction. What you will do Develop and maintain AI-driven applications tailored for logistics challenges Apply machine learning techniques to optimize delivery workflows Work closely with teams to identify and solve data-related problems Requirements Extensive experience with Python Background in artificial intelligence, data science, and machine learning Ability to build solutions that improve operational efficiency

peec is building and improving the backend systems that support its products. As a Senior Python Engineer based in Berlin, this position centers on designing, developing, and maintaining complex backend infrastructure to ensure reliable and high-quality delivery. Role overview This role involves full ownership of backend systems, from initial planning and architecture through to implementation and deployment. The work spans API design, distributed systems, and ongoing performance improvements. What you will do Lead the end-to-end development of backend systems, including scoping, architecture, execution, and delivery. Design and implement scalable APIs and distributed systems that meet high performance standards. Work closely with Product, Frontend, and Design teams to translate product ideas into solid backend solutions. Explore and apply modern backend technologies to improve architecture and the developer experience. Help evolve backend architecture and infrastructure to support fast iteration and growth. Identify areas for system and developer experience improvement, and drive initiatives to address them. Requirements Significant experience with Python and backend development is essential. Strong skills in API design, distributed systems, and collaborating across teams will be important for success in this role.

About dltHubdltHub is at the forefront of innovation, leveraging the dlt library, a leading open-source Python tool for data loading with approximately 4 million downloads monthly. We are on a mission to develop the next generation of AI-native data tools built upon our robust open-source foundation.Our team is agile and thrives on experimentation, automation, and user-centric AI workflows. Founded by seasoned data scientists and machine learning engineers, we have our roots in Berlin and a burgeoning presence in the United States.We are supported by prominent investors including Bessemer Venture Partners, Foundation Capital, Dig Ventures, and technical pioneers from Hugging Face, Instana, Matillion, and more. Our core values drive us: we communicate with courage, prioritize meaningful projects, automate continuously, maintain high energy, honor our commitments, and achieve success collectively. About the RoleAs we continue to enhance our AI-first data platform, we are seeking a passionate and entrepreneurial software engineer in Berlin. This role offers the opportunity to tackle data platform challenges related to ingestion, storage, performance, reliability, observability, and AI-enhanced developer experiences. If you are driven, eager to collaborate with a motivated team, and prioritize high-quality code and innovative data systems, this position is tailored for you.

SmartRecruiters is seeking a Senior Python Software Engineer to join its Engineering team in Germany. This position centers on designing and building software that advances company objectives. The role involves tackling complex problems and working closely with other engineers. Key responsibilities Design and develop software solutions using Python Collaborate with team members to resolve technical challenges Share ideas to enhance products and internal processes What we’re looking for Deep experience with Python programming Ability to work through complex technical issues Interest in working collaboratively to achieve team goals

Join our dynamic team at dev2 as a Senior Python Software Engineer. In this pivotal role, you will leverage your expertise in Python to develop innovative software solutions that meet the evolving needs of our clients. Your contributions will directly impact project outcomes and enhance our technology stack.We are looking for a passionate individual who thrives in a fast-paced environment, possesses strong problem-solving skills, and is eager to collaborate with cross-functional teams to deliver high-quality software.

Join our dynamic Platform tribe at SumUp, where your expertise will help us create an innovative self-service, AI-Ready Data Platform. Our Data Platform team plays a crucial role in supporting our ambitious ventures into Data, AI, and real-time analytics. We are committed to building robust and scalable infrastructure that empowers our global data community.Your ResponsibilitiesDesign and sustain an exceptional data infrastructure that underpins vital processes.Enhance platform self-service capabilities by developing features for ETL and orchestration.Streamline processes to maximize compute resource efficiency and minimize costs.Implement automation solutions to ensure our infrastructure's availability around the clock.Participate in the design and rollout of new services within our platform ecosystem.

Join Synthflow AI, a revolutionary no-code platform designed for deploying voice AI agents that transform contact center operations and business process outsourcing (BPO) at scale. We empower mid-market and enterprise companies to efficiently manage routine calls, ultimately saving valuable time and resources for their teams.Our voice AI agents have made a significant impact:Saved over 5 million hours in contact center operationsIncreased call answering rates by 35% compared to non-AI operatorsHandled 45 million calls with an impressive 99.9% uptimeSupported by leading investors such as Accel, Atlantic Labs, and Singular, we are proud to serve over 1,000 customers and drive an industry shift towards advanced and accessible conversational AI.About the Role:We are seeking a Senior Python Engineer to develop robust, production-grade software and backend systems, focusing on databases, scalable APIs, and core engineering principles.Your Key Responsibilities:Design, develop, and maintain our backend infrastructure, ensuring scalability, performance, and reliability.Create and enhance data models and persistence layers with a strong emphasis on PostgreSQL best practices, including schema design, migrations, query optimization, indexing, and transactions.Develop and support enterprise-grade APIs using FastAPI, focusing on versioning, authentication/authorization, and observability.Participate in pair programming to foster code quality and team collaboration.Manage deployments and environments with Kubernetes and GitOps practices (ArgoCD).Conduct code reviews, offering constructive feedback and ensuring adherence to best practices.Contribute to end-to-end product delivery, including occasional front-end feature development when necessary.Qualifications:5+ years of experience as a Python developer, successfully delivering production systems.Proven expertise in building scalable and extensible data infrastructures.Strong familiarity with relational databases, particularly PostgreSQL (performance tuning, migrations, transactions).Experience in developing and maintaining enterprise-grade APIs using FastAPI.Solid understanding of engineering fundamentals and best practices.

Distribusion is the premier global marketplace for ground transportation, providing travelers with effortless online access to a variety of ground transport options from search to ticket purchase. Our state-of-the-art B2B technology platform connects bus, rail, and ferry operators across 70+ countries to major online retailers, including Google Maps and Booking.com.We are redefining the future of travel by constructing the largest global network of transport providers and retailers. Having achieved a remarkable 10x growth in the past year, we are one of the fastest-growing startups in the travel sector. Supported by four prominent venture capital firms (TQ Ventures, Creandum, Northzone, and Lightrock), and following our recent Series C funding round of $80 million, we are poised to expand even further.We are on the lookout for a talented Senior Python Software Engineer to join our Demand Team. This team is crucial to our product, acting as the first touchpoint for end customers. Therefore, delivering a seamless and efficient search experience is vital for driving sales, ensuring customers can swiftly find optimal choices and successfully complete their bookings.Key Responsibilities:Write clean, efficient, and thoroughly documented Python code.Develop and maintain backend services and APIs for web projects.Implement backend solutions for business-critical applications.Debug, test, and optimize applications for enhanced performance and scalability.Ensure system uptime of at least 99.9%.Engage in code reviews and contribute to best practices.Work Environment:Our headquarters is located in Berlin, where the team frequently meets. However, we embrace a remote-first culture, with team members dispersed globally.

Transform the Travel Experience Become a pivotal part of GetYourGuide, where we connect travelers with extraordinary experiences across the globe. Our commitment to delivering unique activities people can trust empowers millions to embark on unforgettable journeys. Are you ready to make your mark? If you're eager to explore your potential within a community of innovators, consider joining us at our Berlin headquarters or any of our offices worldwide, from New York to Bangkok. Take your first step by visiting getyourguide.careers. Team Mission As part of the Supply Data Engineering team within the strategic Flywheel Data Engineering group, our goal is to establish robust data foundations and self-service capabilities for the supply side of our marketplace. We aggregate, integrate, and structure vital internal and external datasets into trustworthy tables, metrics, and data products, facilitating supplier onboarding, enhancing their success, and optimizing supply operations—all through close collaboration with Supply, Product, and Data teams. Your Mission Own the Data Pipeline: Provide reliable, high-quality data solutions that empower supply decision-making. Act as a Trusted Advisor: Collaborate with Supply, Product, and Data teams to translate business needs into tangible outcomes and promote the utilization of self-service data capabilities. Champion Technical Excellence: Actively manage and enhance the team’s data ecosystem with a long-term focus on architecture, testing, monitoring, performance, and cost efficiency. Enhance Existing Systems: Lead the design of new capabilities and continuously refine deployed systems through strategic refactoring, reliability enhancements, and simplification. Elevate Team Performance: Raise engineering standards through comprehensive code reviews, design contributions, and hands-on support—removing obstacles for fellow engineers and sharing knowledge to enhance team capability. Define Best Practices: Influence technical standards and decisions with an emphasis on simplicity, reusability, and maintainability.

Jobgether is looking for a Senior Data Engineer based in Germany. This position centers on building and refining data systems that support business intelligence and analytics. Role overview The Senior Data Engineer will design and implement data solutions that help the company gain actionable insights. Collaboration with teams across the organization is part of the day-to-day, with a focus on improving data architecture and supporting analytics functions. What you will do Develop and maintain scalable data pipelines Optimize data flow across systems Ensure data integrity for reliable analytics Work closely with colleagues from different departments to enhance data-driven decision making Impact Your work will help Jobgether use data more effectively to guide strategy and improve how the business operates.

Intellectsoft is looking for a Senior Python Developer to join a remote team supporting a BaFin-regulated fintech client based in Germany. The client specializes in Crypto-as-a-Service infrastructure for institutional partners. This role is open to candidates located in Georgia and offers the chance to work fully remotely. What you will do Develop scalable backend systems using Python to support financial services Create and maintain integrations that bridge traditional finance with Web3 technologies Operate within a high-security environment shaped by regulatory requirements Who this role suits This position fits experienced Python developers who are comfortable tackling complex backend projects and financial integrations, particularly in regulated environments. Intellectsoft works with clients across North America, Latin America, the Nordics, the UK, and Europe, delivering digital solutions in sectors such as fintech, healthcare, education technology, construction, and hospitality. Clients include startups as well as Fortune 500 companies like Jaguar Motors, Universal Pictures, and Harley-Davidson. More information can be found at www.intellectsoft.net.

Join Skalar as a Senior Data Engineer, where you will be instrumental in shaping data solutions that drive impactful analytics. You will design, implement, and optimize data pipelines, ensuring the integrity and availability of our data. As a key member of our engineering team, you will collaborate closely with data scientists and analysts to deliver high-quality data products.

Distribusion Technologies stands at the forefront of the global ground transportation marketplace, offering travelers unparalleled access to diverse transport options online, from seamless search functionalities to straightforward ticket purchasing. Our state-of-the-art B2B technology platform bridges the gap between bus, rail, and ferry operators in over 70 countries and leading online retailers such as Google Maps and Booking.com.As we continue to redefine the travel landscape, we are expanding the largest global network of transportation providers and retailers. After experiencing a remarkable 10x growth last year, we are recognized as one of the fastest-growing startups in the travel sector. With the support of four prominent venture capital firms (TQ Ventures, Creandum, Northzone, and Lightrock) and our recent successful $80 million Series C funding round, we are poised for significant advancements.We are currently seeking a dynamic Senior Python Software Engineer to join our Portal Team. In this pivotal role, you will play a crucial part in developing our core B2B product, which connects global carriers, retailers, and travel partners to Distribusion’s robust infrastructure. The Portal serves as the central interface within our ecosystem, facilitating high-performance booking and analytics tools, as well as identity management and operational dashboards. You will navigate multiple domains, influence architectural decisions, integrate complex workflows, and collaborate with teams across Data, Platform, and E-commerce to create a seamless and scalable user experience. We are on the lookout for a developer who thrives on ownership, enjoys tackling complex challenges, and is passionate about building the foundation of how the world books ground transportation.

jobgether is looking for a Senior Full-Stack Engineer based in Germany with strong experience in Python and React. This role centers on building and improving software that supports the company’s platform. Role overview This position involves developing features from initial idea through to deployment. Collaboration with colleagues from different teams is a key part of the job, with a shared focus on delivering reliable code and a smooth user experience. What you will do Work with Python and React to create and maintain platform features Contribute to project planning, development, and release Collaborate with cross-functional teams to meet project goals Help ensure code quality and a positive experience for users