Web Scraping Engineer

About YipitData:YipitData stands as a premier market research and analytics firm focused on the disruptive economy, recently securing up to $475M from The Carlyle Group, achieving a valuation exceeding $1B.We diligently analyze billions of alternative data points daily, delivering accurate and detailed insights across ridesharing, e-commerce marketplaces, payments, and more. Our on-demand insights team leverages proprietary technology to identify, license, clean, and analyze data that the world's largest investment funds and corporations rely upon.For over three consecutive years, we have been honored as one of Inc’s Best Workplaces. As a rapidly growing technology organization backed by The Carlyle Group and Norwest Venture Partners, we have offices in New York City, Austin, Miami, Denver, Mountain View, Seattle, Hong Kong, Shanghai, Beijing, Guangzhou, and Singapore. We promote a people-centric culture that emphasizes mastery, ownership, and transparency.Why You Should Apply NOW:High Impact: Your contributions will significantly influence vital reports and strategic decisions across multiple business units.Exciting Challenges: Engage in the design of resilient web scrapers, navigate complex website structures, and optimize extensive data extraction processes.Growth Opportunities: As an early member of our expanding Web Scraping Engineering team, you will have a substantial voice in shaping our strategies, processes, and team culture.About The Role:We are on the lookout for a Web Scraping Engineer to join our dynamic engineering team. In this hands-on position, you will take charge of designing, building, and maintaining robust web scrapers that fuel critical reports and enhance customer experiences across our organization. You will tackle complex, high-impact scraping challenges and collaborate closely with cross-functional teams to ensure our data ingestion processes are resilient, efficient, and scalable, delivering high-quality data to our products and stakeholders.

Mindrift connects professionals with AI projects for leading technology companies. The platform focuses on advancing Generative AI by involving specialists from around the world. Role overview This Freelance Python Data Scraping Engineer position supports the Tendem project. The role is remote and part-time, open to candidates based in Hyderabad, Telangana, India, or those working remotely from there. Work centers on improving hybrid AI and human-driven data scraping workflows. As an AI Pilot, collaboration with Tendem Agents is a key part of the job. The position blends hands-on technical work with critical thinking and quality control, aiming for precise and actionable web data extraction. What you will do Manage end-to-end data extraction on complex websites to deliver accurate, structured datasets. Work with internal tools such as Apify and OpenRouter, plus custom methods, to collect, validate, and process data to meet project needs. Adjust scraping strategies for dynamic, interactive, and JavaScript-heavy web content, responding to changes in site behavior as they arise. Maintain high data quality through validation, consistency checks, compliance with formatting standards, and thorough verification before delivery. Optimize large-scale scraping by batching or parallel processing, monitoring for failures, and adapting to changes in site structure. Requirements Minimum 3 years of experience in data engineering, web scraping, automation, or similar fields. Strong Python skills and experience with web scraping frameworks. Demonstrated ability to extract data from complex and dynamic web applications. Keen analytical skills and attention to detail for data quality. Clear written and verbal communication abilities. Compensation Contributors may earn up to $12 per hour, depending on experience and pace. Compensation varies by project scope, complexity, and expertise required. How to get started Apply to this posting and complete the qualification process to join. Projects are matched to technical strengths, and schedules are flexible. Tasks include coding, automation, and refining AI outputs, all aimed at improving AI capabilities and practical applications.

Role overview Jobgether is hiring a Senior Python Data Scraping Engineer for a remote freelance role based in India. The focus is on building and maintaining large-scale web data extraction pipelines. Responsibilities include collecting structured data from dynamic websites and managing complex, high-volume datasets. What you will do Design and implement Python-based web scraping systems that can scale. Adjust scraping strategies as website structures and technologies change. Work in workflows that combine AI tools with human oversight. Validate the accuracy and quality of extracted data. Collaborate with AI-driven agents to provide reliable datasets for analytics and AI projects. Requirements Advanced skills in Python programming for web scraping. Experience working with dynamic websites and managing large datasets. Ability to update techniques as web technologies evolve. Strong attention to detail and a focus on data quality. Comfort working independently in a remote freelance setting. Location This position is fully remote and open to candidates based in India.

We are seeking a talented Python Scraping Engineer to join our dynamic team at Smart Working Solutions. In this fully remote role, you will leverage your expertise in Python and web scraping technologies to develop reliable data extraction solutions that will help drive our business forward.

Join our dynamic team at Weblee Technologies as a Web Design Engineer! We are looking for a creative and skilled individual who is passionate about designing user-friendly and aesthetically pleasing web interfaces. In this role, you will collaborate with cross-functional teams to create innovative web solutions that enhance user experience.

ABOUT THE ROLEWe are looking for a dynamic Web Engineering Manager to spearhead and nurture our web development team based in Pune, as an integral part of our global Web organization. This position emphasizes effective people management, project delivery, and overall team well-being, empowering engineers to excel while ensuring alignment with international standards and technical directives.In this role, you will collaborate closely with Lead Developers, QA teams, and regional stakeholders to guarantee seamless delivery across an extensive portfolio of corporate and marketing websites, which includes both legacy systems and a contemporary web stack.As part of the onboarding process, the successful candidate will engage in a two-week orientation at our London office to align on operational methodologies, standards, tools, and key stakeholders.KEY RESPONSIBILITIESTeam Leadership & People ManagementLead and support a talented team of web developers based in Pune.Facilitate regular one-on-ones, performance evaluations, and career development initiatives.Cultivate a positive, inclusive, and high-performance team culture.Assist in onboarding and mentoring new team members.Serve as the primary people manager for the Pune web development team.Delivery & ExecutionOversee daily project delivery, capacity planning, and prioritization of workloads.Ensure timely delivery of work adhering to established quality standards.Collaborate with Project Management, QA, and DevOps teams to facilitate smooth releases.Identify potential delivery risks and escalate issues as needed.Help establish and maintain consistent development processes.Collaboration & AlignmentWork closely with Lead Developers to align with architectural and technical standards.Engage with global stakeholders from London, the US, and Europe.Support recruitment efforts for the Pune team alongside global leadership.Act as a local point of escalation for delivery or personnel-related matters.Continuous ImprovementAssist in enhancing development practices and team workflows over time.Promote the adoption of innovative tools, platforms, and processes.Encourage knowledge sharing and continuous learning within the team.

COMPANY OVERVIEWKKR is a premier global investment firm specializing in alternative asset management and comprehensive capital markets and insurance solutions. We are committed to delivering attractive investment returns through a patient and disciplined approach, by leveraging our world-class talent and fostering growth in our portfolio companies and the communities we serve. Our investment funds span private equity, credit, and real assets, and we collaborate with strategic partners to manage hedge funds. Our insurance subsidiaries, under the Global Atlantic Financial Group, provide a suite of retirement, life, and reinsurance products. TECHNOLOGY ORGANIZATION OVERVIEW The Technology team at KKR is instrumental in establishing and maintaining the firm’s technological infrastructure, which includes a globally distributed system, robust information security, and advanced application and data platforms. We cultivate a culture of technological excellence by automating workflows, democratizing data through modern platforms, and pioneering research into Generative AI tools and services. Technology is a critical business enabler at KKR, propelling us towards expansive growth and transformative business processes. Our dedicated Program Management function, along with Product Managers, ensures disciplined execution across various tech teams, consistently delivering outstanding outcomes that meet business needs. TEAM OVERVIEW The Enterprise Systems Engineering team, a vital part of KKR’s Technology organization, specializes in the design, implementation, and continuous enhancement of technology platforms that support essential corporate functions, including Compliance, Legal, Audit, Risk, Finance, Human Capital, Enterprise Services, KKR Public Websites, and Digital Experience. We form close partnerships with functional leaders and stakeholders to gain insights into evolving needs and deliver technology solutions that yield measurable outcomes.Our focus includes:Customizing and implementing enterprise SaaS platforms and tailored applications.Integrating systems across corporate domains to facilitate seamless data flow and automation.Ensuring platform stability, data integrity, and stringent controls in support of corporate objectives.

Join our innovative team at Oleria Security as an App Engineer - Web & Design. In this role, you will be responsible for designing and developing web applications that contribute to our security solutions. We are looking for a passionate engineer who thrives in a collaborative environment and is eager to create secure, user-friendly applications.

Role overview ixigo is looking for a Web Software Engineer based in Gurugram. The position centers on designing and developing web applications that enhance the way users experience ixigo’s products. What you will do Collaborate with colleagues from different teams to deliver dependable and thoughtfully built web solutions Take part in the full development process, from initial concept through to product launch Contribute ideas and code to features that directly impact user engagement

Join our dynamic team at Squircle IT Consulting Services as a Web Application Server Engineer. In this role, you will be responsible for designing, implementing, and maintaining web applications that empower our clients. You will work closely with cross-functional teams to deliver high-quality software solutions that meet business needs.

Join Endeavor IT Solution as a Web Development Intern and kickstart your career in the tech industry! This internship is tailored for engineering students who are eager to gain hands-on experience in web development. You will work alongside our talented team, contributing to innovative projects while enhancing your skills in various web technologies.

We are seeking a talented Screen Scraping Developer to join our team remotely. The ideal candidate will possess the skills to scrape live web pages, effectively functioning as an API, to capture registration processes and present data seamlessly within our application.

Join Us in Shaping the Future at Dun & BradstreetAt Dun & Bradstreet, we harness the power of data to pave the way for a brighter tomorrow. As a premier global authority in business decisioning data and analytics, we empower companies around the globe to grow, mitigate risks, and innovate. With a legacy spanning over 180 years, organizations have relied on us to convert uncertainty into opportunity. Our diverse and global team champions creativity, collaboration, and bold ideas. Are you ready to make a significant impact? Explore the exciting opportunities waiting for you at dnb.com/careers.Job Overview:We are seeking a talented Senior Software Engineer to enhance our Global Product Data (GPD) team in Hyderabad. In this pivotal role, you will be instrumental in designing and maintaining automated web scraping pipelines that extract structured data from a variety of online sources, converting raw data into production-ready datasets for our Master Data Repository (MDR).This position is part of a strategic move to internalize web scraping and data acquisition capabilities, reducing reliance on external vendors. You will collaborate closely with data engineering and product teams to ensure the delivery of high-quality, reliable, and timely data.

Join our dynamic team at Quince as an Engineering Manager specializing in web development. In this pivotal role, you will lead a team of talented engineers, driving innovation and excellence in our web applications. Your leadership will be crucial in shaping our development strategies, improving code quality, and ensuring timely project delivery. If you are passionate about technology and have a knack for mentoring and guiding teams, we want to hear from you!

Role overview Oleria Security is seeking a Design Engineer focused on Web & Marketing for its Bangalore office. This full-time role combines creative design with hands-on technical work to help advance the company’s brand and marketing objectives. Key responsibilities Design visually appealing and user-friendly assets for web and marketing initiatives. Collaborate with teams across the organization to enhance user experience. Develop materials that effectively communicate Oleria Security’s brand message. Offer design ideas and practical solutions that support marketing campaigns. Collaboration This position works with cross-functional groups to make sure marketing materials reach the right audience and align with company-wide goals.

Join our dynamic team as a Lead Software Engineer specializing in web development. In this pivotal role, you will be responsible for designing, developing, and maintaining cutting-edge web applications. Your expertise will guide our team in implementing best practices, ensuring high performance and responsiveness of applications, and collaborating with cross-functional teams to deliver innovative solutions.

Join our innovative team at Weblee Technologies as a Web Designer and Developer. In this dynamic role, you will be responsible for creating visually appealing websites and applications that enhance user experience. You will collaborate with cross-functional teams to design, develop, and optimize web solutions that meet our clients' needs.

Join Weblee Technologies as a Web Developer and Designer. We are looking for a creative and skilled individual who can design and develop user-friendly websites. As part of our dynamic team, you will be responsible for creating visually appealing and functional web applications that meet our clients' needs.You will work closely with our project managers and clients to understand their requirements and translate them into engaging digital experiences. If you have a passion for web design and development, we want to hear from you!

Join our innovative team at Weblee Technologies as a Web Designer & Developer. We are seeking a creative individual who possesses a strong grasp of design principles and web development skills to create visually appealing and user-friendly websites. Your role will involve collaborating with cross-functional teams to ensure the best user experience.