Data Engineer - Web Scraping Specialist

Join SoFi as a Senior Web Platform Engineer and be at the forefront of building innovative web solutions that empower our users. You will work on enhancing and maintaining our web platform, ensuring high performance, scalability, and security. Collaborate with cross-functional teams to implement new features and optimize existing ones, using the latest technologies and best practices in web development.

Join our innovative team at Peregrine Technologies as a Web Engineer. In this role, you will be responsible for designing, developing, and maintaining high-performance web applications that enhance the user experience and drive business success. You will collaborate with cross-functional teams to implement new features, optimize existing functionalities, and ensure the scalability and security of our web platforms.

Are you passionate about creating engaging user interfaces and enhancing user experiences? Join our innovative team at Sesame as a Web UI/UX Engineer. In this dynamic role, you will collaborate with designers and developers to build intuitive web applications that delight our users. Your expertise in user-centered design principles and your ability to translate concepts into wireframes and prototypes will be key to your success.

Overview:Join the dynamic Data Acquisition team at OpenAI, part of our Foundations organization, where you will play a crucial role in the data collection processes that power our model training initiatives. Our team is at the forefront of managing web crawling and GPTBot services, collaborating closely with departments such as Data Processing, Architecture, and Scaling. We are seeking a talented Software Engineer who is passionate about data acquisition and eager to make a significant impact.Key Responsibilities:Lead and innovate engineering projects focused on data acquisition, including web crawling, data ingestion, and search functionalities.Collaborate effectively with cross-functional teams, including Data Processing, Architecture, and Scaling, to ensure seamless data flow and operational efficiency.Partner with the legal team to navigate compliance and data privacy challenges.Design and implement highly scalable distributed systems capable of processing petabytes of data.Architect algorithms for efficient data indexing and robust search capabilities.Build and manage backend services for data storage, including working with key-value databases and ensuring synchronization.Implement solutions within a Kubernetes Infrastructure-as-Code environment and conduct regular system health checks.Conduct experiments and analyze data to derive insights that enhance system performance.Qualifications:Bachelor's, Master's, or PhD in Computer Science or a related field.4+ years of professional software development experience.Familiarity with large-scale web crawlers is a plus.Deep understanding of large stateful distributed systems and data processing methodologies.Proficient in Kubernetes and knowledgeable about Infrastructure-as-Code principles.Eager to explore and implement new technologies and approaches.Proven ability to manage multiple tasks and adapt to shifting priorities.Excellent written and verbal communication skills.About OpenAI:At OpenAI, we are pioneers in AI research and deployment, dedicated to ensuring that the advancements in artificial intelligence benefit humanity as a whole. Our mission is to push the boundaries of AI capabilities while adhering to safe and responsible deployment practices. Join us in our commitment to harnessing the power of AI for positive global impact.

About AnthropicAt Anthropic, our vision is to develop AI systems that are reliable, interpretable, and controllable. We strive to ensure that AI operates safely and benefits not just our users but society as a whole. Our rapidly expanding team consists of dedicated researchers, engineers, policy experts, and business leaders united in the mission to create beneficial AI systems.The Creative Studio at Anthropic serves as our in-house creative team, responsible for expressing our brand identity across Anthropic and Claude. We value craftsmanship, combining hand-made elements with AI where appropriate. As our team grows, we seek talented individuals committed to excellence to join us. The work we do is significant, and there is ample opportunity to influence its direction.About the RoleWe are in search of a Design Engineer to join our Brand Web team, operating at the crossroads of design and development to construct and uphold Anthropic's web properties. In this position, you will play a vital role in delivering high-quality web experiences for anthropic.com and claude.com. This includes creating interactive marketing pages, data visualizations, and internal tools that ensure our content operations function seamlessly. You will collaborate closely with designers, engineers, and our brand and content teams to translate design concepts into refined, production-ready code. Rapid prototyping is a fundamental aspect of our workflow, and you will be expected to work efficiently and effectively.ResponsibilitiesLead the implementation of design system components and page templates across our web properties, focusing on delivering interactive experiences, data visualizations, and tools that effectively convey complex information.Engage with our headless CMS architecture to maintain and enhance content modeling, structured data workflows, and the publishing experience.Work hand-in-hand with designers to connect static designs with dynamic, interactive web experiences.Contribute to internal tools that empower teams to author, manage, and publish content at scale.Participate in migration and site architecture projects, taking on defined responsibilities within broader initiatives led by senior team members.Quickly transition from concept to working prototype, iterating closely with the team.Support tooling and workflow enhancements that enable faster delivery by the broader team, and contribute to documentation and knowledge sharing.

About UsAt Rox, we are dedicated to empowering individuals to achieve their greatest potential. Our innovative platform enhances sales efforts through autonomous revenue agents, allowing sellers to prioritize their expertise in selling. Just as coding agents transformed engineering, revenue agents amplify customer interactions.We are revolutionizing the revenue stack by developing the world’s first revenue operating system, encompassing everything from the application layer to systems of context. At Rox, we envision a future where humans evolve into orchestrators while agents handle the complete customer lifecycle.Our solutions support Global 2000 leaders across sectors such as banking, construction, and AI, partnering with industry giants like Ramp and Cognition.Our success stems from a united belief in our mission and an unwavering commitment to making it a reality.The TeamOur world-class team is the backbone of our innovative approach to redefining business operations.Our team members have:Founded and successfully exited companiesHeld top roles at Google, AWS, Confluent, and New RelicWon gold medals in international mathematics competitionsPublished groundbreaking research papersWe are proud to be backed by leading investors, having raised $50 million from Sequoia (Alfred Lin), General Catalyst (Hemant Taneja), Google Ventures, Elad Gil, and Chris Ré.Core PrinciplesTaste: Craft beautiful experiences.We meticulously focus on every detail, striving to ensure that each interaction not only helps sellers accomplish their tasks but also enhances their experience. We are relentless in our pursuit of excellence, always exploring new ways to delight our sellers.Obsession: Commit unreasonably.We are dedicated to our craft, responding to customer needs proactively and driving value even before they ask. Our commitment to continuous learning and self-improvement is unwavering.Action: Get it done.Execution is key; we prioritize thoughtful yet swift decision-making and immediate delivery. Trust is essential in our field, and we earn it through our actions.

About RoxAt Rox, we are dedicated to empowering individuals to excel in their work.Our innovative platform enhances seller performance with autonomous revenue agents that handle manual tasks, allowing sales professionals to concentrate on their primary role: selling. Just as coding agents exponentially increased engineering productivity, our revenue agents elevate customer engagement.We are revolutionizing the revenue landscape by developing the world’s first revenue operating system, integrating everything from the application layer to the system of context. At Rox, we envision a future where humans become orchestrators, while our agents efficiently manage the complete customer lifecycle.Rox serves Global 2000 leaders across sectors such as banking, hardware, construction, and sovereign AI, while also supporting prominent AI innovators like Ramp and Cognition.This success is built on a steadfast belief in our mission and an unwavering commitment to achieving it.Join Our TeamOur world-class team is the cornerstone of our success, tasked with redefining business operations.Team members have:Founded and exited successful startupsHeld top positions at Google, AWS, Confluent, and New RelicAchieved gold medals in IMO and IOIPublished groundbreaking research papersWe are proud to be backed by top-tier investors, having raised $50M from Sequoia (Alfred Lin), General Catalyst (Hemant Taneja), Google Ventures, Elad Gil, and Chris Ré.Core PrinciplesTaste: Craft exceptional experiences.Every detail matters; we ensure that each interaction is optimized for the seller’s success and satisfaction. We strive for perfection and continuously seek ways to delight our users.Obsession: Commit without reservation.We are dedicated to our craft, proactively addressing customer needs and constantly improving our skills and products.Action: Deliver results.Execution is key; we prioritize thoughtful, rapid decision-making and immediate delivery, fostering a culture of trust.

Join dstaff as a Web Application Security Engineer and become a key player in fortifying our digital infrastructure. In this role, you will be responsible for identifying and mitigating security vulnerabilities in our web applications, ensuring the integrity and confidentiality of our data.You will collaborate with cross-functional teams to integrate security practices into our development lifecycle and implement robust security measures. Your expertise will help us stay ahead of potential threats and safeguard our users' information.

At Exa, we are on a mission to create a groundbreaking search engine from the ground up, designed to empower every AI application. Our team develops robust infrastructure to efficiently crawl the web, trains cutting-edge embedding models for indexing, and engineers high-performance vector databases in Rust for seamless searching. We also manage a state-of-the-art $5M H200 GPU cluster that powers tens of thousands of machines.As a Software Engineer specializing in Web Crawling, you will take on the exciting challenge of building a web crawler that operates at a scale comparable to Google, revolutionizing the way we access and utilize information online.

Accordion’s AI Data Specialist role sits within the Data & Analytics team, supporting private equity clients and their portfolio companies across sectors such as Retail, Consumer Packaged Goods, Healthcare, Media & Entertainment, Technology, and Logistics. This position is based in any of Accordion’s U.S. offices, including Atlanta, Boston, Charlotte, Chicago, Dallas, Los Angeles, New York, or San Francisco, and follows a hybrid work model with up to two remote days per week. Candidates should reside near their selected office. Immigration sponsorship is not available. Role overview The AI Data Specialist works as part of an AI-enhanced delivery team, collaborating with AI engineers, data scientists, and product managers. The team’s mission is to transform financial operations for companies backed by private equity. The Data Specialist ensures that data is integrated, navigates complex data environments, and helps make information actionable to support client financial operations. Collaboration and focus This role involves close work with the Accordion Intelligence Lab, a group of software and AI engineers dedicated to advancing agentic-AI technologies. The Lab develops and maintains AI systems that underpin Accordion’s consulting work, including agentic architectures and production observability frameworks. AI Data Specialists contribute to building scalable, reliable tools that directly impact private equity clients and their portfolio companies. About the team The Data & Analytics group at Accordion brings together industry knowledge, advanced technology, and analytical skills to solve complex business problems. Team members deliver intelligent solutions and work in a collaborative, supportive environment alongside more than 1,400 finance and technology professionals.

Join our innovative team at atlas as a Web Engineer, where you will be integral in shaping the future of web technologies. In this role, you will collaborate with cross-functional teams to develop high-quality web applications, ensuring optimal performance and user experience.Your expertise in modern web frameworks and programming languages will be essential as you tackle exciting challenges and contribute to our ongoing projects. We are looking for a passionate individual who thrives in a dynamic environment and is eager to learn and grow.

About UsAt Imprint, we are transforming the landscape of co-branded credit cards and financial products to be more innovative, rewarding, and brand-centric. We collaborate with distinguished companies such as Crate & Barrel, Rakuten, Booking.com, H-E-B, Fetch, and Brooks Brothers to launch contemporary credit programs that enhance customer loyalty, unlock savings, and stimulate growth. Our platform seamlessly integrates advanced payment infrastructure, intelligent underwriting, and a user-friendly experience, enabling brands to offer impactful financial products without the need to operate as a bank.Co-branded cards represent over $300 billion in annual spending in the U.S., yet many are still reliant on outdated banking systems. Imprint stands out as the modern alternative: adaptable, technology-driven, and tailored for today’s consumers. Supported by renowned investors such as Kleiner Perkins, Thrive Capital, and Khosla Ventures, we are assembling a top-tier team to revolutionize payment methods and foster brand growth. If you're eager to work in a dynamic environment, tackle challenging problems, and make a significant impact, we invite you to join us.Our TeamThe Data Engineering team at Imprint is tasked with developing and expanding the data infrastructure that underpins product innovation, analytics, operations, and machine learning throughout the organization. We are responsible for creating the pipelines, platforms, and processes that enable our stakeholders to trust and leverage data effectively.We are seeking a Data Engineer to advance our modern data architecture and deliver dependable, scalable data solutions. Your contributions will directly influence decision-making and innovation across various business areas—from financial operations to real-time personalization.

At Scribd, Inc., we are dedicated to enhancing human understanding through our innovative products — Scribd®, Slideshare®, Everand™, and Fable — which empower billions around the world to transform access into knowledge, application, and expertise.Culture at Scribd, Inc.We foster a bold and authentic culture that encourages open debate and commitment, allowing our employees to embrace challenges and prioritize customer satisfaction. We believe that optimal performance arises from a balance of individual flexibility and meaningful community connection.Our Scribd Flex program enables team members to select work styles and locations that enhance their productivity, while also committing to intentional in-person interactions that promote collaboration and a vibrant company culture. All employees, regardless of their location, are required to attend occasional in-person meetings.What do we seek in our new hires? At Scribd, Inc., we value “GRIT.” This concept, reflecting the intersection of passion and perseverance toward long-term goals, forms the core of our expectations for every employee. It also serves as a practical framework for our work: achieving Goals, delivering Results, contributing Innovative ideas and solutions, and strengthening our Team through collaboration and positive attitudes.About the TeamOur Web Platform team is a vital part of the Developer Platform organization, driving our company-wide web modernization initiatives.Our mission is to empower product teams by providing a scalable and well-architected API foundation that supports modern web architectures. We collaborate across teams to minimize dependencies, enhance developer experience, and establish robust integration patterns between frontend and backend systems.With a high degree of autonomy and a long-term vision, the platform we develop will influence how teams design, deploy, and evolve our web products.About the RoleWe are actively seeking a Senior Software Engineer for the Web API Platform to contribute to the establishment of a company-wide API platform that accelerates web modernization at Scribd.

Join Our Team at RoxAt Rox, we are dedicated to empowering individuals to maximize their potential in the workplace.Our innovative platform equips sales professionals with autonomous revenue agents, allowing them to concentrate on their core strengths—selling. Just as coding agents revolutionized engineering, our revenue agents are designed to enhance customer engagements tenfold.We are pioneering the concept of a revenue operating system, transforming the traditional revenue stack from application layers to comprehensive systems of context. With Rox, human roles are evolving into orchestrators while our agents manage the entire customer lifecycle seamlessly.We proudly support Global 2000 leaders across sectors such as banking, hardware, construction, and advanced AI solutions. Our clients include prominent AI innovators like Ramp and Cognition.Our shared commitment to our mission drives our relentless pursuit of excellence in everything we do.

At Plaid, we envision a future where financial interactions are seamless and intuitive. We are committed to driving this evolution by crafting innovative tools and experiences that empower thousands of developers to build their own applications. Our platform supports millions of users in achieving healthier financial lives by connecting their financial accounts with the apps and services they love. Collaborating with industry leaders like Venmo, SoFi, numerous Fortune 500 companies, and major banks, Plaid's extensive network encompasses over 12,000 financial institutions across the United States, Canada, the UK, and Europe. Established in 2013 and headquartered in San Francisco, we also have offices in New York, Washington D.C., London, and Amsterdam. #LI-HybridAs a Senior Data Engineer, you will play a pivotal role in our Data Engineering team, which is focused on developing robust golden datasets to enhance our insights-driven product offerings. Data-driven decision-making is at the heart of Plaid's culture, and your expertise will help us scale our data systems while ensuring data accuracy and integrity. You will provide essential tooling and guidance to various teams across engineering, product, and business, enabling them to access and utilize data effectively. Our engineers rely heavily on SQL and Python to create data workflows, employing tools such as DBT, Airflow, Redshift, ElasticSearch, Atlanta, and Retool for orchestrating data pipelines and defining workflows. Collaborating closely with engineers, product managers, business intelligence, and data analysts, you will contribute to shaping Plaid's data strategy and fostering a data-first mindset. We thrive on an engineering culture that values individual contributions, encouraging bottom-up ideation and empowering our talented team. We seek engineers passionate about making a meaningful impact for our users and customers while growing together as a cohesive team, delivering MVPs, and continuously improving our processes.In this high-impact position, you will empower business leaders to make faster, more informed decisions based on the datasets you create. You will have the opportunity to define the ownership and scope of internal datasets and visualizations across Plaid, an area we aim to develop further by establishing SLAs. This role presents a unique chance to enhance your technical skills and learn best practices from our strong Data Engineering team and the broader Data Platform team. You will collaborate extensively across all teams at Plaid, from Engineering to Product to Marketing and Finance, forging strong cross-functional partnerships.

At Discord, we bring together over 200 million users each month, primarily united by their passion for video gaming. With over 90% of our users engaging in gaming experiences, we collectively spend 1.5 billion hours each month immersed in countless unique titles. We envision Discord as a pivotal platform in the gaming landscape, dedicated to enhancing communication and camaraderie among players before, during, and after their gaming sessions.Central to Discord's mission is the commitment to Safety. Our dedicated Safety team works tirelessly to create an environment where users feel a sense of belonging, effectively reducing exposure to spam and various forms of platform abuse. We are eager to welcome individuals who share our enthusiasm for fostering a safe and welcoming space for all our users.As an Embedded Data Specialist, you will serve as a vital source of data expertise for stakeholders, adept at managing diverse requests. You will skillfully balance immediate needs with the development of long-term tools, while being mindful of the maintenance requirements of your creations, including providing coaching. You will anticipate quality assurance for your code, ensuring that corrections are based on functionality rather than errors.

The OpportunityJoin Enigma at a pivotal moment as we prepare to unveil our next generation of small business intelligence products, informed by years of industry experience. We are in search of a seasoned Senior Data Engineer who will play a crucial role in shaping the future of these innovative products. Your contributions will significantly enhance the accuracy, scalability, and predictive analytics of our data platform, directly impacting our clients' key business strategies.The RoleAs a Senior Data Engineer, your primary responsibility will be to design, construct, and manage our foundational small business data product. You'll work closely with cross-functional teams to develop new product functionalities, starting from raw data sources, through scalable data pipelines and models, to the customer interface. Our culture emphasizes customer focus and complete ownership, measuring your success by the value delivered and the technical excellence of our product.What You’ll DoIdentify and address the unmet needs of our customers through thorough data exploration, prototyping, and evaluation, culminating in the development of robust and maintainable data pipelines.Construct scalable and maintainable data systems that narrate the stories of millions of U.S. small businesses, utilizing tools like Databricks SQL, PySpark, and MLFlow.Leverage our pioneering data infrastructure designed for iterative development and experimentation, utilizing Dagster for orchestration and LakeFS for data version control.Employ a data-driven, first-principles approach to problem-solving, favoring straightforward and elegant solutions while being capable of deploying advanced models and techniques when necessary.Motivate your colleagues to excel while nurturing a collaborative and supportive team environment.What Makes This Role Exciting?Ownership and Impact: Collaborate directly with customers and internal teams to develop product features in a high-trust environment.Technical Challenge: Solve complex data and engineering challenges, balancing innovation speed with scalability and reliability.Learning: Engage with unique, impactful data challenges that demand creative solutions and cutting-edge methodologies.

About Us:At Notion, we empower individuals and teams to create stunning productivity tools tailored to their unique workflows. In a digital landscape crowded with applications and tabs, our platform unifies documents, notes, projects, calendars, and emails—all enhanced with AI capabilities to streamline tasks and provide quick insights. Trusted by millions, including industry leaders like Toyota, Figma, and OpenAI, Notion is chosen for its versatility, helping users optimize their time and resources.We believe in the value of in-person collaboration, making it a cornerstone of our company culture. To foster teamwork, all employees are required to work from our San Francisco office on Mondays, Tuesdays, and Thursdays—our designated Anchor Days. Certain roles may necessitate additional in-office presence.About The Role:As our company expands at an impressive pace, we are on the lookout for skilled Data Engineers to join our finance team. In this role, you will be instrumental in constructing robust datasets and data pipelines that underpin our financial reporting. Your work will bridge our product, financial, and operational systems, creating reliable processes that drive our growth. If you have a passion for data analytics, complex modeling, and innovative problem-solving, we would love to have you on board.What You'll Achieve:Develop essential datasets that act as the definitive sources for Notion’s financial reporting, integrating information from financial systems, business databases, and our product.Collaborate closely with our Finance, Monetization Engineering, Business Intelligence, and Data Science teams to address critical financial analysis and reporting requirements.Design, construct, and oversee data pipelines that not only meet current demands but are also scalable as our data needs grow.Facilitate widespread access to high-quality financial data across the Finance, Staff, and go-to-market teams.Skills You'll Need to Bring:A minimum of 4 years of experience as a data engineer, focusing on building core datasets and supporting various business sectors, particularly in product and high-volume data environments. A strong enthusiasm for analytics use cases, data modeling, and tackling intricate data challenges is essential.Experience in creating integrations and reporting datasets for payment, financial, and business systems such as Stripe, NetSuite, Adaptive, Anaplan, Salesforce, or equivalent.A proactive mindset, with the ability to gather and synthesize high-impact needs from business partners, designing and implementing suitable technical solutions.

Front Careers seeks a Senior Data Engineer based in San Francisco, CA. This position centers on building and maintaining data systems that power analytics and reporting throughout the organization. Key responsibilities Design and maintain data architecture to meet analytics and reporting requirements Develop, optimize, and support data pipelines for reliability and scalability Collaborate with teams across the company to improve data quality, accessibility, and security Enhance and maintain the data infrastructure supporting company operations Role overview This role focuses on ensuring that data flows efficiently and securely, enabling teams to access the information they need for business decisions. The Senior Data Engineer will play a central part in shaping how data is managed and used at Front Careers.