Senior Software Engineer - Data Pipeline

Scale GP is at the forefront of enterprise Generative AI, offering cutting-edge APIs and infrastructure for knowledge retrieval, inference, evaluation, and intelligent automation. We empower critical workflows for leading enterprises, transforming intricate data and models into dependable, production-ready AI systems. Join our newly formed Agentic Data Produ…

Join our innovative team as a Senior Software Engineer, where you will integrate traditional software engineering with agent management and system development.At Gravity Climate (gravityclimate.com), we are leaders in carbon and energy management. Our cutting-edge product empowers businesses to accurately measure their carbon footprints, ensure compliance with regulations, and optimize energy consumption to achieve significant cost savings while minimizing emissions. Major players across various industries—including finance, technology, consumer goods, and manufacturing—trust Gravity to manage their carbon reporting and energy efficiency initiatives. With a robust funding foundation and a new office in the vibrant Jackson Square area of San Francisco, we are experiencing tremendous growth.This is a critical moment at the intersection of technology and environmental responsibility, and we are seeking visionary engineers who are passionate about the future of software development.Key Responsibilities:In this hybrid role, you will not only be responsible for delivering high-quality software but also for mentoring and guiding a team of agents in the development process. You will utilize tools such as Opus 4.5 through Claude Code & Conductor, focusing on code refinement and continuous improvement. While prior experience with these tools is not mandatory, a willingness to learn and adapt is essential. You will also manage context switching effectively to maintain productivity.Additionally, you will contribute to the establishment of our software development infrastructure. This involves refining system prompts, developing automated verification mechanisms for code self-assessment by agents, and creating specialized skills for our agents that align with our codebase and domain expertise—this is how we achieve scalability and efficiency.Your role will also include staying abreast of emerging technologies, ensuring that we remain at the forefront of innovation. For instance, understanding the significance of Claude Code and Opus 4.5 will be critical for our competitive advantage.Finally, your expertise in agentic frameworks, evaluations, and best practices will ensure that the Gravity Agent remains the most effective tool available for our clients.Why Choose Gravity?Navigating the complexities of carbon accounting requires exceptional data models, and we have established ourselves as the best in the industry due to our automation capabilities. Our success is driven by our commitment to innovation and sustainability, making this an exciting time to join our team.

Join our dynamic team at Lever as we redefine the landscape of talent acquisition. We are seeking an Innovative Solutions Engineer to help us in our mission to build cutting-edge hiring software utilized by industry leaders like Netflix, Yelp, and Spotify.Lever, founded a decade ago, is at the forefront of transforming how companies recruit and hire top talent. Our commitment to innovation has earned us recognition as the #1 workplace in San Francisco and a top company to work for across the United States. We pride ourselves on our people-first culture and the competitive advantage our talented team brings to our organization.

Redpanda Data is building the Agentic Data Plane (ADP), a platform that connects AI agents with enterprise data and systems. The ADP supports real-time, autonomous reasoning and action by agentic applications, powered by Redpanda's multi-modal data streaming engine. Major organizations across industries, including Activision Blizzard, Cisco, Moody's, Texas Instruments, Vodafone, and two of the top five U.S. banks, rely on Redpanda to process hundreds of terabytes of data every day. Backed by investors such as Lightspeed, GV, and Haystack VC, Redpanda operates as a globally distributed, people-first company. Role overview The Principal Software Engineer will architect and develop the Agentic Data Plane, which serves as the control and execution layer for AI agents interacting with enterprise data. This system enables agents to access, analyze, and act on data in real time, while providing human operators with oversight and control for secure operations. The ADP brings together Redpanda's low-latency streaming technology, a distributed query engine for real-time context, a library of over 300 data connectors, and a global policy and observability framework. This framework enforces access controls, records agent actions, and supports replayable audits. What you will do Design and build the core architecture of the Agentic Data Plane, focusing on secure and efficient data interaction for AI agents. Integrate streaming, query, and policy enforcement components to support real-time, autonomous agent operations. Monitor developments in the agentic AI field and translate research into engineering proposals and product strategies. Work closely with Engineering, Product, and Go-To-Market teams, as well as key customers, to shape the direction of the ADP.

Transforming AI Through Self-ImprovementAt Letta, we are pioneering the development of self-improving artificial intelligence systems. Unlike traditional AI, which can be inflexible and limited, our agents are designed to learn continuously from their experiences and adapt accordingly.Founded by the innovators behind MemGPT at UC Berkeley’s Sky Computing Lab—the home of industry-changing technologies like Spark and Ray—Letta is supported by prominent figures in AI infrastructure, including Jeff Dean and Clem Delangue. Our agents are already making a difference in production environments for esteemed companies such as 11x and Bilt Rewards, evolving and enhancing their capabilities daily.We are forming a top-tier team of researchers and engineers to tackle AI's most significant challenge: creating machines that can reason, remember, and learn as humans do.Note: This position requires in-person attendance in our downtown San Francisco office, five days a week.

About Collective Collective helps solo entrepreneurs run their businesses with less hassle. Our technology and team of advisors make it easier for members to handle incorporation, accounting, bookkeeping, and taxes, all in one place. The goal: give self-employed professionals the same tax advantages as large companies, so they can focus on what matters most to them. Our work has been highlighted in Forbes, Business Insider, Yahoo, Bloomberg, Financial Times, and TechCrunch. Investors backing Collective include General Catalyst, Sound Ventures (Ashton Kutcher and Guy Oseary), QED Investors, and Google's Gradient Ventures. Many have also supported brands like YouTube, Substack, and Lyft. Role Overview: AI Data Engineer Collective is looking for a Senior AI Data Engineer to build an internal data agent platform. The mission: make company data accessible to employees, so anyone can ask questions in plain English and get reliable insights or dashboards in return. This role blends AI systems, data infrastructure, and product engineering. The work goes beyond building a system, it requires attention to data integrity, consistency, and reliability throughout. The team is searching for someone who can set a technical direction, design the data architecture, and help drive adoption of these new tools across the company. What You Will Do Develop Agent Infrastructure Build systems for data retrieval, planning, and execution using real company data Enable multi-step reasoning and analytical workflows Connect the agent platform with internal tools Create an End-to-End Analytics System Develop a natural language interface for the entire data stack Allow employees to run complex analyses without SQL Manage and Enhance the Data Layer Work with product engineering teams to design and improve clean, consistent data models Continuously raise the quality and accessibility of company data Location San Francisco

Why Choose Flux?At Flux, we are transforming the landscape of hardware development by pioneering the world's first AI Hardware Engineer. Our mission is to democratize access to cutting-edge hardware creation and transform the way electronics are designed and manufactured globally.About the RoleWe are on a quest to build the first AI Hardware Engineer, software capable of designing tangible, manufacturable electronics from simple prompts. As a Senior Software Engineer focusing on Agentic Development, you will be responsible for creating the system's intelligence, developing agentic workflows, reasoning graphs, and integrations that empower both novice and experienced engineers in their journey from concept to manufacturable hardware.This position fuses advanced web engineering with AI systems design. You will leverage TypeScript and LangGraph within the Flux web application and backend platform, embedding intelligent features directly into the design environment.Key ResponsibilitiesDevelop and implement agentic reasoning capabilities using TypeScript (LangGraph).Integrate AI functionalities into the Flux web application (React/Redux) and Chat UI.Implement telemetry and logging for monitoring runtime health, performance, and cost efficiency.Collaborate closely with product and electrical engineers to refine intelligent ECAD workflows.Conduct evaluations and translate insights into engineering decisions.Contribute to the development of agentic patterns, conventions, and best practices within the engineering organization.Required Qualifications8+ years of experience in software engineering, with at least 1 year in developing generative-AI or agentic systems.Proficient in TypeScript, React, and Node.js.Experience with LangChain, LangGraph, or similar orchestration frameworks.Proven track record in building asynchronous, distributed systems in a production environment.Experience in integrating LLM-based systems into complex user-facing web products.Strong communication skills and comfort working in a fully remote, asynchronous team setting.Previous experience in a startup or in 0→1 product development environments.Preferred QualificationsFamiliarity with retrieval systems, evaluation pipelines, or agent benchmarking.

Peregrine Technologies builds AI-powered tools for public safety, government, and private organizations. Our platform connects fragmented data, giving clients immediate access to information that supports better decisions and outcomes. Today, we serve hundreds of clients across more than 30 states and two countries, reaching over 125 million people. Backed by leading Silicon Valley investors, we are expanding into enterprise and international markets. About the Engineering Team Peregrine’s engineers focus on building solutions with empathy for users. Understanding real-world product use is central to our approach, and engineers work closely with onsite colleagues to learn about the varied needs our clients face. Collaboration and hands-on problem solving are core to how we work. Role Overview: Senior Software Engineer, Data Governance This role sits within our core engineering group. The team tackles projects such as enabling real-time collaboration on complex maps and building backend systems that handle billions of data points at scale. The Data Governance team designs and builds services, systems, and product features that help clients manage their data assets from start to finish within Peregrine. The team’s work centers on secure, precise data access and strong audit controls. Location San Francisco, CA

Our mission: to eliminate every barrier to mental health.At Spring Health, we are committed to transforming mental healthcare by dismantling every obstacle that hinders individuals from receiving the assistance they require, precisely when they need it. Our clinically validated technology, Precision Mental Healthcare, empowers us to provide tailored solutions—ranging from therapy and coaching to medication—adequately meeting each individual's needs.We proudly collaborate with over 450 partners, from innovative startups to renowned Fortune 500 companies, serving a diverse clientele of 10 million individuals. Our esteemed clients include well-known brands like Microsoft, Target, and Delta Airlines, all of whom rely on us to deliver exceptional outcomes for their employees worldwide. Through our groundbreaking platform, we have achieved a net positive ROI for employers and are the only entity in our space to have received external validation for delivering net savings for our clients.Our growth has been bolstered by significant investments from leading firms such as Generation Investment, Kinnevik, Tiger Global, Northzone, and RRE Ventures. With our latest Series E funding round, our valuation has surged to $3.3 billion. We are just at the beginning of our journey—join us as we work towards making mental healthcare accessible for everyone, everywhere.Multiple openings exist within the Care Innovation team, and based on your expertise and our team needs, we will connect you with the right group! In this prominent role as a Senior Software Engineer, you will spearhead a new product initiative. This key technical position within a high-impact team affords you the autonomy to direct technical choices and shape product development trajectories. You will work closely with R&D leadership to design and implement high-priority projects that align with our mission to revolutionize mental healthcare.This is an extraordinary opportunity to become a foundational member of a motivated, purpose-driven team focused on developing innovative solutions that create a lasting impact in mental healthcare.

About DecagonDecagon stands at the forefront of conversational AI, empowering brands to create unparalleled customer experiences. Our platform enables leading enterprises such as Avis Budget Group, Block’s Cash App, Square, Chime, Oura Health, and Hunter Douglas to deploy AI agents that facilitate personalized, engaging interactions across voice, chat, email, SMS, and beyond.We envision a world where customer interactions transcend traditional support tickets and hold music, fostering quicker resolutions, richer dialogues, and stronger relationships. Our esteemed investors, including a16z, Accel, Bain Capital Ventures, Coatue, and Index Ventures, share this vision and support our mission.As an in-office company, we are united by our commitment to excellence and rapid execution. Our core values — Just Get It Done, Invent What Customers Want, Winner’s Mindset, and The Polymath Principle — guide our collaborative work environment and team growth.About the TeamThe Agent Orchestration team is responsible for developing the foundational execution layer that drives every agent interaction. We create systems that coordinate model reasoning, assess agent behavior, and ensure reliability in real-world customer scenarios. Our focus on accuracy, dependability, and speed enables Decagon agents to plan, decide, and act effectively across millions of interactions.About the RoleAs a Senior Software Engineer on the Agent Orchestration team, you will design and develop systems that govern the performance of Decagon agents in real-world conditions. You will tackle complex challenges associated with distributed systems and lay the groundwork for enhancing agent performance, reliability, and quality.This role offers significant autonomy and the opportunity to make a direct impact on the architecture of one of the most sophisticated agent platforms globally.In this role, you will:Develop the execution engine that supports intelligent agents at scale.Design systems that synchronize model reasoning and actions while maintaining speed and accuracy.Enhance reliability through improved testing, observability, and safety measures.Establish foundations that enable teams to evaluate and enhance agent behavior safely and continuously.

Join Decagon as a Senior Software Engineer specializing in Agent Development. In this pivotal role, you will be responsible for designing, implementing, and optimizing software solutions that enhance our agent systems. You will collaborate with cross-functional teams to drive innovative projects, ensuring the highest quality and performance of our applications.

About Tenable:Tenable® is at the forefront of Exposure Management. With 44,000 organizations worldwide counting on us to mitigate cyber risk, we empower 65% of the Fortune 500, 45% of the Global 2000, and numerous governmental agencies. Join us in redefining the cybersecurity landscape!Why Choose Tenable?When you ask our team about what makes Tenable a fantastic workplace, the answer is simple: our people! Together, we innovate cutting-edge cybersecurity solutions, fostering a culture of inclusion, respect, and excellence. As part of the #OneTenable team, you'll collaborate with some of the industry's most talented professionals while having access to the resources necessary to make a meaningful impact. We strive to deliver exceptional results and celebrate our victories together!Note: Applicants must be authorized to work in the U.S. without sponsorship. We currently cannot sponsor work visas.Location: Must reside in or be willing to relocate to the San Francisco/San Jose area.Your Role:At Tenable, we're not just creating exceptional products; we're cultivating a premier engineering culture dedicated to hiring top talent. We are seeking a Senior Software Engineer to be part of a pioneering initiative within our engineering team. In this role, you will design and implement the Agentic AI Platform, an orchestration layer that empowers autonomous agents to reason, plan, and execute intricate security workflows within our Exposure Management ecosystem.This role is highly hands-on and has a significant impact. You won't merely be utilizing LLMs; you'll be constructing distributed systems, state machines, and tools that transform generative models into reliable, enterprise-grade agents.While this role is primarily remote, occasional in-person meetings may be necessary.Your Opportunity:Construct the Agentic Backbone: Design and implement backend services that facilitate multi-agent workflows, enabling AI agents to break down complex user requests, manage states, and execute tasks across distributed microservices....

About Redwood MaterialsRedwood Materials is at the forefront of transforming the global battery supply chain by optimizing the recovery, reuse, and recycling of battery materials. Since our inception in 2017, we have pioneered large-scale energy storage solutions and are producing battery materials domestically for the first time, all sourced from existing batteries.Senior Software Engineer, Energy PlatformJoin our dynamic team focused on developing a cutting-edge software platform that drives intelligent control and market engagement for Battery Energy Storage Systems (BESS). Our work encompasses a comprehensive tech stack, including compute infrastructure, telemetry, asset modeling, alerting, feature ingestion, storage for forecasting and optimization, simulation, operational control orchestration, and integration with energy markets. We pride ourselves on our lean structure and our commitment to delivering high-quality, rapid software solutions. Our culture emphasizes collaboration, trust, continuous learning, and engineering excellence, making it a truly enjoyable environment to innovate and create.Key Responsibilities:Contribute to BESS data engineering by developing and maintaining telemetry pipelines and storage systems for effective forecasting and optimization.Design, build, and manage simulation and training infrastructures for large-scale simulation execution.Create and operate real-time forecasting and optimization workflows that streamline BESS operations.Facilitate energy market integration, including data ingestion and bid management processes.Collaborate across the full tech stack and team boundaries to ensure cohesive product functionality.Develop high-quality, well-tested software and continuously enhance it through iterative improvements.Utilize AI tools to expedite development while ensuring superior system design and code quality.

ABOUT THE ROLE:As an AI Engineer at Varick, you will take charge of designing and optimizing the intelligence layer within our enterprise operations. This involves creating agent systems that efficiently handle thousands of transactions, make classification decisions, and learn from human interactions.This position is suited for engineers with extensive experience in LLMs, agent architectures, and evaluation systems. You have successfully developed agent workflows that operate in production, not merely in demo environments. Your expertise includes prompt engineering, retrieval, tool calling, multi-agent orchestration, and the evaluation frameworks necessary for deploying trustworthy AI systems in enterprise settings.WHAT YOU'LL DO:• Design and implement agent architectures that tackle complex enterprise workflows, focusing on multi-step reasoning, tool calling, and exception handling.• Develop and sustain evaluation systems that ensure agent quality, accuracy, safety, and groundedness.• Create robust prompt systems, retrieval pipelines, and context engineering strategies to ensure reliable agent performance.• Establish feedback loops that empower agents to learn from human corrections and enhance their functionality over time.• Optimize inference costs and latency for production workloads to ensure efficiency.• Define and uphold best practices for agent reliability, observability, and governance.• Stay updated with the latest models, frameworks, and research to ensure impactful deployments into production.WHAT WE'RE LOOKING FOR:• At least 3 years of software engineering experience, with a minimum of 1–2 years dedicated to LLM applications or AI systems in a production environment.• Practical experience in constructing agent workflows featuring tool calling, retrieval, and multi-step reasoning.• A profound understanding of prompt engineering, context engineering, and methods to elicit reliable behavior from LLMs.• Experience in developing evaluation systems to assess AI output quality.• Proficient in Python with a solid foundation in backend engineering principles.• You have delivered AI features to actual users and have navigated challenges such as hallucinations, edge cases, accuracy drops, and cost management.• Must be based in San Francisco.

Role Overview Join our innovative team as a Senior Data Engineer focused on Solutions Architecture, where you will architect, develop, and sustain our critical data infrastructure that enhances our renewable energy operations. In this dynamic role, you will bridge the realms of renewable energy and data, creating pipelines that handle various data types, from real-time asset performance metrics to intricate trading and risk analytics. This hybrid position allows you to contribute to the advancement of clean energy initiatives while utilizing a modern data stack, including Snowflake, Dagster, dbt, Modal, and GitLab. Key Responsibilities Analytics Infrastructure & Data Warehouse Management Design, implement, and manage scalable data infrastructures to meet the analytical and reporting demands of the enterprise. Oversee Snowflake instances, focusing on performance optimization, security measures, and planning for increasing data volumes. Enhance query performance and resource efficiency to minimize costs and accelerate processing times. Data Pipeline Development & Orchestration Create and orchestrate sophisticated ETL/ELT workflows utilizing Dagster to ensure dependable, automated data processing for asset management and energy trading. Construct robust data pipelines capable of managing high-throughput, time-sensitive energy market data, along with asset generation and performance metrics. Establish workflow automation and dependencies for essential business operations. Data Transformation & Analytics Support Design and maintain dbt models to convert raw data into business-ready analytical datasets and dimensional models. Implement efficient SQL transformations for complex calculations related to energy markets and asset performance. Facilitate advanced analytics efforts through effective data preparation and feature engineering. Data Quality & Governance Develop comprehensive data validation, testing, and monitoring frameworks to ensure accuracy and consistency across all energy and financial data assets. Establish data lineage tracking and privacy controls to comply with regulatory standards in the energy industry. Create alerting and monitoring systems for data pipelines, encompassing error management, SLA monitoring, and incident response.

Join SoFi as a Senior Software Engineer in our Data Foundations team, where you will play a pivotal role in shaping our data architecture and enhancing our data-driven capabilities. You will work closely with cross-functional teams to develop robust data solutions that empower our business decisions and improve customer experiences.As a Senior Software Engineer, you will leverage your expertise in data engineering, software development, and cloud technologies to build scalable data pipelines and maintain high-quality data infrastructure. Your contributions will directly impact our ability to deliver innovative financial solutions.

About Probably GeneticProbably Genetic is revolutionizing the lives of patients with severe and complex diseases. Our advanced data platform empowers drug developers and patient advocacy organizations to create and launch innovative treatments. By leveraging cutting-edge technology, we identify undiagnosed patients online, analyze their conditions using machine learning and home testing, and facilitate compliant communication with them. Our mission is to ensure that patients gain access to diagnoses, clinical trials, and treatments at the earliest opportunity.We are a dedicated team of passionate problem solvers, driven by a purpose that transcends individual interests. By prioritizing patient welfare, we are developing groundbreaking solutions in healthcare, with a roadmap full of innovations in bioinformatics, AI, and drug development. We invite you to join our lean, talented team and contribute to our vision.Probably Genetic has secured multiple funding rounds from top-tier Silicon Valley investors, including Threshold, Khosla, and Y Combinator. We offer competitive salaries, comprehensive benefits, and meaningful equity opportunities for early-stage team members.About the RoleWe are seeking a founding Data Engineer who is enthusiastic about shaping the future of data utilization to enhance patient outcomes. In this pivotal role, you will establish our data engineering architecture and construct the pipelines that drive internal insights and commercial data products. Your contributions will be instrumental in fostering clarity, impact, and growth throughout our organization.What You Will DoCollaborate closely with the Head of Engineering and Head of Product to transform complex data challenges into elegant, scalable solutions.Build reliable, maintainable infrastructure on AWS using Terraform to accommodate our expanding data requirements.Design data tables and pipelines tailored to the specific needs of our customers and internal teams.Implement state-of-the-art data pipelines with built-in observability from day one.Analyze and visualize data using BI tools to facilitate informed business decisions and provide customized insights to clients.Communicate your work and its impact across teams — presenting findings, receiving feedback, and continuously enhancing processes.Who You AreWe are eager to connect with candidates from diverse backgrounds who are committed to learning, growth, and making a meaningful impact. Here are a few attributes that will enable you to thrive in this role:Proficiency in data engineering principles and practices.Experience with cloud platforms, particularly AWS.Strong analytical skills and familiarity with data visualization tools.Ability to collaborate effectively in a team environment.

Why join Brex?At Brex, we are revolutionizing financial management with our AI-driven spend platform, designed to empower companies of all sizes — from innovative startups to established enterprises like DoorDash and Compass. Our integrated services, including corporate cards, banking, and global payments, along with intuitive travel and expense software, help organizations manage their spending confidently, reduce costs, and enhance operational efficiency globally.Joining our team means embracing challenges, pushing boundaries, and collaborating with some of the brightest minds in the tech industry. We are dedicated to fostering a diverse and inclusive workplace, where your potential is only limited by your ambition. Our commitment is reflected in the tools, resources, and support we provide for your professional growth.Engineering Excellence at BrexOur engineering teams are focused on creating scalable systems with precision and speed. We operate with a high degree of autonomy and collaborative spirit, tackling complex technical challenges and driving excellence from architecture to deployment. Here, engineering is not just a job — it’s a craft, and our builders are empowered to become leaders.Work EnvironmentThis position is based in our San Francisco office, where we foster a hybrid work culture that balances the vibrant energy of in-person collaboration with the flexibility of remote work. Currently, we require team members to be in the office for at least three coordinated days each week: Monday, Wednesday, and Thursday. Additionally, we offer the opportunity for up to four weeks of fully remote work each year!Your RoleAs a Senior Software Engineer on our Product Data Platform (PDP) team, you will play a crucial role in developing and maintaining the high-performance data, query, and search infrastructure that underpins essential product experiences at Brex. Your mission will be to create systems that are customizable, scalable, and reliable for finance teams, requiring the development of optimized, production-grade backend solutions.In this role, you'll focus on data-intensive, distributed systems at scale, leading architectural and performance enhancements across the platform to improve latency, correctness, and scalability across various product surfaces. You’ll also contribute to AI-powered data products, including a dynamic reporting experience that allows customers to interact intelligently with their financial data. Your contributions will ensure that these systems remain fast, reliable, and designed for growth.

Join DoorDash as a Staff Software Engineer specializing in Data Engineering, where you will play a critical role in designing and implementing data solutions that drive business insights and enhance operational efficiency. You will collaborate with cross-functional teams to create robust data pipelines and leverage cutting-edge technology to manage large-scale datasets.