Senior Staff Software Engineer & Data Scientist - Large Model Evaluation

Join Waymo, a pioneer in autonomous driving technology, dedicated to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, our mission is to enhance mobility access and eliminate traffic-related fatalities. The Waymo Driver has already facilitated over ten million rides, showcasing its capability after autonomously navigating over 100 million miles on public roads and trillions in simulations across more than 15 U.S. states.As part of the Release Evaluation team, you will play a key role in ensuring the safety of each Waymo Driver version prior to deployment. We develop automated systems to address rare and exceptional scenarios in autonomous driving, effectively identifying critical signals under constraints of time and resources. The Sampling and Efficiency team utilizes advanced importance sampling techniques and machine learning to enhance the statistical efficiency of our evaluation pipelines.

Waymo is at the forefront of autonomous driving technology, aspiring to become the world's most reliable driver. Originating as the Google Self-Driving Car Project in 2009, we are dedicated to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access while significantly reducing traffic-related fatalities. With over ten million rider-only trips facilitated by our technology, the Waymo Driver has autonomously navigated more than 100 million miles on public roads and tens of billions in simulation across more than 15 states in the U.S.Our Release Evaluation organization is committed to ensuring each iteration of the Waymo Driver is safe for public deployment. We construct automated pipelines to identify rare and exceptional scenarios in autonomous driving, effectively searching for critical insights under limited time and resource constraints. Within Release Evaluation, the Sampling and Efficiency team employs advanced importance sampling techniques combined with machine learning to maximize the statistical efficiency of our discovery pipelines.

Waymo is at the forefront of autonomous driving technology, dedicated to becoming the world’s most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has been committed to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access and save lives lost to traffic accidents. The Waymo Driver not only powers our fully autonomous ride-hailing service but can also be adapted to various vehicle platforms and applications. With over ten million rider-only trips under its belt, the Waymo Driver has driven autonomously over 100 million miles on public roads and completed tens of billions of miles in simulation across more than 15 U.S. states.The Planner Evaluation team tackles one of the primary challenges in autonomous driving: assessing and elevating the quality of the software steering the car. We seek data-driven software engineers and data scientists who are passionate about autonomous vehicles and adept at utilizing complex data to inform decision-making. If this excites you, we encourage you to apply!This position follows a hybrid work model and reports to an Engineering Manager.

Waymo is a pioneering force in autonomous driving technology, dedicated to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo focuses on developing the Waymo Driver—The World's Most Experienced Driver™—to enhance mobility access and significantly reduce traffic-related fatalities. The Waymo Driver powers our fully autonomous ride-hailing service and is adaptable across various vehicle platforms and applications. With over ten million rider-only trips completed and extensive experience from driving more than 100 million miles on public roads, complemented by simulations totaling tens of billions of miles across over 15 U.S. states, we are at the forefront of our industry.The Large Model Evaluation team plays a crucial role in advancing Waymo's AI ambition. As we leverage advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs), our goal is to create cutting-edge AI systems that navigate the complexities of real-world driving scenarios. Our progress hinges on our ability to measure performance effectively. With robust evaluation being essential for deploying large models, the challenges we face are particularly intricate and safety-critical. We are seeking engineers with a quantitative mindset to explore and establish innovative methods for assessing the machine learning models utilized in the Waymo Driver.

Waymo is leading the way in autonomous driving technology with a mission to be the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo is dedicated to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance accessibility to transportation while preventing countless lives lost in traffic accidents. Our innovative Waymo Driver underpins our fully autonomous ride-hailing service and is adaptable to various vehicle platforms and applications. With over ten million rider-only trips facilitated by our technology, we have driven more than 100 million miles on public roads and logged tens of billions of miles in simulation across more than 15 U.S. states.The Perception Evaluation team at Waymo is pioneering advancements in autonomous driving, prioritizing the safety and reliability of our self-driving systems. Our team leverages state-of-the-art tools and methodologies to rigorously evaluate the performance of our perception systems, which are essential for the safe and effective operation of autonomous vehicles. We are currently seeking a Software Engineer to be instrumental in shaping the future of transportation and enhancing the quality and reliability of Waymo's autonomous vehicles.In this hybrid role, you will report directly to an Engineering Manager.

Waymo is at the forefront of autonomous driving technology, committed to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has dedicated itself to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access while significantly reducing traffic-related fatalities. Our Waymo Driver powers a fully autonomous ride-hailing service and is adaptable to a diverse array of vehicle platforms and applications. With over ten million rider-only trips and experience from more than 100 million miles driven autonomously on public roads, Waymo is shaping the future of transportation.Our simulation environment is among the most sophisticated ever created, utilizing deterministic logic, physical dynamics, and cutting-edge Generative AI to establish a training ground for the Waymo Driver. The Simulator Evaluation team tackles the challenging question: How can we mathematically validate that a virtual world is perceptibly 'real'?We are on the lookout for a Staff Software Engineer who will take on the role of Technical Architect in this field. You will operate at the intersection of software engineering and AI, ensuring our simulated environments—whether governed by explicit rules or foundational models—accurately reflect reality.In this Staff-level position, you will report directly to a Senior Staff Software Engineering Manager and function as a Technical Lead, connecting intricate technical metrics with overarching product strategies.Your Responsibilities Will Include:Architecting Evaluation Standards: Define the 'Definition of Done' for simulation realism, anticipating product objectives (e.g., operational capabilities in snow or highway driving) and designing the evaluation roadmap to ensure our simulation fidelity evolves in alignment with onboard requirements.Acting as the System Critic: Create comprehensive mathematical frameworks to validate our hybrid virtual world, determining the balance between diverse evaluation needs—from verifying logical rules and dynamics to assessing the distribution quality of generative AI models.Building at Scale: Lead the development of large-scale, extensible evaluation platforms (in C++/Python), ensuring our metric pipelines are robust distributed systems capable of delivering clear, reproducible insights on petabytes of data.Providing Strategic, Cross-Functional Leadership: Serve as the technical liaison across various organizations, closely collaborating with AI research and other simulation teams. The evaluation workflows you design will facilitate rapid innovation and inform research trajectories.

Waymo is at the forefront of autonomous driving technology, striving to become the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has dedicated itself to creating the Waymo Driver—the world's most experienced driver™—with the aim of enhancing mobility access while preventing the countless lives lost in traffic accidents. The Waymo Driver powers our fully autonomous ride-hailing service and is adaptable across various vehicle platforms and use cases. With over ten million rider-only trips under its belt and extensive experience driving more than 100 million miles on public roads, complemented by extensive simulations across 15+ U.S. states, Waymo is leading the charge in safe and efficient transportation solutions.The Planner Evaluation team addresses a pivotal challenge in autonomous driving: enhancing and quantifying the software that drives our vehicles. We seek seasoned data-driven software engineers and data scientists passionate about autonomous vehicles and utilizing complex data to drive informed decisions. This is the perfect opportunity for someone eager to make a significant impact in the field!In this hybrid role, you will report directly to an Engineering Manager.You will:Develop performance metrics and driving quality signals for the Waymo driver utilizing various techniques, including statistics, mathematics, physics, algorithms, and machine learning.Innovatively use simulations and analyze real-world driving logs to assess driving performance.Design and implement robust methods to strengthen the correlation between changes in onboard software and simulated results.Advocate for code quality and best practices within a large and intricate code base.Analyze data and provide insights for enhancing metric quality and interpretability.Collaborate effectively with engineers, data scientists, statisticians, and leadership to deliver evaluation products and facilitate data-driven decisions.Rapidly validate the effectiveness of additional coverage, ensuring solutions are robust for customer teams to manage their own evaluations.

Waymo, an innovative leader in autonomous driving technology, aims to become the world's most trusted driver. Established in 2009 from the Google Self-Driving Car Project, Waymo focuses on developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access and prevent traffic-related fatalities. The Waymo Driver facilitates our fully autonomous ride-hail service and is adaptable to a variety of vehicle platforms and use cases. With over ten million rider-only trips completed and more than 100 million miles driven autonomously on public roads, along with extensive simulation across 15+ U.S. states, Waymo is redefining transportation.The Planner Evaluation team is at the forefront of addressing one of the critical challenges in autonomous driving: assessing and enhancing the quality of the software that operates the vehicles. We are in search of skilled data-centric software engineers and data scientists who are eager to improve our methodologies for characterizing and evaluating modifications to the onboard software stack. If you're enthusiastic about autonomous vehicles and utilizing complex data to inform decision-making, this is the ideal opportunity for you!This hybrid role will report to an Engineering Manager.Your responsibilities will include:Developing metrics to evaluate the performance and driving characteristics of the Waymo Driver using various techniques such as statistics, mathematics, physics, algorithms, and machine learning.Utilizing simulations creatively while analyzing real-world driving logs to assess driving performance.Designing and implementing methods to establish a robust connection between onboard software modifications and simulated results.Promoting code health and adherence to best practices within a large and intricate codebase.Analyzing data to provide actionable insights on enhancing metric quality and interpretability.Collaborating with fellow engineers, data scientists, statisticians, and leadership to produce evaluation tools and support data-driven decisions.

Waymo is at the forefront of autonomous driving technology, driven by the mission to become the world’s most trusted driver. Born from the Google Self-Driving Car Project in 2009, Waymo has dedicated itself to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility and save lives that are often lost in traffic accidents. The Waymo Driver powers our fully autonomous ride-hailing service, which can be adapted for various vehicle platforms and use cases. With over ten million rider-only trips completed and experience accumulated from driving over 100 million miles on public roads, alongside tens of billions of miles in simulation across more than 15 U.S. states, Waymo is redefining transportation.The Perception team is integral to the Waymo Driver's success, crafting cutting-edge technology that enables our autonomous systems to accurately understand their environment, make informed decisions in real-time, and transport passengers safely to their destinations. We engage in research to tackle real-world challenges and collaborate with other innovative teams at Alphabet. Our access to extensive driving data collected through diverse sensors empowers software engineers like you to create and scale multi-modal models and techniques.Our objective is to establish a robust foundation for a high-level Perception pipeline, which is a fundamental component of the self-driving system. We serve as the essential link between Waymo's hardware teams and the broader engineering organization focused on self-driving technology, defining sensor specifications, providing vital feedback, and simplifying system complexities.In this hybrid role, you will report directly to a Technical Lead Manager.

Waymo is at the forefront of autonomous driving technology, striving to become the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has been dedicated to developing the Waymo Driver—The World’s Most Experienced Driver™—with the goal of enhancing mobility access while aiming to reduce the tragic loss of lives due to traffic accidents. The Waymo Driver not only fuels our fully autonomous ride-hailing service but is also adaptable across various vehicle platforms and applications. With over ten million trips solely for riders and extensive experience from autonomously navigating over 100 million miles on public roads, alongside tens of billions of miles in simulation across more than 15 U.S. states, we are leading the charge in this transformative technology.The Large Model Evaluation team plays a pivotal role in advancing Waymo's AI vision. As we integrate cutting-edge Large Language Models (LLMs) and Vision-Language Models (VLMs), our aim is to construct sophisticated AI systems capable of addressing the multifaceted challenges of real-world driving. Central to our achievements is the ability to accurately measure our progress. In a landscape where robust evaluation serves as a critical barrier to deploying large models, the intricacies of this task at Waymo are particularly complex and safety-sensitive. We seek quantitatively driven engineers to innovate and propose novel methodologies for evaluating the ML models utilized in the Waymo Driver.

Be a Part of the Future of Home RoboticsAt sunday, we are pioneering the development of personal robots designed to liberate individuals from the burden of repetitive tasks. Our mission is to democratize access to generalized robotics, allowing families to reclaim precious time spent on mundane activities.After investing 18 months in assembling an exceptional team, securing investment, and validating our technology, we are on the lookout for passionate professionals eager to join us during this exciting phase of growth. If you are ready to channel your expertise into cutting-edge robotics innovation, we want to hear from you.Your RoleAs a Full Stack Engineer focused on ML Data & Evals, you will serve as the vital link between raw data and robotic intelligence. You’ll create the foundational infrastructure for our ML team to evaluate and deploy models effectively. Your contributions will expedite the research-to-production cycle, facilitating on-robot evaluations and visualizing model performance in intricate, real-world settings.Key ResponsibilitiesDesign Interactive Data Systems: Develop interfaces and engines that integrate robot and Skill Capture Gloves™ data, enabling efficient human-in-the-loop workflows, from episode annotation to seamless transitions between autonomous operation and manual teleoperation.Conduct Evaluations & Benchmarking: Create high-performance tools to compare model-driven motions with human-captured ground truths, aiding in the assessment of model advancements across various tasks.Data Processing & Orchestration: Design processing services that convert raw captures and sensor data into formats suitable for machine learning, ensuring a smooth transition from our global data collection systems to the models that power our robots.Adaptability in a Startup Environment: While your primary focus will be the ML Platform, our Full Stack Engineers should be versatile, readily adjusting to shifting priorities and contributing across different areas of the stack as we grow.

Role overview Nuro is looking for a Senior Middleware Software Engineer to help build and support the software that powers its autonomous delivery platform. This role focuses on developing and maintaining middleware systems that connect different parts of the technology stack. The work directly impacts the reliability and efficiency of Nuro’s services.

About NuroNuro is a pioneering self-driving technology company dedicated to making autonomy accessible to everyone. Established in 2016, we are developing the most scalable autonomous vehicle system, integrating advanced AI with automotive-grade hardware. Nuro licenses its core technology, the Nuro Driver™, to facilitate a diverse array of applications, including robotaxis, commercial fleets, and personal vehicles. With years of proven self-driving deployments, our technology provides automakers and mobility platforms with a clear path to commercial-scale autonomous vehicles, fostering a safer, more connected future.Position OverviewWe are seeking an experienced engineering leader to assume the role of Director for our Evaluation Platform team. This critical platform supports the simulation, evaluation, and validation processes that are essential for the development of autonomous technologies and ensuring readiness for driverless deployment.The successful candidate will establish the technical vision, execution strategy, and organizational framework for a multidisciplinary team focused on simulation, evaluation infrastructure, and software-driven operational workflows.The evaluation platform integrates data, simulation, metrics, and analysis workflows, empowering teams to:Rapidly iterate to enhance autonomy performanceAssess autonomy performance and risk with credibility and rigorMake informed, data-driven decisions regarding driverless deploymentKey ResponsibilitiesDevelop and oversee the technical vision and roadmap for the Evaluation Platform, encompassing simulation, evaluation infrastructure, and software quality operations.Lead, mentor, and nurture a multi-team organization (including managers and senior individual contributors), promoting a culture of technical excellence, ownership, and continuous improvement.Build and scale simulation and evaluation capabilities that cater to both rapid autonomy iteration and rigorous validation scenarios.Take ownership of the autonomy evaluation and validation platform, collaborating closely with Autonomy and Systems leadership to ensure reliability, scalability, and trust from engineering and validation teams.Represent the Evaluation Platform at the executive level, effectively communicating tradeoffs, progress, risks, and readiness.

P-1137 Summary At Databricks, we are dedicated to empowering data teams to tackle the world's most challenging issues, from detecting security threats to advancing cancer treatments. We achieve this by developing and operating the premier data and AI infrastructure platform, enabling our clients to concentrate on the high-impact challenges central to their missions. Founded in 2013 by the original creators of Apache Spark, Databricks has evolved from a modest office in Berkeley, CA, to a global leader with over 1,500 employees. Thousands of organizations, ranging from startups to Fortune 100 companies, rely on Databricks for their mission-critical workloads, establishing us as one of the fastest-growing SaaS firms globally. Our engineering teams are focused on creating highly technical products that address significant real-world needs. We continuously push the limits of data and AI technology while maintaining the resilience, security, and scalability essential for our customers' success on our platform. Customers entrust Databricks with their most valuable data, and our Trust & Safety division is committed to creating the most reliable data analytics and machine learning platform worldwide. Security Engineering plays a vital role in safeguarding customer data from malicious threats. We seek senior leaders to shape the vision and define the strategic approach within this critical domain. The impact you will have: Enhance the safety of Databricks for our customers by identifying and addressing critical vulnerabilities in our infrastructure and services. Attract top-tier talent from across the industry. Advocate for the security engineering discipline across the organization, wielding a significant voice to drive data-driven decisions. Represent Databricks at academic and industry conferences and events.

Aerospike is a leading real-time database solution designed for mission-critical applications and workloads, including machine learning, generative AI, and autonomous systems. Our platform powers millions of transactions per second with millisecond latency, offering a significantly lower total cost of ownership compared to traditional databases.Renowned organizations such as Adobe, Airtel, Barclays, Criteo, DBS Bank, Experian, Grab, HDFC Bank, PayPal, Sony Interactive Entertainment, The Trade Desk, and Wayfair trust Aerospike to manage their customer 360 insights, fraud detection, real-time bidding, profile stores, recommendation engines, and more.At Aerospike, we dream big and deliver even bigger. Our mission is to harness the power of the world's real-time data with a database engineered for infinite scale, speed, and sustainability.If you’re ready to shape the future of data, we invite you to join our team.As we expand rapidly, we are heavily investing in our Cloud Platform, a sophisticated multi-cloud, multi-tenant solution that delivers the capabilities of Aerospike to our customers with unparalleled simplicity, speed, and scalability.Senior Software Engineer, CloudWe are looking for a talented Senior Software Engineer to join our Cloud team. In this pivotal role, you will design and develop the infrastructure orchestration, control systems, and operational frameworks that underpin Aerospike Cloud. Your contributions will directly influence the reliability, scalability, and security of production database clusters that serve clients across the globe.

Nuro develops self-driving technology, aiming to make autonomous vehicles accessible for everyone. Since 2016, the team has blended advanced AI with automotive-grade hardware, licensing the Nuro Driver™ to support robotaxis, commercial fleets, and personal vehicles. Partnerships with automakers and mobility platforms help bring these autonomous solutions to commercial scale. The Senior Software Engineer - Software Update Infrastructure role is part of the Fleet Connectivity team based in Mountain View, California. This position focuses on building and maintaining systems that support release infrastructure and over-the-air (OTA) updates for Nuro’s autonomous vehicles. The work covers both cloud and robotics stacks, ensuring reliable network connectivity, telemetry, and software updates that are vital to Nuro’s mission. What you will do Design, build, test, deploy, and maintain software for release infrastructure and OTA updates. Manage project priorities, deadlines, and deliverables within the team and when working with other groups. Write high-quality, testable code that meets strict code quality standards. Monitor release system performance by defining and tracking KPIs, and implement optimizations to improve build costs, times, and storage usage. Create testing and validation programs to ensure system architecture and data flows are reliable. Contribute to documentation and participate in on-call rotations during business hours. Requirements 5+ years of experience working with large-scale distributed systems. Proficiency in C++ or Go. Comfortable working in Unix/Linux environments and with Bash scripting. Ability to translate broad requirements into detailed system designs. Strong knowledge of data structures, algorithms, and distributed system software design. Bachelor’s degree in Computer Science, Electrical Engineering, or a closely related field. Bonus points Experience with SQL. Familiarity with Google Cloud Platform (GCP). UI development experience. Background in OTA updates execution and management.

Waymo is an innovative leader in autonomous driving technology, dedicated to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, we have developed the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access and save lives lost in traffic accidents. Our Waymo Driver powers a fully autonomous ride-hail service, having completed over ten million rider-only trips, backed by vast experience driving over 100 million miles on public roads and tens of billions in simulation across more than 15 U.S. states.Within our Driver Understanding and Evaluation (DUE) team, we strive to develop comprehensive metrics for analyzing the Waymo Driver's behavior in real-world scenarios.In this hybrid role, you will report directly to a Software Engineering Manager.

About Nuro Nuro is a pioneering self-driving technology firm dedicated to making autonomous driving accessible to everyone. Established in 2016, we are developing the world’s most scalable driver by merging advanced AI with automotive-grade hardware. Our flagship technology, the Nuro Driver™, is licensed to facilitate a variety of applications, from robotaxis to commercial fleets and personal vehicles. With years of proven self-driving deployments, we offer automakers and mobility platforms a clear path to achieving commercial-scale autonomous vehicles, enhancing safety and connectivity for a better future. Join Our Team Our team is committed to empowering engineers to revolutionize transportation by equipping them with the tools and infrastructure necessary for productivity and success. We aim to achieve this through: Identifying and resolving engineering process challenges. Creating innovative tools and infrastructure for enhanced efficiency. Educating engineers on best practices. Utilizing cutting-edge technologies to amplify productivity. Our passionate engineers are dedicated to solving complex problems and making a positive impact on the future of transportation. Role Overview Collaborate with engineers across Nuro to pinpoint inefficiencies in the engineering workflow. Define, analyze, and monitor metrics relating to engineering velocity and system health. Innovatively address daily developer challenges by creating new tools or enhancing existing ones. Develop scalable infrastructure for continuous code integration, testing, and release, while optimizing for coverage, speed, and cost, addressing the unique complexities of Nuro’s machine learning and hardware tests. Oversee and enhance essential libraries and APIs for interaction with complex systems, managing code dependencies and the overall structure of Nuro’s codebase. Utilize Generative AI to enhance engineering productivity. Provide guidance to developers on best practices and newly implemented tools.

Harness is at the forefront of AI-driven software delivery, founded by the visionary technologist Jyoti Bansal, who previously established AppDynamics, leading to its acquisition by Cisco for an impressive $3.7 billion. With approximately $570 million raised in funding and a valuation of $5.5 billion, Harness is supported by prominent investors such as Goldman Sachs, Menlo Ventures, IVP, Unusual Ventures, and Citi Ventures. As AI transforms code generation, the critical bottleneck has shifted to the post-coding phase, including testing, deployments, application security, reliability, compliance, and cost optimization. Harness leverages AI and automation to streamline these processes, empowering teams to deliver software more efficiently while ensuring security and governance throughout the software delivery lifecycle.Utilizing Harness AI and the Software Delivery Knowledge Graph, our platform integrates deep contextual insights and intelligent automation across the software delivery landscape, embedding governance and policy-driven controls throughout.In just the past year, Harness has facilitated over 185 million deployments, 82 million builds, and 18 trillion flag evaluations, alongside conducting 8 million security scans, optimizing 9.1 billion tests, and protecting 3 trillion API calls. We have effectively managed $2.8 billion in cloud expenditures, assisting clients such as United Airlines, Morningstar, and Choice Hotels in accelerating their release cycles by up to 75%, reducing cloud costs by up to 60%, and achieving a remarkable 10x improvement in DevOps efficiency.With a diverse global workforce spanning 14 offices in 25 countries, Harness is pioneering the future of AI-driven software delivery, and we are on the lookout for extraordinary talent to join us on this exciting journey.Position SummaryJoin us as a Senior Software Engineer in a dynamic, high-growth startup environment. As part of the Platform team, you will architect, design, develop, and deliver top-notch software solutions that significantly influence our product offerings.About the RoleDesign and implement scalable platform featuresCreate comprehensive software functional specifications and design documentationQuickly grasp complex systems and code, taking ownership of critical components, including their qualityCollaborate with cross-functional teams to drive innovation and efficiency