Senior Software Engineer for Large Model Evaluation

Waymo is a pioneering force in autonomous driving technology, dedicated to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo focuses on developing the Waymo Driver—The World's Most Experienced Driver™—to enhance mobility access and significantly reduce traffic-related fatalities. The Waymo Driver powers our fully autonomous ride-hailing service and is adaptable across various vehicle platforms and applications. With over ten million rider-only trips completed and extensive experience from driving more than 100 million miles on public roads, complemented by simulations totaling tens of billions of miles across over 15 U.S. states, we are at the forefront of our industry.The Large Model Evaluation team plays a crucial role in advancing Waymo's AI ambition. As we leverage advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs), our goal is to create cutting-edge AI systems that navigate the complexities of real-world driving scenarios. Our progress hinges on our ability to measure performance effectively. With robust evaluation being essential for deploying large models, the challenges we face are particularly intricate and safety-critical. We are seeking engineers with a quantitative mindset to explore and establish innovative methods for assessing the machine learning models utilized in the Waymo Driver.

Waymo is at the forefront of autonomous driving technology, striving to become the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has been dedicated to developing the Waymo Driver—The World’s Most Experienced Driver™—with the goal of enhancing mobility access while aiming to reduce the tragic loss of lives due to traffic accidents. The Waymo Driver not only fuels our fully autonomous ride-hailing service but is also adaptable across various vehicle platforms and applications. With over ten million trips solely for riders and extensive experience from autonomously navigating over 100 million miles on public roads, alongside tens of billions of miles in simulation across more than 15 U.S. states, we are leading the charge in this transformative technology.The Large Model Evaluation team plays a pivotal role in advancing Waymo's AI vision. As we integrate cutting-edge Large Language Models (LLMs) and Vision-Language Models (VLMs), our aim is to construct sophisticated AI systems capable of addressing the multifaceted challenges of real-world driving. Central to our achievements is the ability to accurately measure our progress. In a landscape where robust evaluation serves as a critical barrier to deploying large models, the intricacies of this task at Waymo are particularly complex and safety-sensitive. We seek quantitatively driven engineers to innovate and propose novel methodologies for evaluating the ML models utilized in the Waymo Driver.

Waymo is at the forefront of autonomous driving technology, striving to become the world’s most reliable driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has dedicated itself to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access and prevent countless lives lost to traffic accidents. Our cutting-edge Waymo Driver is the backbone of our fully autonomous ride-hailing service and can be implemented across various vehicle platforms and applications. Having completed over ten million rider-only trips, our technology has accumulated experience from autonomously driving over 100 million miles on public roads and tens of billions of miles in simulations across more than 15 U.S. states.The Large Model Evaluation team is pivotal to Waymo's AI mission. As advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) continue, we are constructing state-of-the-art AI systems capable of addressing the full spectrum of real-world driving complexities. Central to our success is our capacity to measure progress effectively. Robust evaluation is crucial for deploying any large model, and at Waymo, the challenge is particularly intricate and safety-critical. We seek quantitatively-minded engineers to innovate and propose new methods for assessing the machine learning models integrated into the Waymo Driver.

Waymo is at the forefront of autonomous driving technology with a vision to become the world's most reliable driver. Originating from the Google Self-Driving Car Project in 2009, Waymo is dedicated to developing the Waymo Driver—The World's Most Experienced Driver™—which aims to enhance mobility access while significantly reducing traffic-related fatalities. The Waymo Driver is the backbone of our fully autonomous ride-hail service and is adaptable across various vehicle platforms and applications. To date, the Waymo Driver has successfully conducted over ten million rider-only trips, leveraging its experience from over 100 million miles driven on public roads and billions of miles in simulation across more than 15 U.S. states.We are in search of passionate and skilled engineers to join our team and spearhead the post-training process for large language models (LLMs) in the realm of autonomous driving. You will collaborate with an exceptional team of researchers and engineers to innovate and elevate the next generation of advanced AI models.

Waymo is at the forefront of autonomous driving technology, dedicated to becoming the world’s most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has been committed to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access and save lives lost to traffic accidents. The Waymo Driver not only powers our fully autonomous ride-hailing service but can also be adapted to various vehicle platforms and applications. With over ten million rider-only trips under its belt, the Waymo Driver has driven autonomously over 100 million miles on public roads and completed tens of billions of miles in simulation across more than 15 U.S. states.The Planner Evaluation team tackles one of the primary challenges in autonomous driving: assessing and elevating the quality of the software steering the car. We seek data-driven software engineers and data scientists who are passionate about autonomous vehicles and adept at utilizing complex data to inform decision-making. If this excites you, we encourage you to apply!This position follows a hybrid work model and reports to an Engineering Manager.

Join Waymo, a pioneer in autonomous driving technology, dedicated to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, our mission is to enhance mobility access and eliminate traffic-related fatalities. The Waymo Driver has already facilitated over ten million rides, showcasing its capability after autonomously navigating over 100 million miles on public roads and trillions in simulations across more than 15 U.S. states.As part of the Release Evaluation team, you will play a key role in ensuring the safety of each Waymo Driver version prior to deployment. We develop automated systems to address rare and exceptional scenarios in autonomous driving, effectively identifying critical signals under constraints of time and resources. The Sampling and Efficiency team utilizes advanced importance sampling techniques and machine learning to enhance the statistical efficiency of our evaluation pipelines.

Waymo is leading the way in autonomous driving technology with a mission to be the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo is dedicated to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance accessibility to transportation while preventing countless lives lost in traffic accidents. Our innovative Waymo Driver underpins our fully autonomous ride-hailing service and is adaptable to various vehicle platforms and applications. With over ten million rider-only trips facilitated by our technology, we have driven more than 100 million miles on public roads and logged tens of billions of miles in simulation across more than 15 U.S. states.The Perception Evaluation team at Waymo is pioneering advancements in autonomous driving, prioritizing the safety and reliability of our self-driving systems. Our team leverages state-of-the-art tools and methodologies to rigorously evaluate the performance of our perception systems, which are essential for the safe and effective operation of autonomous vehicles. We are currently seeking a Software Engineer to be instrumental in shaping the future of transportation and enhancing the quality and reliability of Waymo's autonomous vehicles.In this hybrid role, you will report directly to an Engineering Manager.

Waymo is at the forefront of autonomous driving technology, committed to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has dedicated itself to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access while significantly reducing traffic-related fatalities. Our Waymo Driver powers a fully autonomous ride-hailing service and is adaptable to a diverse array of vehicle platforms and applications. With over ten million rider-only trips and experience from more than 100 million miles driven autonomously on public roads, Waymo is shaping the future of transportation.Our simulation environment is among the most sophisticated ever created, utilizing deterministic logic, physical dynamics, and cutting-edge Generative AI to establish a training ground for the Waymo Driver. The Simulator Evaluation team tackles the challenging question: How can we mathematically validate that a virtual world is perceptibly 'real'?We are on the lookout for a Staff Software Engineer who will take on the role of Technical Architect in this field. You will operate at the intersection of software engineering and AI, ensuring our simulated environments—whether governed by explicit rules or foundational models—accurately reflect reality.In this Staff-level position, you will report directly to a Senior Staff Software Engineering Manager and function as a Technical Lead, connecting intricate technical metrics with overarching product strategies.Your Responsibilities Will Include:Architecting Evaluation Standards: Define the 'Definition of Done' for simulation realism, anticipating product objectives (e.g., operational capabilities in snow or highway driving) and designing the evaluation roadmap to ensure our simulation fidelity evolves in alignment with onboard requirements.Acting as the System Critic: Create comprehensive mathematical frameworks to validate our hybrid virtual world, determining the balance between diverse evaluation needs—from verifying logical rules and dynamics to assessing the distribution quality of generative AI models.Building at Scale: Lead the development of large-scale, extensible evaluation platforms (in C++/Python), ensuring our metric pipelines are robust distributed systems capable of delivering clear, reproducible insights on petabytes of data.Providing Strategic, Cross-Functional Leadership: Serve as the technical liaison across various organizations, closely collaborating with AI research and other simulation teams. The evaluation workflows you design will facilitate rapid innovation and inform research trajectories.

Waymo is a pioneering company in autonomous driving technology, dedicated to becoming the world’s most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has established the Waymo Driver—The World’s Most Experienced Driver™—with a mission to enhance mobility access and save lives lost in traffic accidents. The Waymo Driver powers our fully autonomous ride-hailing service and can be integrated across various vehicle platforms and applications. Having completed over ten million rider-only trips, our technology has driven more than 100 million miles on public roads and tens of billions in simulations across over 15 U.S. states.The Perception team is responsible for developing systems that learn the spatial-temporal representations and semantic meanings of the environment surrounding our autonomous vehicles (AVs). We collaborate closely with downstream teams to optimize and integrate our work into the Waymo Driver, conduct research to solve real-world challenges, and work alongside research teams at Alphabet. With access to millions of miles of diverse driving data from various sensors, we empower engineers like you to (1) create methods for efficient continuous learning from extensive real-world data, (2) develop scalable models and training methodologies, (3) analyze real-world behaviors to create systems that can navigate complexities, and (4) optimize models for both onboard and offboard hardware.In this hybrid role, you will report to a Technical Lead Manager.

Waymo is at the forefront of autonomous driving technology, striving to become the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has dedicated itself to creating the Waymo Driver—the world's most experienced driver™—with the aim of enhancing mobility access while preventing the countless lives lost in traffic accidents. The Waymo Driver powers our fully autonomous ride-hailing service and is adaptable across various vehicle platforms and use cases. With over ten million rider-only trips under its belt and extensive experience driving more than 100 million miles on public roads, complemented by extensive simulations across 15+ U.S. states, Waymo is leading the charge in safe and efficient transportation solutions.The Planner Evaluation team addresses a pivotal challenge in autonomous driving: enhancing and quantifying the software that drives our vehicles. We seek seasoned data-driven software engineers and data scientists passionate about autonomous vehicles and utilizing complex data to drive informed decisions. This is the perfect opportunity for someone eager to make a significant impact in the field!In this hybrid role, you will report directly to an Engineering Manager.You will:Develop performance metrics and driving quality signals for the Waymo driver utilizing various techniques, including statistics, mathematics, physics, algorithms, and machine learning.Innovatively use simulations and analyze real-world driving logs to assess driving performance.Design and implement robust methods to strengthen the correlation between changes in onboard software and simulated results.Advocate for code quality and best practices within a large and intricate code base.Analyze data and provide insights for enhancing metric quality and interpretability.Collaborate effectively with engineers, data scientists, statisticians, and leadership to deliver evaluation products and facilitate data-driven decisions.Rapidly validate the effectiveness of additional coverage, ensuring solutions are robust for customer teams to manage their own evaluations.

Waymo, an innovative leader in autonomous driving technology, aims to become the world's most trusted driver. Established in 2009 from the Google Self-Driving Car Project, Waymo focuses on developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access and prevent traffic-related fatalities. The Waymo Driver facilitates our fully autonomous ride-hail service and is adaptable to a variety of vehicle platforms and use cases. With over ten million rider-only trips completed and more than 100 million miles driven autonomously on public roads, along with extensive simulation across 15+ U.S. states, Waymo is redefining transportation.The Planner Evaluation team is at the forefront of addressing one of the critical challenges in autonomous driving: assessing and enhancing the quality of the software that operates the vehicles. We are in search of skilled data-centric software engineers and data scientists who are eager to improve our methodologies for characterizing and evaluating modifications to the onboard software stack. If you're enthusiastic about autonomous vehicles and utilizing complex data to inform decision-making, this is the ideal opportunity for you!This hybrid role will report to an Engineering Manager.Your responsibilities will include:Developing metrics to evaluate the performance and driving characteristics of the Waymo Driver using various techniques such as statistics, mathematics, physics, algorithms, and machine learning.Utilizing simulations creatively while analyzing real-world driving logs to assess driving performance.Designing and implementing methods to establish a robust connection between onboard software modifications and simulated results.Promoting code health and adherence to best practices within a large and intricate codebase.Analyzing data to provide actionable insights on enhancing metric quality and interpretability.Collaborating with fellow engineers, data scientists, statisticians, and leadership to produce evaluation tools and support data-driven decisions.

Waymo is at the forefront of autonomous driving technology, driven by the mission to become the world’s most trusted driver. Born from the Google Self-Driving Car Project in 2009, Waymo has dedicated itself to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility and save lives that are often lost in traffic accidents. The Waymo Driver powers our fully autonomous ride-hailing service, which can be adapted for various vehicle platforms and use cases. With over ten million rider-only trips completed and experience accumulated from driving over 100 million miles on public roads, alongside tens of billions of miles in simulation across more than 15 U.S. states, Waymo is redefining transportation.The Perception team is integral to the Waymo Driver's success, crafting cutting-edge technology that enables our autonomous systems to accurately understand their environment, make informed decisions in real-time, and transport passengers safely to their destinations. We engage in research to tackle real-world challenges and collaborate with other innovative teams at Alphabet. Our access to extensive driving data collected through diverse sensors empowers software engineers like you to create and scale multi-modal models and techniques.Our objective is to establish a robust foundation for a high-level Perception pipeline, which is a fundamental component of the self-driving system. We serve as the essential link between Waymo's hardware teams and the broader engineering organization focused on self-driving technology, defining sensor specifications, providing vital feedback, and simplifying system complexities.In this hybrid role, you will report directly to a Technical Lead Manager.

Waymo is at the forefront of autonomous driving technology, aspiring to become the world's most reliable driver. Originating as the Google Self-Driving Car Project in 2009, we are dedicated to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access while significantly reducing traffic-related fatalities. With over ten million rider-only trips facilitated by our technology, the Waymo Driver has autonomously navigated more than 100 million miles on public roads and tens of billions in simulation across more than 15 states in the U.S.Our Release Evaluation organization is committed to ensuring each iteration of the Waymo Driver is safe for public deployment. We construct automated pipelines to identify rare and exceptional scenarios in autonomous driving, effectively searching for critical insights under limited time and resource constraints. Within Release Evaluation, the Sampling and Efficiency team employs advanced importance sampling techniques combined with machine learning to maximize the statistical efficiency of our discovery pipelines.

Be a Part of the Future of Home RoboticsAt sunday, we are pioneering the development of personal robots designed to liberate individuals from the burden of repetitive tasks. Our mission is to democratize access to generalized robotics, allowing families to reclaim precious time spent on mundane activities.After investing 18 months in assembling an exceptional team, securing investment, and validating our technology, we are on the lookout for passionate professionals eager to join us during this exciting phase of growth. If you are ready to channel your expertise into cutting-edge robotics innovation, we want to hear from you.Your RoleAs a Full Stack Engineer focused on ML Data & Evals, you will serve as the vital link between raw data and robotic intelligence. You’ll create the foundational infrastructure for our ML team to evaluate and deploy models effectively. Your contributions will expedite the research-to-production cycle, facilitating on-robot evaluations and visualizing model performance in intricate, real-world settings.Key ResponsibilitiesDesign Interactive Data Systems: Develop interfaces and engines that integrate robot and Skill Capture Gloves™ data, enabling efficient human-in-the-loop workflows, from episode annotation to seamless transitions between autonomous operation and manual teleoperation.Conduct Evaluations & Benchmarking: Create high-performance tools to compare model-driven motions with human-captured ground truths, aiding in the assessment of model advancements across various tasks.Data Processing & Orchestration: Design processing services that convert raw captures and sensor data into formats suitable for machine learning, ensuring a smooth transition from our global data collection systems to the models that power our robots.Adaptability in a Startup Environment: While your primary focus will be the ML Platform, our Full Stack Engineers should be versatile, readily adjusting to shifting priorities and contributing across different areas of the stack as we grow.

Reliable Robotics develops automated aviation systems to improve safety and accessibility in air transportation. The team includes engineers from aerospace, robotics, and autonomous vehicle backgrounds, all working to advance aviation technology and make air travel more dependable. Role overview The Senior Guidance, Navigation, and Control (GNC) Modeling Engineer focuses on creating and improving physics-based models that underpin both rapid design changes and certification-level validation of GNC software. This role plays a key part in ensuring the accuracy and reliability of automated flight systems. What you will do Develop and refine physics-based models to support GNC software Run simulation analyses to test and validate GNC algorithms Compare simulation outputs with real-world data to verify model accuracy Contribute to iterative design and certification processes with high-fidelity modeling Location This position is located in Mountain View, CA.

Waymo is at the forefront of autonomous driving technology, striving to become the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo is dedicated to advancing the Waymo Driver—The World's Most Experienced Driver™—to enhance mobility and save lives that are often lost to traffic accidents. The Waymo Driver not only powers our fully autonomous ride-hail service but is also adaptable across various vehicle platforms and use cases. With over ten million rider-only trips completed, our technology has driven autonomously for more than 100 million miles on public roads, supported by tens of billions of miles in simulation across 15+ U.S. states.Our Compute Team plays a pivotal role in delivering the compute platform that runs the software stack of fully autonomous vehicles. We specialize in designing high-performance custom silicon, developing cutting-edge system-level compute architectures that maximize performance, power efficiency, and minimize latency. Our multidisciplinary team thrives on collaboration, working closely with various engineering teams to ensure both hardware and software are optimized for peak performance. We are looking for passionate and talented individuals to join us in developing one of the highest-performing automotive compute platforms in the world.In this hybrid role, you will report to a Hardware Engineering Manager.

Overview:At DeepMind, we believe that Artificial Intelligence has the potential to be one of humanity's most transformative inventions. Our team consists of elite scientists, engineers, and machine learning specialists dedicated to pushing the boundaries of AI technology. We are committed to using our innovations for the greater good, promoting scientific discovery, and collaborating with partners to tackle critical challenges while prioritizing safety and ethics.About the Role:This impactful position plays a crucial role in shaping the efficiency of AI model serving through low precision and sparse models. You will be instrumental in driving the hardware roadmap at Google, focusing on innovative numerics that are essential for the future of AI. This is a remarkable opportunity to contribute to an area that is increasingly vital in advancing AI technology.Your Responsibilities:Determine the precision, numerics, and sparsity formats for production-level AI models.Lead groundbreaking research initiatives and facilitate the transition of high-impact ideas into production environments.

Role overview Nuro is looking for a Senior Middleware Software Engineer to help build and support the software that powers its autonomous delivery platform. This role focuses on developing and maintaining middleware systems that connect different parts of the technology stack. The work directly impacts the reliability and efficiency of Nuro’s services.

About NuroNuro is a pioneering self-driving technology company dedicated to making autonomy accessible to everyone. Established in 2016, we are developing the most scalable autonomous vehicle system, integrating advanced AI with automotive-grade hardware. Nuro licenses its core technology, the Nuro Driver™, to facilitate a diverse array of applications, including robotaxis, commercial fleets, and personal vehicles. With years of proven self-driving deployments, our technology provides automakers and mobility platforms with a clear path to commercial-scale autonomous vehicles, fostering a safer, more connected future.Position OverviewWe are seeking an experienced engineering leader to assume the role of Director for our Evaluation Platform team. This critical platform supports the simulation, evaluation, and validation processes that are essential for the development of autonomous technologies and ensuring readiness for driverless deployment.The successful candidate will establish the technical vision, execution strategy, and organizational framework for a multidisciplinary team focused on simulation, evaluation infrastructure, and software-driven operational workflows.The evaluation platform integrates data, simulation, metrics, and analysis workflows, empowering teams to:Rapidly iterate to enhance autonomy performanceAssess autonomy performance and risk with credibility and rigorMake informed, data-driven decisions regarding driverless deploymentKey ResponsibilitiesDevelop and oversee the technical vision and roadmap for the Evaluation Platform, encompassing simulation, evaluation infrastructure, and software quality operations.Lead, mentor, and nurture a multi-team organization (including managers and senior individual contributors), promoting a culture of technical excellence, ownership, and continuous improvement.Build and scale simulation and evaluation capabilities that cater to both rapid autonomy iteration and rigorous validation scenarios.Take ownership of the autonomy evaluation and validation platform, collaborating closely with Autonomy and Systems leadership to ensure reliability, scalability, and trust from engineering and validation teams.Represent the Evaluation Platform at the executive level, effectively communicating tradeoffs, progress, risks, and readiness.

P-1137 Summary At Databricks, we are dedicated to empowering data teams to tackle the world's most challenging issues, from detecting security threats to advancing cancer treatments. We achieve this by developing and operating the premier data and AI infrastructure platform, enabling our clients to concentrate on the high-impact challenges central to their missions. Founded in 2013 by the original creators of Apache Spark, Databricks has evolved from a modest office in Berkeley, CA, to a global leader with over 1,500 employees. Thousands of organizations, ranging from startups to Fortune 100 companies, rely on Databricks for their mission-critical workloads, establishing us as one of the fastest-growing SaaS firms globally. Our engineering teams are focused on creating highly technical products that address significant real-world needs. We continuously push the limits of data and AI technology while maintaining the resilience, security, and scalability essential for our customers' success on our platform. Customers entrust Databricks with their most valuable data, and our Trust & Safety division is committed to creating the most reliable data analytics and machine learning platform worldwide. Security Engineering plays a vital role in safeguarding customer data from malicious threats. We seek senior leaders to shape the vision and define the strategic approach within this critical domain. The impact you will have: Enhance the safety of Databricks for our customers by identifying and addressing critical vulnerabilities in our infrastructure and services. Attract top-tier talent from across the industry. Advocate for the security engineering discipline across the organization, wielding a significant voice to drive data-driven decisions. Represent Databricks at academic and industry conferences and events.