Research Program Manager - Model Evaluations and Safety

Our MissionAt Reflection AI, we are committed to creating open superintelligence that is accessible to everyone. Our team is dedicated to developing open weight models tailored for individuals, agents, enterprises, and nation states. Our diverse group of AI experts comes from prestigious organizations such as DeepMind, OpenAI, Google Brain, Meta, Character.A…

Anthropic is looking for a Research Engineer focused on model evaluations. This position involves research and development to assess and strengthen the performance of AI models. Teams are based in San Francisco and New York City, and the role supports remote work with required travel. Key responsibilities Design and implement evaluations for Anthropic's AI models Collaborate with team members to enhance model performance Contribute to research that pushes the boundaries of AI systems Location Remote-friendly (travel required) San Francisco, CA New York City, NY

Team OverviewThe Human Data team at OpenAI is at the forefront of identifying and mitigating risks associated with advanced AI systems. Our mission is to enhance model reliability and public trust by designing thorough evaluations, uncovering vulnerabilities, and collaborating closely with researchers.Role OverviewAs a Technical Program Manager, you will spearhead initiatives aimed at assessing the safety and robustness of OpenAI’s models through innovative experimentation and methodical evaluation. Your role will involve orchestrating efforts across research and engineering teams, translating ambiguous risk signals into actionable research programs that will shape the future of AI model development and deployment.We seek candidates who possess technical acumen, thrive in uncertain environments, and are passionate about pioneering the future of safe AI.This position is based in San Francisco, CA, employing a hybrid work model of three days in the office each week, with relocation assistance available for new hires.Key ResponsibilitiesLead programs that investigate unexpected model behaviors and identify potential failure modes.Convert ambiguous risk signals into clear priorities and actionable research agendas.Design and execute innovative evaluations, experiments, and red-teaming initiatives.Collaborate with research, product, and deployment teams to integrate findings into the model training and deployment pipelines.Establish repeatable systems for monitoring model performance and interpreting emerging behavior patterns.Ideal Candidate ProfileProven experience in technical program management with exceptional organizational and communication abilities.Familiarity with large language models, prompt engineering, or model evaluation methodologies.Ability to manage fast-paced, high-uncertainty projects, shaping them from inception.Creative and resourceful in developing novel methods for evaluating model behavior and performance.Skilled in coordinating effectively across both technical and non-technical stakeholders to ensure alignment and execution.About OpenAIOpenAI is a pioneering AI research and deployment company committed to ensuring that general-purpose artificial intelligence benefits all of humanity. We continually push the boundaries of AI capabilities and strive to deploy them safely through our innovative products. Our mission is to harness the extraordinary potential of AI responsibly and equitably for a better future.

Role OverviewAt Intrinsic Safety, we are pioneering the development of AI systems capable of making critical decisions in high-stakes environments such as risk investigations, fraud detection, and identity verification. Our dedicated team in San Francisco is at the forefront of tackling complex challenges where traditional AI solutions often fall short.We are in search of a Research Engineer to play a pivotal role in shaping our model evaluation strategies. You will be responsible for creating benchmarks, datasets, and evaluation frameworks that accurately assess our systems’ performance in real-world scenarios. This position bridges research, product development, and engineering, focusing on rigorous evaluations that reflect actual customer workflows and identify key failure points to propel the next generation of AI advancements.

Join aiedu as a Senior Lead in Research & Evaluation, where you will drive impactful research initiatives that shape educational practices and policies. In this role, you will lead a team of researchers in designing and executing comprehensive evaluations that inform our strategic direction. Your expertise will be critical in analyzing data, generating insights, and communicating findings to stakeholders.

We are seeking a highly motivated and detail-oriented Research Program Manager to join our dynamic team at mercor. The ideal candidate will play a pivotal role in overseeing various research initiatives, coordinating projects, and ensuring that all objectives are met efficiently and effectively.This position offers an exciting opportunity to work in a fast-paced environment where innovation and collaboration are key. You will be responsible for managing project timelines, budgets, and resources while fostering a culture of continuous improvement within the team.

Join Perplexity as a Research Engineering Manager, where you will spearhead a team of exceptional AI researchers and engineers dedicated to crafting the advanced models that power our innovative products. Our talented team has pioneered some of the most sophisticated models in agentic research, query understanding, and other critical domains that demand precision and depth. As we broaden our user base and expand our product offerings, our proprietary models are increasingly essential for delivering a premium experience to the world's most discerning users.You will explore our extensive datasets of conversational and agentic queries, applying state-of-the-art training methodologies to enhance AI model performance. Through proactive technical and organizational leadership, you will empower your team to create cutting-edge models for the applications that are most significant to our business and our users.

Zyphra is a cutting-edge artificial intelligence firm headquartered in the vibrant city of San Francisco, California.Position Overview:As a Research Scientist specializing in Model Architectures, you will play a pivotal role in Zyphra’s AI Architecture Research Team. Your responsibilities will include the design and thorough evaluation of innovative model architectures and training methodologies aimed at enhancing essential modeling capabilities (e.g., loss per flop or loss per parameter) and tackling core limitations inherent in current models. You will collaborate closely with our pre-training team to ensure that your findings are seamlessly integrated into our next-generation models.Qualifications:A strong research acumen and intuition.Proven ability to navigate research projects from initial conception to execution and final write-up.Exceptional implementation and prototyping skills, with the capability to swiftly transform ideas into experimental outcomes.A collaborative spirit and the ability to thrive in a fast-paced research environment.A deep curiosity and enthusiasm for understanding intelligence.Requirements:Experience with long-term memory, RAG/retrieval systems, dynamic/adaptive computation, and alternative credit assignment strategies.Knowledge of reinforcement learning, control theory, and signal processing techniques.A passion for exploring and critically evaluating unconventional ideas, with the ability to maintain a unique perspective.Familiarity with modern training pipelines and the hardware necessities for designing efficient architectures compatible with GPU hardware.Strong understanding of experimental methodologies for conducting rigorous ablations and hypothesis testing.High proficiency in PyTorch and Python programming.Ability to quickly assimilate into large pre-existing codebases and contribute effectively.Prior publication of machine learning research in reputable venues.Postgraduate degree in a scientific discipline (e.g., Computer Science, Electrical Engineering, Mathematics, Physics).Why Join Zyphra?We emphasize a structured research methodology that systematically addresses ambitious challenges in AI.

Zyphra is an innovative artificial intelligence company located in the heart of San Francisco, California.The Opportunity:Join our dynamic team as a Research Engineer - Audio & Speech Models, where you will play a pivotal role in advancing Zyphra’s Audio Team. You will be instrumental in developing cutting-edge open-source text-to-speech and audio models. Your contributions will span the full spectrum of the model training process, from data collection and processing to the design of innovative architectures and training approaches.Your Responsibilities:Conduct large-scale audio training operationsOptimize the performance of our training infrastructureCollect, process, and evaluate audio datasetsImplement architectural and methodological improvements through rigorous testingWhat We Seek:A strong research mindset with the ability to navigate projects from ideation to implementation and documentation.Proficiency in rapid prototyping and implementation, allowing for swift experimentation.Effective collaboration skills in a fast-paced research environment.A quick learner who is eager to embrace and implement new concepts.Excellent communication abilities, enabling you to contribute to both research and engineering tasks at scale.Preferred Qualifications:Expertise in training audio models, such as text-to-speech, ASR, speech-to-speech, or emotion recognition.Experience with training audio autoencoders.Solid understanding of signal processing, particularly in audio.Familiarity with diffusion models, consistency models, or GANs.Experience with large-scale (multi-node) GPU training environments.Strong understanding of experimental methodologies for conducting rigorous tests and ablations.Interest in large-scale, parallel data processing pipelines.Competence in PyTorch and Python programming.Experience contributing to large, established codebases with rapid adaptation.

Our MissionAt Reflection AI, we are dedicated to creating accessible open superintelligence for everyone.Our team is composed of top-tier AI researchers and innovators from prestigious organizations like DeepMind, OpenAI, Google Brain, Meta, Character.AI, Anthropic, and more. We are committed to building open weight models for individuals, enterprises, and even nation states.About the RolePerform essential comparative analyses to deepen our insights into model capabilities.Design and enhance evaluation systems and processes that establish robust feedback loops between data, evaluations, and model behavior.Create generalizable evaluation frameworks that effectively capture reasoning, alignment, and practical usefulness.Collaborate closely with pre-training, post-training, and applied teams to translate insights into tangible model improvements.Expand the boundaries of measurable metrics, utilizing synthetic evaluations, human feedback, and real-world interaction data.About YouProficient in statistical analysis and experimental design, with the ability to rigorously measure model advancements.Knowledgeable in LLM evaluation methodologies, including static benchmarks, human preference evaluations, and agentic tasks.Possess a high degree of agency and thrive in a fast-paced startup atmosphere, prioritizing impact over rigid processes.Eager to work in a pioneering lab, shaping how we measure and accelerate the development of more capable models.Collaborative, detail-oriented, and driven by the desire to create effective feedback loops that enhance model performance.What We Offer:We believe in building superintelligence that is genuinely open, starting from the ground up. Joining Reflection means you will be part of a small, talent-dense team where you will help shape our future and push the boundaries of open foundational models.You will have the opportunity to engage in the most impactful work of your career, knowing that you and your loved ones are well-supported.Competitive Compensation: Salary and equity structured to attract and retain top global talent.Health & Wellness: Comprehensive medical, dental, vision, life, and disability insurance.

Anthropic is seeking a Technical Program Manager for Research Initiatives to coordinate and deliver advanced projects in artificial intelligence. This position is based in San Francisco, CA or New York City, NY. Role overview This role centers on managing research projects that push the boundaries of AI. The Technical Program Manager will oversee cross-functional teams, keeping projects on track and ensuring research goals are met. What you will do Lead and organize research initiatives focused on artificial intelligence Coordinate teams from different disciplines to achieve project objectives Track project timelines and deliverables, ensuring milestones are reached Requirements Experience managing technical or research-focused projects Ability to work with cross-functional teams Strong organizational and communication skills

About Our TeamThe Safety Systems team at OpenAI is dedicated to advancing safety protocols to ensure our cutting-edge models can be deployed responsibly, ultimately benefiting society. We are at the forefront of OpenAI's commitment to creating and deploying safe Artificial General Intelligence (AGI), fostering a culture rooted in trust and transparency.The Pretraining Safety team aspires to develop safer, more capable base models while facilitating early and reliable safety assessments during the training phase. Our objectives include:Establishing upstream safety evaluations to track the emergence of unsafe behaviors and goals;Creating safer priors through strategic pretraining and mid-training interventions that enhance downstream alignment;Designing safe-by-design architectures that improve control over model capabilities.Additionally, we conduct foundational research to comprehend how behaviors develop, generalize, and can be accurately measured throughout the training process.About the RoleThe Pretraining Safety team is trailblazing the integration of safety into models prior to their post-training and deployment stages. In this position, you will engage with the complete model development lifecycle, focusing on pre-training:Identifying safety-relevant behaviors as they emerge in base models;Assessing and mitigating risk without waiting for extensive training runs;Designing architectures and training setups that prioritize safer behavior;Enhancing models by integrating comprehensive, early safety signals.Our collaborative efforts span across OpenAI’s safety ecosystem—from Safety Systems to Training—to ensure our safety foundations are robust, scalable, and grounded in real-world considerations.Your Responsibilities Will Include:Developing innovative techniques to predict, measure, and assess unsafe behavior in early-stage models;Crafting data curation strategies that refine pretraining priors and mitigate downstream risk;Investigating safe-by-design architectures and training configurations to enhance controllability;Collaborating with cross-functional teams to ensure adherence to safety standards.

Join Cartesia as a Model Architecture ResearcherAt Cartesia, our vision is to revolutionize AI by creating interactive intelligence that is seamlessly integrated into your daily life. Unlike current models, our goal is to develop systems capable of processing extensive streams of audio, video, and text—1 billion text tokens, 10 billion audio tokens, and 1 trillion video tokens—directly on devices.As pioneers in innovative model architectures, our founding team, which originated from the Stanford AI Lab, has developed State Space Models (SSMs)—a groundbreaking foundation for training efficient, large-scale models. Our diverse team merges deep expertise in model innovation with a design-focused engineering approach, allowing us to create and deploy state-of-the-art models and applications.Backed by leading investors such as Index Ventures, Lightspeed Venture Partners, and many others, including industry veterans and advisors, we are poised to shape the future of AI.Your ContributionIn this role, you will drive forward-thinking research in neural network architecture, focusing on alternative models like state space models, efficient transformers, and hybrid architectures.Create innovative architectures that enhance model performance, inference speed, and adaptability in various environments, from cloud infrastructures to on-device implementations.Develop advanced capabilities for models, including statefulness, long-range memory, and novel conditioning mechanisms to boost expressiveness and generalization.Analyze architectural decisions and their effects on model characteristics such as scalability, robustness, latency, and energy consumption.Create frameworks and tools to assess architectural advancements, benchmarking their performance in both research and production contexts.Collaborate with interdisciplinary teams to translate architectural insights into scalable systems that deliver real-world impact.Your QualificationsExtensive experience in architecture design with a focus on advanced models such as state space models, transformers, and RNN/CNN variants.In-depth understanding of the interplay between architectural designs and system constraints, particularly in cloud and on-device deployments.Strong proficiency in the design and evaluation of neural network architectures.

At Scale AI, we are the premier partner for data and evaluation in the rapidly evolving field of artificial intelligence. Our commitment to advancing the assessment and benchmarking of large language models (LLMs) positions us at the forefront of AI innovation. We are dedicated to creating leading-edge LLM evaluation methodologies that set new benchmarks for model performance. Our research teams collaborate with the top AI laboratories in the industry to provide high-quality data, accelerate progress in generative AI research, and inform what excellence looks like in this domain. As a Staff Machine Learning Research Scientist on our LLM Evals team, you will spearhead the creation of novel evaluation methodologies, metrics, and benchmarks to assess the strengths and weaknesses of cutting-edge LLMs. Your work will shape our internal strategies and influence the broader AI research community, making this role essential for establishing best practices in data-driven AI development.

About the TeamThe Safety Systems team is dedicated to ensuring the responsible deployment of our advanced AI models for societal benefit. We lead OpenAI's mission to develop and implement safe AGI, prioritizing transparency and trust in our AI systems.The Model Safety Research team is focused on pioneering research to enhance the robustness and safety of AI models. Our goal is to tackle the evolving safety challenges that arise as AI becomes increasingly powerful and prevalent across various applications. Key areas of focus include the enforcement of nuanced safety policies, model robustness against adversarial threats, addressing privacy and security concerns, and ensuring trustworthiness in critical safety domains.We are committed to understanding real-world deployment and maximizing the benefits of AI while ensuring its safe and responsible use.About the RoleOpenAI is on the lookout for a passionate and experienced Senior Researcher specializing in AI safety. This role will guide research initiatives aimed at enabling safe AGI and will involve working on projects that enhance the safety, alignment, and robustness of our AI systems against adversarial threats. You will play a pivotal role in shaping the future of safe AI at OpenAI, significantly contributing to our mission of deploying safe AGI.In this role, you will:Engage in cutting-edge research on AI safety topics such as Reinforcement Learning from Human Feedback (RLHF), adversarial training, and system robustness.Implement innovative methods within OpenAI’s core model training processes and drive safety enhancements across our products.Define research directions and strategies to bolster the safety, alignment, and robustness of our AI systems.Collaborate with cross-functional teams, including Trust & Safety, legal, and policy experts, to ensure our products uphold the highest safety standards.Continuously assess and analyze the safety of our models and systems, pinpointing risks and proposing effective mitigation strategies.You might thrive in this role if you:Have a strong enthusiasm for AI safety and a solid background in safety research.Possess excellent analytical skills and the ability to think critically about complex safety challenges.Are adept at collaborating with diverse teams and communicating findings effectively.Have a proactive approach to problem-solving and a commitment to ethical AI deployment.

Join Anthropic as a Safeguards Enforcement Analyst, where you will play a pivotal role in ensuring safety evaluations within our innovative AI systems. This role focuses on analyzing compliance with safeguards and developing strategies to enhance safety protocols. Collaborate with cross-functional teams to assess risks and implement robust solutions that align with our commitment to responsible AI.

About Our TeamThe Safety Systems team is seeking a dedicated Technical Program Manager who will play a pivotal role in optimizing our comprehensive safety framework and integrating diverse safety research and mitigations into ChatGPT and our API. This position is essential for the secure deployment of our innovative models by synthesizing contributions from various stakeholders, including research, product development, engineering, legal, and policy teams, to ensure all risks are effectively monitored, mitigated, or resolved.About the RoleIn the position of Safety Engineering Technical Program Manager, you will oversee critical responsibilities such as tracking progress in safety engineering and managing risk assessments. You will also supervise key data infrastructure initiatives, acting as a crucial connector to enhance the implementation of OpenAI's safety systems. Moreover, you will develop and execute a computing roadmap for your team, ensuring that our primary objectives are adequately resourced while capitalizing on new opportunities for significant safety infrastructure investments. Your primary focus will be to establish a foundational layer that supports the safety of all our models and products.This role is located in San Francisco, CA. We operate on a hybrid work model, requiring employees to be in the office three days a week, and we provide relocation assistance to new hires.Responsibilities:Manage key risk domains and engage with relevant stakeholders.Collaborate directly with safety engineers, engineering managers, and product managers to establish a unified safety infrastructure.Oversee data and computational infrastructure, including capacity planning and data residency.Design and implement essential internal programs, including incident management and processes for regularly updating safety mitigations.Prioritize and manage a portfolio of infrastructure requests from internal teams.Ideal Candidates Will:Hold a Bachelor's or Master's degree in Computer Science or Computer Engineering, or possess substantial engineering expertise.Demonstrate a proven history of delivering complex technical projects on time and to high standards.Exhibit strong technical skills and have effectively collaborated with top-tier engineering and research teams.Show expertise in creating and implementing straightforward, scalable processes that address intricate challenges.Possess excellent communication and interpersonal skills to work across various teams.

About Our TeamJoin the Safety Systems team at OpenAI, where we are dedicated to ensuring that our cutting-edge models are deployed safely in the real world, positively impacting society. We are at the forefront of OpenAI's mission to develop and implement safe Artificial General Intelligence (AGI), emphasizing a culture of trust, transparency, and responsibility in AI.The Safety Research team is focused on advancing our capabilities to implement robust and safe behaviors in AI models and systems. As we make strides in AI capabilities, our safety approaches must evolve to effectively address the changing landscape of risks. This vigilance is essential not only for preventing harmful misuse but also for ensuring that potential misalignments do not result in adverse outcomes. Our research is grounded in current methodologies while also being adaptable to future systems.As we expand our team, we are looking for innovative research methods that enhance safety for AGI and beyond. This includes exploratory research into improving safety common sense and generalizable reasoning, developing evaluations to identify misalignment or hidden objectives of AI, and creating new strategies to support human oversight during long-term tasks.About the RoleIn your capacity as a Technical Lead, you will spearhead our strategic initiatives aimed at mitigating potential risks arising from misalignment or significant errors. Your responsibilities will encompass:Establishing visionary goals and milestones for new research endeavors, alongside crafting rigorous evaluations to monitor progress.Leading or driving research into new exploratory areas to validate the feasibility and scalability of our safety approaches.Collaborating across safety research and related teams to ensure that diverse technical strategies converge to deliver robust safety outcomes.We seek individuals with a proven track record in practical research concerning safety and alignment, particularly within the realms of AI and large language models (LLMs), who have successfully led substantial research initiatives in the past.This role is situated in San Francisco, CA, with a hybrid work model of three days in the office each week. We also offer relocation assistance to new employees.

Join Anthropic as a Senior Research Scientist on our Reward Models team, where you will spearhead groundbreaking research aimed at enhancing our understanding of human preferences at scale. Your innovative contributions will directly influence how our AI models, including Claude, align with human values and optimize for user needs. You will delve into the forefront of reward modeling for large language models, designing novel architectures and training methodologies for Reinforcement Learning from Human Feedback (RLHF). Your research will explore advanced evaluation techniques, including rubric-based grading, and tackle challenges such as reward hacking. Collaboration is key, as you'll work alongside teams in Finetuning, Alignment Science, and our broader research organization to ensure your findings result in tangible advancements in AI capabilities and safety. This role offers you an opportunity to address critical AI alignment challenges, leveraging cutting-edge models and substantial computational resources to further the science of safe and capable AI systems.

Lila Sciences is forming a dedicated AI safety team to address the unique risks and challenges posed by scientific superintelligence. The company seeks a Senior or Principal Technical Program Manager to guide the operational side of AI safety research, helping to shape how the team approaches complex and evolving problems. Role overview This Technical Program Manager position connects research, engineering, model development, policy, and executive leadership. The work involves translating fast-moving research into structured, accountable plans. While this is not a research role, curiosity about the technical aspects of AI safety is important. The team values clear communication and the ability to bring clarity and structure as the organization expands. What you will do Act as the primary communication link between the AI safety team and technical, research, and scientific groups. Share complex results and coordinate resource needs. Establish information flows to keep teams connected. Promote accountability within cross-functional, distributed teams, building consensus and trust through open communication and sound judgment. Support rapid experimentation and iteration by refining and applying effective program management practices. Create clear documentation and reports to communicate vision, track progress, and ensure alignment with company objectives. Accurately represent program status and risks, even in uncertain or shifting situations. Requirements Bachelor’s or Master’s degree in Computer Science, Engineering, Life Sciences, or a related discipline. Minimum of 6 years of program or project management experience in technology or life sciences. Demonstrated success in program management, leading cross-functional teams, and delivering projects. Strong analytical and problem-solving abilities, with skill in turning technical requirements into actionable plans. Excellent written and verbal communication skills, including experience preparing executive-level documents, roadmaps, and updates. Location This position is based in Cambridge, MA or San Francisco, CA, USA.