The Behavior team at Motional is responsible for developing the prediction, planning, and sim agent capabilities that enable our autonomous vehicles to navigate complex, dynamic environments safely, comfortably, and confidently. Our work spans understanding and anticipating the behavior of other road users, making intelligent driving decisions, and generating safe, efficient, and natural vehicle trajectories. We combine cutting-edge machine learning, robotics, large-scale data, and rigorous evaluation to advance the capabilities of our production autonomous driving system.

We are looking for a highly motivated, technically strong engineer to lead evaluation initiatives across Prediction and Planning. In this role, you will define how we measure, understand, and improve autonomous driving behavior, bridging the gap between individual model performance and real-world system outcomes. You will drive the development of advanced evaluation methodologies, performance metrics, data mining strategies, and scalable tools to identify system limitations, assess dataset coverage, and guide model development.

You will work closely with machine learning engineers, roboticists, safety, and cross-functional teams to tackle challenging, open-ended problems and establish the quantitative foundations that drive improvements in safety, comfort, and driving performance. Your work will directly influence how we develop, evaluate, and deploy the next generation of autonomous driving capabilities.

What You’ll Do

• Act as the central authority on evaluation for our Behaviors teams, bridging the gap between model development and system-level autonomous behavior.
• Drive the evaluation strategy for our prediction and planning models, including open-loop and simulation-based assessments.
• Design and implement advanced metrics to measure model impact on overall system behavior, leveraging ML-based approaches for robust, end-to-end model evaluation.
• Incorporate concepts such as counterfactual reasoning and reaction time analysis into our evaluation frameworks to better understand system behavior.
• Develop the strategy for data mining and assess the coverage of our training and evaluation datasets across Prediction and Planning.
• Build necessary tooling and pipelines, identify gaps in our processes, and collaborate with machine learning engineers and roboticists to solve open-ended problems.
• Cross-functional collaboration: Partner with Safety, Infrastructure, Operations, Legal, and the other Systems teams to drive design improvements and ensure features generate a net positive effect on system safety, comfort, and assertiveness and meet our launch criteria.

What We’re Looking For:

• Bachelor’s degree and 7+ years of experience in Computer Science, Systems Engineering, Robotics, or a similar technical field (or Master’s/PhD and 5+ years of experience).
• Strong foundation in probability/statistics, data science, and scientific rigor applied to complex, multi-agent systems.
• Proficiency in Python, SQL, and data analysis (visualization, statistics) in production environments.
• Experience defining performance targets and acceptance criteria spanning safety, comfort, and assertiveness, producing the analysis and evidence to justify them.
• Demonstrated ability to tackle technical challenges, analyze cross-disciplinary problems, and propose innovative, maintainable solutions.
• Proven ability to lead initiatives, drive alignment, and ensure stakeholder engagement throughout the development process.
• Working expertise in hazard and safety analysis methods such as STPA, HAZOP, FMEA, and hazard analysis and risk assessment.

Ideally, You Also Have:

• Experience using statistical and ML methods to quantify performance, detect regressions, and reveal patterns of system behavior at scale.
• Deep experience with Prediction and/or Planning for autonomous robotics, including evaluating ML-based systems in production environments.
• Experience with public AV datasets and benchmarks (e.g., Waymo Open Dataset, nuScenes, etc.) and a deep understanding of standard industry metrics for trajectory forecasting and motion planning.
• Prior experience using or building data pipelines at scale.
• Excellent communication skills, capable of articulating complex concepts to technical peers and executive leadership alike, with a proven ability to collaborate across distributed teams.
• Familiarity with SOTIF, ISO 26262, UL4600, and AVSC behavior competencies and other industry standards.