Pinterest is the world's leading visual search and discovery platform, serving over 500 million monthly active users globally on their journey from inspiration to action. As we scale Generative AI experiences — including Pinterest Assistant and Canvas — ensuring these products are safe, fair, and trustworthy is paramount. We are looking for a Senior Data Scientist to help lead Pinterest's responsible AI mandate within the Advanced Technology Group (ATG). In this role, you will design and build the data science foundations for automated adversarial testing of our GenAI products — developing attack strategies, evaluation frameworks, and harm-detection methodologies that operate at scale. You will work in a highly collaborative and cross-functional environment, partnering with ML Engineers, Trust & Safety specialists, Policy teams, and Product Managers. You are expected to develop a deep understanding of generative AI vulnerabilities, and generate insights and robust methodologies to proactively surface and mitigate risks. The results of your work will directly influence product safety, policy compliance, and user trust across Pinterest.

What you'll do

• Design and develop automated adversarial testing methodologies — including single-turn, multi-turn, and multimodal attack strategies — to proactively identify vulnerabilities in Pinterest's Generative AI products.
• Build and calibrate hybrid evaluation pipelines combining LLM-based judges, classifiers, and rule-based systems to accurately detect safety violations, policy breaches, bias, and representational harms.
• Develop and operationalize harm taxonomies grounded in industry standards and Pinterest's Responsible AI and Trust & Safety threat models.
• Design adaptive refinement loops that learn from attack outcomes (near-misses, partial failures) to iteratively surface deeper and previously unknown vulnerabilities.
• Bring scientific rigor and statistical methods to the evaluation of AI safety — including benchmark dataset construction, evaluation calibration, and success-metric definition (vulnerability severity, coverage breadth, pre-launch risk reduction).
• Work cross-functionally to build relationships, proactively communicate key findings, and collaborate closely with ML engineers, Trust & Safety specialists, policy teams, product managers, and legal partners to ensure safe product launches.
• Relentlessly focus on impact, whether through influencing product safety strategy, advancing responsible AI metrics, or improving critical evaluation processes.
• Mentor and up-level junior data scientists and cross-functional partners on adversarial evaluation, responsible AI methodologies, and safety-aware data science practices.

What we're looking for

• 5+ years of experience analyzing data in a fast-paced, data-driven environment with proven ability to apply scientific methods to solve real-world problems on web-scale data.
• Strong interest and hands-on experience in one or more of: AI safety, adversarial machine learning, red teaming, responsible AI, or trust & safety.
• Deep familiarity with large language models (LLMs), generative AI systems, and their failure modes — including prompt injection, jailbreaks, bias, and safety violations.
• Experience designing and calibrating evaluation frameworks for AI systems — including LLM-as-judge, classifier-based evaluation, and benchmark dataset construction.
• Strong quantitative programming (Python) and data manipulation skills (SQL/Spark); experience with ML pipelines and large-scale experimentation.
• Familiarity with AI safety taxonomies and frameworks (e.g., OWASP LLM Top 10, MITRE ATLAS) is strongly preferred.
• Ability to work independently, drive ambiguous projects end-to-end, and operate with high ownership.
• Excellent written and verbal communication skills, with the ability to explain complex technical findings to both technical and non-technical partners.
• A team player eager to partner across Responsible AI, Trust & Safety, Product, Engineering, Policy, and Legal to turn safety insights into action.

This position is not eligible for relocation assistance.

#LI-NM4 

Pinterest is the world's leading visual search and discovery platform, serving over 500 million monthly active users globally on their journey from inspiration to action. As we scale and unify our foundational AI models across every recommendation surface, rigorously measuring the system-level impact of these advances is essential to our ability to ship with confidence and compound gains over time.

We are looking for a Senior Data Scientist to serve as the measurement backbone for Pinterest's foundational model initiatives within the Advanced Technology Group (ATG). In this role, you will design the evaluation frameworks, experimentation strategies, and system-level metrics that quantify how improvements to our largest and most complex models translate into real user and business outcomes. You will work in a highly collaborative and cross-functional environment, partnering with ML Engineers, Applied Scientists, Product Managers, and ML Platform engineers. 

You are expected to develop a deep understanding of Pinterest's recommendation ecosystem, from representation learning to retrieval, ranking, reinforcement learning, and beyond, and bring the analytical rigor necessary to decompose, attribute, and communicate the impact of foundational model changes across this interconnected system. The results of your work will directly influence model investment decisions, launch criteria, and the pace of innovation across ATG and Pinterest.

What You'll Do

• Design and execute system-level measurement frameworks for foundational model improvements spanning offline evaluation benchmarks, online A/B experiments, and longitudinal impact tracking across surfaces.
• Define, and own the success metrics that quantify foundational model value.
• Build causal inference methodologies to isolate the incremental impact of individual model components within a complex, multi-model production system where changes co-occur and interact.
• Work cross-functionally to build relationships, proactively communicate key findings, and collaborate closely with ML Engineers, Applied Scientists, Homefeed and Surface teams to ensure measurement rigor is embedded in every model launch.
• Relentlessly focus on impact, whether through sharpening investment decisions with data, raising the bar for launch criteria, accelerating experimentation velocity, or surfacing hidden inefficiencies in the model ecosystem.

What We're Looking For

• 5+ years of experience analyzing data in a fast-paced, data-driven environment with proven ability to apply scientific methods to solve real-world problems on web-scale data.
• Strong interest and hands-on experience in one or more of: ML system evaluation, recommender system measurement, A/B experimentation at scale, causal inference
• Deep familiarity with large-scale recommendation or ranking systems and their evaluation including an understanding of how representation learning, retrieval, ranking, and re-ranking stages interact and compound in production.
• Experience designing and executing A/B experiments for complex ML systems,  including multi-surface holdouts, metric decomposition, long-run effect estimation, and interference/spillover mitigation.
• Strong quantitative programming (Python) and data manipulation skills (SQL/Spark); experience with ML pipelines, feature stores, and large-scale experimentation platforms.
• Ability to work independently, drive ambiguous projects end-to-end, and operate with high ownership in a fast-moving research-to-production environment.
• Excellent written and verbal communication skills, with the ability to translate complex system-level findings into clear narratives for technical and non-technical partners including leadership-level investment recommendations.
• A team player eager to partner across teams to turn measurement insights into better models and faster launches.

This position is not eligible for relocation assistance.

#LI-NM4

The Responsible AI team is part of the Advanced Technologies Group (ATG), Pinterest’s advanced machine learning team. ATG’s goal is to keep Pinterest at the forefront of machine learning technology across multiple use cases including recommendations, ranking, content understanding, and more. It is an applied team that works horizontally across the company on state of the art AI and ML and works on directly bringing that technology to the product in collaboration with product engineering teams. The team also publishes its work in applied research conferences, but the main goal of the team is to have a direct impact on business metrics.

At Pinterest our goal is to inspire pinners (our users) to live the life they love. The product is powered by state of the art ML algorithms which are used to understand both the billions of visually rich items on the platform and the interests of our 550M+ monthly active users and recommend inspiring personalized content to them. What this means is that ML at Pinterest is not only multi-modal utilizing image, text, and graph signals as input, but it needs to operate at a very large scale and often in a real time interactive user experience.

Pinterest is known for being a positive and inspirational place on the internet and we believe that inspiration begins with representation and belonging. The Responsible AI team is a horizontal team that collaborates across the company on various initiatives. These range from Generative AI alignment, evaluation, and mitigations - ensuring these models are safe, bias-free, and aligned with Pinterest’s vision and policies - to championing ML Fairness and developing user-facing features that enhance our product for all users. This includes launching groundbreaking features such as user-controllable skin tone, hair pattern search refinements, and more recently, body type.

We are looking for a Sr Machine Learning Engineer. You’ll help shape forward-thinking projects, develop advanced ML approaches rooted in fairness and equitability, and pioneer responsible AI safeguards for emerging technologies. By joining this team, you’ll make a lasting impact on our Pinners, our business, and the evolution of ethical AI at Pinterest.

What you'll do:

• Design and drive projects in the responsible AI frontier to identify, avoid, and mitigate bias across a wide range of ML applications at Pinterest. These include but are not limited to Gen AI product evaluations, foundation models safety fine tuning and alignment, and Red Teaming. 
• Collaborate with other engineering teams (trust and safety, ML Platform, Content Understanding, Search, Homefeed,) to leverage their platforms and signals and work with them to collaborate on the adoption and evaluation of Responsible AI practices and ML Fairness tooling across Pinterest.
• Mentor junior engineers on the Responsible AI team and across the company on the R-AI space.
• Work with the team and senior leaders at the company to define and drive technical strategy in this area.

What we're looking for:

• Extensive, real-world experience applying advanced ML methods to production systems, with a strong track record in responsible technology - spanning fairness, ethics, and broader societal considerations.
• Deep familiarity with cutting-edge ML architectures (e.g., transformer-based models, 2-tower architectures, LLMs and VLMS) and their applications in large-scale Search and Recommender Systems.
• Proven ability to measure, deploy, and refine fairness interventions and broader Responsible AI solutions at scale, bridging state-of-the-art research with tangible product impact.
• 4+ years working experience in the engineering teams that build large-scale ML-driven user-facing products
• 1+ years experience leading cross-team engineering efforts.
• Masters or PhD in Comp Sci or related fields

Nice to have:

• Publications at top ML conferences
• Experience using Cursor, Copilot, Codex, or similar AI coding assistants for development, debugging, testing, and refactoring
• Familiarity with LLM-powered productivity tools for documentation search, experiment analysis, SQL/data exploration, and engineering workflow acceleration
• Expertise in scalable realtime systems that process stream data
• Passion for applied research and the Pinterest product

Relocation Statement:

• This position is not eligible for relocation assistance. Visit our PinFlex page to learn more about our working model.

In-Office Requirement Statement:

• We let the type of work you do guide the collaboration style. That means we're not always working in an office, but we continue to gather for key moments of collaboration and connection.
• This role will need to be in the office for in-person collaboration 1-2 times per quarter and therefore can be situated anywhere in the United States. 

#LI-REMOTE
#LI-AK7