Jobs › Companies › Relativity › Product Manager - AI Evaluations Tooling

Sobre este puesto de Product Manager - AI Evaluations Tooling en Relativity

Relativity · Presencial · Illinois

Posting Type

Remote/Hybrid

Job Overview

Relativity's AI portfolio spans 20+ products and we are moving quickly into agentic capabilities. As that portfolio grows, so does the opportunity to give every team building AI at Relativity a shared, standardized way to understand and improve their own AI performance. The Evaluations Tooling team exists to build that foundation: a platform that standardizes how we evaluate AI, accelerates how quickly teams can ship it with confidence, and turns AI quality into shared evidence that applied scientists, application teams, product managers, and legal experts can all act on.

We are looking for a Product Manager to own the product strategy and roadmap for Relativity's evaluation platform. Your mission is to enable every team at Relativity building AI products to understand, measure, and improve their own AI performance. That means productizing what applied scientists have already validated into a platform that application teams, PMs, and legal subject-matter experts can use directly; building a world-class agentic testing system that lets us ship new agentic capabilities quickly and confidently; and using production monitoring and team-facing dashboards to move the organization from reacting to quality problems to proactively getting ahead of them.

This is a platform PM role with unusually direct leverage. A successful candidate is energized by deeply technical customers, comfortable making build-vs-buy calls in a fast-moving vendor landscape, and able to hold a categorical bar ("every AI product ships behind elite evals") while sequencing pragmatically to get there.

Job Description and Requirements

Role Responsibilities  

  • Own the product vision, strategy, and roadmap for the Evaluations Platform across its pillars: offline evaluation pipeline, online evaluation and production monitoring, eval discovery and reuse, SME authoring and approval, and agentic evaluation.  
  • Serve every team building AI at Relativity. Run continuous discovery with applied scientists, application engineering teams, and product PMs to understand how each of them experiences AI quality today and design the platform so teams can measure and improve their own performance without routing every question through an applied scientist.  
  • Lead the orchestration of a world-class agentic testing system: trajectory evaluation, tool-call correctness, intermediate-state rubrics, long-running judge orchestration, and simulation of multi-step flows. Make it fast and safe to ship new agentic capabilities and make the platform a competitive advantage in how quickly Relativity can iterate on agents.  
  • Work closely with legal subject-matter experts to define the tasks our AI is evaluated against, what a correct outcome looks like, where the hard cases are, and how judgment criteria should be expressed so they can be encoded, scored, and reused across products. Build the SME authoring and approval workflow around how experts work.  
  • Own the team-facing quality dashboards. Every AI product team should be able to open a view and understand their own performance, where they stand against their quality bar, and what changed since last release, with no engineering help required. 
  • Lead build-vs-buy evaluations for tracing infrastructure and commercial eval tooling and run a recurring review as the vendor landscape shifts.  
  • Partner with engineering and applied science on judge calibration so LLM-as-judge is trustworthy enough to gate deployments. 
  • Clearly articulate technical tradeoffs to stakeholders ranging from applied scientists to attorneys to executive leadership. 

 Preferred qualifications:

  • Experience with LLM-based products and a working understanding of how they are evaluated (rubrics, LLM-as-judge, offline vs. online evaluation). Direct experience with agentic systems is a plus, not a requirement.  
  • Experience as a platform, developer-tools, or data/ML PM serving technical customers, or a demonstrated ability to learn a technical domain quickly and earn credibility with engineers and scientists.  
  • Comfort bringing non-engineering domain experts into technical workflows and translating their judgment into something a system can act on.  
  • Strong analytical instincts: sets metrics, tests assumptions, and changes course when the evidence says to. 

 

Minimum qualifications:

  • 4+ years of product management experience at a technology company, with at least 2 years on technical, platform, infrastructure, or data/ML products. 
  • Solid understanding of the software development lifecycle and agile practices. 
  • Experience conducting user discovery with technical audiences and translating findings into roadmap decisions. 
  • Excellent written and verbal communication skills, with the ability to explain complex technical concepts to both technical and non-technical audiences. 

. 

Relativity is committed to competitive, fair, and equitable compensation practices.

This position is eligible for total compensation which includes a competitive base salary, an annual performance bonus, and long-term incentives.

The expected salary range for this role is between following values:

$115,000 and $173,000

The final offered salary will be based on several factors, including but not limited to the candidate's depth of experience, skill set, qualifications, and internal pay equity. Hiring at the top end of the range would not be typical, to allow for future meaningful salary growth in this position. 

Required Skills:

Agile Methodology, Cross-Functional Teamwork, Market Research, Market Strategy, Product Lifecycle, Product Lifecycle Management (PLM), Product Management, Product Strategies, Stakeholder Management, Team Leadership
¿Listo para postularte en Relativity?
Postúlate en Relativity

Cómo se compara este salario de Product Manager

Este puesto paga $144,000/yr — en línea con el rango típico para los puestos de Product Manager.

$114,900 la mediana de $161,236 $224,500

Rango típico $133,006–$195,457/yr, a partir de 287 ofertas comparables de Product Manager en JobsRadar (salario anualizado en USD). Ver datos salariales de Product Manager →

Sobre Relativity

We’re solving big data challenges in the legal tech industry, and we’re always looking for more people to join us on the journey. At Relativity, you'll learn cross-functional skills to grow your career and have the chance to make a big impact on our customers, our industry, and our communities. We admire and value our employees, so it’s no surprise that our hiring process is designed to help us really get to know you – and for you to get to know us, too.

Ver todos los empleos en Relativity →

Empleos similares

Reddit
Lead Product Adoption Strategist, SMB
Reddit
⚡ Postúlate pronto Chicago, IL; New York City, NY Presencial $182,000–$254,800
● Nuevo 👁 Visto ✓ Postulado hace 9h
Reddit
Senior Product Manager, Go-To-Market Systems
Reddit
⚡ Postúlate pronto Chicago, IL; Los Angeles, CA;... Presencial $154,700–$216,600
● Nuevo 👁 Visto ✓ Postulado hace 9h
Thinkahead
Business Product Manager
Thinkahead
⚡ Postúlate pronto 1200 Technology Way, Libertyvi... Híbrido
● Nuevo 👁 Visto ✓ Postulado hace 18h
Okta
Staff Product Marketing Manager - Comp Intel
Okta
⚡ Postúlate pronto Bellevue, WA, United States; C... Presencial $166,000–$227,000
● Nuevo 👁 Visto ✓ Postulado hace 21h
Qualtrics
Analyst, Technical Customer Manager - Product & Services
Qualtrics
⚡ Postúlate pronto Provo, Utah, United States Híbrido
● Nuevo 👁 Visto ✓ Postulado hace 22h
Qualtrics
Principal Technical Customer Manager - Product & Services
Qualtrics
⚡ Postúlate pronto Chicago, Illinois, United Stat... Híbrido $121,500–$147,500
● Nuevo 👁 Visto ✓ Postulado hace 22h
Qualtrics
Senior Analyst, Technical Customer Manager - Product & Services
Qualtrics
⚡ Postúlate pronto Provo, Utah, United States Híbrido
● Nuevo 👁 Visto ✓ Postulado hace 22h
Samsara
Principal Product Manager
Samsara
⚡ Postúlate pronto Remote - CA; Remote - Chicago;... Híbrido $137,200–$245,000
● Nuevo 👁 Visto ✓ Postulado hace 1d
Samsara
Lead Product Manager
Samsara
⚡ Postúlate pronto Remote - CA; Remote - Chicago;... Híbrido $130,602–$219,500
● Nuevo 👁 Visto ✓ Postulado hace 1d

Regístrate para recibir sugerencias adaptadas a los empleos que abres y las búsquedas que guardas.

Más empleos en Relativity

Ver todos los empleos en Relativity →

Postúlate ahora
🤖

Un momento — para

JobsRadar se creó para personas reales que están pasando un mal momento en su búsqueda de empleo — no para solicitudes automatizadas. Estás haciendo clic demasiado rápido y ahora estás bloqueado temporalmente.

Vuelve más tarde. Si de verdad estás buscando empleo, cuentas con nosotros — solo compórtate como una persona.

Catch your next role the second it’s posted.

Create a free account and we’ll watch the boards for you — the instant a job matches your search, it lands in your inbox or Telegram. No digging, no refreshing.

Create free account

Free forever · takes 30 seconds · already have one?

Toma ventaja en tu búsqueda de empleo.

Únete a nuestro canal de Telegram para lo que te ayuda a conseguir el puesto — referencias salariales, el pulso semanal del mercado y avisos de nuevas funciones. Sin spam, solo señal.

Únete al canal — es gratis