Jobs › Companies › FurtherAI › Senior / Staff AI Engineer — Evals & Model Quality

Über diese Senior / Staff AI Engineer — Evals & Model Quality Stelle bei FurtherAI

FurtherAI · Vor Ort · San Francisco

About Us

At FurtherAI, we’re building the AI agents that run insurance — a trillion-dollar industry still powered by manual work. By combining frontier agentic AI with deep domain expertise, we’re reshaping how the industry’s most critical work gets done.

We’ve raised more than $30M from top investors (Andreessen Horowitz, YC, Nexus, South Park Commons), grown more than XY00% in the last 12 months, and work with some of the largest names in insurance — top 10 carriers/brokers, Fortune 100 companies, and public companies.

We are a deeply technical company. Both founders are engineers, and engineers have an outsized say on product direction.

Now, we’re looking for an exceptional AI Engineer to own how we evaluate and continuously improve the quality of AI agents across our platform.

Why Join Us

  • Outsized ambition: We’re out to build the definitive AI company for insurance — and we’re moving faster than anyone in the space.

  • Elite team: Founders and engineers from the best companies and schools in the world — ex-Apple AI Research, Brex, Coinbase, and Uber; 4 YC founders and 6 ex-founders; grads of Stanford, Berkeley, IITs, and Cornell.

  • Massive market impact: Insurance is the backbone of global commerce. Our work reshapes how the industry operates.

  • Founder’s mindset: Own big pieces of the product from day one — ideal if you want to start your own company one day.

What You’ll Do

  • Own evaluations across FurtherAI: Build the evaluation frameworks and infrastructure that measure the accuracy, quality, and reliability of our AI agents across workflows, Assistant, document processing, and computer use.

  • Build high-quality datasets: Develop golden datasets, benchmarks, and regression suites, working closely with Forward Deployed Engineers and domain experts to capture real-world insurance tasks and edge cases.

  • Evaluate complex agent behavior: Design evaluations for multi-step agent trajectories, tool use, reasoning, and end-to-end task completion — not just individual LLM responses.

  • Build automated grading systems: Develop LLM-as-judge evaluations, deterministic graders, human evaluation workflows, and rubrics that accurately reflect real-world quality.

  • Make development evaluation-driven: Integrate offline and online evaluations into development and release processes so engineers can measure improvements, catch regressions, and ship confidently.

  • Drive model evaluation and selection: Continuously benchmark frontier and open-weight models across accuracy, latency, cost, and reliability. Use results to inform model selection, upgrades, and production releases.

  • Build continuous improvement loops: Turn production failures, customer feedback, and agent traces into new evaluation cases and measurable improvements in agent quality.

Representative Projects

  • Build an end-to-end evaluation framework for FurtherAI agents, measuring success across multi-step workflows, tool calls, and complex insurance tasks.

  • Create a library of golden datasets and benchmarks for common insurance workflows, reusable across customer implementations and out-of-the-box AI skills.

  • Build an automated benchmarking system that evaluates new frontier and open-weight model releases and identifies the best models for different tasks.

  • Develop production evaluation pipelines that detect quality regressions, identify failure patterns, and turn them into actionable improvements.

What We’re Looking For

  • 5+ years of software engineering experience, with strong hands-on experience building production AI/ML systems.

  • Experience building evaluation frameworks, LLM-based applications, model benchmarking systems, or applied AI infrastructure.

  • Strong understanding of modern LLMs, agent architectures, evaluation methodologies, and model performance tradeoffs.

  • Strong Python engineering skills and the ability to build reliable, production-grade systems.

  • Ability to independently own ambiguous technical problems, from research and experimentation through production.

  • Clear communicator — both in person and in writing.

  • Bonus: Experience with LLM-as-judge, human evaluation systems, agent trajectory evaluations, or large-scale benchmarking.

  • Bonus: Experience with open-weight models, fine-tuning, post-training, or model routing.

  • Bonus: Background in B2B SaaS and 0-to-1 experience.

  • Above all: drive, grit, and ownership. If you excel here, other requirements are secondary.

At FurtherAI, we set a high bar. We’re not looking for someone who just wants a job. We’re looking for someone who wants to build something transformative. If you thrive in environments where the pace is fast, expectations are high, and the rewards are outsized, you’ll love it here.

You can look forward to the following benefits:

  • Fully covered, best-in-class health, dental, and vision benefits

  • Competitive compensation, meaningful stock options, company 401(k)

  • Unlimited PTO

  • Outstanding in-office culture in the heart of San Francisco, 5x per week

  • Lunch and dinner onsite

  • Team events, such as happy hours and off-sites

  • Pre-tax commuter benefits

Bereit, sich bei FurtherAI zu bewerben?
Bei FurtherAI bewerben

Wie sich dieses Gehalt für AI Engineer vergleicht

Diese Stelle zahlt $325,000/yr — über der üblichen Spanne für AI Engineer Stellen.

$158,500 dem Median $228,500 $320,000

Übliche Spanne $195,250–$264,500/yr, aus 521 vergleichbaren AI Engineer Anzeigen auf JobsRadar (Vergütung auf USD hochgerechnet). Gehaltseinblicke für AI Engineer ansehen →

Ähnliche Jobs

OpenAI
Applied AI Engineer, Cyber
OpenAI
⚡ Früh bewerben San Francisco Hybrid $234,000–$335,000
● Neu 👁 Gesehen ✓ Beworben vor 4 Std.
Firecrawl
Machine Learning Engineer
Firecrawl
⚡ Früh bewerben San Francisco HQ; Toronto Hub Hybrid $250,000–$290,000
● Neu 👁 Gesehen ✓ Beworben vor 4 Std.
Cohere
Senior ML Systems Engineer, Frameworks & Tooling
Cohere
⚡ Früh bewerben London · standortgebunden CA$250,000–CA$535,000
● Neu 👁 Gesehen ✓ Beworben vor 4 Std.
Chime Financial, Inc
Staff AI/ML Engineer
Chime Financial, Inc
⚡ Früh bewerben San Francisco, CA, USA Hybrid $246,000–$340,000
● Neu 👁 Gesehen ✓ Beworben vor 5 Std.
Okta
Senior Software Engineer, Applied AI
Okta
⚡ Früh bewerben San Francisco, California Vor Ort $165,000–$226,000
● Neu 👁 Gesehen ✓ Beworben vor 5 Std.
Okta
Staff Software Engineer, Applied AI
Okta
⚡ Früh bewerben San Francisco, California Vor Ort $194,000–$267,000
● Neu 👁 Gesehen ✓ Beworben vor 5 Std.
Okta
Principal Software Engineer, Applied AI
Okta
⚡ Früh bewerben San Francisco, California Vor Ort $224,000–$308,000
● Neu 👁 Gesehen ✓ Beworben vor 5 Std.
DO
Staff Applied AI Engineer
Doximity
⚡ Früh bewerben San Francisco, CA or Remote (U... · standortgebunden $170,000–$248,000
● Neu 👁 Gesehen ✓ Beworben vor 6 Std.
Zipline
Staff Platform Software Engineer, AI Enablement
Zipline
⚡ Früh bewerben South San Francisco, Californi... Vor Ort
● Neu 👁 Gesehen ✓ Beworben vor 7 Std.

Registrieren für Vorschläge, die auf die von Ihnen geöffneten Jobs und gespeicherten Suchen zugeschnitten sind.

Mehr Jobs bei FurtherAI

Alle Jobs bei FurtherAI ansehen →

Jetzt bewerben
🤖

Moment — langsam

JobsRadar wurde für echte Menschen gebaut, die eine schwere Zeit bei der Jobsuche haben — nicht für automatisierte Anfragen. Sie klicken viel zu schnell und sind jetzt vorübergehend blockiert.

Kommen Sie später wieder. Wenn Sie wirklich auf Jobsuche sind, stehen wir hinter Ihnen — verhalten Sie sich einfach wie ein Mensch.

Catch your next role the second it’s posted.

Create a free account and we’ll watch the boards for you — the instant a job matches your search, it lands in your inbox or Telegram. No digging, no refreshing.

Create free account

Free forever · takes 30 seconds · already have one?

Verschaffe dir einen Vorsprung bei der Jobsuche.

Tritt unserem Telegram-Kanal bei für das, was dir hilft, die Stelle zu bekommen — Gehaltsbenchmarks, den wöchentlichen Marktpuls und neue Feature-Drops. Kein Spam, nur Signal.

Dem Kanal beitreten — kostenlos