About the Role

AI now touches almost every conversation we have with patients — consultations, medical support, health coaching, AI assistants, and patient experience support. As we scale, we need one standardised way to answer: are these conversations safe, accurate, and high quality?

That’s what you’ll build. You’ll own the development of a standardised evaluations layer across all of our support workstreams — clinical interactions (consultations, medical support), health coach conversations, AI assistant conversations, PX support, and health insights (wellness vs. clinical). You’ll build the services that monitor what is said to our patients and flag risky conversations and insights. You’ll also own how our AI assistants integrate with our in-house patient data models, so every assistant responds with accurate, well-grounded patient context.

This is a rare junior role: real ownership of a safety-critical system from day one, with the support of experienced teams around you. You’ll report to the Head of Data and work closely with the ML Engineering, Data Engineering and Data Science teams.

What You'll Own

• Design and build a standardised evaluation framework covering clinical interactions, health coach conversations, AI assistant conversations, PX support, and health insights
• Build and run services that monitor patient-facing conversations and automatically flag risky content for review and escalation
• Develop eval datasets, metrics, and thresholds with clinicians, coaches, and support leads — turning “what good looks like” into something measurable
• Own the technical integration between our AI assistants and our data models, making patient & business context available to assistants safely, and consistently
• Operate your systems in production: monitoring, alerting, and iterating as models and prompts evolve
• Own and automate guardrail updates based on internal and patient feedback from interactions with our third-party support tools
• Share results with the teams who own each workstream, and help them improve

About You

• A degree in computer science, engineering, data science, or a related field — or equivalent practical experience
• Strong Python skills and the fundamentals to build reliable services, not just notebooks
• Hands-on experience with LLMs: prompt engineering, and ideally exposure to eval frameworks or LLM-as-judge approaches (coursework, internships, or personal projects all count)
• A careful, safety-first mindset — you care about what an AI system says to a real patient, and you sweat the edge cases
• Clear communication — you can explain eval results to a clinician as well as an engineer
• Comfortable with ambiguity: this layer doesn’t exist yet, and you’ll help define it

Nice to have

• Experience with eval tooling (e.g. promptfoo, Ragas, LangSmith, or in-house harnesses)
• Exposure to healthcare, regulated industries, or other high-stakes AI applications
• Familiarity with GCP, SQL, or modern data stacks

Why join Eucalyptus?

Euc is also behind a growing family of digital healthcare clinics (Pilot, Kin, Software, Compound) across men’s health and well-being, fertility, skincare, and preventative health.

Here’s what makes joining Euc unique:

• What’s next - Our goal for the next three years is to support 1 million patients globally to live better for longer. We’re launching into new conditions, demographics, and geographies as we build a truly preventive healthcare ecosystem.
• Build something world-changing - We’re on the path to becoming the world’s largest international digital healthcare company. It will be challenging, fast-paced, and deeply rewarding.
• Make real impact - You will deliver work that directly shapes patient outcomes and scales evidence-based care across markets.
• Accelerate your growth - You will have high ownership, continuous feedback, and dedicated development support.
• Join a motivated team - You will collaborate with talented peers to solve complex clinical and operational problems at scale.

What’s ahead in Australia?

• Grow with us globally - Get mentorship from leaders across Australia, the UK, Germany, and Japan, collaborate with teams in the Philippines and South Africa, and explore new markets or travel internationally. ****Recognised on Hatch’s Hotlist as one of Australia’s top employers, we support early-career professionals to become future global leaders.
• Shape your career - Access learning budgets, conferences, certifications, peer shadowing, and a global knowledge-sharing culture.
• Work where well-being matters - Experience catered wellness talks, exercise classes, whoops to track your wellbeing, free barista coffees, funded social clubs, and quarterly rooftop parties.
• Innovate with purpose - Use state-of-the-art tools and contribute to bold, impactful solutions in healthcare.
• Support at every stage - Benefit from parental and miscarriage leave, to health, professional development, and personal days, and our Employee Assistance Program.