What's the opportunity?

We’re looking for an experienced SDET with strong hands-on experience in agentic AI systems who will contribute to accelerating agentic AI adoption across RBC. This role offers the opportunity to build production-grade solutions, work collaboratively with Lines of Business teams, and develop deep expertise in the Borealis agentic AI platform.

At RBC Borealis, you’ll be joining a team that bridges Lines of Business with our cutting-edge Borealis agentic AI platform. You’ll work directly with business stakeholders, platform engineers, and technical teams to implement high-impact solutions while building your capability in emerging agentic technologies with guidance from our Principal Engineers.

What will you do?

• Pioneer the future of autonomous testing by designing and implementing intelligent test systems across all levels (unit, integration, end-to-end), becoming the go-to expert on AI-driven testing procedures, agent orchestration, and risk mitigation in cutting-edge applications.
• Shape groundbreaking test strategies for AI systems: Take ownership of test specifications for LLM-powered applications, architect innovative prompt validation frameworks, and establish observability practices that push the boundaries of quality assurance.
• Engineer next-generation test solutions using cutting-edge tools like Playwright, Stagehand, Locust, and Allure, seamlessly integrated with LLM orchestration frameworks to create self-healing, adaptive test systems that evolve alongside your applications.
• Build and command fleets of intelligent SDET Agents that revolutionize testing workflows—agents that generate their own test cases, dynamically validate prompts, and autonomously triage defects with unprecedented efficiency.
• Orchestrate the complete lifecycle of AI application testing: Lead autonomous test execution, harness AI-driven defect detection and classification, unlock actionable insights through LLM-powered analysis, and establish world-class observability practices.
• Drive innovation through collaboration: Partner closely with AI/ML engineers, prompt engineers, and developers to ensure bulletproof testing of intelligent systems, pushing reliability and performance to new heights in unexplored territory.
• Build the knowledge foundation for tomorrow: Document test cases for LLM behaviors and agent interaction patterns with crystal clarity, creating a knowledge base that empowers your team and advances the state of the art in AI testing.

What do you need to succeed?

Must Haves:

• 3+ years of Quality Engineering experience in diverse environments (cloud, distributed, APIs, databases, AI/ML systems), with proven ability to resolve complex cross-functional issues in agent-based architectures.
• 3–5 years of hands-on automation experience designing efficient test strategies for autonomous systems, reducing manual testing through intelligent agents, and covering component, integration, and end-to-end scenarios.
• Deep expertise in testing AI agent applications: Proficiency in testing LLM-based agents, multi-agent systems, prompt engineering validation, output consistency testing, and handling non-deterministic behavior.
• AI framework proficiency: Hands-on experience with LangChain, LangGraph, CrewAI, AutoGen, or similar frameworks for building and testing agent workflows.
• DevOps proficiency in CI/CD, GitHub Actions, JIRA, and qTest, with experience integrating AI model evaluation into deployment pipelines.
• Automation tools expertise in Playwright, Locust, Python httpx, and AI-specific evaluation tools like promptfoo
• Programming skills in Python, TypeScript, SQL, and source control tools (Git), with strong understanding of asynchronous programming for agent orchestration.
• Experience in Agile/Iterative development, defining test strategies for AI systems, evaluating LLM outputs, and reviewing artifacts (agent code, prompt templates, automation scripts).

Nice to have:

• Knowledge of prompt engineering techniques: Including few-shot learning, chain-of-thought prompting, ReAct patterns, and validation of prompt robustness.
• Knowledge in AI observability: Experience with platforms like Langfuse or Arize Phoenix for tracing agent execution, debugging multi-step reasoning, analyzing LLM calls, and monitoring performance metrics.
• Understanding of Retrieval Augmented Generation (RAG) pipelines and testing strategies for vector databases, embedding quality, and retrieval accuracy.

What’s in it for you?

We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.

• A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable
• Leaders who support your development through coaching and managing opportunities
• Ability to make a difference and lasting impact
• Work in a dynamic, collaborative, progressive, and high-performing team
• A world-class training program in financial services
• Flexible work/life balance options
• Opportunities to do challenging work

#LI-POST

#TechPJ

Job Skills

Additional Job Details

Note: Applications will be accepted until 11:59 PM on the day prior to the application deadline date above

Our Employment Opportunities

At RBC, we are guided by living shared values of Client First, Integrity, Collaboration, Respect and Excellence and winning together as One RBC. We believe an inclusive workplace that has diverse perspectives is core to our continued growth as one of the largest and most successful banks in the world. Maintaining a workplace where our employees feel supported to perform at their best, effectively collaborate, drive innovation, and grow professionally helps to bring our Purpose to life and create value for our clients and communities. RBC strives to deliver this through policies and programs intended to foster a workplace based on respect, belonging and opportunity for all.

RBC is presently inviting candidates to apply for this existing vacancy. Applying to this posting allows you to express your interest in this current career opportunity at RBC. Qualified applicants may be contacted to review their resume in more detail.