Your role
As a Sr. SDET in Agentic QA, you will own the test automation and quality frameworks that support Dialpad’s AI Voice Agent services. You will develop automated tests for end-to-end product experiences, from frontend UI to backend services to APIs to audio/text interactions. You will test orchestration flows, agent configuration experiences, and guardian safeguards to create robust automated coverage for functionality, performance, reliability, UX, and more. 

In this role, you will develop substantial amounts of automated test infrastructure and partner deeply with the development team to make our fast-growing AI platform more testable, more stable, and more delightful for customers.

This position is based at one of Dialpad’s Canadian offices and reports to a QA Eng Manager in the United States.

What you’ll do

• Own end-to-end quality for agentic features and workflows, including strategy, development, execution, and release qualification.
• Design and build automation tooling and frameworks for AI/LLM-driven systems, including prompt flows, agent orchestration, and tool integrations.
• Develop and maintain evaluation frameworks (evals) to measure response quality, accuracy, and hallucination rates.
• Drive automation coverage (80%+ for critical AI workflows) using deterministic + probabilistic validation approaches.
• Integrate AI quality checks into CI/CD pipelines with fast feedback cycles (<15 minutes for PR validation).
• Build tooling for LLM observability and debugging, including prompt tracing and response analysis.
• Partner with Applied AI teams on prompt engineering, model selection, and evaluation strategies.
• Design and execute performance and load tests for AI services (latency, throughput, cost efficiency).
• Identify and mitigate risks related to hallucinations, bias, safety, and edge cases.
• Define and track AI quality KPIs (task success rates, precision/recall, latency, etc.).
• Participate in design and architecture reviews to ensure systems are testable, observable, and resilient.
• Mentor engineers and contribute to raising the bar on AI quality engineering practices.

What you’ll bring

• 5+ years of experience in software engineering or SDET roles with an emphasis on software development.
• Strong programming skills in Python (preferred), Java, or JavaScript.
• Experience testing distributed, cloud-native SaaS systems and APIs.
• Demonstrated proficiency in coding with AI agents to accelerate development and improve code quality.
• Hands-on exposure to LLMs or AI/ML systems (e.g., OpenAI, Claude, Gemini, or similar platforms).
• Understanding of non-deterministic systems and probabilistic testing approaches.
• Experience building test frameworks and scalable automation systems.
• Familiarity with AI evaluation techniques (benchmarking, golden datasets, human-in-the-loop validation).
• Experience with CI/CD pipelines (e.g., Jenkins, GitHub Actions).
• Strong collaboration skills with the ability to work across distributed teams and time zones.
• Bachelor’s degree in Computer Science or equivalent practical experience.
• Backend: Python, Go, Google Cloud Platform, Cloud Run / App Engine, Kubernetes, Datastore, Redis, ElasticSearch.
• Frontend: Vue3, React.
• AI Stack: LLM APIs, LiveKit, prompt orchestration frameworks, evaluation tooling.

Your role
As a Sr. SDET in Agentic QA, you will own the test automation and quality frameworks that support Dialpad’s AI Voice Agent services. You will develop automated tests for end-to-end product experiences, from frontend UI to backend services to APIs to audio/text interactions. You will test orchestration flows, agent configuration experiences, and guardian safeguards to create robust automated coverage for functionality, performance, reliability, UX, and more. 

In this role, you will develop substantial amounts of automated test infrastructure and partner deeply with the development team to make our fast-growing AI platform more testable, more stable, and more delightful for customers.

This position is based at one of Dialpad’s Canadian offices and reports to a QA Eng Manager in the United States.

What you’ll do

• Own end-to-end quality for agentic features and workflows, including strategy, development, execution, and release qualification.
• Design and build automation tooling and frameworks for AI/LLM-driven systems, including prompt flows, agent orchestration, and tool integrations.
• Develop and maintain evaluation frameworks (evals) to measure response quality, accuracy, and hallucination rates.
• Drive automation coverage (80%+ for critical AI workflows) using deterministic + probabilistic validation approaches.
• Integrate AI quality checks into CI/CD pipelines with fast feedback cycles (<15 minutes for PR validation).
• Build tooling for LLM observability and debugging, including prompt tracing and response analysis.
• Partner with Applied AI teams on prompt engineering, model selection, and evaluation strategies.
• Design and execute performance and load tests for AI services (latency, throughput, cost efficiency).
• Identify and mitigate risks related to hallucinations, bias, safety, and edge cases.
• Define and track AI quality KPIs (task success rates, precision/recall, latency, etc.).
• Participate in design and architecture reviews to ensure systems are testable, observable, and resilient.
• Mentor engineers and contribute to raising the bar on AI quality engineering practices.

What you’ll bring

• 5+ years of experience in software engineering or SDET roles with an emphasis on software development.
• Strong programming skills in Python (preferred), Java, or JavaScript.
• Experience testing distributed, cloud-native SaaS systems and APIs.
• Demonstrated proficiency in coding with AI agents to accelerate development and improve code quality.
• Hands-on exposure to LLMs or AI/ML systems (e.g., OpenAI, Claude, Gemini, or similar platforms).
• Understanding of non-deterministic systems and probabilistic testing approaches.
• Experience building test frameworks and scalable automation systems.
• Familiarity with AI evaluation techniques (benchmarking, golden datasets, human-in-the-loop validation).
• Experience with CI/CD pipelines (e.g., Jenkins, GitHub Actions).
• Strong collaboration skills with the ability to work across distributed teams and time zones.
• Bachelor’s degree in Computer Science or equivalent practical experience.
• Backend: Python, Go, Google Cloud Platform, Cloud Run / App Engine, Kubernetes, Datastore, Redis, ElasticSearch.
• Frontend: Vue3, React.
• AI Stack: LLM APIs, LiveKit, prompt orchestration frameworks, evaluation tooling.

As Marqeta’s  Manager of Service Delivery & Partner Performance you'll be at the forefront of transforming customer support into a competitive advantage. This role puts you in the driver's seat to revolutionize how we deliver exceptional cardholder experiences through data-driven insights, cutting-edge technology, and strategic partnerships.

You'll have the autonomy to identify opportunities, architect solutions, and implement change that measurably elevates our support operations and cardholder’s experience. Working with talented cross-functional teams and strategic BPO partners, you'll not only meet KPIs and SLAs—you'll redefine what excellence looks like in fintech customer support.

This is your opportunity to make a tangible impact on cardholders while building a high-performing support ecosystem. You'll shape performance standards, drive operational transformation, and create the frameworks that scale as Marqeta grows. If you thrive on turning data into action, building collaborative partnerships, and seeing your strategies deliver real results, this role is your platform to shine.

We work Flexible First. This role can be performed remotely anywhere within Ontario or British Columbia, Canada. We’d love for you to join us!

This position is not for an existing vacancy.

The Impact You’ll Have

• Performance Fanatic: You don’t sit around and wait for things to happen. You strategize, plan, and monitor a path to operational success based on the right data.
• BPO Vendor Management: Manage the operational success of multiple programs across multiple products through building and maintaining effective relationships with operational BPO site leadership
• Performance Analysis & Reporting: Monitor and analyze contact center data (KPIs, metrics) to identify trends and areas for improvement.
• Process Optimization: Identify gaps, develop and implement sustainable and scalable solutions  to enhance operational efficiency, workflows, and customer satisfaction.
• Quality Assurance & Compliance: Define what great looks like and collaborate with QA and Complaints teams to ensure adherence to quality standards and compliance requirements.
• Technology Implementation: Utilize and drive enhancements through new tooling and AI solutions across our telephony, IVR and CRM system to improve agent efficiency and effectiveness 
• Leadership & Coaching: Motivate staff, provide coaching, and influence positive outcomes to cultivate a high-performing culture.
• Escalation Management: Resolve complex customer issues during moments of truth.
• Cardholder Experience Optimization: Foster strong relationships with key Program stakeholders to collaborate on enhancing cardholder experience through sharing expertise, CSAT/QA results and data drive operational efficiencies
• Change Management: Central role to new program launches and existing program refreshes
• Marqeta Culture Carrier: Champion culture cross-functionally through actively embodying Marqeta values, enhancing engagement, and facilitating culture-building activities 

Who You Are

• 7+ years of Customer Support Operations with equivalent years of experience in BPO industry or management of BPOs
• Strong communication skills (interpersonal, verbal, presentation, written, email) and track record of creating and executing strategic operational efficiencies 
• Demonstrated ability to drive customer satisfaction and experience using people, processes or technology to deliver exception outcomes
• Proven ability to influence and manage stakeholders (internal and external) to achieve positive outcomes
• Experience using data to drive insights and recommend changes for improvement
• Strong technical aptitude including Google Suite, Salesforce, Slack and contact center telephony systems (ex. Amazon Connect)
• Familiarity and first-hand knowledge in fintech, banking or payments
• Positive attitude, team player, adaptable, and resourceful
• Detail-oriented, organized, and adept at managing competing priorities
• Exercises sound judgement and decision-making across diverse scenarios
• Ability to travel globally

Nice-To-Haves

• Deep knowledge of payment processing, program management and/or issuing of debit and credit cards
• Demonstrated ability to manage and operate Amazon Connect
• Strong operational and customer-facing documentation skills with the ability to produce clear, engaging internal and external support guides
• Experience implementing and growing communication channels beyond phone and email
• Proven success in integrating AI solutions into BPO operations from generative AI tooling to agentic implementation of voice or chatbots

Your Manager

• Jena Baker, Ops Director of Global Customer Support Operations

Typical Process

• Application Submission
• Recruiter video call
• Hiring manager video call
• Virtual “Onsite” consisting of 4-5, 45 min calls

Compensation and Benefits

Marqeta is a Flex First company which allows you to choose your best working environment, whether that be from home or at a company office. To support Flex First, we calibrate pay to a competitive value according to working location. 

When determining salaries, we consider several factors including, but not limited to, skills, prior experience, and work location. The new-hire base salary range for this position, reflected in CAD,  is: 114,500 - 143,000

We also believe in recognizing the contributions of our people. That's why we award annual bonuses to eligible employees, rewarding both individual performance and the success of the entire company.

Along with monetary compensation, Marqeta offers

• Multiple health insurance options
• Flexible vacation time
• Retirement savings program with company contribution
• Equity in a publicly-traded company 
• Monthly stipend to support our remote work model
• Annual “development dollars” to support our people growth and development
• Family-forming benefits and up to 20 weeks of Parental Leave

Join Horace Mann and Unlock Your Financial Potential

Ready to take control of your financial future? At Horace Mann, we are here to support you in achieving your goals and making a meaningful difference in the lives of the educators who trust you. Join us and take the first step toward building a brighter, more prosperous future.

If you're ready to take the next step toward realizing your financial goals, we invite you to join our team as an Exclusive Insurance Agent. With Horace Mann’s expertise and your ambition, together, we can help you unlock your full potential.

What We Offer:

• Competitive, performance-based [1099] compensation with an industry-leading 48-month incentive package
• Earning incentives tied to your activity and success during the first 48 months
• Quarterly production incentives for the first 48 months, rewarding consistent performance
• Dedicated Service Representative to handle client service work, allowing you to focus on building your business
• Exclusive territory in a niche market designed to increase your opportunities for success
• Cutting-edge technology and ongoing training to support and grow your operations
• A comprehensive multiline product portfolio to meet a variety of client needs
• Market and relationship-building programs to help you establish and grow your network
• No external office required for 2 years

Your Path to Success:

Several factors will contribute to your success in this role, including:

• A commitment to identifying and implementing solutions that help educators achieve affordable insurance solutions and financial security

• A focus on achieving market access and building strong relationships
• The ability to confidently present products to both groups and individuals
• Active engagement in networking, community, and industry events
• A dedication to investing time and resources to ensure the long-term success of your business

What We’re Looking For:

• Strong interpersonal and business management skills to build and manage your agency
• 2-5 years of experience in the insurance and financial services industry (preferred)
• Resident State General Lines Insurance Licenses:
• Life and Health Insurance License
• Property and Casualty Insurance License
• Ability to obtain FINRA Series 6 & 63 licenses (if applicable in your region)

#LI-MH1

#VIZI#