This role is for one of the Weekday's clients

Salary range: Rs 1200000 - Rs 2000000 (ie INR 12-20 LPA)

Experience: 2+ yrs

Location: Bengaluru

Job Type: Full-Time

We are looking for an AI Product & Evals Analyst to drive evaluation, quality measurement, product analytics, and enterprise reporting for AI-powered voice and conversational systems. This role sits at the intersection of Product, AI Engineering, Data Analytics, and Customer Success, where you will define how AI performance is measured, monitored, and continuously improved.

You will be responsible for building robust evaluation frameworks, analyzing conversational intelligence, creating enterprise-grade dashboards, and delivering actionable insights that enhance AI quality, customer experience, and business outcomes. The ideal candidate combines strong analytical skills with an AI-first mindset and enjoys leveraging modern AI tools, automation workflows, and data-driven decision-making to scale enterprise AI products.

Requirements

Key Responsibilities

• Design and manage AI evaluation frameworks to measure the quality and performance of voice and conversational AI systems.
• Define conversation-level and interaction-level metrics to monitor AI effectiveness, accuracy, and user experience.
• Analyze business KPIs such as automation rates, containment, customer satisfaction, conversion, operational efficiency, and cost optimization.
• Build and maintain enterprise dashboards, reports, and visualizations using modern business intelligence tools.
• Perform data analysis using SQL and Python to identify trends, detect regressions, and generate actionable insights.
• Develop reusable analytics frameworks, reporting models, and self-service intelligence solutions for internal and enterprise users.
• Leverage AI tools, LLMs, and workflow automation to streamline analytics, reporting, and root cause analysis.
• Collaborate with Product, AI Engineering, Customer Success, and Operations teams to improve product quality and enterprise adoption.
• Support enterprise reporting, usage analytics, operational reconciliation, and performance tracking initiatives.
• Translate complex AI behavior into clear business recommendations and communicate insights effectively to technical and non-technical stakeholders.

What Makes You a Great Fit

• 2–5 years of experience in Product Analytics, Data Analytics, AI Quality, or similar analytical roles.
• Strong proficiency in SQL and Python for data analysis, reporting, and investigative analytics.
• Experience designing dashboards using tools such as Power BI, Tableau, Looker, or similar visualization platforms.
• Excellent analytical thinking with the ability to define metrics, interpret data, and solve complex business problems.
• Strong understanding of conversational systems, workflow-driven products, and AI-powered applications.
• Ability to collaborate effectively with cross-functional teams including Product, Engineering, Operations, and Customer Success.
• Excellent communication and storytelling skills with the ability to convert technical findings into business insights.
• AI-first mindset with experience using modern AI tools, copilots, and automation platforms to improve productivity.
• Exposure to Conversational AI, LLMs, prompt engineering, voice technologies such as STT, or AI evaluation methodologies will be an added advantage.
• Self-driven, detail-oriented, and passionate about building scalable analytics frameworks for enterprise AI products.