About this QA Architect - R01571456 role at Brillio
Brillio · Onsite · Bangalore, Karnataka, India
QA Architect
Job requirements
Experience Range: With at least 7 years of quality assurance experience, including substantial hands-on work with data science and machine learning testing frameworks Key Responsibilities:
Design and implement automated testing strategies for AI and data science outputs, ensuring accuracy and reliability across models and pipelinesDevelop and maintain robust evaluation and validation frameworks for backend and frontend components, leveraging statistical and machine learning techniquesCollaborate with data scientists and engineers to define test cases, hypotheses, and statistical metrics for model assessment and improvementIntegrate advanced statistical tests such as T-Test, Z-Test, and regression analyses into automated QA workflows to validate model performanceUtilize tools like Great Expectations, Evidently AI, and specific machine learning frameworks (TensorFlow, PyTorch, Sci-Kit Learn, CNTK, Keras, MXNet) to monitor, track, and report on model drift, anomalies, and forecast accuracyOptimize testing processes for scalability and efficiency using Python, PySpark, R, and related technologies in large-scale data environmentsConfigure and manage testing infrastructure using platforms such as KubeFlow and BentoML to streamline deployment and evaluation cyclesTroubleshoot and resolve issues in automated testing pipelines, driving continuous improvement and high-quality deliverablesRequired Skills:
Advanced proficiency in Python and PySpark for test automation and statistical analysisExpertise in statistical testing methods including Hypothesis Testing, T-Test, Z-Test, and Regression (Linear, Logistic)Strong experience with machine learning frameworks such as TensorFlow, PyTorch, Sci-Kit Learn, CNTK, Keras, and MXNetHands-on knowledge of Great Expectations and Evidently AI for data validation and monitoringProficiency in SAS and SPSS for statistical computing and analysisDeep understanding of probabilistic graph models and classification algorithms including Decision Trees and SVMExperience with forecasting techniques including Exponential Smoothing, ARIMA, and ARIMAXFamiliarity with distance metrics such as Hamming, Euclidean, and Manhattan DistanceAdvanced skills in R and R Studio for statistical modeling and QA scriptingExperience configuring testing platforms such as KubeFlow and BentoMLPreferred Skills:
Experience automating evaluation pipelines for AI/ML in production environmentsExpertise in integrating QA processes with CI/CD workflows and cloud-native architecturesKnowledge of emerging ML testing tools and frameworks beyond industry standardsAbility to develop custom statistical metrics for model evaluationExperience with QA automation for distributed systems at scaleDesired Qualifications:
Bachelor's degree in Computer Science, Data Science, Statistics, Mathematics, or a quantitative disciplineCertification in Quality Assurance, Data Science, or Machine Learning (e.g., ISTQB Advanced Test Analyst, TensorFlow Developer Certificate)Certification in statistical analysis tools or platforms (e.g., SAS Certified Specialist, SPSS Certification)
