Senior AI Quality Assurance Engineer

Role Overview

An AI Quality Assurance Engineer ensures the accuracy, reliability, and compliance of AI systems and models before deployment. This role is critical for validating AI-driven applications, detecting defects, and safeguarding against risks such as bias, hallucinations, and non-deterministic behavior. The position involves designing robust test strategies for machine learning models and collaborating with data scientists, ML engineers, and product teams to maintain high-quality standards.

Key Responsibilities

· Testing & Validation

o Develop and execute comprehensive test plans for AI models and systems.

o Perform manual and automated tests to validate AI functionality under various scenarios.

o Conduct bias detection, edge-case testing, and stress testing for AI models.

· Automation & Tools

o Design and implement automated testing frameworks for AI workflows.

o Utilize AI-powered QA tools and self-healing test scripts to improve efficiency.

· Compliance & Governance

o Ensure AI systems adhere to ethical guidelines, regulatory standards, and internal policies.

o Maintain documentation such as Generative AI Bill of Materials (BoM) for transparency and audits.

· Collaboration

o Work closely with ML engineers, data scientists, and product managers to resolve defects.

o Participate in code reviews and contribute to continuous improvement initiatives.

· Monitoring & Reporting

o Analyze test results, identify defects, and track quality metrics.

o Provide risk assessments and recommendations for AI deployments.

Required Skills & Qualifications

· Education: Bachelor’s in Computer Science, Software Engineering, or related field.

· Experience:

o 3–7 years in QA, with at least 2 years in AI/ML testing.

o Familiarity with non-deterministic testing and AI-specific QA challenges.

· Technical Expertise:

o Proficiency in Python and testing frameworks (e.g., PyTest, Selenium).

o Knowledge of ML frameworks (TensorFlow, PyTorch) and MLOps tools.

o Experience with CI/CD pipelines and cloud platforms (AWS, Azure, GCP).

· Soft Skills:

o Strong analytical and problem-solving abilities.

o Excellent communication and collaboration skills.

Preferred Skills

· Understanding of ethical AI principles and bias testing.

· Familiarity with low-code/no-code automation tools and AI-driven QA platforms.

· Knowledge of regulatory compliance for AI systems.

#LI-DNI

#LifeAtBottomline

We are looking for a detail-oriented Software QA Automation Engineer to join our team. In this role, you will be responsible for building and maintaining automated test scripts that ensure the quality, reliability, and performance of our applications. You will collaborate closely with developers, QA team members, and product managers to deliver high-quality software in a fast-paced environment.

Responsibilities: 

• Develop, maintain, and execute automated test scripts for web, API, and/or mobile applications. 

• Collaborate with QA engineers, developers, and product managers to ensure adequate test coverage for new and existing features. 

• Participate in Agile ceremonies including sprint planning, standups, and sprint reviews to understand product requirements and testing needs. 

• Identify, document, and track software defects with clear, reproducible steps. 

• Perform regression, functional, and exploratory testing using both automation and manual testing approaches as needed. 

• Contribute to the design and improvement of automation frameworks and QA processes. 

• Support continuous integration and deployment pipelines by integrating automated testing into CI/CD workflows. 

Position Requirements: 

• Bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience. 

• 5+ years of hands-on experience in software testing and automation. 

• Familiarity using AI tools as part of the automation process (Cursor preferred). 

• Proficiency in one or more programming or scripting languages such as TypeScript, Java, Python, JavaScript, or C#. 

• Experience with test automation tools such as Playwright, Cypress, Selenium, or Appium. 

• Experience with API testing tools such as Postman, RestAssured, or similar technologies. 

• Knowledge of CI/CD tools and version control systems such as Git, Jenkins, GitHub Actions, or GitLab CI. 

• Strong understanding of software testing methodologies, QA best practices, and Agile development processes. 

• Strong analytical and problem-solving skills with attention to detail. 

Preferred Qualifications:

• Experience with performance or load testing tools such as JMeter, Gatling, or Locust. 

• Experience working with cloud platforms such as AWS, Azure, or Google Cloud Platform. 

• Basic understanding of containerization technologies such as Docker or Kubernetes. 
• ISTQB or equivalent QA certification. 

Who we’re looking for: We are seeking an experienced AI/ML Engineer with a minimum of 5 years of expertise to join our innovative technology team. In this pivotal role, you will be responsible for designing, training, and optimizing AI models, with a strong focus on Natural Language Processing (NLP) for real-time content summarization, recommendation, and conversational UIs. You will implement cutting-edge techniques like Retrieval-Augmented Generation (RAG) and fine-tune transformer-based models, collaborating closely with backend and data engineering teams to ensure seamless deployment and maintenance of brand tone and content accuracy in all AI outputs.

What you’ll be doing:

• Design, train, and optimize AI models specifically for real-time content summarization and recommendation.
• Implement advanced Natural Language Processing (NLP) techniques to enhance conversational UI and predictive interactions.
• Apply Retrieval-Augmented Generation (RAG) for intelligent response generation from dynamic content sources.
• Fine-tune transformer-based models (e.g., BERT, T5) using domain-specific datasets to achieve optimal performance.
• Collaborate closely with backend and data engineering teams to streamline model deployment within a production-grade environment.
• Ensure that brand tone and content accuracy are meticulously maintained throughout all AI-generated outputs.

What you’ll need:

• Bachelor's degree in Computer Science, Artificial Intelligence, Data Science, or a related technical field.
• 5–7 years of progressive experience in AI/ML engineering, with a strong and proven focus on Natural Language Processing (NLP).
• Strong practical experience with leading Machine Learning frameworks such as TensorFlow, PyTorch, and Hugging Face.
• Experience with LangChain and LangGraph.
• Experience with MLOps pipelines and platforms like AWS SageMaker, Azure AI Search, or similar.
• Solid understanding of Large Language Models (LLMs), embeddings, and semantic search technologies.
• Expert proficiency in Python and robust data handling libraries such as Pandas and NumPy.
• Familiarity with deploying AI models via microservices or APIs.
• Preferred Qualifications:
• Understanding of branding, tone, and the user experience (UX) implications in AI-generated content.
• Experience working with headless CMS (Content Management Systems) and structured content datasets.
• Knowledge of ethical AI principles and prompt engineering best practices.

Additional Competencies:

• AI/ML Engineering
• Natural Language Processing (NLP)
• Large Language Models (LLMs)
• MLOps & Deployment
• Python & Data Science
• Transformer Models
• Content Summarization & Recommendation
• Brand & Content Integrity

About us:

Artefact is a new generation of a data service provider, specialising in data consulting and data-driven digital marketing, dedicated to transforming data into business impact across the entire value chain of organisations. We are proud to say we’re enjoying skyrocketing growth.

Our broad range of data-driven solutions in data consulting and digital marketing are designed to meet our clients’ specific needs, always conceived with a business-centric approach and delivered with tangible results. Our data-driven services are built upon the deep AI expertise we’ve acquired with our 1000+ client base around the globe.

We have 1000 employees across 20 offices who are focused on accelerating digital transformation. Thanks to a unique mix of company assets: State of the art data technologies, lean AI agile methodologies for fast delivery, and cohesive teams of the finest business consultants, data analysts, data scientists, data engineers, and digital experts, all dedicated to bringing extra value to every client.

About the role:

As a Data Analyst, your role will encompass:

• Conducting projects to accompany the transformation of your clients’ businesses through the effective collection, processing, and visualisation of data
• Extracting valuable insights from our clients’ marketing-related data sources.
• Designing dashboards for marketing decision-making while taking into account the business needs
• Accompanying our clients in the conception and implementation of data architectures and data pipelines, from collection to monitoring.
• Actively contributing to the expertise level and competencies of the Data & Analytics team
• Closely collaborate with the other divisions (Media & Activation, Creation, Consulting, Data Science and Data Engineer) to provide comprehensive services to your clients

Qualifications

Having an academic level of education (Bachelor or Master), in a data-driven working environment, you have experience or have knowledge in :

• A minimum of 1 year of work experience in a data-driven working environment
• 1 or more data collection (Google Analytics, Adobe Analytics, SQL database, CRM database, transactional data)
• 1 or 2 programming languages among SQL, Python and/or R.
• Must have extensive experience in either Tableau or Power BI. Experience in both is preferred but not essential.

Ideally, you have the knowledge or take interested in one or more of the following technologies:

• 1 Google Cloud Platform resource (Big Query, Google Cloud Storage) or its equivalent in Azure or AWS
• You know how to connect to an API through the knowledge of the different authentication method (OAuth, HTTP Basic, API key)
• 1 advertising tool, linked to tracking and/or analytics in a marketing context (Google Analytics, Adobe Analytics, DMP, DSP, Adservers, DV 360…)
• 1 ETL tool (Airflow, Dataiku, Dataflow, Dataprep…)
• 1 web application (R-Shiny, Python Flask) that you have industrialized

As a bonus, you have:

• Basic knowledge of statistics
• Basic knowledge of machine learning and data engineering

Why you should join us

• Artefact is the place to be: come and build the future of marketing
• Progress: every day offers new challenges and new opportunities to learn
• Culture: join the best team you could ever imagine
• Entrepreneurship: you will be joining a team of driven entrepreneurs. We won’t give up until we make a huge dent in this industry!

Come join us!

Artefact is a new generation of data service providers specialising in data consulting and data-driven digital marketing. It is dedicated to transforming data into business impact across the entire value chain of organisations. We are proud to say we’re enjoying skyrocketing growth.

The backbone of our consulting missions, today our Data consulting team has more than 400 consultants covering all Artefact's offers (and more): data marketing, data governance, strategy consulting, product owner…

What you will be doing?
As a Data Engineer, your role involves crafting and maintaining robust data pipelines, utilising Python and SQL, to ensure efficient extraction, transformation, and loading (ETL) of data.

Your responsibilities will include:

• Data Pipeline Development: Building and optimising data pipelines to facilitate seamless data flow across systems and platforms.
• Database Management: Managing databases, ensuring their integrity, and implementing data storage and retrieval solutions.
• Cloud Services Integration: Leveraging cloud services such as MS Azure, GCP, and AWS to architect and deploy scalable data solutions.
• Machine Learning Integration: Collaborating with teams to integrate machine learning models into data pipelines for enhanced data processing.
• Utilising Spark & Kafka: Implementing and working with Spark and Kafka for real-time data processing and analytics.

What we are looking for?

• Proficiency in Python, SQL, and database management.
• Experience with Data Pipelines ETL, Cloud Services (MS Azure, GCP preferred), ML Modeling, Spark & Kafka.
• Proven problem-solving skills and a solution-oriented mindset.
• Excellent communication skills to collaborate effectively within teams and with stakeholders.
• Strong business acumen with an interest in business-facing roles.
• Adaptability and a start-up mentality to thrive in a dynamic environment.
• Candidates with similar skill sets and experiences have excelled in technology firms or consultancy firms.
• Successful candidates often possess Computer Science, Electronics, and Communication Engineering degrees.