Certified Tester AI Testing (CT-AI)

The ISTQB® AI Testing (CT-AI) certification extends understanding of artificial intelligence and/or deep (machine) learning, most specifically testing AI-based systems and using AI in testing.

Audience

The Certified Tester AI Testing certification is aimed at anyone involved in testing AI-based systems and/or AI for testing. This includes people in roles such as testers, test analysts, data analysts, test engineers, test consultants, test managers, user acceptance testers, and software developers. This certification is also appropriate for anyone who wants a basic understanding of testing AI-based systems and/or AI for testing, such as project managers, quality managers, software development managers, business analysts, operations team members, IT directors, and management consultants. To gain this certification, candidates must hold the Certified Tester Foundation Level certificate.

Content

ISTQB® Certified Tester – AI Testing (CT-AI)

Introduction to AI

Definition of IA and AI Effect

Narrow, General and Super AI

AI-Based and Conventional Systems

AI Technologies

AI Development Frameworks

Hardware for AI-Based Systems

AI as a Service (AIaaS)

Pre-Trained Models

Standards, Regulations and AI

Quality Characteristic for AI Based Systems

Flexibility and Adaptability

Autonomy

Evolution

Bias

Ethics

Side Effects and Reward Hacking

Transparency, Interpretability and Explainability

Safety and AI

Machine Learning (ML) – Overview

Forms of ML

ML Workflow

Selecting a Form of ML

Factors Involved in ML Algorithm Selection

Overfitting and Underfitting

ML Data

Data Preparation as Part of the ML Workflow

Training, Validation and Test Datasets in the ML Workflow

Dataset Quality Issues

Data Quality and its Effect on the ML Model

Data Labelling for Supervised Learning

ML Functional Performance Metrics

Confusion Matrix

Add ML Functional performance Metrics for Classification, Regression and Clustering

Limitations of ML Functional Performance Metrics

Selecting ML Functional Performance Metrics

Benchmark Suites for ML Performance

ML Neural Networks and Testing

Neural Networks

Coverage Measures for Neural Networks

Testing AI-Based Systems – Overview

Specification of AI-Based Systems

Test Levels of AI-Based systems

Test Data for Testing AI-Based Systems

Testing for Automation Bias in AI-Based Systems

Documenting an AI-Based Component

Testing for Concept Drift

Selecting a Test Approach for an ML System

Testing AI-Specific Quality Characteristics

Challenges Testing Self-Learning Systems

Testing Autonomous Self-Learning Systems

Testing for Algorithmic, Sample and Inappropriate Bias

Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems

Challenges Testing Complex AI-Based Systems

Testing Transparency Interpretability and Explainability of AI-Based Systems

Test Oracles for AI-Based Systems

Test objectives and Acceptance Criteria

Methods and Techniques for the Testing of AI-Based Systems

Adversarial Attacks and Data Poisoning

Pairwise Testing

A/B Testing

Back-to-Back Testing

Metamorphic Testing (MT)

Experience-Based Testing of AI-Based Systems

Selecting Test Techniques for AI-Based Systems

Testing Environments for AI-Based Systems

Test Environments for AI-Based Systems

Virtual Test Environments for Testing AI-Based Systems

Using AI for Testing

AI Technologies for Testing

Using AI to Analyze Defect Reports

Using AI to Test Case Generation

Using AI for the Optimization of Regression Test Suites

Using AI for Defect Prediction

Using AI for Testing User Interfaces

Exam Structure

No. of Questions: 40

Total Points: 47

Passing Score: 31

Exam Length (mins): 60

Business Outcomes

Individuals who hold the ISTQB® Certified Tester – AI Testing certification should be able to accomplish the following business outcomes:

Describe the purpose of test automation

Understand test automation through the software development lifecycle

Understand the Configuration of an Infrastructure to Enable Test Automation

Learn the evaluation process for selecting the right tools and strategies

Understand design concepts for building modular and scalable test automation solutions

Select an approach, including a pilot, to plan test automation deployment within the software development lifecycle

Design and develop (new or modified) test automation solutions that meet technical needs

Consider scope and approach of test automation and maintenance of testware

Understand how automated tests integrate within CI/CD pipelines

Understand how to collect, analyze, and report on test automation data in order to inform stakeholders

Verify the test automation infrastructure

Define continuous improvement opportunities for test automation

More Information

The other AI-related syllabi from A4Q, AiU, and CSTQB/KSTQB will be valid until October 12th, 2022.

Accredited courses require accreditation of training materials, as described in the ISTQB® Accreditation Process.

Holders of the A4Q, AiU, and KSTQB & CSTQB AIT previous versions continue to hold a valid certification.

Training is available from Accredited Training Providers (classroom, virtual, and e-learning). We highly recommend attending accredited training as it ensures that an ISTQB® Member Board has assessed the materials for relevance and consistency against the syllabus. Self-study, using the syllabus and recommended reading material, is also an option when preparing for the exam. Holders of this certification may choose to proceed to other Core, Agile, or Specialist stream certifications.

Materials that you can download

Certified Tester AI Testing (CT-AI)

Exam Structure

Business Outcomes

More Information

Materials that you can download

Syllabus

Sample Exams

Exam Structures and Rules

General Files