Model Evaluation & Red Teaming
Certify your expertise in rigorous evaluation and red teaming of generative AI models: metrics, adversarial testing, bias and vulnerability detection.
Key skills
- ✓Evaluate the performance and limitations of LLM models with quantitative metrics
- ✓Conduct red teaming campaigns to identify vulnerabilities
- ✓Detect and measure biases, hallucinations and behavioural drifts
- ✓Design adversarial testing protocols and document risks
The question bank for this exam is prepared after your purchase. The exam is available no later than 48 hours after payment; you will receive an email as soon as it opens.
Is this right for you?
✓ This certification is right for you if:
- →ML Engineers working on generative models
- →AI Safety Specialists and alignment researchers
- →AI security auditors and consultants
- →AI product managers and governance teams
✗ This certification is not suitable if:
- →Beginners with no prior experience with language models → prefer PROVA Prompt Engineering Foundations
- →AI Safety experts seeking a research-level certification → prefer upcoming PROVA AI Alignment & Safety (Master 801)
Prerequisites & eligibility
Open registration, technical prerequisites verified via self-assessment questionnaire.
Purchase options
Exam voucher, LMS pathway, bundle packs — build your order
Options d'achat
Composez votre parcours — voucher d'examen, préparation LMS, packs entreprise.
Skills assessed
Quantitative evaluation
Mastery of metrics (perplexity, BLEU, ROUGE, BERTScore) and standardised benchmarks for LLMs.
Operational red teaming
Design and execution of adversarial attacks (prompt injection, jailbreak, exfiltration).
Bias detection
Identification and quantification of social, cultural and representational biases in outputs.
Risk documentation
Writing structured evaluation reports compliant with European standards (AI Act).
Exam format
Curriculum
Fundamentals of model evaluation
Classic evaluation metrics (accuracy, precision, recall, F1-score, perplexity, log-likelihood). Standardised benchmarks for LLMs: MMLU, HellaSwag, TruthfulQA, BBH, HumanEval. Limits of automatic metrics and need for human evaluation. A/B evaluation protocols and Elo comparisons. NLG-specific metrics: BLEU, ROUGE, METEOR, BERTScore, BLEURT. Evaluation of consistency, fluency and relevance. Evaluation frameworks: HELM, Eleuther AI Harness, OpenAI Evals.
Red teaming and adversarial testing
Taxonomy of adversarial attacks on LLMs. Prompt injection (direct and indirect). Jailbreaking techniques and guardrail bypass. Adversarial suffix attacks (GCG, AutoPrompt). Training data exfiltration and membership inference. Context poisoning attacks. Model inversion and parameter extraction. Red teaming tools: PyRIT (Microsoft), Garak, Promptfoo, OWASP LLM Top 10. Design of red teaming campaigns: scenarios, personas, success metrics. Documentation and reporting of discovered vulnerabilities.
Bias and hallucination detection
Taxonomy of biases in models: representation bias, stereotypes, cultural and linguistic biases. Quantitative bias measurement: disparate impact, demographic parity, equalized odds. Audit tools: Fairlearn, AI Fairness 360, WinoBias, StereoSet. Detection and quantification of factual hallucinations. Fact verification methods: retrieval-augmented generation, automatic fact-checking. Analysis of temporal and logical consistency. Evaluation of robustness to phrasing variations. Consistency tests and model calibration.
Documentation and compliance
Writing model cards according to the Mitchell et al. standard. Datasheets for datasets. Structured red teaming reports: methodology, findings, recommendations. European AI Act compliance: obligations for high-risk systems, technical documentation, traceability. AI risk management system. Communication of limitations and risks to non-technical stakeholders. Versioning and traceability of evaluations. Integration into CI/CD pipelines. Governance and accountability of models in production.
How to fund your certification
3 ways to cover the cost of your certification
OPCO — Employer funding
Ask your employer to cover the cost through your OPCO (employee training plan).
Personal — €450 incl. VAT
Secure online payment by credit card (Stripe).
Business rate: €585 excl. VAT
Business — custom quote
Teams, cohorts or private catalogue: a quote tailored to your volumes and goals.
Frequently asked questions
Do I need to master model fine-tuning to take this certification?
No, but a solid understanding of the internal workings of LLMs is required. The exam assesses your ability to test and audit existing models, not to train them. Knowledge of PyTorch/TensorFlow is a plus but not mandatory.
Which tools are covered in the exam?
PyRIT (Microsoft), Garak, Promptfoo, OWASP LLM Top 10, AI Fairness 360, Fairlearn, as well as evaluation frameworks like HELM and Eleuther AI Harness.
Is this certification recognised for AI Act audits?
Yes. The PROVA framework integrates the requirements of the European AI Act for high-risk system evaluation. Certified individuals are trained in the documentation and traceability obligations imposed by the regulation.
Your certification pathway
Before, during, after — the recommended progression
Droits du candidat
Transparence
Les critères de décision et le barème sont documentés et accessibles avant l'examen.
Droit d'appel
Toute décision peut être contestée dans les 30 jours. Examen par un tiers indépendant.
Plainte
Toute personne peut signaler un dysfonctionnement. Formulaire public accessible sans compte.
