Yiga AI

Course 10 of 13

AI Safety, Evaluation & Monitoring

Red-teaming, Evals, Observability, and more.

Yiga AI

Advanced en

Advanced · Course · 3 weeks at 5 hrs/week

Duration
3 weeks
Effort
5 hrs/week
Lessons
16
Hands-on exercises
16
Projects
1
Language
en

What you’ll learn

Skills you can put to work.

LLMs for Developers

Prompt Engineering for Production

Retrieval-Augmented Generation (RAG)

Building AI Agents

Syllabus

Every module, step by step.

4 modules · 21 lessons · 14 hours · 1 graded project

840 minutes of guided lesson time.

Sign in and enrol to unlock every lesson. The first lesson is open as a preview.

Module 1 · 5 lessons · 3 hoursRed-teamingModule 1 of 4 in AI Safety, Evaluation & Monitoring.

  1. Overview and set-up

    Overview and set-up — Red-teaming in AI Safety, Evaluation & Monitoring.

    Lesson

    40 min

  2. Core concepts

    Core concepts — Red-teaming in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  3. Worked example

    Worked example — Red-teaming in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  4. Hands-on practice

    Hands-on practice — Red-teaming in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  5. Module quiz: Red-teaming

    Check what you learned in Red-teaming before moving on.

    Quiz · Locked

    20 min
Module 2 · 5 lessons · 3 hours LockedEvalsModule 2 of 4 in AI Safety, Evaluation & Monitoring.

  1. Overview and set-up

    Overview and set-up — Evals in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  2. Core concepts

    Core concepts — Evals in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  3. Worked example

    Worked example — Evals in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  4. Hands-on practice

    Hands-on practice — Evals in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  5. Module quiz: Evals

    Check what you learned in Evals before moving on.

    Quiz · Locked

    20 min
Module 3 · 5 lessons · 3 hours LockedObservabilityModule 3 of 4 in AI Safety, Evaluation & Monitoring.

  1. Overview and set-up

    Overview and set-up — Observability in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  2. Core concepts

    Core concepts — Observability in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  3. Worked example

    Worked example — Observability in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  4. Hands-on practice

    Hands-on practice — Observability in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  5. Module quiz: Observability

    Check what you learned in Observability before moving on.

    Quiz · Locked

    20 min
Module 4 · 6 lessons · 5 hours LockedCost controlModule 4 of 4 in AI Safety, Evaluation & Monitoring.

  1. Overview and set-up

    Overview and set-up — Cost control in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  2. Core concepts

    Core concepts — Cost control in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  3. Worked example

    Worked example — Cost control in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  4. Hands-on practice

    Hands-on practice — Cost control in AI Safety, Evaluation & Monitoring.

    Lesson · Locked

    40 min

  5. Module quiz: Cost control

    Check what you learned in Cost control before moving on.

    Quiz · Locked

    20 min

  6. Graded project: AI Safety, Evaluation & Monitoring

    Apply everything from AI Safety, Evaluation & Monitoring in one graded, portfolio-ready build.

    Graded project · Locked

    120 min

Applied learning project

Make the learning visible.

Use Graded project: AI Safety, Evaluation & Monitoring to apply this course to a practical challenge. You’ll finish with a clear solution and the reasoning behind your choices.

Apply the course’s core skillMake and explain practical decisionsCreate evidence you can share
Mariam K., Software & cloud

Your instructor

Mariam K.

Software & cloud

Helps new developers build reliable products through clear explanations and guided practice.

Diana Nsiiza, Yiga learner

“I use what I learned in my work now, not in some distant future project.”

Diana Nsiiza

Data analyst, Entebbe

Continue with

Your next best step.

Following the order of AI Engineer (LLM Applications).

See the full chain

Next lesson

Overview and set-up

AI Safety, Evaluation & Monitoring

40 min · Resume

Course 11 in the track

Fine-Tuning Open Models

Hugging Face, LoRA/QLoRA, Datasets, and more.

Advanced · Course · 4 weeks at 5 hrs/week

Course 12 in the track

Speech & Vision with AI APIs

Whisper, TTS, Image understanding, and more.

Intermediate · Course · 3 weeks at 5 hrs/week

Course 13 in the track

Prompt Engineering for Production

Structured outputs, Evaluation, Guardrails, and more.

Intermediate · Course · 3 weeks at 5 hrs/week

Learner reviews

What learners say.

Write a review

Sign in to share your experience with this course.

Sign in to review

Loading reviews…

Previous courseAI App Development with Next.js & SupabaseNext courseFine-Tuning Open Models

Keep going

Other courses in this track.

View full track
LLMs for Developers cover illustration
AI

17 lessons · 14.1h · Intermediate

LLMs for Developers

APIs (OpenAI, Anthropic, Gemini), Tokens & cost, Streaming, and more.

Yiga Plus
Building AI Agents cover illustration
AI

24 lessons · 19.3h · Intermediate

Building AI Agents

Tool use, Planning, Memory, and more.

Yiga Plus