6.1 How to evaluate a RAG system: methods and metrics

Автор: Evidently AI

Загружено: 2025-05-20

Просмотров: 1714

Описание:

00:03 Intro
00:24 What is RAG?
01:03 How to evaluate RAG? Look at both sides.
01:34 Evaluating retrieval quality: ground truth-based evals.
02:23 Evaluating retrieval quality: labeling relevance.
02:55 Evaluating retrieval quality: LLM-labeled relevance / context quality judge.
03:55 Evaluating generation quality: ground truth-based evals.
04:36 Using synthetic data to create the test dataset.
05:10 Evaluating generation quality: faithfulness, completeness, etc.
06:09 Stress-testing RAG.
06:42 Recap

COURSE PLAYLIST
Full playlist: • Course: LLM evaluation for builders
Instructor: Elena Samuylova, CEO Evidently AI.

LINKS
Ranking metrics https://www.evidentlyai.com/ranking-m...
RAG evaluation https://www.evidentlyai.com/blog/open...

EVIDENTLY
Sign up for Evidently Cloud https://www.evidentlyai.com/register
Support Evidently on GitHub https://github.com/evidentlyai/evidently

6.1 How to evaluate a RAG system: methods and metrics

Доступные форматы для скачивания:

Скачать видео mp4

Информация по загрузке:

Скачать аудио mp3

Похожие видео

6.2. Tutorial: Building and evaluating a RAG system

6.2. Tutorial: Building and evaluating a RAG system

Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)

Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)

Учебное пособие по Microsoft 365 Copilot для начинающих (2026 год)

Учебное пособие по Microsoft 365 Copilot для начинающих (2026 год)

Session 7: RAG Evaluation with RAGAS and How to Improve Retrieval

Session 7: RAG Evaluation with RAGAS and How to Improve Retrieval

1. Introduction to LLM evaluations in 10 key ideas

1. Introduction to LLM evaluations in 10 key ideas

Google закрыл RAG (Сделайте это вместо него)

Google закрыл RAG (Сделайте это вместо него)

RAG Evaluation Metrics Explained: Context Precision, Recall, Relevancy & Faithfulness

RAG Evaluation Metrics Explained: Context Precision, Recall, Relevancy & Faithfulness

RAGAS: How to Evaluate a RAG Application Like a Pro for Beginners

RAGAS: How to Evaluate a RAG Application Like a Pro for Beginners

Все стратегии RAG объясняются за 13 минут (без лишних слов)

Все стратегии RAG объясняются за 13 минут (без лишних слов)

LLM-as-a-judge: evaluating LLMs with LLMs

LLM-as-a-judge: evaluating LLMs with LLMs

RAG Evaluation Sucks: Here's a Totally New Way to Do It - e17 RAG Masters

RAG Evaluation Sucks: Here's a Totally New Way to Do It - e17 RAG Masters

2 метода улучшения поиска в RAG

2 метода улучшения поиска в RAG

Do Reranking Models Actually Improve RAG?

Do Reranking Models Actually Improve RAG?

How to evaluate an LLM-powered RAG application automatically.

How to evaluate an LLM-powered RAG application automatically.

Course: LLM evaluation for builders

Course: LLM evaluation for builders

Mastering RAG Evaluation: Metrics and Methods | Retrieval-Augmented Generation

Mastering RAG Evaluation: Metrics and Methods | Retrieval-Augmented Generation

Finally a Local RAG That WORKS!! (+ FULL RAG Pipeline)

Finally a Local RAG That WORKS!! (+ FULL RAG Pipeline)

AI vs Oscar Winning Actor (Same Scene)

AI vs Oscar Winning Actor (Same Scene)

RAG Evaluation Is Broken! Here's Why (And How to Fix It) - Yuval Belfer and Niv Granot

RAG Evaluation Is Broken! Here's Why (And How to Fix It) - Yuval Belfer and Niv Granot

Advanced RAG techniques for developers

Advanced RAG techniques for developers