Topic lessonTopic: Applied container, data, cost and AI labs
Exercise RAG evaluation|Citation, version and support
Retrieved documents, displayed citations and generated claims provide different evidence.
Identify an unsupported answer even when a citation appears.
The cases use fictional inputs.
What to predict
Retrieved documents, displayed citations and generated claims provide different evidence.
- Retrieved evidence → Source versionAlign conditions
- Source version → Evidence for claimCompare outcomes
- Evidence for claim → DecisionVerify the decision
- Retrieved evidence
- Citation source present / No matching source
- Source version
- Intended version / Older version
- Evidence for claim
- Passage supports claim / Passage support unverified
- Decision
- Identify an unsupported answer even when a citation appears.
Compare the cases
| Case | Conditions | Decision |
|---|---|---|
| Trial 1 | Retrieved evidence Citation source present, Source version Intended version, Evidence for claim Passage supports claim | This synthetic claim is supported. Check additional claims individually. |
| Trial 2 | Retrieved evidence No matching source, Source version Intended version, Evidence for claim Passage supports claim | No retrieved evidence. Do not accept the answer as verified. |
| Trial 3 | Retrieved evidence Citation source present, Source version Older version, Evidence for claim Passage supports claim | A citation exists but uses an older version. Recheck the intended version. |
| Trial 4 | Retrieved evidence Citation source present, Source version Intended version, Evidence for claim Passage support unverified | The citation is present but the passage does not substantiate the claim. |
Acceptance and limits
Compares predefined evidence states. It does not connect to an LLM or retrieve real documents.
Read the initial conditions and operational goal.
Change one condition and compare the decision.
Record the model boundary and outstanding checks.
Change a condition
Compares predefined evidence states. It does not connect to an LLM or retrieve real documents.
This synthetic claim is supported. Check additional claims individually.
{"unverified":"No retrieved evidence. Do not accept the answer as verified.","stale":"A citation exists but uses an older version. Recheck the intended version.","unsupported":"The citation is present but the passage does not substantiate the claim.","supported":"This synthetic claim is supported. Check additional claims individually."}Check your understanding
Choose an option to see the result and explanation.
Check, then explain in your own words
- Read the initial conditions and operational goal.
- Change one condition and compare the decision.
- Record the model boundary and outstanding checks.
Check your understanding: Identify an unsupported answer even when a citation appears.
Compares predefined evidence states. It does not connect to an LLM or retrieve real documents.
Record the model boundary and outstanding checks.
Sources and conditions checked
The official references used in this lesson are collected here with their scope, conditions and corresponding sections. Independent expert review and live AWS comparison have not been performed.
Scope and conditionsRetrieved documents, displayed citations and generated claims provide different evidence.
Used in this lessonWhat to predict Acceptance and limits