Module 4 Quiz: Full-Stack AI Delivery

Six questions. Click an option to check it.

---

Question 1

In a RAG system, where should tenant or permission filters be applied?

1. After retrieval, on the top-k results
2. Inside the retrieval query, before similarity ranking
3. In the prompt, by instructing the model to ignore other tenants
4. In the UI

---

Question 2

Why must the prompt include an explicit instruction for the "answer not in the passages" case?

1. To reduce token usage
2. Otherwise the model answers from training data and the user cannot tell it from a grounded answer
3. Because the vector store requires it
4. To make citations shorter

---

Question 3

What is the key difference between a test and an evaluation?

1. Evaluations are written in YAML
2. A test is pass/fail on deterministic code; an evaluation produces a score on model-backed behaviour against a threshold
3. Tests run in CI, evaluations run locally
4. There is no difference

---

Question 4

Which source of evaluation cases is the most valuable once the feature is live?

1. Synthetic questions generated by a model
2. Real questions flagged by users through the "this was wrong" button
3. Questions copied from a public benchmark
4. Questions written by the engineer who built the feature

---

Question 5

An evaluation gate flips between pass and fail on the same commit. What is the right fix?

1. Add retries until it passes
2. Disable the gate
3. Widen the evaluation set or move the threshold outside the noise band
4. Raise the temperature

---

Question 6

Why run the evaluation set nightly even when nothing in the repository changed?

1. To keep the CI runners warm
2. Because providers change models behind stable names, and drift shows up in the scores before users report it
3. To generate more synthetic cases
4. Because GitHub requires a scheduled workflow