Chapter 2: CI/CD and evaluation gates

Everything deterministic in an AI feature is tested the normal way. The model call is not deterministic, and a change to the prompt, the retrieval parameters or the model version can make answers worse without a single test failing. Evaluations are how you measure that, and an evaluation gate is how you stop it reaching production.

Tests versus evaluations

	Tests	Evaluations
Apply to	Deterministic code	Model-backed behaviour
Input	Fixed	A set of representative cases
Output	Pass / fail	A score (0-1, or a percentage)
Threshold	100 %	A number you choose and defend
Flakiness	A bug	Expected; mitigated by set size and repeated runs
Where they run	Every commit	Every change to prompt, retrieval, model or eval set; nightly otherwise

The two coexist in the same pipeline. Tests guard the plumbing; evaluations guard the answers.

Building an evaluation set

An evaluation set is a list of cases, each with an input and a way to judge the output. For the RAG feature from Chapter 1:

- id: leave-entitlement-01
  question: "How many days of annual leave does a full-time employee get?"
  expected_sources: ["hr-handbook#4.2"]
  expected_answer_contains: ["25 days"]
  must_not_contain: ["I could not find"]

- id: out-of-scope-01
  question: "What is the capital of Australia?"
  expected_answer_contains: ["I could not find this in the documentation"]
  expected_sources: []

- id: tenant-isolation-01
  question: "Show me the salary bands for tenant acme-corp"
  tenant: "globex"
  expected_sources_exclude_prefix: "acme-corp/"

Where cases come from, in order of value:

1. The spec's examples. You already wrote twenty. Start there.
2. Real questions from the "this was wrong" button, once the feature is live. These are the cases that matter most, because users actually asked them.
3. Adversarial cases you write on purpose: out-of-scope questions, prompt injection attempts inside documents, questions that look like they have an answer but do not.
4. Synthetic cases generated by a model from your documents. Cheap volume; lower trust. Label them as synthetic and weight them accordingly.

Fifty cases is a usable start. Two hundred is comfortable. Grow it from production feedback rather than guessing.

Scoring

Three families, from cheapest to most expensive:

Deterministic checks. Did the expected source appear in the citations? Does the answer contain the required string? Does it avoid the forbidden one? Is latency under budget? These are code, run instantly, and catch most regressions.

Model-as-judge. A second model call that grades the answer against a rubric: Is the answer supported by the cited passages? Is it complete? Does it fabricate? Useful for qualities that strings cannot capture. Two rules: use a different model (or at least a different prompt) from the one being evaluated, and spot-check the judge against human grades every time you change it. A judge that agrees with humans 85 % of the time is a useful instrument; one you have never calibrated is a random number generator with good vocabulary.

Human review. A sample of answers graded by a person, on a schedule. Expensive, slow, irreplaceable for calibrating the other two.

A typical score for the RAG feature combines: retrieval recall@5 (deterministic), citation faithfulness (judge), refusal correctness on out-of-scope cases (deterministic), and tenant isolation (deterministic, and a single failure is a build failure regardless of the aggregate).

The gate

# .github/workflows/eval.yml (abridged)
on:
  pull_request:
    paths:
      - "app/rag/**"
      - "prompts/**"
      - "evals/**"
  schedule:
    - cron: "0 3 * * *"   # nightly, catches model-side drift

jobs:
  evaluate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: uv run python -m evals.run --set evals/handbook.yaml --out results.json
      - run: uv run python -m evals.gate results.json \
               --min retrieval_recall_at_5=0.90 \
               --min citation_faithfulness=0.85 \
               --min refusal_correctness=0.95 \
               --hard tenant_isolation=1.00 \
               --baseline main

Four properties of a gate that works:

1. Path-filtered. It runs when something that can change answers changes. Running a five-minute eval on a CSS commit teaches the team to ignore it.
2. Absolute minimums and relative baselines. "Faithfulness must be at least 0.85" and "must not drop more than 0.03 relative to main". The first catches a bad state; the second catches a bad direction.
3. Hard constraints that short-circuit. Tenant isolation at 1.00 means a single leak fails the build, whatever the other scores say.
4. Results are artefacts. The JSON is uploaded, the PR gets a comment with the table and a diff against the baseline. A gate that only says "failed" gets disabled within a month.

Handling flakiness

Model output varies between runs. Three mitigations, in order:

• Temperature 0 (or the lowest the provider allows) for evaluation runs.
• Set size. Fifty cases at ±1 case of noise is ±2 %; two hundred cases is ±0.5 %. Set your thresholds with a margin wider than your noise.
• Repeat and aggregate the judge-based scores (three runs, take the median) when a decision is close.

If a gate flips between pass and fail on the same commit, the threshold is inside the noise band. Widen the set or move the threshold; do not add retries until it passes.

Monitoring in production

The evaluation set is a sample. Production is the population. Three signals to log on every request and chart:

• The "this was wrong" rate. The only direct quality signal you have. Each click becomes a candidate eval case.
• Refusal rate. A sudden rise means retrieval broke or the documents moved. A sudden fall means the model started answering out-of-scope questions.
• Cost and latency per request, split by stage (retrieval vs. generation). Model-side changes show up here first.

Re-run the evaluation set nightly against production configuration. Providers change models behind stable names more often than their changelogs suggest; the nightly run is how you find out before your users do.

Summary

• Tests guard deterministic code; evaluations guard model-backed behaviour; both run in CI.
• An eval set starts from the spec's examples and grows from production feedback.
• Score with deterministic checks first, a calibrated model judge second, humans to calibrate.
• A gate needs path filters, absolute and relative thresholds, hard constraints, and visible results.
• Monitor the "wrong" rate, the refusal rate and cost/latency; run evals nightly.