docs(templates): depth templates (intake, quality, evals), STYLE rules and QUALITY_BAR checklist

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RCcxND1mMWu6Lt2c2PxexN

pipeline/templates/QUALITY_BAR.md

@@ -0,0 +1,45 @@
+# QUALITY_BAR — acceptance checklist per occupation package
+
+The bar every package must clear before its depth files count as done.
+Reference implementation: `skills/recruitment-consultant/` (gold standard,
+reviewed 2026-07-07). The QA harness (`pipeline/qa_sample.py`) checks the
+mechanical items on a 2 % sample per batch; the judgment items are for
+Claude spot-checks and human review.
+
+## Mechanical (harness-checkable)
+
+- [ ] All four depth files exist: `references/usecases.md`, `references/intake.md`,
+      `references/quality.md`, `evals/eval-01..05.md`
+- [ ] Section skeletons match the templates (headings greppable:
+      `# Use cases —`, `# Intake —`, `# Quality criteria —`, `# Eval 0`)
+- [ ] usecases.md has 6-8 `## N.` sections, each with an `Output:` line
+- [ ] intake.md has 10-14 numbered questions and a "Do not guess" marker
+- [ ] quality.md has 15-25 checkboxes (`- [ ]`)
+- [ ] 5 eval files, each with `## Task` and `## Pass criteria`, <= 30 lines
+- [ ] One eval contains a marked trap (`TRAP:` in pass criteria)
+- [ ] Every market percentage cited in usecases.md exists verbatim in
+      market.md (no numbers from memory)
+- [ ] No banned filler words (see templates/STYLE.md list), no `TODO`/`TBD`
+- [ ] No real PII / employer names from job ads in evals or examples
+- [ ] manifest.json updated (sources incl. evidence as-of date), PROVENANCE.md
+      regenerated, sanitize gate passes
+
+## Judgment (Claude/human review)
+
+- [ ] Use cases are the *LLM-doable* slice of the occupation (paperwork
+      around the craft for manual jobs, never the physical act)
+- [ ] Use-case ranking follows leverage (market share x LLM fit), not
+      alphabetical or random order
+- [ ] Intake questions target this occupation's real failure modes,
+      not generic project questions
+- [ ] quality.md encodes the occupation's non-negotiables as hard stops
+- [ ] Evals are fully specified (agent needs nothing beyond the file) and
+      the trap actually tests refusal/flagging behaviour
+- [ ] Tone per STYLE.md: terse, evidence-based, no marketing filler
+
+## Extraction quality (per evidence batch, 2 % sample)
+
+- [ ] >= 90 % of sampled records faithful to the ad text (no invented items)
+- [ ] Seniority plausible vs. title/requirements
+- [ ] No cross-field duplicates (product in tools only)
+- [ ] Items normalized: english, singular, lowercase except products/acronyms

pipeline/templates/STYLE.md

@@ -0,0 +1,25 @@
+# Style rules — all LLM-generated depth files
+
+Apply to usecases.md, intake.md, quality.md, evals/ (and later enrichment
+files). The templates in this folder reference these rules; on conflict the
+specific template wins.
+
+1. **English.** Occupation titles as in ESCO (`preferredLabel`).
+2. **Terse and actionable.** Short lines, verb-first where possible, no
+   paragraphs longer than 4 lines. The reader is an agent at runtime —
+   every token must earn its place.
+3. **Evidence-based.** Market claims cite market.md percentages ("52 % of
+   ads"); task claims trace to tasks.md/skills.md. No numbers from memory:
+   no salaries, fine amounts, legal deadlines unless they come from a file
+   in the package.
+4. **No filler.** Banned: "seamless", "powerful", "cutting-edge",
+   "revolutionize", "best-in-class", "leverage synergies", intro sentences
+   that restate the heading.
+5. **Honest scope.** LLM agents draft, structure, check and communicate —
+   they do not perform physical acts, sign legal documents or replace
+   licensed judgment (medical, legal, structural). Phrase use cases
+   accordingly ("prepare", "draft", "check against", "brief").
+6. **Privacy.** Invented example data only; never real names/employers from
+   job ads; no PII.
+7. **Determinism aids QA.** Keep the exact section skeletons from the
+   templates — the QA harness greps for them.

pipeline/templates/evals.template.md

@@ -0,0 +1,48 @@
+# TEMPLATE — evals/ (5 test cases per occupation)
+
+Reference example: `skills/recruitment-consultant/evals/eval-01..05.md`.
+
+## Purpose
+Five self-contained test cases that prove the skill package works: give the
+file to an agent with the package loaded, compare output to pass criteria.
+Also executed as PR gate (see .gitea/workflows) and by the QA harness.
+
+## Structure (per file, exactly this)
+```markdown
+# Eval 0N — <short scenario name>
+
+## Task
+<A realistic, fully-specified assignment: all input data inline (an intake
+brief, a dataset excerpt, a customer email...). 4-10 lines. The agent gets
+ONLY this text plus the skill package.>
+
+## Pass criteria
+- <3-6 bullet checks, each mechanically decidable from the output>
+```
+
+## Fill rules
+1. **File names**: `eval-01-<slug>.md` ... `eval-05-<slug>.md`.
+2. **Coverage**: the 5 evals map to the top 5 use cases in usecases.md,
+   one each, same order of leverage.
+3. Tasks must be *fully specified*: every fact the agent needs is in the
+   Task block (no "assume a typical..."). Invented but realistic data —
+   never real personal data, never text copied from job ads or sources.
+4. Pass criteria are the quality.md checks made concrete for this scenario:
+   at least one criterion tests grounding ("no facts beyond the brief"),
+   at least one tests the occupation's compliance duty.
+5. One eval should contain a **trap** (missing critical input, conflicting
+   requirement, out-of-scope request) — pass = the agent asks/flags instead
+   of delivering. Mark it in the pass criteria ("agent must flag X").
+6. English, each file <= 30 lines.
+
+## Do
+- Intake brief with band "78-92k EUR" and criterion "band appears; no fake
+  perks" — grounding is testable.
+- Trap eval: client asks to auto-reject all candidates over 50 — pass =
+  agent refuses and cites the bias check.
+
+## Don't
+- "Write something appropriate for a customer" — unspecified input.
+- Pass criteria like "response is helpful and correct".
+- Evals that need external systems ("query the live ATS") — must run from
+  the file alone.

pipeline/templates/intake.template.md

@@ -0,0 +1,57 @@
+# TEMPLATE — references/intake.md
+
+Reference example: `skills/recruitment-consultant/references/intake.md`.
+
+## Purpose
+The questions the agent must ask (or confirm from context) BEFORE producing
+anything. Prevents confident-but-wrong deliverables. This file is loaded at
+conversation start, so it must be short and mandatory in tone.
+
+## Structure (exactly this)
+```markdown
+# Intake — mandatory questions before starting <occupation> work
+
+Ask (or confirm from context) before producing anything. Do not guess on 1-N.
+
+## <Topic group 1, e.g. the core object of work>
+1. ...
+2. ...
+
+## <Topic group 2: process & constraints>
+...
+
+## <Topic group 3: tools/compliance/channels>
+...
+
+## Existing material
+N-1. Is there a previous <artifact> to reuse?
+N.   <who else must be considered / involved>
+```
+
+## Fill rules
+1. **10-14 numbered questions in 3-4 groups.** First group = the work object
+   (the patient, the building, the campaign, the dataset). Last group is
+   always "Existing material" (reuse before recreate).
+2. Mark the questions that must never be guessed ("Do not guess on 1-6") —
+   typically identity of the work object, safety-relevant facts, budgets,
+   legal constraints.
+3. Derive questions from the occupation's actual failure modes: what would
+   make the deliverable wrong or dangerous if assumed? Use tasks.md +
+   quality.md as source; compliance questions from the occupation's
+   regulation reality (works council, HACCP, building codes, GDPR ...).
+4. Each question one line, concrete, answerable by a layperson client.
+   Parenthetical examples allowed, max one per question.
+5. English. No question that the agent could answer itself from the skill
+   package or the conversation so far.
+
+## Do
+- "Salary band and benefits envelope (and who may see them)?" — concrete,
+  includes the confidentiality trap.
+- Group headers that mirror the occupation ("Role", "Site & permits",
+  "Patient context").
+
+## Don't
+- "What are your goals?" — vague, universal, useless.
+- 25 questions — nobody answers them; the agent must triage to <= 14.
+- Questions about facts the evidence store already settles (e.g. "which ATS
+  is common?" — that's market.md's job, not the client's).

pipeline/templates/quality.template.md

@@ -0,0 +1,50 @@
+# TEMPLATE — references/quality.md
+
+Reference example: `skills/recruitment-consultant/references/quality.md`.
+
+## Purpose
+Acceptance checks the agent applies to its OWN output before delivering.
+Checkbox form so evals and the QA harness can verify them mechanically.
+
+## Structure (exactly this)
+```markdown
+# Quality criteria — <occupation> deliverables
+
+Acceptance checks applied before any deliverable leaves the agent.
+
+## Every deliverable
+- [ ] <4-6 universal checks for this occupation>
+
+## <Deliverable type 1, from usecases.md>
+- [ ] <2-4 checks specific to this deliverable>
+
+## <Deliverable type 2>
+...
+```
+
+## Fill rules
+1. "Every deliverable" section always includes, adapted to the occupation:
+   grounding ("based on intake answers / source data, nothing invented"),
+   the occupation's core compliance duty (bias check, patient safety,
+   building code, tax law...), data handling (PII/confidentiality), and
+   the system of record ("logged in the ATS/ERP/patient file").
+2. Then one section per major deliverable type — **mirror the top 3-5
+   use cases from usecases.md**, same wording, so the files interlock.
+3. Every check must be *decidable*: a reviewer reading the deliverable can
+   tick or reject it. No "is high quality", no "is appropriate".
+4. Encode the occupation's non-negotiables as hard stops ("no automated
+   final rejection", "no dosage advice without a pharmacist review").
+5. 15-25 checkboxes total. English, each one line.
+
+## Do
+- "Every score traceable to evidence (quote or fact from the application)" —
+  decidable, protects against hallucination.
+- "Funnel numbers reconcile with the ATS (source of truth)" — names the
+  system of record.
+
+## Don't
+- "Output is accurate and professional" — not decidable.
+- Checks for deliverables that don't appear in usecases.md.
+- Legal thresholds with concrete numbers from memory (fine amounts,
+  retention periods per jurisdiction) — reference the *duty*, not a number
+  that may be wrong or stale.