Files
skillfactor-pipeline/docs/decisions.md
2026-07-07 11:33:43 +02:00

33 lines
1.9 KiB
Markdown

# Decisions log
Short rationale for choices that are not obvious from the code
(rule from auftrag_fable.md §Regeln).
## 2026-07-07 (Fable 5 — quality foundation day)
1. **Ollama structured output + strict prompt, both.** The JSON schema is
passed as `format` to `/api/generate` (server-side shape enforcement)
AND the prompt demands strict JSON with 3 few-shots. Belt and braces:
the schema pins keys/types, the few-shots pin content quality.
Result: 20/20 schema-valid on the first model pass (gemma3:27b, ø 6.5 s).
2. **Validator repairs instead of second prompt iteration.** Testset showed
95 % content-clean; the remaining defects (skill phrases leaking into
qualifications, mojibake from broken ad encodings, acronym casing) are
deterministic and cheaper to fix in `validate()` than by prompt tuning
on a 27B model. After hardening: 20/20 clean.
3. **`evidence_job` gets an `occupation_slug` column** (nullable ALTER +
backfill) instead of per-occupation tables or a rebuild-per-run design —
p3b becomes incremental MERGE; existing 350 recruiter rows stay valid.
4. **Mass extraction runs as a detached script, not in the Claude loop**
(~180 k ads x 6.5 s ≈ 14 GPU-days): Claude only spot-checks 2 % samples
via `qa_sample.py`. Budget enforcement lives in code
(`progress.spend_request()` raises), not in instructions.
5. **Templates as `pipeline/templates/*.template.md` + shared STYLE.md.**
Skeleton headings are greppable on purpose — QUALITY_BAR's mechanical
checks and the PR gates rely on exact section markers.
6. **Git history squash discovered mid-session** (orphan branch
`clean-history` renamed to master, 11:21 local — not done by Claude).
Old 16-commit history preserved as tag `backup/pre-squash-history`;
new work committed on top of the squashed master. Delete the tag once
the squash is confirmed intentional and nothing is missing.