skillfactor-pipeline/docs/reference-standard.md

Reference-quality package — the checklist

Derived from the artificial-intelligence-engineer reference run (2026-07-09, branch quality/reference-standard). A package is reference quality when every box below holds; qa/lint_package.py enforces the machine-checkable subset as the publish gate.

Market evidence

• Every stored ad passed the relevance gate (blocklist + gemma3 judge against the ESCO description); rejects logged in rejected-ads.jsonl
• Corpus ≥ 100 relevant ads total, or the report says so explicitly
• Regional sections (US / UK / EU-DACH); regions under 30 ads state "insufficient evidence" instead of ranking noise
• No foreign-role tools (designer/marketing stack) in any top-10 ranking

Agent-skill mapping

• Every entry tiered core or adjacent; core = touches a top-20 market hard skill or an essential ESCO competence
• Max 10 entries per source repo; overflow only in the audit log
• Per-source attribution complete: repo URL, commit, license, retrieval date; links point at a pinned commit (tree/<sha>)
• Nothing copied from sources — name, one-line summary, link only

SKILL.md

• Intro is one grammatically clean paragraph (rewrite cached in data/rewrites/); raw ESCO text reachable via the sources footer
• Essential competences complete — never a silently cut alphabetical list
• "Hot technologies" derived from the post-gate market ranking; omitted below 30 gated ads; never an alphabetical O*NET dump
• O*NET proxy caveat present wherever the crosswalk match is not exact
• Curated content (core workflow, intro) lives in overlays/<slug>/ and survives regeneration

Counts & provenance

• Every displayed number traces to generated stats.json — nothing hard-coded in UI or READMEs
• PROVENANCE.md present with all five source categories and pinned colors; manifest carries esco_uri, onet_soc (+ match type), collar
• Raw fetch data and extraction outputs exist on disk and were never edited or deleted

Process

• qa/lint_package.py <slug> exits clean
• Both eval suites at ship level (evals/ad-relevance ≥ 27/30, evals/skill-tiering ≥ 18/20)
• Regeneration is a single command (pipeline/regen_flagship.py <slug>) and reproduces the package from stored data