Files

skillfactor-pipeline 69a7ba6ea7 feat(qa): package linter as publish gate + v1 full-catalog heatmap

lint_package.py checks contamination, alphabetical truncation, count
consistency, provenance completeness and intro grammar; wired into both
publish paths (p4_publish + batch stage_packages), SKIP_LINT=1 reserved
for labeled v1-grade republishes. Heatmap over 3,039 v1 packages:
1,406 clean / 1,633 with findings; C2 truncation is systemic (generator),
contamination affects 2.6% of crawled packages; nothing requires
re-fetching.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDKeXvpT6tENSvyQGLV1Uq

2026-07-09 06:54:09 +02:00

2.8 KiB

Raw Blame History

v1 quality heatmap — full-catalog lint (3,039 packages)

Date: 2026-07-09 · Linter: qa/lint_package.py (phase 3) · Baseline: git tag v1 Raw results: data/v1-lint-results.json (per-package findings).

Distribution

	Packages
Clean	1,406 (46%)
With findings	1,633 (54%)

Check	Findings	Affected packages	Nature
C2 truncation (cut lists / alphabetical dumps)	2,006	1,632	systemic — generator defect
C1 contamination (foreign-role tools in market top-10)	21	5	data defect (no relevance gate)
C5 grammar (raw ESCO intro)	5	5	content defect (no rewrite step)
C3 count mismatch	0	0	—
C4 provenance	0	0	—

Reading the numbers

The flagship's problems are SYSTEMIC, not isolated. 54% of the catalog carries the same truncation defect (essential[:15] + alphabetical O*NET hot-tech dump) — one generator bug, thousands of hits. Both causes are already fixed on quality/reference-standard (full lists; market-derived hot-tech via p3c).
C1 can only fire where market data exists. Only 189 packages had a market.md at lint time (the evidence crawl is ongoing). 5 of 189 ≈ 2.6% of crawled packages show top-10 contamination; the ungated crawl keeps producing v1-grade corpora, so expect this rate to hold until re-gating.
Provenance discipline held everywhere (C4 = 0), and counts are consistent package-internally (C3 = 0 — the landing-page "83" defect was a hard-coded UI number, structurally fixed via stats.json).

Regeneration cost split (per the phase-5 decision brief)

Fix class	Packages	Method	Cost
C2 truncation + intro rewrite	1,632 (+ all others for intro polish)	re-run p2 SKILL.md generation (overlays preserve curation)	local only — no API calls; gemma3 intro rewrites ~3,039 × 1 call, cached; ~2–4 h GPU
C1 contamination	5 known + future crawled	re-gate stored raw ads (`ad_gate.py --all`, running) + re-extract relevant-only + re-aggregate	local only — raw ads are on disk for all 189 crawled packages; zero re-fetching
C5 grammar	5	covered by the intro rewrite step	local only
EU/DACH corpora	opt-in per package	`eu_fetch.py` + `data/eu-queries.json`	API cost — ~8–16 JSearch requests per occupation; budget-gated

Bottom line: nothing in v1 requires re-fetching. The entire cleanup is re-classification and re-compilation from stored raw data plus local generation — the only API spend is the deliberate EU/DACH expansion.

Publish gate status

p4_publish now refuses to push a lint-failing package (SKIP_LINT=1 escape hatch exists for explicitly-labeled v1-grade republishes, e.g. the ongoing evidence crawl's catalog refresh).

2.8 KiB Raw Blame History Unescape Escape