skillfactor — Occupational Skill Library (MVP)

skillfactor ("KI mit Berufserfahrung") builds ready-to-use occupational skill packages for AI agents: public occupation databases (ESCO, O*NET) + live job-ad evidence (JSearch/Adzuna APIs) + expert knowledge (Wikipedia/Wikidata, literature) are compiled into one git repo per occupation and published to a git-native marketplace (Gitea).

Architecture

flowchart LR
    A[ESCO / O*NET / Crosswalk] --> DB[(MSSQL 'skillfactor')]
    J[Job ads via JSearch/Adzuna APIs] --> DB
    W[Wikipedia / Wikidata / literature] --> G
    DB --> G[Generator]
    G --> S[skills/<slug>/ packages]
    S --> M[Gitea marketplace<br/>orgs: skills-core / skills-community / tenant-acme]

• Evidence store: Microsoft SQL Server (database skillfactor, local instance) — project requirement: the whole stack runs on MSSQL (Gitea itself included). No SQLite.
• Pipeline: one script per phase in pipeline/ (p1_load.py, …), orchestrated by pipeline/run_all.py. Downloads are cached in data/raw/ (not committed).
• Skill package layout (per occupation, language: English):

skills/<slug>/
├── SKILL.md            # frontmatter: name, description (synonyms + task
│                       # triggers), core workflow; < 300 lines
├── references/
│   ├── profile.md      # occupation profile (ESCO), ESCO URI, ISCO code
│   ├── tasks.md        # O*NET task statements + detailed work activities
│   ├── skills.md       # ESCO skills (essential/optional)
│   └── tools.md        # O*NET tools & technology
└── manifest.json       # ids, sources, version, layer:"core", attribution

Setup

1. Python >= 3.11, python -m venv .venv, pip install -r requirements.txt
2. Copy .env (never committed): Gitea URL/token, JSearch/Adzuna keys, MSSQL credentials (see .env keys in pipeline/db.py)
3. python pipeline/run_all.py

Data sources & licensing

• ESCO (European Commission) — occupations/skills/relations. Usage per the ESCO conditions.
• O*NET (U.S. Department of Labor, CC BY 4.0) — task statements, tools & technology, detailed work activities. Attribution included in every manifest.json and below.
• Job ads exclusively via official APIs (JSearch, Adzuna) — no HTML scraping.
• Wikipedia/Wikidata content is paraphrased, never copied; source URLs cited.

This product includes information from the O*NET Database by the U.S. Department of Labor, Employment and Training Administration (USDOL/ETA), used under the CC BY 4.0 license. skillfactor is not endorsed by USDOL/ETA.

Data load smoke test (Phase 1)

Table	Rows
esco_occupation	3,039
esco_skill	13,939
esco_occ_skill	126,051
esco_isco_group	619
onet_occupation	1,016
onet_task	18,796
onet_software	31,821
onet_task_dwa	23,850
onet_dwa	2,087
crosswalk_esco_onet	4,210

Note (O*NET 30.3): the former "Technology Skills" file is now called "Software Skills"; tools data loads from there.

Status / TODO

• Phase 0 — setup (repo, venv, MSSQL evidence store, Gitea Actions + runner)
• Phase 1 — load ESCO/O*NET/crosswalk into MSSQL
• Phase 2 — generate ~100 core skill packages (exactly 100 published)

Phase 2 spot check (5 samples read manually)

recruitment-consultant, accountant, marketing-manager, ict-help-desk-agent, market-research-analyst:

• structure complete (SKILL.md < 300 lines, 4 reference files, manifest)
• ESCO description/competences and O*NET tasks/software are consistent with the occupation; crosswalk matches are plausible (recruiter → 13-1071.00)
• known limitation: O*NET "hot technologies" for broad SOC codes contain generic office software — the job-ad market evidence (phase 3) is the corrective for that
• first selection draft was ISCO-11..13-heavy (alphabetical fill); fixed via round-robin bucket selection across ISCO prefixes (11:7, 12:17, 13:11, 24:23, 25:13, 26:6, 33:24, 35:8, 42:6)
• Phase 3 — recruiter deep enrichment (glossary, literature, usecases, intake, quality, 5 evals)
• Phase 4 — published to Gitea: orgs skills-core/skills-community (public) + tenant-acme (private), 100 repos, marketplace, PR gates green (sanitize + eval on self-hosted act_runner, host mode)
• Phase 5 — screenshots in docs/screenshots/

Market evidence (phase 3) — DONE

101 unique recruiter ads (us+gb) fetched via JSearch (11 of 200 monthly requests), entities extracted (1,859) into the MSSQL evidence store, aggregated at the ≥20 % threshold into the recruiter package: ATS 54 %, LinkedIn/LinkedIn Recruiter 26 %, candidate sourcing 48 %, full-cycle recruiting 33 %, candidate screening 27 %, talent pipelining 29 %. Remaining occupations follow the staged plan in docs/batch-plan.md (~30 occupations/month within the free tier).

Open TODO

• Structured-interview article on Wikipedia describes the survey-research method; the hiring context in glossary.md is our own framing.

Environment notes (this deployment)

• Gitea 1.24.5 on Microsoft SQL Server, served at https://zeiterfassung.cloud/skillfactor/ (Caddy sub-path setup).
• Gitea Actions: github.server_url does not include the sub-path (Gitea limitation) — Caddy additionally forwards the git smart-HTTP endpoints of the three skill orgs without the prefix so actions/checkout can clone.
• act_runner 0.6.1, host mode, label windows; started via Tools\gitea\act_runner\start-actrunner.ps1 (no autostart).