# skillfactor — Occupational Skill Library (MVP)

**skillfactor** ("KI mit Berufserfahrung") builds ready-to-use occupational skill
packages for AI agents: public occupation databases (ESCO, O*NET) + live job-ad
evidence (JSearch/Adzuna APIs) + expert knowledge (Wikipedia/Wikidata,
literature) are compiled into one git repo per occupation and published to a
git-native marketplace (Gitea).

## Architecture

```mermaid
flowchart LR
    A[ESCO / O*NET / Crosswalk] --> DB[(MSSQL 'skillfactor')]
    J[Job ads via JSearch/Adzuna APIs] --> DB
    W[Wikipedia / Wikidata / literature] --> G
    DB --> G[Generator]
    G --> S[skills/<slug>/ packages]
    S --> M[Gitea marketplace<br/>orgs: skills-core / skills-community / tenant-acme]
```

- **Evidence store: Microsoft SQL Server** (database `skillfactor`, local
  instance) — project requirement: the whole stack runs on MSSQL (Gitea itself
  included). No SQLite.
- **Pipeline:** one script per phase in `pipeline/` (`p1_load.py`, …),
  orchestrated by `pipeline/run_all.py`. Downloads are cached in `data/raw/`
  (not committed).
- **Skill package layout** (per occupation, language: English):

```
skills/<slug>/
├── SKILL.md            # frontmatter: name, description (synonyms + task
│                       # triggers), core workflow; < 300 lines
├── references/
│   ├── profile.md      # occupation profile (ESCO), ESCO URI, ISCO code
│   ├── tasks.md        # O*NET task statements + detailed work activities
│   ├── skills.md       # ESCO skills (essential/optional)
│   └── tools.md        # O*NET tools & technology
└── manifest.json       # ids, sources, version, layer:"core", attribution
```

## Setup

1. Python >= 3.11, `python -m venv .venv`, `pip install -r requirements.txt`
2. Copy `.env` (never committed): Gitea URL/token, JSearch/Adzuna keys, MSSQL
   credentials (see `.env` keys in `pipeline/db.py`)
3. `python pipeline/run_all.py`

## Data sources & licensing

- **ESCO** (European Commission) — occupations/skills/relations. Usage per the
  [ESCO conditions](https://esco.ec.europa.eu/en/use-esco/download).
- **O*NET** (U.S. Department of Labor, CC BY 4.0) — task statements, tools &
  technology, detailed work activities. Attribution included in every
  `manifest.json` and below.
- Job ads exclusively via official APIs (JSearch, Adzuna) — no HTML scraping.
- Wikipedia/Wikidata content is paraphrased, never copied; source URLs cited.

> This product includes information from the O*NET Database by the U.S.
> Department of Labor, Employment and Training Administration (USDOL/ETA),
> used under the CC BY 4.0 license. skillfactor is not endorsed by USDOL/ETA.

## Data load smoke test (Phase 1)

| Table | Rows |
|---|---|
| esco_occupation | 3,039 |
| esco_skill | 13,939 |
| esco_occ_skill | 126,051 |
| esco_isco_group | 619 |
| onet_occupation | 1,016 |
| onet_task | 18,796 |
| onet_software | 31,821 |
| onet_task_dwa | 23,850 |
| onet_dwa | 2,087 |
| crosswalk_esco_onet | 4,210 |

Note (O*NET 30.3): the former "Technology Skills" file is now called
**"Software Skills"**; tools data loads from there.

## Status / TODO

- [x] Phase 0 — setup (repo, venv, MSSQL evidence store, Gitea Actions + runner)
- [x] Phase 1 — load ESCO/O*NET/crosswalk into MSSQL
- [x] Phase 2 — generate ~100 core skill packages (exactly 100 published)

### Phase 2 spot check (5 samples read manually)

`recruitment-consultant`, `accountant`, `marketing-manager`,
`ict-help-desk-agent`, `market-research-analyst`:
- structure complete (SKILL.md < 300 lines, 4 reference files, manifest)
- ESCO description/competences and O*NET tasks/software are consistent with
  the occupation; crosswalk matches are plausible (recruiter → 13-1071.00)
- known limitation: O*NET "hot technologies" for broad SOC codes contain
  generic office software — the job-ad market evidence (phase 3) is the
  corrective for that
- first selection draft was ISCO-11..13-heavy (alphabetical fill); fixed via
  round-robin bucket selection across ISCO prefixes (11:7, 12:17, 13:11,
  24:23, 25:13, 26:6, 33:24, 35:8, 42:6)
- [x] Phase 3 — recruiter deep enrichment (glossary, literature, usecases,
      intake, quality, 5 evals)
- [x] Phase 4 — published to Gitea: orgs `skills-core`/`skills-community`
      (public) + `tenant-acme` (private), 100 repos, marketplace, PR gates
      green (sanitize + eval on self-hosted act_runner, host mode)
- [x] Phase 5 — screenshots in `docs/screenshots/`

### Market evidence (phase 3) — DONE

101 unique recruiter ads (us+gb) fetched via JSearch (11 of 200 monthly
requests), entities extracted (1,859) into the MSSQL evidence store,
aggregated at the ≥20 % threshold into the recruiter package:
**ATS 54 %**, LinkedIn/LinkedIn Recruiter 26 %, candidate sourcing 48 %,
full-cycle recruiting 33 %, candidate screening 27 %, talent pipelining 29 %.
Remaining occupations follow the staged plan in `docs/batch-plan.md`
(~30 occupations/month within the free tier).

### Open TODO
- Structured-interview article on Wikipedia describes the survey-research
  method; the hiring context in `glossary.md` is our own framing.

### Environment notes (this deployment)

- Gitea 1.24.5 on **Microsoft SQL Server**, served at
  `https://zeiterfassung.cloud/skillfactor/` (Caddy sub-path setup).
- Gitea Actions: `github.server_url` does not include the sub-path (Gitea
  limitation) — Caddy additionally forwards the git smart-HTTP endpoints of
  the three skill orgs without the prefix so `actions/checkout` can clone.
- act_runner 0.6.1, host mode, label `windows`; started via
  `Tools\gitea\act_runner\start-actrunner.ps1` (no autostart).