skillfactor-pipeline/ARCHITECTURE.md

SkillFactor — Architecture (binding target picture)

Status: binding as of 2026-07-10 (branch quality/reference-standard). SkillFactor is a git-based Occupational Skill Layer: profession profiles with competences + practitioner knowledge, preloaded into AI agents as "work experience". Three knowledge layers, one platform-neutral skill interface, two enrichment paths.

1. Three knowledge layers

Layer	Content	Visibility	Storage
1 — Community profession profiles	professions/<slug>/competences/<skill-slug>.md, vocabulary.json, index.md — anonymized, curated, license-clean (CC-BY-SA attribution)	public	this repo (Gitea), PR-based intake
2 — Organisation overlay	company standards, processes, calculation rules, preferred partners	private per company	one private Gitea repo per org, structural mirror of layer 1 (same profession slugs, same competence slugs) — specific knowledge extends/overrides general knowledge
3 — Project/customer knowledge	projects/<customer-slug>/<project-slug>/lessons/<date>-<slug>.md — customer/project reference explicitly wanted here	private, per project folder	inside the org repo; access via Gitea team permissions per project folder

Double tagging as the connective tissue: every layer-2/3 lesson references layer-1 competence URIs (ESCO/O*NET) in its frontmatter. Retrieval works both ways: "What do we know about customer A?" (by folder) and "What do we know about acceptance processes?" (by competence — pulls customer-A/B lessons up).

Loading cascade for agents: profession profile (1) → org overlay (2) → project context (3); more specific beats more general ("in general X, for customer A deviating Y").

Promotion logic (knowledge rises by proving itself):

• Same insight in ≥ 2 projects → suggested promotion into the org overlay.
• Proven org knowledge without company specifics → anonymized second distillate as a PR to the community.
• Hard rule: nothing flows automatically out of projects/. The path into layer 1 is ALWAYS an explicitly confirmed, generalized, anonymized second distillate.

flowchart TB
    subgraph L1["Layer 1 — Community (public repo)"]
      P1["professions/<slug>/competences/*.md"]
    end
    subgraph ORG["Org repo (private, per company)"]
      L2["Layer 2 — overlay mirror<br/>professions/&lt;slug&gt;/competences/*.md"]
      L3["Layer 3 — projects/&lt;customer&gt;/&lt;project&gt;/lessons/*.md"]
    end
    AGENT((Agent)) -->|1. load| P1
    AGENT -->|2. overlay| L2
    AGENT -->|3. project context| L3
    L3 -->|">=2 projects: promotion suggestion"| L2
    L2 -->|"confirmed, generalized,<br/>anonymized 2nd distillate — PR"| P1
    CHAT["Path A: conversation distillate"] -->|PR, user-confirmed| L3
    CHAT -->|anonymized PR| P1
    M365["Path B: M365 mining<br/>(SQL Server, read-only)"] -->|PR collection,<br/>review queue| L3

2. The skill as a universal interface (multi-platform, CRITICAL)

Usable as an agent skill in Claude, ChatGPT and Gemini:

• Canonical format is the git repo itself: plain Markdown + YAML frontmatter + JSON. NO Claude-specific constructs in content files (no tool names, no Anthropic-only syntax). Platform binding lives exclusively in thin adapters.
• Adapters as build targets from the same content:
  • Claude (adapters/claude/): agent-skill package per profession — SKILL.md (trigger description + the contribution behaviour of §3, referencing competence files as lazily loaded references) + marketplace.json (repo as plugin marketplace).
  • ChatGPT (adapters/openai/): export bundle per profession — system instruction generated from the same source + competence files as knowledge upload; write path via Action/OpenAPI onto the MCP gateway.
  • Gemini (adapters/gemini/): analogous bundle; write path via function calling onto the same gateway.
• MCP as the vendor-neutral bridge: read/write on the repos goes through one MCP server (base: official gitea-mcp, extended with SkillFactor tools, see services/skillfactor-mcp/). One server serves all three platforms; adapters differ only in how they attach.
• The skill is also the enrichment interface: it carries the contribution logic (§3), not just the knowledge.

flowchart LR
    C[(Canonical content:<br/>Markdown + YAML + JSON)] --> A1[adapters/claude<br/>SKILL.md + marketplace.json]
    C --> A2[adapters/openai<br/>instruction + knowledge bundle]
    C --> A3[adapters/gemini<br/>instruction + knowledge bundle]
    A1 --> MCP[skillfactor-mcp gateway<br/>reads/writes via gitea-mcp]
    A2 -->|OpenAPI action| MCP
    A3 -->|function calling| MCP
    MCP --> GIT[(Gitea repos:<br/>community + org)]

3. Enrichment path A — conversation distillate (interactive)

Behaviour that belongs in EVERY profession skill package (adapter level):

1. Selective triggering: suggest a contribution only when something was solved that (a) is not trivially in the profile, (b) is transferable, (c) contained a real aha. Manually triggerable at any time with the canonical prompt: "Save the key insight from this chat as a lesson learned in my SkillFactor repo." (variants: "What should we remember from this chat as a team? File it in the repo." / "Log this as a lesson for my project .")
2. Switch with double distillate: propose up to two placements — the project distillate WITH customer reference → layer 3; a generalized, anonymized distillate (if a transferable core exists) → layer 1 as PR. The user picks: both / one / none.
3. Confirmation on full text: the user always sees the complete final text; what is committed is what was confirmed — word for word. Default assignment is the user's profession, with reasoned alternatives via the competence mapping.
4. Duplicate check before commit: search the target folder for similar lessons first; on a hit, propose updating the existing file instead of creating a new one.
5. Anonymization rules for anything toward layer 1: no names, no company/project references, roles instead of persons ("the client's site manager"), only the transferable pattern. Internally (layers 2/3): roles instead of clear names where possible.
6. Always a PR, never a direct commit to main. Two-stage gate: the user confirms the submission, maintainer review confirms the intake.

4. Enrichment path B — M365 mining via SQL Server (batch)

A connector replicates an organisation's M365 data (mails, Teams chats, meeting transcripts) into a SQL Server. The agent NEVER touches M365 directly; it reads that database read-only (spec: services/m365-mining/).

• Periodic batch per user (each user is assigned a profession): read new items since the last run → extract lesson candidates (same criteria as path A) → suggest project/customer assignment from metadata (subject, Teams channel, calendar context) → same switch, same anonymization → PR collection into the org repo, confirmed by the user via a review queue — no auto-merge.
• Privacy principles: raw data never leaves the SQL Server toward git — distillates only. Person references are replaced by roles at extraction time. Processing only for users with a consent flag in the DB. Every lesson references its source IDs only as opaque audit references, never as content.
• Technique: MCP tool set m365-mining (connection string from env var, parameterized read-only queries). The concrete DB schema is not yet available → expected minimal schema is defined as an assumption (DECISIONS D10) in services/m365-mining/SPEC.md.

Guard rails (non-negotiable)

• Content first; platform binding only in adapters.
• Nothing public may carry customer/company/person references; nothing automatic crosses the trust boundary (layers 2/3 → 1).
• No real credentials/hosts in code or docs — env vars + .env.example.
• Schemas under schemas/ are binding; the verify stage validates all content files against them.