docs(arch): binding architecture + JSON schemas (lesson/competence/vocabulary)

Three knowledge layers with promotion logic, platform-neutral canonical content with thin adapters, MCP gateway as the vendor bridge, two enrichment paths. Schemas encode the trust boundary (customer/project forbidden in layer 1). Neutrality audit: clean (D9). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDKeXvpT6tENSvyQGLV1Uq
2026-07-10 05:37:25 +02:00
parent a2f17bdc12
commit c79801886b
5 changed files with 290 additions and 0 deletions
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -0,0 +1,150 @@
+# SkillFactor — Architecture (binding target picture)
+
+Status: binding as of 2026-07-10 (branch `quality/reference-standard`).
+SkillFactor is a git-based **Occupational Skill Layer**: profession profiles
+with competences + practitioner knowledge, preloaded into AI agents as
+"work experience". Three knowledge layers, one platform-neutral skill
+interface, two enrichment paths.
+
+## 1. Three knowledge layers
+
+| Layer | Content | Visibility | Storage |
+|---|---|---|---|
+| **1 — Community profession profiles** | `professions/<slug>/competences/<skill-slug>.md`, `vocabulary.json`, `index.md` — anonymized, curated, license-clean (CC-BY-SA attribution) | public | this repo (Gitea), PR-based intake |
+| **2 — Organisation overlay** | company standards, processes, calculation rules, preferred partners | private per company | one private Gitea repo per org, **structural mirror of layer 1** (same profession slugs, same competence slugs) — specific knowledge extends/overrides general knowledge |
+| **3 — Project/customer knowledge** | `projects/<customer-slug>/<project-slug>/lessons/<date>-<slug>.md` — customer/project reference explicitly wanted here | private, per project folder | inside the org repo; access via Gitea team permissions per project folder |
+
+**Double tagging as the connective tissue:** every layer-2/3 lesson
+references layer-1 competence URIs (ESCO/O*NET) in its frontmatter.
+Retrieval works both ways: "What do we know about customer A?" (by folder)
+and "What do we know about acceptance processes?" (by competence — pulls
+customer-A/B lessons up).
+
+**Loading cascade for agents:** profession profile (1) → org overlay (2) →
+project context (3); more specific beats more general ("in general X, for
+customer A deviating Y").
+
+**Promotion logic (knowledge rises by proving itself):**
+- Same insight in ≥ 2 projects → suggested promotion into the org overlay.
+- Proven org knowledge without company specifics → anonymized second
+  distillate as a PR to the community.
+- **Hard rule:** nothing flows automatically out of `projects/`. The path
+  into layer 1 is ALWAYS an explicitly confirmed, generalized, anonymized
+  second distillate.
+
+```mermaid
+flowchart TB
+    subgraph L1["Layer 1 — Community (public repo)"]
+      P1["professions/&lt;slug&gt;/competences/*.md"]
+    end
+    subgraph ORG["Org repo (private, per company)"]
+      L2["Layer 2 — overlay mirror<br/>professions/&lt;slug&gt;/competences/*.md"]
+      L3["Layer 3 — projects/&lt;customer&gt;/&lt;project&gt;/lessons/*.md"]
+    end
+    AGENT((Agent)) -->|1. load| P1
+    AGENT -->|2. overlay| L2
+    AGENT -->|3. project context| L3
+    L3 -->|">=2 projects: promotion suggestion"| L2
+    L2 -->|"confirmed, generalized,<br/>anonymized 2nd distillate — PR"| P1
+    CHAT["Path A: conversation distillate"] -->|PR, user-confirmed| L3
+    CHAT -->|anonymized PR| P1
+    M365["Path B: M365 mining<br/>(SQL Server, read-only)"] -->|PR collection,<br/>review queue| L3
+```
+
+## 2. The skill as a universal interface (multi-platform, CRITICAL)
+
+Usable as an agent skill in **Claude, ChatGPT and Gemini**:
+
+- **Canonical format is the git repo itself:** plain Markdown + YAML
+  frontmatter + JSON. NO Claude-specific constructs in content files (no
+  tool names, no Anthropic-only syntax). Platform binding lives exclusively
+  in thin adapters.
+- **Adapters as build targets from the same content:**
+  - *Claude* (`adapters/claude/`): agent-skill package per profession —
+    `SKILL.md` (trigger description + the contribution behaviour of §3,
+    referencing competence files as lazily loaded references) +
+    `marketplace.json` (repo as plugin marketplace).
+  - *ChatGPT* (`adapters/openai/`): export bundle per profession — system
+    instruction generated from the same source + competence files as
+    knowledge upload; write path via Action/OpenAPI onto the MCP gateway.
+  - *Gemini* (`adapters/gemini/`): analogous bundle; write path via
+    function calling onto the same gateway.
+- **MCP as the vendor-neutral bridge:** read/write on the repos goes
+  through one MCP server (base: official `gitea-mcp`, extended with
+  SkillFactor tools, see `services/skillfactor-mcp/`). One server serves
+  all three platforms; adapters differ only in how they attach.
+- **The skill is also the enrichment interface:** it carries the
+  contribution logic (§3), not just the knowledge.
+
+```mermaid
+flowchart LR
+    C[(Canonical content:<br/>Markdown + YAML + JSON)] --> A1[adapters/claude<br/>SKILL.md + marketplace.json]
+    C --> A2[adapters/openai<br/>instruction + knowledge bundle]
+    C --> A3[adapters/gemini<br/>instruction + knowledge bundle]
+    A1 --> MCP[skillfactor-mcp gateway<br/>reads/writes via gitea-mcp]
+    A2 -->|OpenAPI action| MCP
+    A3 -->|function calling| MCP
+    MCP --> GIT[(Gitea repos:<br/>community + org)]
+```
+
+## 3. Enrichment path A — conversation distillate (interactive)
+
+Behaviour that belongs in EVERY profession skill package (adapter level):
+
+1. **Selective triggering:** suggest a contribution only when something was
+   solved that (a) is not trivially in the profile, (b) is transferable,
+   (c) contained a real aha. Manually triggerable at any time with the
+   canonical prompt:
+   **"Speichere die Erkenntnisse aus diesem Chat als Lesson Learned in
+   meinem SkillFactor-Repo."**
+   (variants: "Was sollten wir uns aus diesem Chat als Team merken? Leg es
+   im Repo ab." / "Trag das als Lesson für mein Projekt <Kunde> ein.")
+2. **Switch with double distillate:** propose up to two placements — the
+   project distillate WITH customer reference → layer 3; a generalized,
+   anonymized distillate (if a transferable core exists) → layer 1 as PR.
+   The user picks: both / one / none.
+3. **Confirmation on full text:** the user always sees the complete final
+   text; what is committed is what was confirmed — word for word. Default
+   assignment is the user's profession, with reasoned alternatives via the
+   competence mapping.
+4. **Duplicate check before commit:** search the target folder for similar
+   lessons first; on a hit, propose updating the existing file instead of
+   creating a new one.
+5. **Anonymization rules for anything toward layer 1:** no names, no
+   company/project references, roles instead of persons ("the client's
+   site manager"), only the transferable pattern. Internally (layers 2/3):
+   roles instead of clear names where possible.
+6. **Always a PR, never a direct commit to main.** Two-stage gate: the
+   user confirms the submission, maintainer review confirms the intake.
+
+## 4. Enrichment path B — M365 mining via SQL Server (batch)
+
+A connector replicates an organisation's M365 data (mails, Teams chats,
+meeting transcripts) into a **SQL Server**. The agent NEVER touches M365
+directly; it reads that database read-only (spec:
+`services/m365-mining/`).
+
+- Periodic batch per user (each user is assigned a profession): read new
+  items since the last run → extract lesson candidates (same criteria as
+  path A) → suggest project/customer assignment from metadata (subject,
+  Teams channel, calendar context) → same switch, same anonymization →
+  **PR collection into the org repo, confirmed by the user via a review
+  queue — no auto-merge.**
+- **Privacy principles:** raw data never leaves the SQL Server toward git —
+  distillates only. Person references are replaced by roles at extraction
+  time. Processing only for users with a consent flag in the DB. Every
+  lesson references its source IDs only as opaque audit references, never
+  as content.
+- **Technique:** MCP tool set `m365-mining` (connection string from env
+  var, parameterized read-only queries). The concrete DB schema is not yet
+  available → expected minimal schema is defined as an assumption
+  (DECISIONS D10) in `services/m365-mining/SPEC.md`.
+
+## Guard rails (non-negotiable)
+
+- Content first; platform binding only in adapters.
+- Nothing public may carry customer/company/person references; nothing
+  automatic crosses the trust boundary (layers 2/3 → 1).
+- No real credentials/hosts in code or docs — env vars + `.env.example`.
+- Schemas under `schemas/` are binding; the verify stage validates all
+  content files against them.
--- a/knowledge/DECISIONS.md
+++ b/knowledge/DECISIONS.md
@@ -70,3 +70,26 @@ die Homepage-Statusleiste zeigt den SE-Layer-Stand neben dem Crawl-Zähler.
  Lauf (Netz-Budget); Rest behaelt review_needed=true und steht im REPORT.
 - "Weitere Anreicherung": domaenenspezifische Stufe-2-Quelle je Site-Familie
  (EconStor fuer Finanzen, CORE/DOAB fuer Recht/HR, arXiv fuer SWE/DS).
+
+## D9 - Architektur-Auftrag (2026-07-10)
+- Neutralitaets-Audit: Content unter knowledge/professions enthaelt keine
+  Plattform-Konstrukte. Die Treffer auf "Claude" sind (a) die ehrliche
+  Provenance-Zeile "generator: ... spot-checked by Claude" im Frontmatter
+  (Metadaten, keine Verhaltens-/Tool-Syntax - bleibt) und (b) ein
+  Stack-Exchange-Autor mit Klarnamen "Claude" (CC-BY-SA-Attribution -
+  bleibt zwingend). Verhaltensregeln (Weiche/Destillat/Bestaetigung)
+  liegen ausschliesslich in adapters/, nicht im Content.
+- Schemas: competence.schema.json bildet exakt das Frontmatter des
+  Generators ab; lesson.schema.json verlangt customer/project NUR in
+  Schicht 3 und verbietet beides in Schicht 1 (harte Vertrauensgrenze im
+  Schema selbst).
+- Org-Gitea-URL, reale Org-Namen: nicht vorhanden -> Template nutzt
+  Platzhalter ($GITEA_URL, acme-bau als fiktiver Bauunternehmer).
+
+## D10 - M365-Minimal-Schema (Annahme)
+Reales Connector-Schema liegt nicht vor. Angenommenes Minimal-Schema
+(services/m365-mining/SPEC.md, DDL): m365_users(user_id, upn,
+profession_slug, consent, consent_date), m365_messages(item_id, user_id,
+kind[mail|chat|transcript], subject, body, channel, sent_at, processed),
+m365_project_map(pattern, customer_slug, project_slug, priority).
+Anpassung an das reale Schema = reine Query-Ebene, Spec bleibt gueltig.
--- a/schemas/competence.schema.json
+++ b/schemas/competence.schema.json
@@ -0,0 +1,37 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://zeiterfassung.cloud/skillfactor/schemas/competence.schema.json",
+  "title": "SkillFactor competence-file frontmatter (layer 1)",
+  "type": "object",
+  "required": ["esco_uri", "esco_label", "relation", "onet_soc", "source",
+               "confidence", "qa_count", "generated"],
+  "properties": {
+    "esco_uri": { "type": "string",
+      "pattern": "^http://data\\.europa\\.eu/esco/(skill|occupation)/" },
+    "esco_label": { "type": "string", "minLength": 2 },
+    "relation": { "enum": ["essential", "optional"] },
+    "onet_soc": { "type": "string", "pattern": "^\\d{2}-\\d{4}\\.\\d{2}$" },
+    "source": { "enum": ["stackexchange", "web-research", "model-knowledge"] },
+    "confidence": { "enum": ["high", "medium", "low"] },
+    "qa_count": { "type": "integer", "minimum": 0 },
+    "generator": { "type": "string" },
+    "generated": { "type": "string", "pattern": "^\\d{4}-\\d{2}-\\d{2}$" },
+    "review_needed": { "type": "boolean" },
+    "source_urls": {
+      "type": "array", "items": { "type": "string", "format": "uri" },
+      "$comment": "required when source=web-research"
+    },
+    "retrieved": { "type": "string", "pattern": "^\\d{4}-\\d{2}-\\d{2}$" }
+  },
+  "allOf": [
+    {
+      "if": { "properties": { "source": { "const": "web-research" } } },
+      "then": { "required": ["source_urls"] }
+    },
+    {
+      "if": { "properties": { "source": { "const": "model-knowledge" } } },
+      "then": { "required": ["confidence"] }
+    }
+  ],
+  "additionalProperties": true
+}
--- a/schemas/lesson.schema.json
+++ b/schemas/lesson.schema.json
@@ -0,0 +1,54 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://zeiterfassung.cloud/skillfactor/schemas/lesson.schema.json",
+  "title": "SkillFactor lesson frontmatter (layers 1/2/3)",
+  "type": "object",
+  "required": ["profession", "competences", "date", "source", "status", "layer"],
+  "properties": {
+    "layer": { "enum": [1, 2, 3] },
+    "profession": { "type": "string", "pattern": "^[a-z0-9-]+$" },
+    "competences": {
+      "type": "array", "minItems": 1,
+      "items": {
+        "type": "object",
+        "required": ["esco_uri"],
+        "properties": {
+          "esco_uri": { "type": "string", "pattern": "^http://data\\.europa\\.eu/esco/(skill|occupation)/" },
+          "onet_soc": { "type": "string", "pattern": "^\\d{2}-\\d{4}\\.\\d{2}$" }
+        }
+      }
+    },
+    "customer": { "type": "string", "pattern": "^[a-z0-9-]+$",
+      "$comment": "layer 3 only; forbidden in layer 1" },
+    "project": { "type": "string", "pattern": "^[a-z0-9-]+$",
+      "$comment": "layer 3 only; forbidden in layer 1" },
+    "contributor": { "type": "string",
+      "$comment": "role or handle; never a clear name toward layer 1" },
+    "date": { "type": "string", "pattern": "^\\d{4}-\\d{2}-\\d{2}$" },
+    "source": { "enum": ["conversation", "m365-mail", "m365-chat",
+                          "m365-transcript", "promotion"] },
+    "confidence": { "enum": ["high", "medium", "low"] },
+    "status": { "enum": ["proposed", "accepted", "promoted"] },
+    "source_refs": {
+      "type": "array",
+      "items": { "type": "string" },
+      "$comment": "opaque audit references (e.g. m365:msg:<id>, chat session id); never content"
+    },
+    "promoted_from": { "type": "string",
+      "$comment": "path of the origin lesson when status=promoted" }
+  },
+  "allOf": [
+    {
+      "if": { "properties": { "layer": { "const": 3 } } },
+      "then": { "required": ["customer", "project"] }
+    },
+    {
+      "if": { "properties": { "layer": { "const": 1 } } },
+      "then": {
+        "not": { "anyOf": [ { "required": ["customer"] },
+                            { "required": ["project"] } ] }
+      }
+    }
+  ],
+  "additionalProperties": true
+}
--- a/schemas/vocabulary.schema.json
+++ b/schemas/vocabulary.schema.json
@@ -0,0 +1,26 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://zeiterfassung.cloud/skillfactor/schemas/vocabulary.schema.json",
+  "title": "SkillFactor profession vocabulary",
+  "type": "object",
+  "required": ["profession", "terms"],
+  "properties": {
+    "profession": { "type": "string", "pattern": "^[a-z0-9-]+$" },
+    "source_pairs": { "type": "integer", "minimum": 0 },
+    "terms": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["term", "doc_freq"],
+        "properties": {
+          "term": { "type": "string", "minLength": 2 },
+          "doc_freq": { "type": "integer", "minimum": 1 }
+        }
+      }
+    },
+    "competence_terms": {
+      "type": "array", "items": { "type": "string" }
+    }
+  },
+  "additionalProperties": false
+}