Files
skillfactor-pipeline/docs/batch-plan.md

3.1 KiB

Job-ad enrichment — staged batch plan (JSearch basic: 200 requests/month)

Budget math: 1 request ≈ 10 ads; target ≥ 50 ads per occupation ≈ 5-7 requests (incl. duplicates, us+gb). → ~30 occupations per month in good quality. All responses are cached in data/raw/jobs/ — re-runs cost 0 requests.

Month 1 (started 2026-07): recruiter + batch 1 — HR & IT (25)

recruitment-consultant (done first, per MVP mandate), then: broadcast-technician, computer-scientist, computer-vision-engineer, corporate-training-manager, data-analyst, database-administrator, data-scientist, data-warehouse-designer, equality-and-inclusion-manager, human-resources-manager, ict-help-desk-agent, ict-network-administrator, ict-network-engineer, ict-research-consultant, ict-resilience-manager, ict-security-technician, ict-system-administrator, ict-system-analyst, ict-system-architect, ict-technician, projectionist, telecommunications-engineering-technician, video-technician, webmaster

Month 2: batch 2 — finance, marketing/sales, support (40; ggf. auf 2 Monate splitten)

accountant, administrative-assistant, advertising-specialist, asset-manager, budget-analyst, call-centre-supervisor, commercial-sales-representative, corporate-treasurer, cost-analyst, costume-buyer, court-reporter, customer-contact-centre-information-clerk, data-entry-supervisor, ebusiness-manager, financial-auditor, financial-broker, financial-controller, financial-fraud-examiner, financial-manager, financial-trader, foreign-exchange-broker, hospitality-revenue-manager, host-hostess, insurance-broker, insurance-risk-consultant, insurance-underwriter, legal-administrative-assistant, marketing-assistant, marketing-consultant, market-research-analyst, mutual-fund-broker, network-marketer, office-manager, public-finance-accountant, purchase-planner, securities-broker, ticket-sales-agent, tour-organiser, travel-agent, travel-consultant

aquaculture-harvesting-manager, aquaculture-husbandry-manager, business-analyst, business-intelligence-manager, chemical-production-manager, chief-executive-officer, civil-service-administrative-officer, construction-general-contractor, coroner, customs-and-excise-officer, environmental-policy-officer, event-assistant, event-manager, fire-commissioner, fiscal-affairs-policy-officer, forester, forwarding-manager, government-minister, import-export-specialist, industrial-production-manager, intelligence-officer, judge, lawyer, legal-consultant, logistics-analyst, police-commissioner, procurement-department-manager, programme-manager, public-administration-manager, regulatory-affairs-manager, secretary-general, secretary-of-state, supreme-court-judge, venue-director, waste-management-officer

Scaling options

  • Adzuna (free, ADZUNA_APP_ID/KEY in .env) as breadth channel: up to 50 hits per call, but truncated description snippets → good for tool/title counting, weak for methods/responsibilities. Mark lower evidence quality in the manifest when used.
  • openwebninja paid tier once the monthly full refresh across 100+ occupations becomes production reality.