5c03e07ded
Simpler story (slide decks unread -> three-bullet status email), proper Claude-chat look with avatars and bubbles on both sides. Canonical trigger prompt switched to English everywhere (homepage, architecture, both adapter generators rebuilt) - V7 consistency green, all checks pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDKeXvpT6tENSvyQGLV1Uq
2.5 KiB
esco_uri, esco_label, relation, onet_soc, source, confidence, qa_count, generator, generated
| esco_uri | esco_label | relation | onet_soc | source | confidence | qa_count | generator | generated |
|---|---|---|---|---|---|---|---|---|
| http://data.europa.eu/esco/skill/f9670490-8aa4-4540-b121-d440a8294aab | develop data processing applications | essential | 15-2051.00 | model-knowledge | high | 0 | gemma3:27b (prompt-designed and spot-checked by Claude) | 2026-07-10 |
develop data processing applications — Data Scientist
As a Data Scientist, 'develop data processing applications' isn’t about building full-blown software products; it's about rapidly prototyping and deploying code to transform raw data into analysis-ready formats. Daily tasks involve writing scripts (often in Python or R) to clean messy datasets – handling missing values, outliers, inconsistent formatting – then performing feature engineering: creating new variables from existing ones that improve model performance. Think automating ETL pipelines (Extract, Transform, Load) for recurring data sources, building custom functions to handle specific data quirks, and sometimes wrapping these into reusable modules or microservices.
Typical tools go beyond basic scripting. You'll frequently use libraries like Pandas, NumPy, and scikit-learn in Python; dplyr and tidyr in R. For larger datasets, you’ll leverage distributed computing frameworks like Spark (often with PySpark) or Dask to parallelize processing. Cloud platforms (AWS, Azure, GCP) are common for scaling these applications. Version control (Git) is essential – treating your data transformation code as production-level software even if it's 'throwaway'.
A key pitfall is over-engineering. Data Scientists need to deliver insights quickly; spending months building a perfectly optimized, fully generalized application isn’t usually the priority. Another is neglecting data quality checks within your processing pipeline – assuming input data will always be clean leads to unreliable results. Poorly documented code and lack of testing are also common issues that hinder reproducibility and collaboration.
Good looks like: well-commented, modular code; automated tests verifying data transformations; clear documentation outlining the purpose and assumptions of each script/module; efficient handling of large datasets; and a focus on delivering clean, reliable data to downstream analysis or modeling tasks. It's about being a pragmatic programmer focused on data problems, not just software engineering best practices (though those are important too!).
Weitere Anreicherung
Stage-2 source for future practitioner grounding: arXiv cs.LG/stat.ML (ML preprints).