113 lines
6.0 KiB
Markdown
113 lines
6.0 KiB
Markdown
# Tools & technology — data engineer
|
|
|
|
Source: O*NET 30.3, occupation 15-2051.00 (Data Scientists) — manual nearest-occupation mapping via ISCO group 2511; the official ESCO crosswalk has no entry for this ESCO occupation.
|
|
|
|
| Software | Category | Hot technology |
|
|
|---|---|---|
|
|
| Alteryx software | Business intelligence and data analysis software | yes |
|
|
| Amazon Elastic Compute Cloud EC2 | Data base user interface and query software | yes |
|
|
| Amazon Redshift | Data base user interface and query software | yes |
|
|
| Amazon Web Services AWS software | Data base user interface and query software | yes |
|
|
| Apache Airflow | Procedure management software | yes |
|
|
| Apache Cassandra | Data base management system software | yes |
|
|
| Apache Hadoop | Data base management system software | yes |
|
|
| Apache Hive | Data base management system software | yes |
|
|
| Apache Kafka | Development environment software | yes |
|
|
| Apache Spark | Business intelligence and data analysis software | yes |
|
|
| Atlassian Confluence | Project management software | yes |
|
|
| Atlassian JIRA | Content workflow software | yes |
|
|
| Bash | Operating system software | yes |
|
|
| C | Development environment software | yes |
|
|
| C# | Object or component oriented development software | yes |
|
|
| C++ | Object or component oriented development software | yes |
|
|
| Docker | Application server software | yes |
|
|
| Elasticsearch | Data base management system software | yes |
|
|
| Git | File versioning software | yes |
|
|
| GitHub | Application server software | yes |
|
|
| Go | Development environment software | yes |
|
|
| Google Looker Analytics | Business intelligence and data analysis software | yes |
|
|
| IBM SPSS Statistics | Analytical or scientific software | yes |
|
|
| JavaScript | Web platform development software | yes |
|
|
| JavaScript Object Notation JSON | Web platform development software | yes |
|
|
| Jenkins CI | Enterprise application integration software | yes |
|
|
| Kubernetes | Application server software | yes |
|
|
| Linux | Operating system software | yes |
|
|
| Microsoft Access | Data base user interface and query software | yes |
|
|
| Microsoft Azure software | Development environment software | yes |
|
|
| Microsoft Excel | Spreadsheet software | yes |
|
|
| Microsoft Office software | Office suite software | yes |
|
|
| Microsoft Power BI | Business intelligence and data analysis software | yes |
|
|
| Microsoft PowerPoint | Presentation software | yes |
|
|
| Microsoft SQL Server | Data base user interface and query software | yes |
|
|
| MongoDB | Data base management system software | yes |
|
|
| NoSQL | Data base management system software | yes |
|
|
| Oracle Java | Object or component oriented development software | yes |
|
|
| Perl | Object or component oriented development software | yes |
|
|
| PostgreSQL | Object oriented data base management software | yes |
|
|
| Python | Object or component oriented development software | yes |
|
|
| PyTorch | Data base user interface and query software | yes |
|
|
| R | Object or component oriented development software | yes |
|
|
| Ruby | Development environment software | yes |
|
|
| SAS | Analytical or scientific software | yes |
|
|
| Scala | Object or component oriented development software | yes |
|
|
| Shell script | Operating system software | yes |
|
|
| Snowflake | Data mining software | yes |
|
|
| Splunk Enterprise | Enterprise system management software | yes |
|
|
| Structured query language SQL | Data base user interface and query software | yes |
|
|
| Tableau | Business intelligence and data analysis software | yes |
|
|
| TensorFlow | Analytical or scientific software | yes |
|
|
| Teradata Database | Data base management system software | yes |
|
|
| The MathWorks MATLAB | Analytical or scientific software | yes |
|
|
| UNIX | Operating system software | yes |
|
|
| Amazon Simple Storage Service S3 | Storage networking software | |
|
|
| Amazon Web Services AWS SageMaker | Cloud-based management software | |
|
|
| Apache MXNet | Industrial control software | |
|
|
| Apache Pig | Data base management system software | |
|
|
| BigQuery | Data base user interface and query software | |
|
|
| Business intelligence software | Business intelligence and data analysis software | |
|
|
| Flask | Development environment software | |
|
|
| Geographic information system GIS systems | Geographic information system | |
|
|
| Google Cloud software | Cloud-based management software | |
|
|
| Julia | Development environment software | |
|
|
| Jupyter software | Object or component oriented development software | |
|
|
| Keras | Operating system software | |
|
|
| Kubeflow | Analytical or scientific software | |
|
|
| Management information systems MIS | Enterprise resource planning ERP software | |
|
|
| MapReduce big data software | Business intelligence and data analysis software | |
|
|
| Mathematical software | Analytical or scientific software | |
|
|
| Mlflow | Analytical or scientific software | |
|
|
| Neo4j | Data base user interface and query software | |
|
|
| NumPy | Data base user interface and query software | |
|
|
| OpenAI ChatGPT | Development environment software | |
|
|
| pandas | Data base user interface and query software | |
|
|
| PySpark | Data base user interface and query software | |
|
|
| Qlik Tech QlikView | Business intelligence and data analysis software | |
|
|
| Reporting software | Data base reporting software | |
|
|
| RESTful API | Web platform development software | |
|
|
| Scikit-learn | Development environment software | |
|
|
| SciPy | Object or component oriented development software | |
|
|
| Shiny | Object or component oriented development software | |
|
|
| spaCy | Object or component oriented development software | |
|
|
| StataCorp Stata | Analytical or scientific software | |
|
|
| Statistical software | Analytical or scientific software | |
|
|
| XGBoost | Development environment software | |
|
|
|
|
<!-- market-evidence -->
|
|
|
|
## Market evidence (job-ad analysis, 22 ads, as of 2026-07-11)
|
|
|
|
Share of analyzed job ads mentioning the item (threshold ≥ 20 %). Source: JSearch/Adzuna APIs.
|
|
|
|
### Tools
|
|
|
|
- Python — **41 %**
|
|
- Databricks — **41 %**
|
|
- Apache Spark — **23 %**
|
|
- AWS — **23 %**
|
|
- Spark — **23 %**
|
|
- SQL — **23 %**
|
|
- Snowflake — **18 %**
|
|
- informatica — **18 %**
|
|
|
|
<!-- market-evidence -->
|