# Role Curation Log Generated 2026-04-19T09:25:05.374883+00:00Z by `scripts/curate_roles.py` This log shows, per role, which skills were kept and which were dropped (with reason). Source of truth for keep decisions: `seed_data/role_curation.yaml`. Edit there + re-run. ## Data Scientist **Kept (16 skills):** - `Python` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `SQL` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Pandas` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `NumPy` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `Scikit-learn` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `PyTorch` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Statistical Hypothesis Testing` (mandatory, weight=0.9, level=INTERMEDIATE, source=manual) - `Git` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `TensorFlow` (optional, weight=0.9, level=INTERMEDIATE, source=onet) - `AWS` (optional, weight=0.8, level=INTERMEDIATE, source=onet) - `Tableau` (optional, weight=0.8, level=INTERMEDIATE, source=onet) - `Apache Spark` (optional, weight=0.7, level=INTERMEDIATE, source=onet) - `Feature Engineering` (optional, weight=0.7, level=INTERMEDIATE, source=manual) - `Microsoft Power BI` (optional, weight=0.6, level=INTERMEDIATE, source=onet) - `Matplotlib` (optional, weight=0.6, level=BEGINNER, source=manual) - `Docker` (optional, weight=0.6, level=INTERMEDIATE, source=manual) **Dropped (13 skills):** - `Apache Hadoop` — Spark replaced it - `C++` — Low-level numerics only; not in PK DS postings - `Microsoft Azure` — Pick one cloud — AWS retained as primary - `Microsoft Excel` — R4: not core for DS (Excel is for DA/BI roles) - `Microsoft PowerPoint` — Presentation tool, not measurable - `Java` — Rare in DS workflow - `R` — R4: PK DS market is Python+SQL; R rarely required (Glassdoor PK Data Scientist evidence) - `SAS` — Enterprise legacy - `Snowflake` — R4: warehouse skill — Data Engineer territory - `MATLAB` — Academic only - `Jupyter Notebook` — R4: assumed knowledge, not learnable on slider - `Seaborn` — R4: covered by Matplotlib retention - `Data Structures & Algorithms` — R4: interview prep, not posting requirement (per job-board evidence) ## Data Analyst **Kept (10 skills):** - `SQL` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Microsoft Excel` (mandatory, weight=1.0, level=ADVANCED, source=onet) - `Microsoft Power BI` (optional, weight=0.9, level=INTERMEDIATE, source=manual) - `Python` (mandatory, weight=0.9, level=INTERMEDIATE, source=onet) - `Tableau` (optional, weight=0.8, level=INTERMEDIATE, source=onet+manual) - `Pandas` (optional, weight=0.7, level=INTERMEDIATE, source=manual) - `DAX` (optional, weight=0.7, level=BEGINNER, source=manual) - `Power Query / M Language` (optional, weight=0.6, level=BEGINNER, source=manual) - `Data Visualization` (optional, weight=0.6, level=BEGINNER, source=manual) - `Statistical Hypothesis Testing` (optional, weight=0.5, level=INTERMEDIATE, source=manual) **Dropped (7 skills):** - `AWS` — Junior DA in PK rarely owns cloud infra - `IBM SPSS Statistics` — Enterprise legacy - `Microsoft Access` — Legacy desktop DB - `Microsoft Office` — Too generic - `Microsoft PowerPoint` — Presentation tool - `R` — R4: PK DA postings rarely list R; demoted then dropped - `SAS` — Enterprise legacy; PK market is Python/SQL ## Machine Learning Engineer **Kept (16 skills):** - `Python` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `PyTorch` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `Scikit-learn` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `SQL` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Docker` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `AWS` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Pandas` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `NumPy` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `Git` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `GitHub` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `TensorFlow` (optional, weight=0.9, level=INTERMEDIATE, source=onet+manual) - `MLflow` (optional, weight=0.8, level=INTERMEDIATE, source=manual) - `Kubernetes` (optional, weight=0.7, level=INTERMEDIATE, source=onet+manual) - `FastAPI` (optional, weight=0.7, level=BEGINNER, source=manual) - `Apache Spark` (optional, weight=0.7, level=INTERMEDIATE, source=onet) - `Linux` (optional, weight=0.6, level=INTERMEDIATE, source=onet) **Dropped (26 skills):** - `Apache Hadoop` — Spark replaced it - `C++` — Niche optimization only - `Microsoft Azure` — Pick one cloud — AWS retained - `Microsoft Excel` — Not core to ML eng - `Microsoft Power BI` — BI tool - `Microsoft PowerPoint` — Presentation tool - `Java` — Rare in ML pipelines - `R` — Statistics-heavy — DS territory - `SAS` — Enterprise legacy - `Snowflake` — Data Engineer territory - `Tableau` — BI tool - `MATLAB` — Academic only - `Ansible` — DevOps territory - `Bash` — R4: assumed background skill - `C` — Legacy systems language - `Go` — Rare in ML pipelines - `Terraform` — DevOps territory - `JavaScript` — Frontend, not ML - `Jenkins` — Replaced by GitHub Actions; not ML-eng-specific - `NoSQL` — Generic - `Scala` — Spark+Scala niche, declining - `Splunk Enterprise` — Observability niche - `ONNX` — R4: niche optimization, not in PK ML postings - `Data Structures & Algorithms` — R4: not in postings (interview prep) - `Object-Oriented Programming` — R4: not in postings (interview prep) - `Agile / Scrum` — R4: rarely in ML eng postings ## Backend Developer **Kept (16 skills):** - `Python` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Django` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `Node.js` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `PostgreSQL` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `SQL` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `REST API Design` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `Docker` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Git` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `GitHub` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `JWT / OAuth` (mandatory, weight=0.8, level=BEGINNER, source=manual) - `FastAPI` (optional, weight=0.8, level=INTERMEDIATE, source=manual) - `TypeScript` (optional, weight=0.7, level=INTERMEDIATE, source=onet) - `AWS` (optional, weight=0.7, level=INTERMEDIATE, source=onet) - `NoSQL` (optional, weight=0.6, level=INTERMEDIATE, source=onet) - `Laravel` (optional, weight=0.4, level=BEGINNER, source=manual) - `PHP` (optional, weight=0.4, level=BEGINNER, source=manual) **Dropped (34 skills):** - `Apache Kafka` — Data Engineer territory - `Jira` — PM tool - `C` — Not Python/Node-stack backend - `C#` — Microsoft .NET stack — separate market - `C++` — Game/systems, not web backend - `CSS` — Frontend - `XML` — Legacy - `Angular` — Frontend - `HTML` — Frontend - `Terraform` — DevOps territory - `JavaScript` — no reason given (please add to role_curation.yaml drop_reasons) - `JSON` — Assumed knowledge - `Jenkins` — DevOps territory - `Kubernetes` — R4: senior-only in PK; Docker covers entry-level - `Linux` — R4: assumed background - `.NET` — Microsoft enterprise stack — separate market - `Microsoft Azure` — Pick one cloud — AWS retained - `Java` — Spring stack — separate market - `React` — Frontend - `Spring Boot` — Java stack - `Spring Framework` — Java stack - `Flask` — R4: declining; FastAPI/Django dominate PK postings - `Redis` — R4: rarely listed by name in PK postings (assumed within Postgres/MongoDB choice) - `pytest` — R4: rarely listed by name in postings - `Celery` — R4: rarely listed by name - `nginx` — R4: rarely listed by name - `Pydantic` — R4: rarely listed by name (implicit with FastAPI) - `Data Structures & Algorithms` — R4: interview prep, not posting requirement - `Object-Oriented Programming` — R4: assumed for interviews, not posting requirement - `Agile / Scrum` — R4: not in posting requirement lists - `CI/CD Pipelines` — R4: not as concept skill in postings - `GitHub Actions` — R4: rare standalone in PK BE postings - `.NET / C#` — R4: separate Pakistani market, deserves its own role tier - `Unit Testing` — R4: not in posting requirement lists ## Frontend Developer **Kept (14 skills):** - `React` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `JavaScript` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `TypeScript` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `HTML` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `CSS` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `REST API Design` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `Git` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `GitHub` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Next.js` (mandatory, weight=0.9, level=INTERMEDIATE, source=manual) - `Tailwind CSS` (optional, weight=0.8, level=BEGINNER, source=manual) - `Redux` (optional, weight=0.7, level=BEGINNER, source=manual) - `Node.js` (optional, weight=0.7, level=INTERMEDIATE, source=onet) - `Vite` (optional, weight=0.5, level=BEGINNER, source=manual) - `TanStack Query` (optional, weight=0.4, level=BEGINNER, source=manual) **Dropped (33 skills):** - `AWS` — DevOps territory - `Apache Kafka` — Data Eng - `Jira` — PM tool - `C#` — Backend - `C++` — Not FE - `Docker` — R4: rarely listed for FE-only roles - `Go` — Not FE - `Angular` — R4: separate role market in PK - `GraphQL` — R4: rare in PK FE postings - `JSON` — Assumed - `Jenkins` — DevOps - `jQuery` — Legacy - `Kubernetes` — DevOps territory - `Linux` — no reason given (please add to role_curation.yaml drop_reasons) - `Microsoft Azure` — DevOps territory - `MongoDB` — Backend/data - `MySQL` — Backend/data - `NoSQL` — Backend/data - `Java` — Backend - `PHP` — Backend (separate Full Stack stack) - `PostgreSQL` — Backend/data - `Python` — Backend - `Spring Boot` — Java backend - `Spring Framework` — Java backend - `SQL` — Backend/data - `Vue.js` — R4: separate role market in PK; React dominates - `WordPress` — CMS — separate small-agency market - `Vitest` — R4: testing rarely in PK FE postings - `Playwright` — R4: testing rarely in PK FE postings - `Web Accessibility (a11y)` — R4: rarely a posted requirement in PK - `Responsive Design` — R4: assumed within React+Tailwind work - `Data Structures & Algorithms` — R4: interview prep, not posting requirement - `Agile / Scrum` — R4: not in FE posting requirement lists ## Full Stack Developer **Kept (18 skills):** - `JavaScript` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `TypeScript` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `React` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Node.js` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Python` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `PostgreSQL` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `REST API Design` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `SQL` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Docker` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Git` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `GitHub` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `HTML` (mandatory, weight=0.9, level=INTERMEDIATE, source=onet) - `CSS` (mandatory, weight=0.9, level=INTERMEDIATE, source=onet) - `Next.js` (optional, weight=0.8, level=INTERMEDIATE, source=manual) - `Django` (optional, weight=0.8, level=INTERMEDIATE, source=manual) - `MongoDB` (optional, weight=0.8, level=INTERMEDIATE, source=onet) - `Tailwind CSS` (optional, weight=0.7, level=BEGINNER, source=manual) - `AWS` (optional, weight=0.7, level=INTERMEDIATE, source=onet) **Dropped (34 skills):** - `Apache Kafka` — Data Eng - `Jira` — PM tool - `C` — Not web full-stack - `C#` — .NET niche — separate stack - `C++` — Not web full-stack - `XML` — Legacy - `Angular` — Pick one FE framework — React retained - `Terraform` — DevOps - `JSON` — Assumed knowledge - `Jenkins` — DevOps - `Kubernetes` — no reason given (please add to role_curation.yaml drop_reasons) - `Linux` — R4: assumed background - `.NET` — .NET niche — separate stack - `Microsoft Azure` — Pick one cloud — AWS retained - `NoSQL` — Generic — MongoDB covers it - `Java` — Spring niche — separate stack - `Spring Boot` — Java stack - `Spring Framework` — Java stack - `Go` — Niche for full-stack web in PK - `GraphQL` — R4: rare in PK FS postings - `jQuery` — Legacy - `MySQL` — PostgreSQL covers SQL needs - `PHP` — R4: separate Laravel/PHP stack market — Backend covers - `Vue.js` — Pick one FE framework — React retained - `WordPress` — CMS — separate market - `Vite` — R4: assumed within Next.js/React tooling - `Vitest` — R4: testing rarely in PK postings - `JWT / OAuth` — R4: covered by Backend role; FS focuses on stack - `Data Structures & Algorithms` — R4: interview prep, not posting requirement - `Object-Oriented Programming` — R4: not in posting requirement lists - `Agile / Scrum` — R4: not in posting requirement lists - `CI/CD Pipelines` — R4: not as concept skill in PK FS postings - `GitHub Actions` — R4: rare standalone for FS - `Laravel` — R4: separate Laravel/PHP stack market — Backend covers ## Data Engineer **Kept (14 skills):** - `Python` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `SQL` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Apache Spark` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `Apache Airflow` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `ETL / ELT` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `AWS` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Git` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `Snowflake` (optional, weight=0.8, level=INTERMEDIATE, source=onet+manual) - `dbt` (optional, weight=0.8, level=INTERMEDIATE, source=manual) - `Apache Kafka` (optional, weight=0.7, level=BEGINNER, source=manual) - `Google BigQuery` (optional, weight=0.9, level=INTERMEDIATE, source=manual) - `Docker` (optional, weight=0.7, level=INTERMEDIATE, source=manual) - `Microsoft Azure` (optional, weight=0.6, level=INTERMEDIATE, source=onet) - `Data Modeling` (optional, weight=0.6, level=INTERMEDIATE, source=manual) **Dropped (14 skills):** - `Microsoft Access` — Legacy desktop DB - `Microsoft Excel` — R4: rare for DE postings - `Microsoft Office` — Too generic - `Microsoft Outlook` — Email client - `Microsoft Power BI` — BI Analyst territory - `Microsoft PowerPoint` — Presentation tool - `Tableau` — BI Analyst territory - `Java` — R4: declining for DE; Python+Scala dominate - `R` — BI/stats territory; Python+SQL covers DE - `Delta Lake` — R4: rare in PK entry-level DE postings - `Apache Iceberg` — R4: rare in PK entry-level DE postings; Delta Lake (also dropped) was the safer pick but neither needed - `Databricks` — R4: senior-only in PK postings - `Parquet` — R4: rarely listed by name in postings (implicit with Spark/Airflow) - `Data Structures & Algorithms` — R4: interview prep, not posting requirement ## DevOps Engineer **Kept (16 skills):** - `Docker` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `Kubernetes` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `Terraform` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `AWS` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `Linux` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Bash Scripting` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `CI/CD Pipelines` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `Git` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `GitHub` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Python` (mandatory, weight=0.8, level=INTERMEDIATE, source=onet) - `GitHub Actions` (optional, weight=0.8, level=INTERMEDIATE, source=manual) - `Microsoft Azure` (optional, weight=0.7, level=BEGINNER, source=onet+manual) - `Ansible` (optional, weight=0.7, level=BEGINNER, source=onet+manual) - `Prometheus` (optional, weight=0.6, level=BEGINNER, source=manual) - `Grafana` (optional, weight=0.6, level=BEGINNER, source=manual) - `Jenkins` (optional, weight=0.5, level=INTERMEDIATE, source=onet+manual) **Dropped (16 skills):** - `Amazon Web Services AWS CloudFormation` — Terraform won - `Jira` — R2: PM tool consistency - `Bash` — Duplicate of Bash Scripting - `C` — Not DevOps day-to-day - `C++` — Not DevOps day-to-day - `Go` — K8s/Terraform internals only - `JavaScript` — Not DevOps - `Microsoft Active Directory` — Windows enterprise niche - `Microsoft PowerShell` — Windows admin only - `Java` — Rare in modern DevOps - `SQL` — R4: debugging-only, not core skill - `Google Cloud Platform` — R4: rare in PK DevOps postings (AWS/Azure dominate) - `Linux Administration` — R2: duplicate of Linux - `ArgoCD` — R4: senior-only in PK postings - `Helm` — R4: senior-only in PK postings - `Computer Networking Fundamentals` — R4: assumed background, not in posting requirement lists ## AI Engineer (GenAI / LLM) **Kept (17 skills):** - `Python` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `HuggingFace Transformers` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `LangChain` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `OpenAI API` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `RAG` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `Prompt Engineering` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `PyTorch` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Docker` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Git` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `GitHub` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `AWS` (optional, weight=0.8, level=INTERMEDIATE, source=onet) - `Microsoft Azure` (optional, weight=0.7, level=INTERMEDIATE, source=onet) - `Anthropic Claude API` (optional, weight=0.7, level=BEGINNER, source=manual) - `pgvector` (optional, weight=0.7, level=BEGINNER, source=manual) - `FastAPI` (optional, weight=0.7, level=BEGINNER, source=manual) - `LangGraph` (optional, weight=0.6, level=BEGINNER, source=manual) - `Sentence Transformers` (optional, weight=0.6, level=BEGINNER, source=manual) **Dropped (26 skills):** - `Ansible` — DevOps - `Apache Hadoop` — Big data — Data Eng - `Apache Spark` — Big data — Data Eng - `Bash` — Background - `C` — Legacy - `C++` — Niche - `Go` — Rare in AI eng - `Terraform` — DevOps - `JavaScript` — Not AI eng - `Jenkins` — DevOps - `Kubernetes` — R4: senior-only in PK AI eng postings - `Linux` — Background - `NoSQL` — Generic — vector DBs are the relevant DB type - `Java` — Rare in AI eng - `R` — Statistics, not LLM eng - `Scala` — Big data, not LLM eng - `Splunk Enterprise` — Observability niche - `SQL` — R4: rare for AI Eng day-to-day - `TensorFlow` — PyTorch dominates research/LLM - `LlamaIndex` — R4: overlaps LangChain; rarely both required in PK postings - `Pinecone` — R4: open-source vector DBs (pgvector, Chroma) winning - `LLM Evaluations` — R4: concept rarely listed in postings - `LLM Observability` — R4: concept rarely listed in postings - `LLM Fine-tuning` — R4: rare in entry-level PK postings - `Object-Oriented Programming` — R4: interview prep - `Data Structures & Algorithms` — R4: interview prep ## Business Intelligence Analyst **Kept (10 skills):** - `Microsoft Power BI` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet+manual) - `DAX` (mandatory, weight=1.0, level=INTERMEDIATE, source=manual) - `SQL` (mandatory, weight=1.0, level=INTERMEDIATE, source=onet) - `Microsoft Excel` (mandatory, weight=1.0, level=ADVANCED, source=onet) - `Power Query / M Language` (mandatory, weight=0.9, level=BEGINNER, source=manual) - `Tableau` (optional, weight=0.7, level=INTERMEDIATE, source=onet+manual) - `Snowflake` (optional, weight=0.7, level=BEGINNER, source=manual) - `Python` (optional, weight=0.6, level=INTERMEDIATE, source=onet) - `Star Schema Modeling` (optional, weight=0.6, level=BEGINNER, source=manual) - `Data Storytelling` (optional, weight=0.5, level=BEGINNER, source=manual) **Dropped (10 skills):** - `AWS` — BI Analyst consumes from cloud DWH, doesn't own infra - `Microsoft Azure` — Same as AWS — BI Analyst doesn't own cloud infra - `Microsoft Office` — Too generic — Microsoft Excel covers it - `Microsoft PowerPoint` — Presentation, not BI engineering - `Oracle Cloud` — Enterprise niche - `R` — R4: PK BI market is Power BI+SQL+Excel; R rarely required - `SAP` — ERP — niche - `SAS` — Enterprise legacy - `Looker` — R4: rare in PK BI postings (Power BI/Tableau dominate) - `dbt` — R4: more for Data Eng; rarely in PK BI postings