Deploy CV parser dashboard with dataset 2 model

.gitignore

@@ -0,0 +1,16 @@
+# Python
+__pycache__/
+*.py[cod]
+.venv/
+venv/
+
+# Streamlit local secrets (NEVER commit — use the Space's secret settings)
+.streamlit/secrets.toml
+
+# Local model weights live on dev machines / the Hub, not in this repo
+exported_models/
+
+# Misc
+.DS_Store
+logs/
+*.log

Dockerfile

@@ -0,0 +1,17 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    STREAMLIT_SERVER_HEADLESS=true \
+    STREAMLIT_BROWSER_GATHER_USAGE_STATS=false
+
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY . .
+
+EXPOSE 7860
+
+CMD ["streamlit", "run", "app.py", "--server.address=0.0.0.0", "--server.port=7860"]

README.md

@@ -1,10 +1,72 @@
 ---
 title: Automated CV Parser
-emoji: 🏆
-colorFrom: purple
-colorTo: red
+emoji: 🧩
+colorFrom: indigo
+colorTo: blue
 sdk: docker
+app_port: 7860
 pinned: false
+short_description: Resume NER — extracts Job Titles, Skills & Education
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# CV Parser Dashboard
+
+A Streamlit app on top of the WQF7007 resume-NER model. Deployed as a Hugging
+Face Space; the model is loaded from the Hub so it can be updated without a
+redeploy.
+
+- **🔎 Live Parser** — paste/upload one CV and watch it get tokenized and
+  classified: sub-word token chips coloured by predicted label, the original text
+  with highlighted entities, and a structured summary.
+- **📊 Analytics** — batch-upload CVs (PDF / DOCX / TXT) for a skills word cloud
+  and top-entity charts across the set.
+- **🔐 Manage Model** — password-gated; teammates upload a new exported model and
+  it's pushed to the Hub repo the app reads from.
+
+## Models — how swapping works
+
+The app loads its model **from a Hugging Face Hub repo** (`PRIMARY_MODEL_ID` in
+`config.py`, default `Zeqhx/cv-parser-ner`). A Space's own disk is wiped on
+restart, so the Hub repo is the durable store. To update the live model:
+
+- **Easiest:** open **Manage Model**, enter the page password, upload a `.zip` of
+  your exported model folder (the `exported_models/…` folder the training notebooks
+  produce). It's validated against the 7-tag scheme and pushed to the Hub repo.
+- **Or:** push files straight to the Hub model repo (web UI / CLI).
+- Then click **🔄 Reload model** in the sidebar to pick up the new weights.
+
+The sidebar picker also has a **Custom HF model ID** box to load any repo live.
+
+## Secrets (Space → Settings → Variables and secrets)
+
+| Name | Purpose |
+|------|---------|
+| `HF_TOKEN` | A Hugging Face **write** token, so Manage Model can push to the Hub. |
+| `MANAGE_PASSWORD` | Password protecting the Manage Model page (page is disabled if unset). |
+| `DASHBOARD_MODEL_ID` | *(optional)* Override the model repo the app loads. |
+
+## Run locally
+
+```bash
+pip install -r requirements.txt
+streamlit run app.py
+```
+
+Local exported models in `../exported_models/…` are auto-offered in the picker
+when present (see `config.MODEL_REGISTRY`).
+
+## Layout
+
+```
+app.py                  # landing page + model-status banner
+config.py               # model registry/resolution, labels, colours, sample CV
+pages/
+  1_Live_Parser.py
+  2_Analytics.py
+  3_Manage_Model.py     # password-gated model uploader -> Hub
+lib/
+  model.py              # load (local/Hub/fallback) + sliding-window inference
+  extract.py            # PDF / DOCX / TXT -> text (adaptive spacing fix)
+  ui.py                 # shared sidebar model picker
+  viz.py                # entity HTML, token chips, word cloud, bar charts
+```

app.py

@@ -0,0 +1,65 @@
+"""CV Parser dashboard — entry page.
+
+Run from the project root:
+    streamlit run dashboard/app.py
+"""
+import streamlit as st
+
+import config
+from lib.ui import model_selector
+
+st.set_page_config(page_title="CV Parser Dashboard", page_icon="🧩", layout="wide")
+
+
+def model_status_banner(lm):
+    if lm.is_fallback:
+        st.warning(
+            f"**Demo mode — no fine-tuned model loaded.** Source: `{lm.source}`. "
+            "The classification head is untrained, so entity predictions are "
+            "**not meaningful** yet. Publish a model from the **Manage Model** page "
+            f"to `{config.PRIMARY_MODEL_ID}`, or pick one in the sidebar.",
+            icon="⚠️",
+        )
+    else:
+        st.success(f"Model loaded — {lm.source}", icon="✅")
+
+
+st.title("🧩 Automated CV Parser")
+st.caption("WQF7007 NLP · Resume NER · extracts Job Titles, Skills & Education")
+
+lm = model_selector()
+model_status_banner(lm)
+
+st.markdown(
+    """
+### What's inside
+
+- **🔎 Live Parser** — paste or upload a single CV and watch it get **tokenized and
+  classified** in real time: sub-word token chips coloured by predicted label, the
+  original text with highlighted entities, and a clean structured summary.
+- **📊 Analytics** — upload a batch of CVs (PDF / DOCX / TXT) and the page builds a
+  **skills word cloud** plus top Job Titles / Skills / Education charts across the set.
+
+Use the sidebar to switch pages.
+"""
+)
+
+with st.expander("ℹ️ How the model is resolved"):
+    st.markdown(
+        f"""
+        The sidebar **Model** picker selects which weights run. Options:
+
+        1. **⭐ Best model (Hub)** — `{config.PRIMARY_MODEL_ID}`. The team's current
+           best model; what the deployed app loads by default.
+        2. **Custom HF model ID** — type any Hub repo id to load it live.
+        3. **Local export** — `exported_models/…` folders (only on dev machines).
+        4. **Demo fallback** — `{config.FALLBACK_MODEL}` with a random head
+           (UI works, predictions don't).
+
+        Teammates update the live model from the **🔐 Manage Model** page (uploads
+        an exported model and pushes it to the Hub repo) — no redeploy needed.
+        After updating, click **🔄 Reload model** in the sidebar.
+
+        Label scheme: `{', '.join(config.LABELS)}`
+        """
+    )

config.py

@@ -0,0 +1,109 @@
+"""Central config for the CV Parser dashboard.
+
+A model "ref" is resolved by lib/model.py as:
+  - a local directory (e.g. exported_models/roberta-base-ner) -> load from disk
+  - any other string with a "/" -> a Hugging Face Hub repo id
+  - None -> FALLBACK_MODEL (roberta-base + random head; flagged as demo in the UI)
+
+On the deployed HF Space there are no local folders, so the app loads the team's
+"best model" repo (PRIMARY_MODEL_ID) from the Hub. Teammates update that repo via
+the password-gated Manage Model page (or by pushing to it directly), and the app
+picks up the new weights — no redeploy needed.
+"""
+import os
+
+# ---- Model resolution -------------------------------------------------------
+# Hugging Face owner + the canonical "best model" repo the app loads by default
+# and that the Manage Model page overwrites.
+HF_OWNER = os.environ.get("HF_OWNER", "Zeqhx")
+PRIMARY_MODEL_ID = os.environ.get("DASHBOARD_MODEL_ID", f"{HF_OWNER}/cv-parser-ner")
+
+# Back-compat single-ref overrides (used by load_model()'s config path).
+MODEL_PATH = os.environ.get("DASHBOARD_MODEL_PATH", "")
+MODEL_ID = PRIMARY_MODEL_ID
+FALLBACK_MODEL = "roberta-base"
+
+DEMO_LABEL = "Demo — untrained roberta-base"
+
+# Toggle registry. Each entry: (label, kind, ref). "local" entries are only
+# offered when the folder exists (dev machines); "hub" entries are always offered.
+MODEL_REGISTRY = [
+    (f"⭐ Best model (Hub: {PRIMARY_MODEL_ID})", "hub", PRIMARY_MODEL_ID),
+    ("RoBERTa — local export", "local", "exported_models/roberta-base-ner"),
+    ("BERT — local export", "local", "exported_models/bert-base-uncased-ner"),
+]
+
+
+def available_models():
+    """Ordered {label: ref} of selectable models, plus the demo fallback.
+
+    ``ref=None`` marks the demo/fallback. Local entries appear only when present
+    on disk; Hub entries always appear (a missing/private repo degrades to demo,
+    which the UI flags).
+    """
+    found = {}
+    local_found = {}
+    hub_found = {}
+    for label, kind, ref in MODEL_REGISTRY:
+        if kind == "local" and not os.path.isdir(ref):
+            continue
+        if kind == "local":
+            local_found[label] = ref
+        else:
+            hub_found[label] = ref
+    found.update(local_found)
+    found.update(hub_found)
+    found[DEMO_LABEL] = None
+    return found
+
+# ---- Inference --------------------------------------------------------------
+MAX_LENGTH = 512
+STRIDE = 128  # matches the project's sliding-window preprocessing
+
+# ---- Label scheme (must match training; see project README) -----------------
+LABELS = [
+    "O",
+    "B-JOB_TITLE", "I-JOB_TITLE",
+    "B-SKILL", "I-SKILL",
+    "B-EDUCATION", "I-EDUCATION",
+]
+ID2LABEL = {i: l for i, l in enumerate(LABELS)}
+LABEL2ID = {l: i for i, l in enumerate(LABELS)}
+
+# Entity types (BIO prefix stripped) + display colours
+ENTITY_TYPES = ["JOB_TITLE", "SKILL", "EDUCATION"]
+ENTITY_COLORS = {
+    "JOB_TITLE": "#ffb703",   # amber
+    "SKILL": "#2a9d8f",       # teal
+    "EDUCATION": "#4361ee",   # blue
+}
+ENTITY_LABELS = {
+    "JOB_TITLE": "Job Title",
+    "SKILL": "Skill",
+    "EDUCATION": "Education",
+}
+
+SUPPORTED_EXTS = [".pdf", ".docx", ".txt"]
+
+SAMPLE_RESUME = """John Carter
+Senior Software Engineer
+
+Summary
+Experienced Software Engineer and Team Lead with 8 years building scalable
+backend systems. Skilled in Python, Java, Kubernetes, and distributed systems.
+
+Experience
+Senior Software Engineer, Acme Corp (2019 - present)
+- Designed microservices using Python, FastAPI and PostgreSQL.
+- Led a team of 5 engineers and introduced CI/CD with Docker and Jenkins.
+
+Data Scientist, Globex (2016 - 2019)
+- Built machine learning models with TensorFlow and scikit-learn.
+
+Education
+Master of Science in Computer Science, Stanford University (2016)
+Bachelor of Engineering in Software Engineering, MIT (2014)
+
+Skills
+Python, Java, SQL, Machine Learning, Kubernetes, Docker, AWS, Leadership
+"""

lib/extract.py

@@ -0,0 +1,86 @@
+"""Extract plain text from uploaded CV files (PDF / DOCX / TXT).
+
+Extraction libs are imported lazily so the app still loads if one is missing;
+the caller gets a clear error string instead of a crash.
+"""
+from __future__ import annotations
+
+import os
+import re
+
+
+def _clean(text: str) -> str:
+    """Light normalisation mirroring the project's preprocessing."""
+    text = text.replace("\x00", " ")
+    text = re.sub(r"\(cid:\d+\)", " ", text)   # unmapped PDF glyphs (icons/ligatures)
+    text = re.sub(r"[ \t]+", " ", text)
+    text = re.sub(r"\n{3,}", "\n\n", text)
+    return "\n".join(line.strip() for line in text.splitlines()).strip()
+
+
+def _space_ratio(text: str) -> float:
+    """Fraction of characters that are spaces. Normal prose ~0.12-0.18;
+    PDFs with glued words ('UniversityofMalaya') drop near ~0.0."""
+    t = text.strip()
+    return (t.count(" ") / len(t)) if t else 0.0
+
+
+def _from_pdf(file) -> str:
+    import pdfplumber
+
+    def extract(pages, **kw):
+        return "\n".join((p.extract_text(**kw) or "") for p in pages)
+
+    with pdfplumber.open(file) as pdf:
+        pages = pdf.pages
+        text = extract(pages)
+        # Some PDFs encode inter-word spaces as gaps smaller than pdfplumber's
+        # default x_tolerance (3), so words come out glued together. Detect that
+        # via a very low space ratio and re-extract with a tighter tolerance,
+        # keeping it only if it genuinely adds spaces.
+        if _space_ratio(text) < 0.08:
+            tight = extract(pages, x_tolerance=1)
+            if _space_ratio(tight) > _space_ratio(text):
+                text = tight
+    return text
+
+
+def _from_docx(file) -> str:
+    import docx
+    document = docx.Document(file)
+    return "\n".join(p.text for p in document.paragraphs)
+
+
+def _from_txt(file) -> str:
+    raw = file.read()
+    if isinstance(raw, bytes):
+        return raw.decode("utf-8", errors="ignore")
+    return raw
+
+
+def extract_text(file, filename: str | None = None):
+    """Return (text, error). Exactly one is non-empty.
+
+    `file` is a file-like object (e.g. a Streamlit UploadedFile).
+    """
+    name = filename or getattr(file, "name", "") or ""
+    ext = os.path.splitext(name)[1].lower()
+    try:
+        if ext == ".pdf":
+            text = _from_pdf(file)
+        elif ext == ".docx":
+            text = _from_docx(file)
+        elif ext == ".txt":
+            text = _from_txt(file)
+        else:
+            return "", f"Unsupported file type: {ext or '(none)'}"
+    except ModuleNotFoundError as e:
+        return "", (f"Missing library for {ext} files ({e.name}). "
+                    f"Install dashboard/requirements.txt.")
+    except Exception as e:  # noqa: BLE001 - surface any parse error to the UI
+        return "", f"Could not read {name}: {e}"
+
+    text = _clean(text)
+    if not text:
+        return "", f"No extractable text in {name} (scanned/image PDF?)."
+    return text, ""

lib/model.py

@@ -0,0 +1,198 @@
+"""Model loading + sliding-window NER inference.
+
+Kept free of any Streamlit imports so it can be unit-tested / reused.
+The Streamlit pages wrap `load_model()` in `st.cache_resource`.
+"""
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass, field
+
+import torch
+from transformers import AutoTokenizer, AutoModelForTokenClassification
+
+import config
+
+
+@dataclass
+class LoadedModel:
+    tokenizer: object
+    model: object
+    id2label: dict
+    source: str            # human-readable description of where weights came from
+    is_fallback: bool      # True => random head, predictions are meaningless
+    device: str = field(default="cpu")
+
+
+def _ensure_label_scheme(model):
+    """If the loaded model lacks our entity labels, overwrite its id2label."""
+    cfg_labels = set(getattr(model.config, "id2label", {}).values())
+    if "B-SKILL" not in cfg_labels:
+        model.config.id2label = dict(config.ID2LABEL)
+        model.config.label2id = dict(config.LABEL2ID)
+    return {int(k): v for k, v in model.config.id2label.items()}
+
+
+_USE_CONFIG = "__use_config__"
+
+
+def _load_local(path: str, device: str) -> LoadedModel:
+    tok = AutoTokenizer.from_pretrained(path)
+    model = AutoModelForTokenClassification.from_pretrained(path)
+    id2label = _ensure_label_scheme(model)
+    model.to(device).eval()
+    return LoadedModel(tok, model, id2label,
+                       source=f"Local folder: {path}",
+                       is_fallback=False, device=device)
+
+
+def _load_hub(model_id: str, device: str) -> LoadedModel:
+    tok = AutoTokenizer.from_pretrained(model_id)
+    model = AutoModelForTokenClassification.from_pretrained(model_id)
+    id2label = _ensure_label_scheme(model)
+    model.to(device).eval()
+    return LoadedModel(tok, model, id2label,
+                       source=f"Hugging Face Hub: {model_id}",
+                       is_fallback=False, device=device)
+
+
+def _load_fallback(device: str) -> LoadedModel:
+    tok = AutoTokenizer.from_pretrained(config.FALLBACK_MODEL, add_prefix_space=True)
+    model = AutoModelForTokenClassification.from_pretrained(
+        config.FALLBACK_MODEL,
+        num_labels=len(config.LABELS),
+        id2label=dict(config.ID2LABEL),
+        label2id=dict(config.LABEL2ID),
+    )
+    model.to(device).eval()
+    return LoadedModel(tok, model, dict(config.ID2LABEL),
+                       source=f"Fallback base model: {config.FALLBACK_MODEL} (untrained head)",
+                       is_fallback=True, device=device)
+
+
+def load_model(ref: str | None = _USE_CONFIG) -> LoadedModel:
+    """Load a NER model from a ref.
+
+    - ``ref=_USE_CONFIG`` (default): resolve per config.py priority
+      (local MODEL_PATH -> MODEL_ID -> fallback). Keeps old callers working.
+    - ``ref`` is a local directory: load that exported folder.
+    - ``ref`` is any other non-empty string: treat as a Hugging Face Hub repo id.
+    - ``ref=None``: go straight to the demo fallback model.
+
+    A local folder that's missing, or a Hub repo that can't be loaded
+    (404 / private / offline), degrades gracefully to the demo fallback.
+    """
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+
+    if ref == _USE_CONFIG:
+        if config.MODEL_PATH and os.path.isdir(config.MODEL_PATH):
+            return _load_local(config.MODEL_PATH, device)
+        if config.MODEL_ID:
+            ref = config.MODEL_ID
+        else:
+            return _load_fallback(device)
+
+    if not ref:
+        return _load_fallback(device)
+    if os.path.isdir(ref):
+        return _load_local(ref, device)
+    try:
+        return _load_hub(ref, device)
+    except Exception:  # noqa: BLE001 - missing/private/offline repo -> demo
+        return _load_fallback(device)
+
+
+@torch.no_grad()
+def predict(text: str, lm: LoadedModel):
+    """Run sliding-window token classification over `text`.
+
+    Returns (tokens, entities):
+      tokens   = [{"text", "label", "type", "start", "end"}]  one per sub-word
+      entities = [{"text", "type", "start", "end"}]           merged BIO spans
+    """
+    text = text or ""
+    if not text.strip():
+        return [], []
+
+    enc = lm.tokenizer(
+        text,
+        max_length=config.MAX_LENGTH,
+        truncation=True,
+        stride=config.STRIDE,
+        return_overflowing_tokens=True,
+        return_offsets_mapping=True,
+        padding=True,
+        return_tensors="pt",
+    )
+    offsets = enc["offset_mapping"]
+    attn = enc["attention_mask"]
+    input_ids = enc["input_ids"].to(lm.device)
+    attn_dev = attn.to(lm.device)
+
+    logits = lm.model(input_ids=input_ids, attention_mask=attn_dev).logits
+    preds = logits.argmax(-1).cpu()
+
+    # Deduplicate overlapping sliding-window tokens by their global char offset.
+    seen: dict[int, tuple] = {}
+    n_windows, seq_len = preds.shape
+    for w in range(n_windows):
+        for i in range(seq_len):
+            s, e = offsets[w][i].tolist()
+            if (s == 0 and e == 0) or attn[w][i] == 0:
+                continue  # special token or padding
+            if s in seen:
+                continue
+            seen[s] = (s, e, int(preds[w][i]))
+
+    tokens = []
+    for s in sorted(seen):
+        _, e, pid = seen[s]
+        label = lm.id2label.get(pid, "O")
+        etype = label.split("-", 1)[1] if "-" in label else None
+        tokens.append({"text": text[s:e], "label": label, "type": etype,
+                       "start": s, "end": e})
+
+    entities = _merge_bio(tokens, text)
+    return tokens, entities
+
+
+def _merge_bio(tokens, text):
+    """Merge consecutive B-/I- tokens of the same type into entity spans."""
+    entities = []
+    cur = None
+    for t in tokens:
+        label = t["label"]
+        if "-" not in label:  # "O"
+            if cur:
+                entities.append(cur)
+                cur = None
+            continue
+        prefix, etype = label.split("-", 1)
+        if prefix == "B" or cur is None or cur["type"] != etype:
+            if cur:
+                entities.append(cur)
+            cur = {"type": etype, "start": t["start"], "end": t["end"]}
+        else:  # I- continuing the same type
+            cur["end"] = t["end"]
+    if cur:
+        entities.append(cur)
+
+    for e in entities:
+        e["text"] = text[e["start"]:e["end"]].strip()
+    return [e for e in entities if e["text"]]
+
+
+def group_entities(entities):
+    """Group merged entities by type, de-duplicating case-insensitively."""
+    grouped = {t: [] for t in config.ENTITY_TYPES}
+    seen = {t: set() for t in config.ENTITY_TYPES}
+    for e in entities:
+        t = e["type"]
+        if t not in grouped:
+            continue
+        key = e["text"].lower()
+        if key in seen[t]:
+            continue
+        seen[t].add(key)
+        grouped[t].append(e["text"])
+    return grouped

lib/ui.py

@@ -0,0 +1,60 @@
+"""Shared Streamlit UI helpers — the cross-page model toggle.
+
+Keeping this in one place means every page shows the same picker and shares the
+same selection (via st.session_state) and the same per-ref cache.
+"""
+import streamlit as st
+
+import config
+from lib.model import load_model
+
+_CUSTOM_LABEL = "✏️ Custom HF model ID…"
+
+
+@st.cache_resource(show_spinner="Loading NER model…")
+def _load_cached(ref):
+    """Cache one LoadedModel per distinct ref (None = demo fallback)."""
+    return load_model(ref=ref)
+
+
+def model_selector():
+    """Render the sidebar model picker and return the selected LoadedModel.
+
+    The choice persists across pages through st.session_state["model_label"].
+    A "Custom HF model ID" option lets anyone load any Hub repo live, and the
+    refresh button clears the cache to pick up a freshly-uploaded model.
+    """
+    options = config.available_models()           # {label: ref}
+    labels = list(options.keys()) + [_CUSTOM_LABEL]
+
+    st.sidebar.subheader("Model")
+    current = st.session_state.get("model_label", labels[0])
+    index = labels.index(current) if current in labels else 0
+    choice = st.sidebar.selectbox(
+        "Active NER model", labels, index=index, key="model_label",
+        label_visibility="collapsed",
+    )
+
+    if choice == _CUSTOM_LABEL:
+        ref = st.sidebar.text_input(
+            "HF model repo id", placeholder="e.g. Zeqhx/cv-parser-ner",
+            key="custom_model_id",
+        ).strip() or None
+    else:
+        ref = options[choice]
+
+    if st.sidebar.button("🔄 Reload model", use_container_width=True,
+                         help="Clear the cache and re-pull (use after updating a model)."):
+        _load_cached.clear()
+        st.rerun()
+
+    lm = _load_cached(ref)
+
+    if lm.is_fallback:
+        st.sidebar.warning("Demo mode — untrained head; predictions are not meaningful.",
+                           icon="⚠️")
+    else:
+        st.sidebar.success("Model loaded", icon="✅")
+    st.sidebar.caption(lm.source)
+
+    return lm

lib/viz.py

@@ -0,0 +1,102 @@
+"""Rendering helpers: entity-highlighted text, token chips, word cloud, charts."""
+from __future__ import annotations
+
+import html
+
+import config
+
+
+def _legend() -> str:
+    items = []
+    for t in config.ENTITY_TYPES:
+        items.append(
+            f'<span style="background:{config.ENTITY_COLORS[t]};color:#fff;'
+            f'padding:2px 8px;border-radius:4px;margin-right:8px;font-size:0.8rem">'
+            f'{config.ENTITY_LABELS[t]}</span>'
+        )
+    return '<div style="margin-bottom:10px">' + "".join(items) + "</div>"
+
+
+def render_entities_html(text: str, entities: list[dict]) -> str:
+    """Original text with entity spans wrapped in coloured marks."""
+    ents = sorted((e for e in entities if e["type"] in config.ENTITY_COLORS),
+                  key=lambda e: e["start"])
+    out, cursor = [], 0
+    for e in ents:
+        if e["start"] < cursor:  # skip any overlap defensively
+            continue
+        out.append(html.escape(text[cursor:e["start"]]))
+        color = config.ENTITY_COLORS[e["type"]]
+        label = config.ENTITY_LABELS[e["type"]]
+        out.append(
+            f'<mark style="background:{color};color:#fff;padding:1px 4px;'
+            f'border-radius:4px" title="{label}">'
+            f'{html.escape(text[e["start"]:e["end"]])}'
+            f'<sub style="font-size:0.6em;opacity:.85"> {label}</sub></mark>'
+        )
+        cursor = e["end"]
+    out.append(html.escape(text[cursor:]))
+    body = "".join(out).replace("\n", "<br>")
+    return (_legend() +
+            f'<div style="line-height:2.1;font-family:system-ui;font-size:0.95rem;'
+            f'border:1px solid #ddd;border-radius:8px;padding:16px;'
+            f'max-height:520px;overflow:auto">{body}</div>')
+
+
+def render_tokens_html(tokens: list[dict], limit: int = 400) -> str:
+    """Sub-word token chips, coloured by predicted label — the 'tokenization view'."""
+    chips = []
+    for t in tokens[:limit]:
+        txt = html.escape(t["text"]) or "·"
+        if t["type"] in config.ENTITY_COLORS:
+            color = config.ENTITY_COLORS[t["type"]]
+            style = f"background:{color};color:#fff"
+        else:
+            style = "background:#eee;color:#555"
+        chips.append(
+            f'<span style="{style};padding:2px 6px;border-radius:4px;margin:2px;'
+            f'display:inline-block;font-family:monospace;font-size:0.8rem">{txt}</span>'
+        )
+    more = "" if len(tokens) <= limit else f'<div style="color:#888;margin-top:8px">… +{len(tokens)-limit} more tokens</div>'
+    return (_legend() +
+            f'<div style="border:1px solid #ddd;border-radius:8px;padding:12px;'
+            f'max-height:420px;overflow:auto">{"".join(chips)}{more}</div>')
+
+
+def wordcloud_figure(freq: dict, title: str = ""):
+    """Return a matplotlib Figure for a frequency dict, or None if unavailable."""
+    if not freq:
+        return None
+    try:
+        from wordcloud import WordCloud
+        import matplotlib.pyplot as plt
+    except ModuleNotFoundError:
+        return None
+    wc = WordCloud(width=900, height=400, background_color="white",
+                   colormap="viridis", prefer_horizontal=0.9)
+    wc.generate_from_frequencies(freq)
+    fig, ax = plt.subplots(figsize=(9, 4))
+    ax.imshow(wc, interpolation="bilinear")
+    ax.axis("off")
+    if title:
+        ax.set_title(title)
+    fig.tight_layout()
+    return fig
+
+
+def top_bar_figure(counter, title: str, color: str, top_n: int = 15):
+    """Horizontal bar chart of the most common items. Returns a Plotly fig or None."""
+    if not counter:
+        return None
+    try:
+        import plotly.graph_objects as go
+    except ModuleNotFoundError:
+        return None
+    items = counter.most_common(top_n)[::-1]
+    labels = [k for k, _ in items]
+    values = [v for _, v in items]
+    fig = go.Figure(go.Bar(x=values, y=labels, orientation="h",
+                           marker_color=color))
+    fig.update_layout(title=title, height=max(300, 28 * len(items) + 80),
+                      margin=dict(l=10, r=10, t=40, b=10))
+    return fig

pages/1_Live_Parser.py

@@ -0,0 +1,80 @@
+"""Live Parser page — tokenize + classify a single CV, visually."""
+import streamlit as st
+
+import config
+from lib.model import predict, group_entities
+from lib.extract import extract_text
+from lib.ui import model_selector
+from lib import viz
+
+st.set_page_config(page_title="Live Parser", page_icon="🔎", layout="wide")
+
+lm = model_selector()
+
+st.title("🔎 Live CV Parser")
+if lm.is_fallback:
+    st.warning("Demo mode: predictions come from an untrained head and are not meaningful.",
+               icon="⚠️")
+
+# ---- Input ------------------------------------------------------------------
+src = st.radio("Input", ["Paste text", "Upload file", "Use sample CV"],
+               horizontal=True, label_visibility="collapsed")
+
+text = ""
+if src == "Paste text":
+    text = st.text_area("Paste CV text", height=260, placeholder="Paste resume text here…")
+elif src == "Upload file":
+    up = st.file_uploader("Upload a CV", type=["pdf", "docx", "txt"])
+    if up is not None:
+        text, err = extract_text(up)
+        if err:
+            st.error(err)
+else:
+    text = config.SAMPLE_RESUME
+    st.info("Using a built-in sample CV.")
+
+run = st.button("Parse CV", type="primary", disabled=not text.strip())
+
+# ---- Output -----------------------------------------------------------------
+if run and text.strip():
+    with st.spinner("Tokenizing and classifying…"):
+        tokens, entities = predict(text, lm)
+
+    grouped = group_entities(entities)
+    c1, c2, c3, c4 = st.columns(4)
+    c1.metric("Sub-word tokens", len(tokens))
+    c2.metric("Job Titles", len(grouped["JOB_TITLE"]))
+    c3.metric("Skills", len(grouped["SKILL"]))
+    c4.metric("Education", len(grouped["EDUCATION"]))
+
+    tab_ent, tab_tok, tab_card = st.tabs(
+        ["🏷️ Highlighted entities", "🔢 Tokenization view", "🗂️ Structured summary"])
+
+    with tab_ent:
+        st.markdown(viz.render_entities_html(text, entities), unsafe_allow_html=True)
+
+    with tab_tok:
+        st.caption("Each chip is one sub-word token produced by the tokenizer, "
+                   "coloured by its predicted label.")
+        st.markdown(viz.render_tokens_html(tokens), unsafe_allow_html=True)
+
+    with tab_card:
+        cols = st.columns(3)
+        for col, etype in zip(cols, config.ENTITY_TYPES):
+            with col:
+                st.subheader(config.ENTITY_LABELS[etype])
+                vals = grouped[etype]
+                if vals:
+                    for v in vals:
+                        st.markdown(f"- {v}")
+                else:
+                    st.caption("— none —")
+
+        import json
+        st.download_button(
+            "⬇️ Download as JSON",
+            data=json.dumps({config.ENTITY_LABELS[t]: grouped[t]
+                             for t in config.ENTITY_TYPES}, indent=2),
+            file_name="parsed_cv.json",
+            mime="application/json",
+        )

pages/2_Analytics.py

@@ -0,0 +1,112 @@
+"""Analytics page — batch-upload CVs and aggregate extracted entities."""
+from collections import Counter
+
+import pandas as pd
+import streamlit as st
+
+import config
+from lib.model import predict, group_entities
+from lib.extract import extract_text
+from lib.ui import model_selector
+from lib import viz
+
+st.set_page_config(page_title="Analytics", page_icon="📊", layout="wide")
+
+lm = model_selector()
+
+st.title("📊 CV Corpus Analytics")
+st.caption("Upload a batch of CVs — select every file in a folder — to see aggregate "
+           "skills and roles across the set.")
+if lm.is_fallback:
+    st.warning("Demo mode: predictions come from an untrained head and are not meaningful.",
+               icon="⚠️")
+
+uploads = st.file_uploader(
+    "Upload CVs (PDF / DOCX / TXT). Tip: open a folder and Ctrl/Cmd-A to select all.",
+    type=["pdf", "docx", "txt"], accept_multiple_files=True,
+)
+
+if not uploads:
+    st.info("Upload one or more CVs to build the analytics.")
+    st.stop()
+
+if st.button(f"Analyze {len(uploads)} file(s)", type="primary"):
+    counters = {t: Counter() for t in config.ENTITY_TYPES}
+    rows = []
+    failures = []
+    progress = st.progress(0.0, text="Processing…")
+
+    for i, up in enumerate(uploads, start=1):
+        text, err = extract_text(up)
+        if err:
+            failures.append((up.name, err))
+        else:
+            _, entities = predict(text, lm)
+            grouped = group_entities(entities)  # de-duped per CV
+            for etype in config.ENTITY_TYPES:
+                for val in grouped[etype]:
+                    counters[etype][val] += 1
+            rows.append({
+                "file": up.name,
+                "job_titles": len(grouped["JOB_TITLE"]),
+                "skills": len(grouped["SKILL"]),
+                "education": len(grouped["EDUCATION"]),
+            })
+        progress.progress(i / len(uploads), text=f"Processed {i}/{len(uploads)}")
+    progress.empty()
+
+    st.session_state["analytics"] = {"counters": counters, "rows": rows,
+                                     "failures": failures, "n": len(uploads)}
+
+# ---- Render (persists across reruns) ----------------------------------------
+data = st.session_state.get("analytics")
+if data:
+    counters, rows, failures = data["counters"], data["rows"], data["failures"]
+
+    a, b, c, d = st.columns(4)
+    a.metric("CVs processed", len(rows))
+    b.metric("Unique job titles", len(counters["JOB_TITLE"]))
+    c.metric("Unique skills", len(counters["SKILL"]))
+    d.metric("Unique education", len(counters["EDUCATION"]))
+    if failures:
+        with st.expander(f"⚠️ {len(failures)} file(s) could not be read"):
+            for name, err in failures:
+                st.write(f"- **{name}** — {err}")
+
+    st.subheader("☁️ Skills word cloud")
+    fig = viz.wordcloud_figure(dict(counters["SKILL"]))
+    if fig is not None:
+        st.pyplot(fig)
+    elif counters["SKILL"]:
+        st.info("Install `wordcloud` + `matplotlib` to see the cloud. Showing top skills below instead.")
+    else:
+        st.caption("No skills extracted.")
+
+    st.subheader("🏆 Most common entities")
+    cols = st.columns(3)
+    for col, etype in zip(cols, config.ENTITY_TYPES):
+        with col:
+            bar = viz.top_bar_figure(counters[etype], config.ENTITY_LABELS[etype],
+                                     config.ENTITY_COLORS[etype])
+            if bar is not None:
+                st.plotly_chart(bar, use_container_width=True)
+            else:
+                top = counters[etype].most_common(15)
+                st.write(f"**{config.ENTITY_LABELS[etype]}**")
+                st.table(pd.DataFrame(top, columns=["entity", "count"]) if top
+                         else pd.DataFrame(columns=["entity", "count"]))
+
+    st.subheader("📋 Per-file breakdown")
+    df = pd.DataFrame(rows)
+    st.dataframe(df, use_container_width=True)
+
+    # Full long-form export of every (file-agnostic) entity count
+    export = pd.concat([
+        pd.DataFrame({"type": config.ENTITY_LABELS[t],
+                      "entity": list(counters[t].keys()),
+                      "count": list(counters[t].values())})
+        for t in config.ENTITY_TYPES
+    ], ignore_index=True)
+    st.download_button("⬇️ Download entity counts (CSV)",
+                       data=export.to_csv(index=False),
+                       file_name="cv_entity_counts.csv", mime="text/csv")

pages/3_Manage_Model.py

@@ -0,0 +1,135 @@
+"""Manage Model — password-gated page for teammates to update the live model.
+
+Upload an exported model (a .zip of the folder produced by the training
+notebooks, i.e. config.json + model.safetensors + tokenizer files). It is
+validated against the project's 7-label scheme and pushed to the Hugging Face
+Hub repo the app loads from (config.PRIMARY_MODEL_ID). The Hub repo is the
+durable store — a Space's own disk is wiped on restart, so we never rely on it.
+
+Secrets (set in the Space settings, never in git):
+  HF_TOKEN         a Hugging Face *write* token
+  MANAGE_PASSWORD  password protecting this page
+"""
+import io
+import json
+import os
+import tempfile
+import zipfile
+from pathlib import Path
+
+import streamlit as st
+
+import config
+
+st.set_page_config(page_title="Manage Model", page_icon="🔐", layout="centered")
+st.title("🔐 Manage Model")
+st.caption("Update the live NER model. Restricted to the team.")
+
+
+def _secret(name, default=""):
+    # st.secrets raises if no secrets file exists; fall back to env.
+    try:
+        if name in st.secrets:
+            return st.secrets[name]
+    except Exception:  # noqa: BLE001
+        pass
+    return os.environ.get(name, default)
+
+
+# ---- Password gate ----------------------------------------------------------
+PASSWORD = _secret("MANAGE_PASSWORD")
+if not PASSWORD:
+    st.error("This page is disabled: no `MANAGE_PASSWORD` secret is configured.")
+    st.stop()
+
+if not st.session_state.get("manage_authed"):
+    pw = st.text_input("Password", type="password")
+    if st.button("Unlock"):
+        if pw == PASSWORD:
+            st.session_state["manage_authed"] = True
+            st.rerun()
+        else:
+            st.error("Wrong password.")
+    st.stop()
+
+# ---- Authed --------------------------------------------------------------
+TARGET_REPO = st.text_input("Target Hub model repo", value=config.PRIMARY_MODEL_ID,
+                            help="The repo the app loads. Overwriting it updates the live model.")
+private = st.checkbox("Keep repo private", value=True)
+
+st.markdown(
+    "Upload a **.zip of your exported model folder** "
+    "(`config.json`, `model.safetensors`, tokenizer files, ideally `label_config.json`). "
+    "This is the folder the training notebooks write to `exported_models/…`."
+)
+up = st.file_uploader("Model .zip", type=["zip"])
+
+REQUIRED = ["config.json"]
+WEIGHTS = ["model.safetensors", "pytorch_model.bin"]
+
+
+def _find_model_dir(root: Path):
+    """Locate the directory holding config.json (zip may have a wrapper folder)."""
+    for cfg in root.rglob("config.json"):
+        return cfg.parent
+    return None
+
+
+def _validate(model_dir: Path):
+    files = {p.name for p in model_dir.iterdir()}
+    if not any(w in files for w in WEIGHTS):
+        return f"No weights file found ({' or '.join(WEIGHTS)})."
+    # Label-scheme check: prefer label_config.json, else config.json id2label.
+    labels = None
+    if (model_dir / "label_config.json").exists():
+        labels = set(json.loads((model_dir / "label_config.json").read_text())
+                     .get("id2label", {}).values())
+    else:
+        cfg = json.loads((model_dir / "config.json").read_text())
+        labels = set(cfg.get("id2label", {}).values())
+    if "B-SKILL" not in labels:
+        return ("Label scheme mismatch — model does not use the project's BIO tags "
+                f"(expected B-SKILL/JOB_TITLE/EDUCATION, got: {sorted(labels) or 'none'}).")
+    return None
+
+
+if up is not None and st.button("Validate & publish", type="primary"):
+    token = _secret("HF_TOKEN")
+    if not token:
+        st.error("No `HF_TOKEN` secret configured — cannot push to the Hub.")
+        st.stop()
+    try:
+        from huggingface_hub import HfApi, create_repo
+    except ModuleNotFoundError:
+        st.error("`huggingface_hub` is not installed (add it to requirements.txt).")
+        st.stop()
+
+    with tempfile.TemporaryDirectory() as td:
+        tdp = Path(td)
+        try:
+            with zipfile.ZipFile(io.BytesIO(up.getvalue())) as zf:
+                zf.extractall(tdp)
+        except zipfile.BadZipFile:
+            st.error("That file is not a valid .zip.")
+            st.stop()
+
+        model_dir = _find_model_dir(tdp)
+        if model_dir is None:
+            st.error("No `config.json` found anywhere in the zip.")
+            st.stop()
+
+        err = _validate(model_dir)
+        if err:
+            st.error(err)
+            st.stop()
+
+        st.success(f"Validated: {sorted(p.name for p in model_dir.iterdir())}")
+        with st.spinner(f"Publishing to {TARGET_REPO}…"):
+            create_repo(TARGET_REPO, repo_type="model", private=private,
+                        exist_ok=True, token=token)
+            HfApi(token=token).upload_folder(
+                folder_path=str(model_dir), repo_id=TARGET_REPO, repo_type="model",
+                commit_message="Update model via Manage Model page",
+            )
+        st.success(f"✅ Published to https://huggingface.co/{TARGET_REPO}")
+        st.info("Go to any page and click **🔄 Reload model** in the sidebar to use it.")

requirements.txt

@@ -0,0 +1,10 @@
+streamlit>=1.32
+transformers>=4.38
+torch>=2.0
+huggingface_hub>=0.23
+pdfplumber>=0.10
+python-docx>=1.1
+wordcloud>=1.9
+matplotlib>=3.7
+plotly>=5.18
+pandas>=2.0