Deploy research dashboard

.dockerignore

@@ -0,0 +1,8 @@
+node_modules
+frontend/node_modules
+frontend/src
+.git
+**/__pycache__
+**/*.pyc
+.env
+backend/data

.gitattributes

@@ -1,35 +0,0 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,6 @@
+node_modules/
+__pycache__/
+*.pyc
+.DS_Store
+.raca/
+frontend/node_modules/

.hfignore

@@ -0,0 +1,8 @@
+node_modules/
+__pycache__/
+.venv/
+*.pyc
+.DS_Store
+.raca/
+.git/
+dist/

CLAUDE.md

@@ -0,0 +1,91 @@
+# Experiment Dashboard Visualizer
+
+Research dashboard with experiment tracking and trace visualization.
+
+- **Live**: https://huggingface.co/spaces/{HF_ORG}/dashboard
+- **GitHub**: `{github-org}/dashboard-repo`
+- **Remotes**: `origin` (GitHub), `space` (HF Space)
+
+## Updating the Website
+
+When the user says "update the website", "sync the dashboard", or "push to the website":
+
+1. **Build the frontend**:
+   ```bash
+   cd tools/visualizer/frontend && npm run build
+   ```
+
+2. **Import experiment data** (reads local files, uploads to HF dataset):
+   ```bash
+   cd tools/visualizer && python3 scripts/import_experiments.py
+   ```
+
+3. **Sync the live Space** (tells the running app to re-download data from HF):
+   ```bash
+   curl -s -X POST https://{HF_ORG}-dashboard.hf.space/api/experiments/sync
+   ```
+
+4. **Push code to HF Space** (deploys new frontend/backend code):
+   ```bash
+   cd tools/visualizer
+   git add -A && git commit -m "update dashboard"
+   git push origin main
+   git push space main
+   ```
+
+Steps 1-3 sync **data** (experiments, summary findings, presets). Step 4 deploys **code** changes.
+
+If only `summary_findings.md` changed (no code changes), steps 2-3 are sufficient.
+
+## Summary Findings
+
+- Source file: `{WORKSPACE}/notes/experiments/summary_findings.md`
+- Only the user writes this file — never edit it
+- Synced to HF via `import_experiments.py` → served at `GET /api/experiments/summary`
+- Shown on the Experiments page via the "Findings / Summary" button
+
+## Experiment Discovery
+
+`scripts/import_experiments.py` auto-discovers all experiments in `notes/experiments/`, skipping `old/`, `_templates/`, and hidden directories. To hide specific experiments, add them to `EXCLUDED_EXPERIMENTS` in the script.
+
+## Stack
+
+- **Backend**: Flask (Python), blueprints in `backend/api/`
+- **Frontend**: React + TypeScript + Tailwind, built with Vite into `frontend/dist/`
+- **Data**: HF datasets (`RACA_DASHBOARD` for experiments, `RACA-VIS-PRESETS` for visualizer presets)
+
+## AdaEvolve Traces
+
+### Required HuggingFace Dataset Columns
+
+| Column | Type | Purpose |
+|--------|------|---------|
+| iteration | int | Iteration number |
+| island_id | int | Which island produced this solution |
+| score | float | Current iteration score |
+| best_score | float | Best score so far |
+| delta | float | Change from previous iteration |
+| adaptation_type | string | "L1_explore", "L1_exploit", "L2_migrate", "L3_meta" |
+| exploration_intensity | float | How exploratory this iteration was |
+| is_valid | bool | Whether solution is valid |
+| task_id | string | Task identifier |
+| prompt_text | string | (optional) Input prompt to model |
+| reasoning_trace | string | (optional) Thinking/reasoning output |
+| program_code | string | (optional) Generated/evolved code |
+
+### Color Mapping
+
+L1_explore=blue, L1_exploit=green, L2_migrate=amber, L3_meta=red, other=gray.
+
+### Adding a Preset
+
+1. Upload dataset to HuggingFace (org: `your-org`)
+2. Add entry to `your-org/RACA-VIS-PRESETS` file `adaevolve_presets.json`:
+   ```json
+   {"id": "8-char-hex", "name": "Descriptive Name", "repo": "org/dataset-name", "split": "train"}
+   ```
+3. Sync: `curl -X POST "https://your-org-agg-trace-visualizer.hf.space/api/presets/sync"`
+
+### Naming Convention
+
+`{Task} {Model}: {Description} ({N} iter)`

Dockerfile

@@ -0,0 +1,25 @@
+FROM python:3.11-slim
+
+# Install nginx
+RUN apt-get update && apt-get install -y --no-install-recommends nginx && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+# Install Python deps (cached unless requirements.txt changes)
+COPY backend/requirements.txt ./backend/
+RUN pip install --no-cache-dir -r backend/requirements.txt
+
+# Copy backend + pre-built frontend (built locally by install.sh, no Node needed)
+COPY backend/ ./backend/
+COPY frontend/dist/ ./frontend/dist/
+
+# Writable dirs for runtime
+RUN mkdir -p /app/backend/data /app/backend/presets && \
+    chmod -R 777 /app/backend/data /app/backend/presets
+
+COPY nginx.conf /etc/nginx/nginx.conf
+COPY start.sh ./
+RUN chmod +x start.sh
+
+EXPOSE 7860
+CMD ["./start.sh"]

README.md

@@ -1,10 +1,36 @@
 ---
-title: Dashboard
-emoji: 👁
-colorFrom: indigo
-colorTo: red
+title: Research Dashboard
+emoji: 📊
+colorFrom: blue
+colorTo: purple
 sdk: docker
 pinned: false
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Research Dashboard
+
+A unified research control pane with two main sections:
+
+## Experiments
+
+Track research experiments, hypotheses, runs, and artifacts:
+
+- **Experiment tracking** - Create/manage experiments with hypothesis statements, status, and completeness scoring
+- **Run history** - Record runs with conditions, models, clusters, metrics, and HF dataset links
+- **Sub-experiments** - Drill into focused sub-studies with markdown reports
+- **HF dataset catalog** - Link and browse all HuggingFace datasets per experiment
+
+Data stored in `your-org/RACA_DASHBOARD`. Supports programmatic import via `/api/experiments/import`.
+
+## Visualizer
+
+Six trace visualization tools:
+
+- **Model Trace** - Analyze reasoning traces from model responses (think tags, backtracks, restarts)
+- **Arena** - Explore multi-agent game episodes and transcripts
+- **RLM** - Navigate hierarchical RLM call traces (GEPA iterations, RLM calls)
+- **RLM Eval** - RLM evaluation trace viewer
+- **Harbor** - View SWE-bench agent trajectories (ATIF + raw message formats)
+- **AdaEvolve** - Explore AdaEvolve optimization traces
+
+Each visualizer loads datasets from HuggingFace and supports preset configurations stored in `your-org/RACA-VIS-PRESETS`.

backend/api/experiments.py

@@ -0,0 +1,730 @@
+import json
+import os
+import uuid
+import tempfile
+import threading
+from datetime import datetime, timezone
+from flask import Blueprint, request, jsonify
+
+bp = Blueprint("experiments", __name__, url_prefix="/api/experiments")
+
+def _resolve_hf_org() -> str:
+    """Resolve HF org from env > .raca/config.yaml > fallback."""
+    org = os.environ.get("HF_ORG")
+    if org and org != "your-org":
+        return org
+    # Walk up from this file looking for .raca/config.yaml
+    from pathlib import Path
+    current = Path(__file__).resolve().parent
+    for _ in range(10):
+        current = current.parent
+        config = current / ".raca" / "config.yaml"
+        if config.exists():
+            try:
+                import yaml
+                with open(config) as f:
+                    cfg = yaml.safe_load(f) or {}
+                org = cfg.get("hf_org", "")
+                if org:
+                    return org
+            except Exception:
+                pass
+    return "your-org"
+
+HF_ORG = _resolve_hf_org()
+DASHBOARD_REPO = f"{HF_ORG}/RACA_DASHBOARD"
+LOCAL_DATA_DIR = os.path.join(os.path.dirname(os.path.dirname(__file__)), "data")
+
+_cache: dict[str, list[dict]] = {}
+_cache_loaded: set[str] = set()
+_dashboard_cache: dict[str, dict] = {}
+_dashboard_cache_loaded: bool = False
+_lock = threading.Lock()
+
+FILES = ["experiments", "runs", "sub_experiments", "experiment_notes", "summary_findings", "activity_logs", "artifacts"]
+
+
+def _ensure_local_dir():
+    os.makedirs(LOCAL_DATA_DIR, exist_ok=True)
+
+
+def _local_path(name: str) -> str:
+    _ensure_local_dir()
+    return os.path.join(LOCAL_DATA_DIR, f"{name}.json")
+
+
+def _download_file(name: str) -> list[dict]:
+    try:
+        from huggingface_hub import hf_hub_download
+        path = hf_hub_download(
+            DASHBOARD_REPO,
+            f"{name}.json",
+            repo_type="dataset",
+        )
+        with open(path) as f:
+            data = json.load(f)
+        with open(_local_path(name), "w") as f:
+            json.dump(data, f, indent=2)
+        return data
+    except Exception:
+        local = _local_path(name)
+        if os.path.exists(local):
+            with open(local) as f:
+                return json.load(f)
+        return []
+
+
+def _upload_file(name: str, data: list[dict]):
+    with open(_local_path(name), "w") as f:
+        json.dump(data, f, indent=2)
+
+    def _do_upload():
+        try:
+            from huggingface_hub import HfApi
+            api = HfApi()
+            try:
+                api.create_repo(DASHBOARD_REPO, repo_type="dataset", exist_ok=True)
+            except Exception:
+                pass
+            with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as f:
+                json.dump(data, f, indent=2)
+                tmp = f.name
+            api.upload_file(
+                path_or_fileobj=tmp,
+                path_in_repo=f"{name}.json",
+                repo_id=DASHBOARD_REPO,
+                repo_type="dataset",
+            )
+            os.unlink(tmp)
+        except Exception as e:
+            print(f"[experiments] HF upload failed for {name}: {e}")
+
+    threading.Thread(target=_do_upload, daemon=True).start()
+
+
+def _get(name: str):
+    """Get cached data, downloading from HF if needed.
+    Returns list[dict] for most files, but dict for activity_logs (keyed by experiment_id).
+    """
+    # Check cache without holding lock during download
+    with _lock:
+        if name in _cache_loaded:
+            data = _cache.get(name, [])
+            return dict(data) if isinstance(data, dict) else list(data)
+        need_download = True
+
+    if need_download:
+        # Download outside the lock so one slow download doesn't block all requests
+        downloaded = _download_file(name)
+        with _lock:
+            if name not in _cache_loaded:  # Double-check after reacquiring
+                _cache[name] = downloaded
+                _cache_loaded.add(name)
+            data = _cache.get(name, [])
+            return dict(data) if isinstance(data, dict) else list(data)
+
+
+def _set(name: str, data: list[dict]):
+    with _lock:
+        _cache[name] = data
+        _cache_loaded.add(name)
+    _upload_file(name, data)
+
+
+def _now() -> str:
+    return datetime.now(timezone.utc).isoformat()
+
+
+def _load_dashboard_state() -> dict:
+    """Load dashboard_state.json (a single dict, not an array) from HF or local fallback."""
+    global _dashboard_cache_loaded
+    with _lock:
+        if _dashboard_cache_loaded:
+            return dict(_dashboard_cache)
+    try:
+        from huggingface_hub import hf_hub_download
+        path = hf_hub_download(
+            DASHBOARD_REPO,
+            "dashboard_state.json",
+            repo_type="dataset",
+        )
+        with open(path) as f:
+            data = json.load(f)
+        # Cache locally
+        local = os.path.join(LOCAL_DATA_DIR, "dashboard_state.json")
+        _ensure_local_dir()
+        with open(local, "w") as f:
+            json.dump(data, f, indent=2)
+        with _lock:
+            _dashboard_cache.clear()
+            _dashboard_cache.update(data if isinstance(data, dict) else {})
+            _dashboard_cache_loaded = True
+        return dict(_dashboard_cache)
+    except Exception:
+        local = os.path.join(LOCAL_DATA_DIR, "dashboard_state.json")
+        if os.path.exists(local):
+            try:
+                with open(local) as f:
+                    data = json.load(f)
+                with _lock:
+                    _dashboard_cache.clear()
+                    _dashboard_cache.update(data if isinstance(data, dict) else {})
+                    _dashboard_cache_loaded = True
+                return dict(_dashboard_cache)
+            except Exception:
+                pass
+        with _lock:
+            _dashboard_cache_loaded = True
+        return {}
+
+
+def _merge_dashboard_state(experiments: list[dict]) -> list[dict]:
+    """Enrich experiment list with live dashboard state fields."""
+    state = _load_dashboard_state()
+    if not state:
+        return experiments
+
+    # Build lookup: dashboard state keyed by experiment id and name
+    exp_states: dict[str, dict] = {}
+    for exp_id, exp_state in state.items():
+        if isinstance(exp_state, dict):
+            exp_states[exp_id] = exp_state
+            # Also index by name if present
+            name = exp_state.get("name", "")
+            if name:
+                exp_states[name] = exp_state
+
+    # Enrich existing experiments
+    seen_ids = set()
+    result = []
+    for exp in experiments:
+        seen_ids.add(exp["id"])
+        ds = exp_states.get(exp["id"]) or exp_states.get(exp.get("name", ""))
+        if ds:
+            exp = {
+                **exp,
+                "live_status": ds.get("status"),
+                "live_message": ds.get("message", ""),
+                "live_jobs": ds.get("jobs", {}),
+                "unreachable_clusters": ds.get("unreachable_clusters", {}),
+                "live_history": ds.get("history", []),
+                "live_started_at": ds.get("started_at"),
+                "live_updated_at": ds.get("updated_at"),
+            }
+        result.append(exp)
+
+    # Add experiments that exist ONLY in dashboard state (not in experiments.json)
+    for exp_id, ds in state.items():
+        if exp_id not in seen_ids and isinstance(ds, dict):
+            seen_ids.add(exp_id)
+            result.append({
+                "id": exp_id,
+                "name": ds.get("name", exp_id),
+                "research_project": ds.get("research_project", ""),
+                "hypothesis": {
+                    "statement": "",
+                    "type": "exploration",
+                    "status": "pending",
+                    "success_criteria": "",
+                },
+                "stage": "active",
+                "completeness": 0,
+                "models": [],
+                "tasks": [],
+                "tags": [],
+                "hf_repos": [],
+                "wandb_url": "",
+                "notes": "",
+                "created": ds.get("started_at", _now()),
+                "updated": ds.get("updated_at", _now()),
+                "run_count": 0,
+                "sub_count": 0,
+                "note_count": 0,
+                "live_status": ds.get("status"),
+                "live_message": ds.get("message", ""),
+                "live_jobs": ds.get("jobs", {}),
+                "unreachable_clusters": ds.get("unreachable_clusters", {}),
+                "live_history": ds.get("history", []),
+                "live_started_at": ds.get("started_at"),
+                "live_updated_at": ds.get("updated_at"),
+            })
+
+    return result
+
+
+# --- Experiments CRUD ---
+
+@bp.route("/", methods=["GET"])
+def list_experiments():
+    experiments = _get("experiments")
+    runs = _get("runs")
+    subs = _get("sub_experiments")
+    notes = _get("experiment_notes")
+
+    # Enrich with counts
+    result = []
+    for exp in experiments:
+        exp_runs = [r for r in runs if r.get("experiment_id") == exp["id"]]
+        exp_subs = [s for s in subs if s.get("experiment_id") == exp["id"]]
+        exp_notes = [n for n in notes if n.get("experiment_id") == exp["id"]]
+        result.append({
+            **exp,
+            "run_count": len(exp_runs),
+            "sub_count": len(exp_subs),
+            "note_count": len(exp_notes),
+        })
+
+    # Merge live dashboard state
+    result = _merge_dashboard_state(result)
+
+    return jsonify(result)
+
+
+@bp.route("/", methods=["POST"])
+def create_experiment():
+    data = request.get_json()
+    name = data.get("name", "").strip()
+    if not name:
+        return jsonify({"error": "name is required"}), 400
+
+    exp_id = data.get("id", name.lower().replace(" ", "_"))
+
+    experiments = _get("experiments")
+    if any(e["id"] == exp_id for e in experiments):
+        return jsonify({"error": f"Experiment '{exp_id}' already exists"}), 409
+
+    experiment = {
+        "id": exp_id,
+        "name": name,
+        "research_project": data.get("research_project", ""),
+        "hypothesis": data.get("hypothesis", {
+            "statement": "",
+            "type": "exploration",
+            "status": "pending",
+            "success_criteria": "",
+        }),
+        "stage": data.get("stage", "idea"),
+        "completeness": data.get("completeness", 0),
+        "models": data.get("models", []),
+        "tasks": data.get("tasks", []),
+        "tags": data.get("tags", []),
+        "hf_repos": data.get("hf_repos", []),
+        "wandb_url": data.get("wandb_url", ""),
+        "notes": data.get("notes", ""),
+        "created": _now(),
+        "updated": _now(),
+    }
+
+    experiments.append(experiment)
+    _set("experiments", experiments)
+    return jsonify(experiment), 201
+
+
+@bp.route("/<exp_id>", methods=["GET"])
+def get_experiment(exp_id):
+    experiments = _get("experiments")
+    exp = next((e for e in experiments if e["id"] == exp_id), None)
+    if not exp:
+        return jsonify({"error": "not found"}), 404
+
+    runs = [r for r in _get("runs") if r.get("experiment_id") == exp_id]
+    subs = [s for s in _get("sub_experiments") if s.get("experiment_id") == exp_id]
+    notes = [n for n in _get("experiment_notes") if n.get("experiment_id") == exp_id]
+
+    # Activity log
+    all_logs = _get("activity_logs")
+    if isinstance(all_logs, dict):
+        activity_log = all_logs.get(exp_id, [])
+    elif isinstance(all_logs, list) and len(all_logs) == 1 and isinstance(all_logs[0], dict):
+        activity_log = all_logs[0].get(exp_id, [])
+    else:
+        activity_log = []
+
+    # Artifacts from manifest
+    all_artifacts = _get("artifacts")
+    artifacts = [a for a in all_artifacts if a.get("experiment_id") == exp_id]
+
+    return jsonify({
+        **exp,
+        "runs": runs,
+        "sub_experiments": subs,
+        "experiment_notes": notes,
+        "activity_log": activity_log,
+        "artifacts": artifacts,
+    })
+
+
+@bp.route("/<exp_id>", methods=["PUT"])
+def update_experiment(exp_id):
+    data = request.get_json()
+    experiments = _get("experiments")
+
+    for exp in experiments:
+        if exp["id"] == exp_id:
+            for key in ["name", "research_project", "hypothesis", "stage",
+                        "completeness", "models", "tasks", "tags", "hf_repos",
+                        "wandb_url", "notes"]:
+                if key in data:
+                    exp[key] = data[key]
+            exp["updated"] = _now()
+            _set("experiments", experiments)
+            return jsonify(exp)
+
+    return jsonify({"error": "not found"}), 404
+
+
+@bp.route("/<exp_id>", methods=["DELETE"])
+def delete_experiment(exp_id):
+    experiments = _get("experiments")
+    experiments = [e for e in experiments if e["id"] != exp_id]
+    _set("experiments", experiments)
+
+    # Also delete associated runs, subs, and notes
+    runs = [r for r in _get("runs") if r.get("experiment_id") != exp_id]
+    _set("runs", runs)
+    subs = [s for s in _get("sub_experiments") if s.get("experiment_id") != exp_id]
+    _set("sub_experiments", subs)
+    notes = [n for n in _get("experiment_notes") if n.get("experiment_id") != exp_id]
+    _set("experiment_notes", notes)
+
+    return jsonify({"status": "ok"})
+
+
+# --- Run records ---
+
+@bp.route("/<exp_id>/runs", methods=["POST"])
+def create_run(exp_id):
+    experiments = _get("experiments")
+    if not any(e["id"] == exp_id for e in experiments):
+        return jsonify({"error": "experiment not found"}), 404
+
+    data = request.get_json()
+    run = {
+        "id": data.get("id", f"run_{uuid.uuid4().hex[:8]}"),
+        "experiment_id": exp_id,
+        "condition": data.get("condition", ""),
+        "model": data.get("model", ""),
+        "cluster": data.get("cluster", ""),
+        "status": data.get("status", "completed"),
+        "hf_dataset": data.get("hf_dataset", ""),
+        "metrics": data.get("metrics", {}),
+        "timestamp": data.get("timestamp", _now()),
+        "notes": data.get("notes", ""),
+    }
+
+    runs = _get("runs")
+    runs.append(run)
+    _set("runs", runs)
+
+    # Touch experiment updated timestamp
+    for exp in experiments:
+        if exp["id"] == exp_id:
+            exp["updated"] = _now()
+    _set("experiments", experiments)
+
+    return jsonify(run), 201
+
+
+@bp.route("/<exp_id>/runs/<run_id>", methods=["PUT"])
+def update_run(exp_id, run_id):
+    data = request.get_json()
+    runs = _get("runs")
+
+    for run in runs:
+        if run["id"] == run_id and run["experiment_id"] == exp_id:
+            for key in ["condition", "model", "cluster", "status",
+                        "hf_dataset", "metrics", "notes"]:
+                if key in data:
+                    run[key] = data[key]
+            _set("runs", runs)
+            return jsonify(run)
+
+    return jsonify({"error": "not found"}), 404
+
+
+@bp.route("/<exp_id>/runs/<run_id>", methods=["DELETE"])
+def delete_run(exp_id, run_id):
+    runs = _get("runs")
+    runs = [r for r in runs if not (r["id"] == run_id and r["experiment_id"] == exp_id)]
+    _set("runs", runs)
+    return jsonify({"status": "ok"})
+
+
+# --- Sub-experiments ---
+
+@bp.route("/<exp_id>/subs", methods=["POST"])
+def create_sub(exp_id):
+    experiments = _get("experiments")
+    if not any(e["id"] == exp_id for e in experiments):
+        return jsonify({"error": "experiment not found"}), 404
+
+    data = request.get_json()
+    name = data.get("name", "").strip()
+    if not name:
+        return jsonify({"error": "name is required"}), 400
+
+    sub_id = data.get("id", f"{exp_id}__{name.lower().replace(' ', '_')}")
+
+    sub = {
+        "id": sub_id,
+        "experiment_id": exp_id,
+        "name": name,
+        "hypothesis": data.get("hypothesis", ""),
+        "status": data.get("status", "active"),
+        "content_md": data.get("content_md", ""),
+        "hf_repos": data.get("hf_repos", []),
+        "created": _now(),
+        "updated": _now(),
+    }
+
+    subs = _get("sub_experiments")
+    subs.append(sub)
+    _set("sub_experiments", subs)
+
+    # Touch experiment updated timestamp
+    for exp in experiments:
+        if exp["id"] == exp_id:
+            exp["updated"] = _now()
+    _set("experiments", experiments)
+
+    return jsonify(sub), 201
+
+
+@bp.route("/<exp_id>/subs/<sub_id>", methods=["PUT"])
+def update_sub(exp_id, sub_id):
+    data = request.get_json()
+    subs = _get("sub_experiments")
+
+    for sub in subs:
+        if sub["id"] == sub_id and sub["experiment_id"] == exp_id:
+            for key in ["name", "hypothesis", "status", "content_md", "hf_repos"]:
+                if key in data:
+                    sub[key] = data[key]
+            sub["updated"] = _now()
+            _set("sub_experiments", subs)
+            return jsonify(sub)
+
+    return jsonify({"error": "not found"}), 404
+
+
+@bp.route("/<exp_id>/subs/<sub_id>", methods=["DELETE"])
+def delete_sub(exp_id, sub_id):
+    subs = _get("sub_experiments")
+    subs = [s for s in subs if not (s["id"] == sub_id and s["experiment_id"] == exp_id)]
+    _set("sub_experiments", subs)
+    return jsonify({"status": "ok"})
+
+
+# --- Experiment Notes ---
+
+@bp.route("/<exp_id>/notes", methods=["POST"])
+def create_note(exp_id):
+    experiments = _get("experiments")
+    if not any(e["id"] == exp_id for e in experiments):
+        return jsonify({"error": "experiment not found"}), 404
+
+    data = request.get_json()
+    title = data.get("title", "").strip()
+    if not title:
+        return jsonify({"error": "title is required"}), 400
+
+    note_id = data.get("id", f"{exp_id}__note_{uuid.uuid4().hex[:8]}")
+
+    note = {
+        "id": note_id,
+        "experiment_id": exp_id,
+        "title": title,
+        "filename": data.get("filename", ""),
+        "content_md": data.get("content_md", ""),
+        "created": _now(),
+        "updated": _now(),
+    }
+
+    notes = _get("experiment_notes")
+    notes.append(note)
+    _set("experiment_notes", notes)
+    return jsonify(note), 201
+
+
+@bp.route("/<exp_id>/notes/<note_id>", methods=["GET"])
+def get_note(exp_id, note_id):
+    notes = _get("experiment_notes")
+    note = next((n for n in notes if n["id"] == note_id and n["experiment_id"] == exp_id), None)
+    if not note:
+        return jsonify({"error": "not found"}), 404
+    return jsonify(note)
+
+
+@bp.route("/<exp_id>/notes/<note_id>", methods=["PUT"])
+def update_note(exp_id, note_id):
+    data = request.get_json()
+    notes = _get("experiment_notes")
+
+    for note in notes:
+        if note["id"] == note_id and note["experiment_id"] == exp_id:
+            for key in ["title", "content_md"]:
+                if key in data:
+                    note[key] = data[key]
+            note["updated"] = _now()
+            _set("experiment_notes", notes)
+            return jsonify(note)
+
+    return jsonify({"error": "not found"}), 404
+
+
+@bp.route("/<exp_id>/notes/<note_id>", methods=["DELETE"])
+def delete_note(exp_id, note_id):
+    notes = _get("experiment_notes")
+    notes = [n for n in notes if not (n["id"] == note_id and n["experiment_id"] == exp_id)]
+    _set("experiment_notes", notes)
+    return jsonify({"status": "ok"})
+
+
+# --- Activity Log ---
+
+@bp.route("/<exp_id>/activity-log", methods=["GET"])
+def get_activity_log(exp_id):
+    """Get activity log entries for an experiment."""
+    all_logs = _get("activity_logs")
+    # activity_logs is a dict keyed by experiment_id (or empty list on first load)
+    if isinstance(all_logs, dict):
+        entries = all_logs.get(exp_id, [])
+    elif isinstance(all_logs, list) and len(all_logs) == 1 and isinstance(all_logs[0], dict):
+        # HF download may wrap dict in a list
+        entries = all_logs[0].get(exp_id, [])
+    else:
+        entries = []
+
+    # Sort by timestamp descending (most recent first)
+    entries.sort(key=lambda e: e.get("timestamp", ""), reverse=True)
+
+    # Optional filters
+    scope = request.args.get("scope")
+    entry_type = request.args.get("type")
+    if scope:
+        entries = [e for e in entries if e.get("scope") == scope]
+    if entry_type:
+        entries = [e for e in entries if e.get("type") == entry_type]
+
+    return jsonify(entries)
+
+
+# --- Artifacts ---
+
+@bp.route("/<exp_id>/artifacts", methods=["GET"])
+def get_artifacts(exp_id):
+    """Get artifact entries for an experiment from manifest data."""
+    all_artifacts = _get("artifacts")
+    artifacts = [a for a in all_artifacts if a.get("experiment_id") == exp_id]
+    return jsonify(artifacts)
+
+
+# --- Summary Findings ---
+
+@bp.route("/summary", methods=["GET"])
+def get_summary():
+    data = _get("summary_findings")
+    if data and len(data) > 0:
+        return jsonify(data[0])
+    return jsonify({"content_md": "", "updated": ""})
+
+
+# --- Sync & Import ---
+
+@bp.route("/sync", methods=["POST"])
+def sync():
+    global _dashboard_cache_loaded
+    with _lock:
+        _cache.clear()
+        _cache_loaded.clear()
+        _dashboard_cache.clear()
+        _dashboard_cache_loaded = False
+    for name in FILES:
+        _get(name)
+    return jsonify({"status": "ok"})
+
+
+@bp.route("/import", methods=["POST"])
+def import_experiments():
+    """Bulk import from experiment.yaml format (as produced by exp-runner)."""
+    data = request.get_json()
+    items = data if isinstance(data, list) else [data]
+    imported = []
+
+    experiments = _get("experiments")
+    runs = _get("runs")
+    subs = _get("sub_experiments")
+    existing_ids = {e["id"] for e in experiments}
+
+    for item in items:
+        exp_id = item.get("name", "").lower().replace(" ", "_").replace("-", "_")
+        if not exp_id:
+            continue
+
+        hypothesis = item.get("hypothesis", {})
+        models = item.get("models", [])
+        model_names = [m.get("id", "") if isinstance(m, dict) else str(m) for m in models]
+
+        if exp_id not in existing_ids:
+            experiment = {
+                "id": exp_id,
+                "name": item.get("name", exp_id),
+                "research_project": item.get("research_project", ""),
+                "hypothesis": {
+                    "statement": hypothesis.get("statement", "") if isinstance(hypothesis, dict) else str(hypothesis),
+                    "type": hypothesis.get("type", "exploration") if isinstance(hypothesis, dict) else "exploration",
+                    "status": hypothesis.get("status", "pending") if isinstance(hypothesis, dict) else "pending",
+                    "success_criteria": hypothesis.get("success_criteria", "") if isinstance(hypothesis, dict) else "",
+                },
+                "stage": "active",
+                "completeness": 0,
+                "models": model_names,
+                "tasks": [],
+                "tags": item.get("observability", {}).get("tags", []) if isinstance(item.get("observability"), dict) else [],
+                "hf_repos": [],
+                "wandb_url": "",
+                "notes": "",
+                "created": item.get("created", _now()),
+                "updated": _now(),
+            }
+            experiments.append(experiment)
+            existing_ids.add(exp_id)
+
+        # Import runs
+        for run_data in item.get("runs", []):
+            run_id = run_data.get("run_id", f"run_{uuid.uuid4().hex[:8]}")
+            if any(r["id"] == run_id and r["experiment_id"] == exp_id for r in runs):
+                continue
+            run = {
+                "id": run_id,
+                "experiment_id": exp_id,
+                "condition": run_data.get("condition", ""),
+                "model": run_data.get("model", ""),
+                "cluster": run_data.get("cluster", ""),
+                "status": run_data.get("status", "completed"),
+                "hf_dataset": run_data.get("hf_dataset", ""),
+                "metrics": run_data.get("metrics", {}),
+                "timestamp": run_data.get("timestamp", _now()),
+                "notes": run_data.get("notes", ""),
+            }
+            runs.append(run)
+
+            # Add HF repo to experiment if present
+            if run.get("hf_dataset"):
+                for exp in experiments:
+                    if exp["id"] == exp_id:
+                        existing_repos = {r["repo"] for r in exp.get("hf_repos", [])}
+                        if run["hf_dataset"] not in existing_repos:
+                            exp.setdefault("hf_repos", []).append({
+                                "repo": run["hf_dataset"],
+                                "description": f"{run['condition']} - {run['model']}",
+                                "date": run["timestamp"][:10] if run["timestamp"] else "",
+                            })
+
+        imported.append(exp_id)
+
+    _set("experiments", experiments)
+    _set("runs", runs)
+    _set("sub_experiments", subs)
+
+    return jsonify({"imported": imported, "count": len(imported)})

backend/api/manifest.py

@@ -0,0 +1,104 @@
+"""Shared manifest query API — loads RACA-PROJECT-MANIFEST from HuggingFace.
+
+Provides both a utility function (for use by other blueprints) and a
+shared /api/manifest endpoint so any frontend tab can browse runs.
+"""
+
+import json
+import logging
+import os
+
+from datasets import load_dataset
+from flask import Blueprint, jsonify, request
+from huggingface_hub import HfApi
+
+ORG_NAME = os.environ.get("HF_ORG", "your-org")
+MANIFEST_REPO = f"{ORG_NAME}/RACA-PROJECT-MANIFEST"
+
+log = logging.getLogger(__name__)
+
+bp = Blueprint("manifest", __name__, url_prefix="/api/manifest")
+
+
+def get_manifest():
+    """Load RACA-PROJECT-MANIFEST dataset. Returns list of dicts, or None on failure."""
+    try:
+        ds = load_dataset(MANIFEST_REPO, split="train")
+        return [row for row in ds]
+    except Exception as e:
+        error_str = str(e).lower()
+        if any(phrase in error_str for phrase in [
+            "doesn't exist", "404", "no data", "corresponds to no data",
+        ]):
+            return None
+        raise
+
+
+def _get_live_repos() -> set[str]:
+    """Return set of dataset repo IDs that actually exist in the org."""
+    try:
+        api = HfApi()
+        return {
+            ds.id for ds in api.list_datasets(author=ORG_NAME)
+        }
+    except Exception as e:
+        log.warning("Failed to list org datasets for liveness check: %s", e)
+        return set()
+
+
+def query_runs(prefix: str, validate: bool = True):
+    """Query manifest for datasets matching a name prefix. Returns list of run dicts.
+
+    Args:
+        prefix: filter by dataset_name prefix (empty string = all).
+        validate: if True, filter out datasets that no longer exist on HF.
+    """
+    manifest = get_manifest()
+    if manifest is None:
+        return []
+
+    live_repos = _get_live_repos() if validate else set()
+
+    runs = []
+    for row in manifest:
+        name = row.get("dataset_name", "")
+        if prefix and not name.startswith(prefix):
+            continue
+        repo = f"{ORG_NAME}/{name}"
+        if validate and live_repos and repo not in live_repos:
+            continue
+        tags = row.get("tags", [])
+        if isinstance(tags, str):
+            try:
+                tags = json.loads(tags)
+            except (json.JSONDecodeError, TypeError):
+                tags = []
+        metadata = row.get("custom_metadata", "{}")
+        if isinstance(metadata, str):
+            try:
+                metadata = json.loads(metadata)
+            except (json.JSONDecodeError, TypeError):
+                metadata = {}
+        runs.append({
+            "dataset_name": name,
+            "repo": repo,
+            "tags": tags,
+            "metadata": metadata,
+            "created_at": row.get("created", ""),
+        })
+    return runs
+
+
+@bp.route("/query", methods=["GET"])
+def query_endpoint():
+    """Generic manifest query — any tab can use this.
+
+    Query params:
+        prefix: filter datasets by name prefix (e.g. 'my-experiment-')
+    """
+    prefix = request.args.get("prefix", "")
+    try:
+        runs = query_runs(prefix)
+        return jsonify(runs)
+    except Exception as e:
+        return jsonify({"error": f"Failed to query manifest: {e}"}), 500

backend/api/model_datasets.py

@@ -0,0 +1,409 @@
+import json
+import os
+import hashlib
+from flask import Blueprint, request, jsonify
+from datasets import load_dataset, Dataset
+
+bp = Blueprint("model_datasets", __name__, url_prefix="/api/model/datasets")
+
+# In-memory cache: id -> {dataset, repo, column, split, n_rows, n_samples}
+_cache: dict[str, dict] = {}
+
+
+def _make_id(repo: str, column: str, split: str) -> str:
+    key = f"{repo}:{column}:{split}"
+    return hashlib.md5(key.encode()).hexdigest()[:12]
+
+
+def _load_hf_dataset(repo: str, split: str) -> Dataset:
+    if os.path.exists(repo):
+        return Dataset.from_parquet(repo)
+    return load_dataset(repo, split=split)
+
+
+def _detect_response_column(columns: list[str], preferred: str) -> str:
+    if preferred and preferred in columns:
+        return preferred
+    for fallback in ["model_responses", "model_response", "response", "responses", "output", "outputs", "completion", "messages"]:
+        if fallback in columns:
+            return fallback
+    # Last resort: return preferred if set, otherwise first column
+    return preferred if preferred else (columns[0] if columns else "")
+
+
+def _detect_prompt_column(columns: list[str], preferred: str) -> str | None:
+    if preferred and preferred in columns:
+        return preferred
+    for fallback in ["formatted_prompt", "prompt", "question", "input", "instruction"]:
+        if fallback in columns:
+            return fallback
+    return None
+
+
+def _compute_question_fingerprint(ds: Dataset, n: int = 5) -> str:
+    """Hash first N question texts to fingerprint the question set."""
+    questions = []
+    for i in range(min(n, len(ds))):
+        row = ds[i]
+        for qcol in ["question", "prompt", "input", "formatted_prompt"]:
+            if qcol in row:
+                questions.append(str(row[qcol] or "")[:200])
+                break
+    return hashlib.md5("||".join(questions).encode()).hexdigest()[:8]
+
+
+def _compute_chat_fingerprint(ds: Dataset, column: str, n: int = 5) -> str:
+    """Hash first N user messages from chat-format data to fingerprint the question set."""
+    prompts = []
+    for i in range(min(n, len(ds))):
+        row = ds[i]
+        messages = row[column]
+        if _is_chat_messages(messages):
+            _, user_prompt, _ = _extract_chat_messages(messages)
+            prompts.append(user_prompt[:200])
+        else:
+            prompts.append("")
+    return hashlib.md5("||".join(prompts).encode()).hexdigest()[:8]
+
+
+def _count_samples(ds: Dataset, column: str) -> int:
+    if len(ds) == 0:
+        return 0
+    first = ds[0][column]
+    if isinstance(first, list):
+        return len(first)
+    return 1
+
+
+def _flatten_evals(evals) -> list[bool]:
+    if not isinstance(evals, list):
+        return [bool(evals)]
+    return [
+        bool(e[-1]) if isinstance(e, list) and len(e) > 0
+        else (bool(e) if not isinstance(e, list) else False)
+        for e in evals
+    ]
+
+
+def _is_chat_messages(value) -> bool:
+    """Check if a column value is in chat format (list of role/content dicts)."""
+    if not isinstance(value, list) or len(value) == 0:
+        return False
+    first = value[0]
+    return isinstance(first, dict) and "role" in first and "content" in first
+
+
+def _extract_chat_messages(messages: list[dict]) -> tuple[str, str, str]:
+    """Extract (system, user_prompt, assistant_response) from chat messages."""
+    system = ""
+    user_prompt = ""
+    assistant_response = ""
+    for msg in messages:
+        role = msg.get("role", "")
+        content = msg.get("content", "") or ""
+        if role == "system":
+            system = content
+        elif role == "user":
+            user_prompt = content
+        elif role == "assistant":
+            assistant_response = content
+    return system, user_prompt, assistant_response
+
+
+def _extract_reasoning(meta: dict | None) -> str | None:
+    """Extract reasoning/thinking content from response metadata's raw_response."""
+    if not meta or not isinstance(meta, dict):
+        return None
+    raw = meta.get("raw_response")
+    if not raw or not isinstance(raw, dict):
+        return None
+    try:
+        msg = raw["choices"][0]["message"]
+        return (
+            msg.get("reasoning_content")
+            or msg.get("thinking")
+            or msg.get("reasoning")
+        )
+    except (KeyError, IndexError, TypeError):
+        return None
+
+
+def _merge_reasoning_into_response(response: str, reasoning: str | None) -> str:
+    """Prepend <think>{reasoning}</think> to response if reasoning exists
+    and isn't already present in the response."""
+    if not reasoning:
+        return response or ""
+    response = response or ""
+    # Don't double-add if response already contains the thinking
+    if "<think>" in response:
+        return response
+    return f"<think>{reasoning}</think>\n{response}"
+
+
+def _analyze_trace(text: str) -> dict:
+    if not text:
+        return dict(total_len=0, think_len=0, answer_len=0,
+                    backtracks=0, restarts=0, think_text="", answer_text="")
+    think_end = text.find("</think>")
+    if think_end > 0:
+        # Keep raw tags so display is 1:1 with HuggingFace data
+        think_text = text[:think_end + 8]  # include </think>
+        answer_text = text[think_end + 8:].strip()
+    else:
+        think_text = text
+        answer_text = ""
+    t = text.lower()
+    backtracks = sum(t.count(w) for w in
+                     ["wait,", "wait ", "hmm", "let me try", "try again",
+                      "another approach", "let me reconsider"])
+    restarts = sum(t.count(w) for w in
+                   ["start over", "fresh approach", "different approach", "from scratch"])
+    return dict(total_len=len(text), think_len=len(think_text),
+                answer_len=len(answer_text), backtracks=backtracks,
+                restarts=restarts, think_text=think_text, answer_text=answer_text)
+
+
+@bp.route("/load", methods=["POST"])
+def load_dataset_endpoint():
+    data = request.get_json()
+    repo = data.get("repo", "").strip()
+    if not repo:
+        return jsonify({"error": "repo is required"}), 400
+
+    split = data.get("split", "train")
+    preferred_column = data.get("column") or ""
+    preferred_prompt_column = data.get("prompt_column") or ""
+
+    try:
+        ds = _load_hf_dataset(repo, split)
+    except Exception as e:
+        return jsonify({"error": f"Failed to load dataset: {e}"}), 400
+
+    columns = ds.column_names
+    column = _detect_response_column(columns, preferred_column)
+    prompt_column = _detect_prompt_column(columns, preferred_prompt_column)
+
+    if column not in columns:
+        return jsonify({
+            "error": f"Column '{column}' not found. Available: {columns}"
+        }), 400
+
+    # Detect chat messages format (list of role/content dicts)
+    is_chat = False
+    if len(ds) > 0 and column in ds.column_names:
+        first_val = ds[0][column]
+        is_chat = _is_chat_messages(first_val)
+
+    n_samples = 1 if is_chat else _count_samples(ds, column)
+    ds_id = _make_id(repo, column, split)
+
+    if is_chat:
+        # For chat format, fingerprint based on the user message content
+        fingerprint = _compute_chat_fingerprint(ds, column)
+    else:
+        fingerprint = _compute_question_fingerprint(ds)
+
+    _cache[ds_id] = {
+        "dataset": ds,
+        "repo": repo,
+        "column": column,
+        "prompt_column": prompt_column,
+        "split": split,
+        "n_rows": len(ds),
+        "n_samples": n_samples,
+        "question_fingerprint": fingerprint,
+        "is_chat": is_chat,
+    }
+
+    short_name = repo.rsplit("/", 1)[-1] if "/" in repo else repo
+
+    return jsonify({
+        "id": ds_id,
+        "repo": repo,
+        "name": short_name,
+        "column": column,
+        "prompt_column": prompt_column,
+        "columns": columns,
+        "split": split,
+        "n_rows": len(ds),
+        "n_samples": n_samples,
+        "question_fingerprint": fingerprint,
+    })
+
+
+@bp.route("/", methods=["GET"])
+def list_datasets():
+    result = []
+    for ds_id, info in _cache.items():
+        result.append({
+            "id": ds_id,
+            "repo": info["repo"],
+            "name": info["repo"].rsplit("/", 1)[-1] if "/" in info["repo"] else info["repo"],
+            "column": info["column"],
+            "split": info["split"],
+            "n_rows": info["n_rows"],
+            "n_samples": info["n_samples"],
+            "question_fingerprint": info.get("question_fingerprint", ""),
+        })
+    return jsonify(result)
+
+
+@bp.route("/<ds_id>/question/<int:idx>", methods=["GET"])
+def get_question(ds_id, idx):
+    if ds_id not in _cache:
+        return jsonify({"error": "Dataset not loaded"}), 404
+
+    info = _cache[ds_id]
+    ds = info["dataset"]
+    column = info["column"]
+
+    if idx < 0 or idx >= len(ds):
+        return jsonify({"error": f"Index {idx} out of range (0-{len(ds)-1})"}), 400
+
+    row = ds[idx]
+    is_chat = info.get("is_chat", False)
+
+    if is_chat:
+        # Chat messages format: extract assistant response, user prompt, system
+        messages = row[column]
+        system_msg, user_prompt, assistant_response = _extract_chat_messages(messages)
+        responses_raw = [assistant_response]
+        prompt_text = user_prompt
+        question = user_prompt
+    else:
+        responses_raw = row[column]
+        if not isinstance(responses_raw, list):
+            responses_raw = [responses_raw]
+
+        # Check for {column}__metadata to recover reasoning/thinking content
+        meta_column = f"{column}__metadata"
+        response_metas = None
+        if meta_column in row:
+            response_metas = row[meta_column]
+            if not isinstance(response_metas, list):
+                response_metas = [response_metas]
+
+        # Merge reasoning from metadata into responses
+        merged_responses = []
+        for i, resp in enumerate(responses_raw):
+            meta = response_metas[i] if response_metas and i < len(response_metas) else None
+            reasoning = _extract_reasoning(meta)
+            merged_responses.append(_merge_reasoning_into_response(resp, reasoning))
+        responses_raw = merged_responses
+
+        # Prompt text from configured prompt column
+        prompt_text = ""
+        prompt_col = info.get("prompt_column")
+        if prompt_col and prompt_col in row:
+            val = row[prompt_col]
+            if isinstance(val, str):
+                prompt_text = val
+            elif isinstance(val, list):
+                prompt_text = json.dumps(val)
+            elif val is not None:
+                prompt_text = str(val)
+
+        question = ""
+        for qcol in ["question", "prompt", "input", "problem", "formatted_prompt"]:
+            if qcol in row:
+                val = row[qcol] or ""
+                if isinstance(val, str):
+                    question = val
+                elif isinstance(val, list):
+                    question = json.dumps(val)
+                else:
+                    question = str(val)
+                break
+
+    eval_correct = []
+    if "eval_correct" in row:
+        eval_correct = _flatten_evals(row["eval_correct"])
+    elif "correct" in row:
+        eval_correct = _flatten_evals(row["correct"])
+
+    # Check extractions with column-aware name
+    extractions = []
+    extractions_col = f"{column}__extractions"
+    for ecol in [extractions_col, "response__extractions"]:
+        if ecol in row:
+            ext = row[ecol]
+            if isinstance(ext, list):
+                extractions = [str(e) for e in ext]
+            break
+
+    metadata = {}
+    if "metadata" in row:
+        metadata = row["metadata"] if isinstance(row["metadata"], dict) else {}
+
+    analyses = [_analyze_trace(r or "") for r in responses_raw]
+
+    return jsonify({
+        "question": question,
+        "prompt_text": prompt_text,
+        "responses": [r or "" for r in responses_raw],
+        "eval_correct": eval_correct,
+        "extractions": extractions,
+        "metadata": metadata,
+        "analyses": analyses,
+        "n_samples": len(responses_raw),
+        "index": idx,
+    })
+
+
+@bp.route("/<ds_id>/summary", methods=["GET"])
+def get_summary(ds_id):
+    if ds_id not in _cache:
+        return jsonify({"error": "Dataset not loaded"}), 404
+
+    info = _cache[ds_id]
+    ds = info["dataset"]
+    n_rows = info["n_rows"]
+    n_samples = info["n_samples"]
+
+    # Support both "eval_correct" and "correct" column names
+    eval_col = None
+    if "eval_correct" in ds.column_names:
+        eval_col = "eval_correct"
+    elif "correct" in ds.column_names:
+        eval_col = "correct"
+
+    if eval_col is None:
+        return jsonify({
+            "n_rows": n_rows,
+            "n_samples": n_samples,
+            "has_eval": False,
+        })
+
+    pass_at = {}
+    for k in [1, 2, 4, 8]:
+        if k > n_samples:
+            break
+        correct = sum(1 for i in range(n_rows)
+                      if any(_flatten_evals(ds[i][eval_col])[:k]))
+        pass_at[k] = {"correct": correct, "total": n_rows,
+                       "rate": correct / n_rows if n_rows > 0 else 0}
+
+    total_samples = n_rows * n_samples
+    total_correct = sum(
+        sum(_flatten_evals(ds[i][eval_col]))
+        for i in range(n_rows)
+    )
+
+    return jsonify({
+        "n_rows": n_rows,
+        "n_samples": n_samples,
+        "has_eval": True,
+        "sample_accuracy": {
+            "correct": total_correct,
+            "total": total_samples,
+            "rate": total_correct / total_samples if total_samples > 0 else 0,
+        },
+        "pass_at": pass_at,
+    })
+
+
+@bp.route("/<ds_id>", methods=["DELETE"])
+def unload_dataset(ds_id):
+    if ds_id in _cache:
+        del _cache[ds_id]
+    return jsonify({"status": "ok"})

backend/api/plan_revisions.py

@@ -0,0 +1,60 @@
+import os
+from flask import Blueprint, jsonify
+from datasets import load_dataset
+
+bp = Blueprint("plan_revisions", __name__, url_prefix="/api/plan-revisions")
+
+HF_REPO = "timchen0618/bcp-plan-revisions-v1"
+
+_cache: dict | None = None
+
+
+def _load():
+    global _cache
+    if _cache is not None:
+        return _cache
+
+    ds = load_dataset(HF_REPO, split="train")
+
+    # Group: condition -> query_id -> sorted list of revision entries
+    grouped: dict[str, dict[str, list]] = {}
+    for row in ds:
+        cond = row["condition"]
+        qid = str(row["query_id"])
+        grouped.setdefault(cond, {}).setdefault(qid, [])
+        grouped[cond][qid].append({
+            "revision_index": row["revision_index"],
+            "source": row["source"],
+            "plan_text": row["plan_text"],
+            "total_revisions": row["total_revisions"],
+            "status": row["status"],
+        })
+
+    # Sort revisions within each group
+    for cond in grouped:
+        for qid in grouped[cond]:
+            grouped[cond][qid].sort(key=lambda r: r["revision_index"])
+
+    conditions = sorted(grouped.keys())
+    _cache = {"conditions": conditions, "data": grouped}
+    return _cache
+
+
+@bp.get("/")
+def get_data():
+    try:
+        result = _load()
+        return jsonify(result)
+    except Exception as e:
+        return jsonify({"error": str(e)}), 500
+
+
+@bp.post("/reload")
+def reload_data():
+    global _cache
+    _cache = None
+    try:
+        result = _load()
+        return jsonify({"status": "ok", "conditions": result["conditions"]})
+    except Exception as e:
+        return jsonify({"error": str(e)}), 500

backend/api/presets.py

@@ -0,0 +1,188 @@
+import json
+import os
+import uuid
+import tempfile
+import threading
+from flask import Blueprint, request, jsonify
+
+bp = Blueprint("presets", __name__, url_prefix="/api/presets")
+
+HF_ORG = os.environ.get("HF_ORG", "your-org")
+PRESETS_REPO = f"{HF_ORG}/RACA-VIS-PRESETS"
+VALID_TYPES = {"model"}
+LOCAL_PRESETS_DIR = os.path.join(os.path.dirname(os.path.dirname(__file__)), "presets")
+
+# In-memory cache: vis_type -> list[dict]
+_cache: dict[str, list[dict]] = {}
+_cache_loaded: set[str] = set()
+_lock = threading.Lock()
+
+
+def _ensure_local_dir():
+    os.makedirs(LOCAL_PRESETS_DIR, exist_ok=True)
+
+
+def _local_path(vis_type: str) -> str:
+    _ensure_local_dir()
+    return os.path.join(LOCAL_PRESETS_DIR, f"{vis_type}_presets.json")
+
+
+def _download_presets(vis_type: str) -> list[dict]:
+    """Download presets from HuggingFace, falling back to local file."""
+    try:
+        from huggingface_hub import hf_hub_download
+        path = hf_hub_download(
+            PRESETS_REPO,
+            f"{vis_type}_presets.json",
+            repo_type="dataset",
+        )
+        with open(path) as f:
+            presets = json.load(f)
+        # Cache locally for offline fallback
+        with open(_local_path(vis_type), "w") as f:
+            json.dump(presets, f, indent=2)
+        return presets
+    except Exception:
+        # Fall back to local cache
+        local = _local_path(vis_type)
+        if os.path.exists(local):
+            with open(local) as f:
+                return json.load(f)
+        return []
+
+
+def _upload_presets(vis_type: str, presets: list[dict]):
+    """Upload presets to HuggingFace (best-effort, non-blocking)."""
+    # Always save locally first
+    with open(_local_path(vis_type), "w") as f:
+        json.dump(presets, f, indent=2)
+
+    def _do_upload():
+        try:
+            from huggingface_hub import HfApi
+            api = HfApi()
+            # Ensure repo exists
+            try:
+                api.create_repo(
+                    PRESETS_REPO,
+                    repo_type="dataset",
+                    exist_ok=True,
+                )
+            except Exception:
+                pass
+            with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as f:
+                json.dump(presets, f, indent=2)
+                tmp = f.name
+            api.upload_file(
+                path_or_fileobj=tmp,
+                path_in_repo=f"{vis_type}_presets.json",
+                repo_id=PRESETS_REPO,
+                repo_type="dataset",
+            )
+            os.unlink(tmp)
+        except Exception as e:
+            print(f"[presets] HF upload failed for {vis_type}: {e}")
+
+    threading.Thread(target=_do_upload, daemon=True).start()
+
+
+def _get_presets(vis_type: str) -> list[dict]:
+    """Get presets for a visualizer type, downloading if needed."""
+    with _lock:
+        if vis_type not in _cache_loaded:
+            _cache[vis_type] = _download_presets(vis_type)
+            _cache_loaded.add(vis_type)
+        return list(_cache.get(vis_type, []))
+
+
+def _set_presets(vis_type: str, presets: list[dict]):
+    """Update presets in cache and sync to HF."""
+    with _lock:
+        _cache[vis_type] = presets
+        _cache_loaded.add(vis_type)
+    _upload_presets(vis_type, presets)
+
+
+@bp.route("/<vis_type>", methods=["GET"])
+def list_presets(vis_type):
+    if vis_type not in VALID_TYPES:
+        return jsonify({"error": f"Invalid type. Must be one of: {VALID_TYPES}"}), 400
+    return jsonify(_get_presets(vis_type))
+
+
+@bp.route("/<vis_type>", methods=["POST"])
+def create_preset(vis_type):
+    if vis_type not in VALID_TYPES:
+        return jsonify({"error": f"Invalid type. Must be one of: {VALID_TYPES}"}), 400
+
+    data = request.get_json()
+    name = data.get("name", "").strip()
+
+    if not name:
+        return jsonify({"error": "name is required"}), 400
+
+    preset = {
+        "id": uuid.uuid4().hex[:8],
+        "name": name,
+    }
+    # Include type-specific fields
+    repo = data.get("repo", "").strip()
+    if not repo:
+        return jsonify({"error": "repo is required"}), 400
+    preset["repo"] = repo
+    preset["split"] = data.get("split", "train")
+
+    if vis_type == "model":
+        preset["column"] = data.get("column", "model_responses")
+
+    presets = _get_presets(vis_type)
+    presets.append(preset)
+    _set_presets(vis_type, presets)
+
+    return jsonify(preset), 201
+
+
+@bp.route("/<vis_type>/<preset_id>", methods=["PUT"])
+def update_preset(vis_type, preset_id):
+    if vis_type not in VALID_TYPES:
+        return jsonify({"error": f"Invalid type. Must be one of: {VALID_TYPES}"}), 400
+
+    data = request.get_json()
+    presets = _get_presets(vis_type)
+
+    for p in presets:
+        if p["id"] == preset_id:
+            if "name" in data:
+                p["name"] = data["name"].strip()
+            if "column" in data:
+                p["column"] = data["column"]
+            if "split" in data:
+                p["split"] = data["split"]
+            if "config" in data:
+                p["config"] = data["config"]
+            _set_presets(vis_type, presets)
+            return jsonify(p)
+
+    return jsonify({"error": "not found"}), 404
+
+
+@bp.route("/<vis_type>/<preset_id>", methods=["DELETE"])
+def delete_preset(vis_type, preset_id):
+    if vis_type not in VALID_TYPES:
+        return jsonify({"error": f"Invalid type. Must be one of: {VALID_TYPES}"}), 400
+
+    presets = _get_presets(vis_type)
+    presets = [p for p in presets if p["id"] != preset_id]
+    _set_presets(vis_type, presets)
+    return jsonify({"status": "ok"})
+
+
+@bp.route("/sync", methods=["POST"])
+def sync_presets():
+    """Force re-download presets from HF."""
+    with _lock:
+        _cache.clear()
+        _cache_loaded.clear()
+    for vt in VALID_TYPES:
+        _get_presets(vt)
+    return jsonify({"status": "ok"})

backend/app.py

@@ -0,0 +1,36 @@
+from flask import Flask
+from flask_cors import CORS
+
+
+def create_app():
+    app = Flask(__name__, static_folder="../frontend/dist", static_url_path="/")
+    CORS(app)
+
+    from backend.api import model_datasets, presets, experiments, manifest, plan_revisions
+    app.register_blueprint(model_datasets.bp)
+    app.register_blueprint(presets.bp)
+    app.register_blueprint(experiments.bp)
+    app.register_blueprint(manifest.bp)
+    app.register_blueprint(plan_revisions.bp)
+
+    @app.route("/api/health")
+    def health():
+        return {"status": "ok"}
+
+    @app.route("/", defaults={"path": ""})
+    @app.route("/<path:path>")
+    def serve_frontend(path):
+        return app.send_static_file("index.html")
+
+    return app
+
+
+app = create_app()
+
+
+def main():
+    app.run(debug=True, port=8080)
+
+
+if __name__ == "__main__":
+    main()

backend/data/activity_logs.json

@@ -0,0 +1,53 @@
+{
+  "onboarding": [
+    {
+      "timestamp": "2026-04-06T00:00:00Z",
+      "type": "milestone",
+      "scope": "experiment",
+      "author": "agent",
+      "message": "Experiment design complete: Qwen3-1.7B on Countdown, 10-sample canary, torch cluster"
+    },
+    {
+      "timestamp": "2026-04-06T00:01:00Z",
+      "type": "milestone",
+      "scope": "experiment",
+      "author": "agent",
+      "message": "Red-team review passed. Key checks: max_tokens=4096 (marginal, monitor for truncation), prompt format verified against reference, evaluation must use CountdownJudge AST-based eval (not string match). Output schema defined with 8 columns including finish_reason."
+    },
+    {
+      "timestamp": "2026-04-06T00:05:00Z",
+      "type": "milestone",
+      "scope": "job",
+      "author": "agent",
+      "message": "Canary job 5550586 submitted to torch (h200_courant, 1x H200, 30min). Qwen3-1.7B on 10 Countdown problems. Results \u2192 timchen0618/onboarding-countdown-qwen3-1.7b"
+    },
+    {
+      "timestamp": "2026-04-06T14:22:00Z",
+      "type": "note",
+      "scope": "job",
+      "author": "agent",
+      "message": "Job 5550712 failed: OOM during vLLM CUDA graph compilation. Fixed: added enforce_eager=True to skip compilation. Resubmitted as job 5552202."
+    },
+    {
+      "timestamp": "2026-04-06T14:22:00Z",
+      "type": "milestone",
+      "scope": "job",
+      "author": "agent",
+      "message": "Job 5552202 submitted to torch (enforce_eager=True fix applied)"
+    },
+    {
+      "timestamp": "2026-04-06T14:36:00Z",
+      "type": "note",
+      "scope": "job",
+      "author": "agent",
+      "message": "Job 5552202 failed: OOM (CPU RAM) from vLLM spawn subprocess. Switched to transformers AutoModelForCausalLM \u2014 no subprocess needed. Resubmitted as job 5552573."
+    },
+    {
+      "timestamp": "2026-04-06T15:05:00Z",
+      "type": "result",
+      "scope": "artifact",
+      "author": "agent",
+      "message": "Job 5553179 completed: 10/10 samples generated. Evaluator bugs fixed (last <answer> block extraction + cascaded step validation). Re-scored: 10/10 correct (100%). Uploaded to timchen0618/onboarding-countdown-qwen3-1.7b."
+    }
+  ]
+}

backend/data/artifacts.json

@@ -0,0 +1 @@
+[]

backend/data/experiment_notes.json

@@ -0,0 +1,42 @@
+[
+  {
+    "id": "onboarding__note__Users_hungting_Desktop_Lab_Projects_notes_experiments_onboarding_EXPERIMENT_README_md",
+    "experiment_id": "onboarding",
+    "title": "EXPERIMENT_README.md",
+    "filename": "EXPERIMENT_README.md",
+    "relative_path": "/Users/hungting/Desktop/Lab/Projects/notes/experiments/onboarding/EXPERIMENT_README.md",
+    "content_md": "# Welcome to RACA\n\nThis is a sample experiment to show you how the dashboard works. You're looking at the **Overview** tab right now \u2014 it displays the experiment's README (this file).\n\nEverything you see here is generated from plain files in `notes/experiments/onboarding/`. You can browse them in your editor anytime.\n\n## How This Dashboard Works\n\nEach experiment has several tabs at the top. Here's what they do:\n\n### Overview (you are here)\n\nDisplays the experiment's README and any notes you've written in the `user/` folder. This is the main landing page for each experiment \u2014 a summary of what the experiment is, what you're investigating, and what you found.\n\n### Red Team Brief\n\nBefore any experiment runs, RACA reviews the design for potential problems \u2014 wrong evaluation metrics, truncated outputs, missing baselines, wasted compute. The brief lives at `red_team_brief.md`. This tab will be empty until you run your first real experiment.\n\n### Timeline\n\nA chronological log of everything that happened: when jobs were submitted, when artifacts were uploaded, when bugs were found and fixed. This is auto-generated from `activity_log.jsonl` \u2014 RACA writes to it as events happen.\n\n### Runs\n\nTracks each job submission \u2014 which model, which cluster, what status (pending, running, completed, failed), and links to the HuggingFace dataset with the results. Empty until you run something.\n\n### Artifacts\n\nLinks to all HuggingFace datasets produced by this experiment \u2014 canary runs, partial results, final data. Each artifact has metadata about what generated it. Empty until artifacts are uploaded.\n\n### Files\n\nAll the markdown and YAML files in the experiment folder. Click any file to read it. This is a quick way to browse the experiment's configuration and notes without leaving the dashboard.\n\n## Folder Structure\n\n```\nnotes/experiments/onboarding/\n  EXPERIMENT_README.md    \u2190 this file (shows in Overview tab)\n  experiment.yaml         \u2190 config: hypothesis, models, tasks\n  flow_state.json         \u2190 current phase (design/running/complete)\n  HUGGINGFACE_REPOS.md    \u2190 links to all uploaded datasets\n  questions.md            \u2190 research questions (read-only)\n  red_team_brief.md       \u2190 created during preflight review\n  activity_log.jsonl      \u2190 timeline entries (auto-generated)\n  user/                   \u2190 YOUR notes \u2014 RACA doesn't touch these\n    README.md             \u2190 your interpretation and observations\n    FINDINGS.md           \u2190 key results and surprises\n    DECISIONS.md          \u2190 design decisions and rationale\n    summary.md            \u2190 one-paragraph summary when done\n```\n\n**Most of this is automated.** RACA creates and updates the experiment files, uploads artifacts, and keeps the timeline current. The only files you write are in `user/` \u2014 that's your space for notes, findings, and decisions.\n\n## What's Next\n\nThis sample experiment hasn't been run yet \u2014 it's just here to show you the structure. When you're ready to run a real experiment, just tell RACA:\n\n> *I want to test whether Qwen3-8B follows complex instructions better than Llama-3.1-8B*\n\nOr try the full guided tutorial:\n\n> */raca:experiment-tutorial*\n",
+    "created": "",
+    "updated": ""
+  },
+  {
+    "id": "onboarding__note__Users_hungting_Desktop_Lab_Projects_notes_experiments_onboarding_HUGGINGFACE_REPOS_md",
+    "experiment_id": "onboarding",
+    "title": "HUGGINGFACE_REPOS.md",
+    "filename": "HUGGINGFACE_REPOS.md",
+    "relative_path": "/Users/hungting/Desktop/Lab/Projects/notes/experiments/onboarding/HUGGINGFACE_REPOS.md",
+    "content_md": "# HuggingFace Repositories\n\n| Dataset | Date | Rows | Purpose |\n|---------|------|------|---------|\n| [onboarding-countdown-qwen3-1.7b \u2014 10 rows, 10/10 correct (2026-04-06)](https://huggingface.co/datasets/timchen0618/onboarding-countdown-qwen3-1.7b) | 2026-04-06 | 10 | Qwen3-1.7B on Countdown (3-4 operands, targets 1-99, seed=42) |\n",
+    "created": "",
+    "updated": ""
+  },
+  {
+    "id": "onboarding__note__Users_hungting_Desktop_Lab_Projects_notes_experiments_onboarding_questions_md",
+    "experiment_id": "onboarding",
+    "title": "questions.md",
+    "filename": "questions.md",
+    "relative_path": "/Users/hungting/Desktop/Lab/Projects/notes/experiments/onboarding/questions.md",
+    "content_md": "# Research Questions\n\n1. Can Qwen3-1.7B solve basic Countdown problems (4 numbers, targets < 100)?\n2. What reasoning strategies does the model use (trial-and-error, systematic search, pattern matching)?\n3. Where does the model fail \u2014 wrong arithmetic, giving up, or invalid expressions?\n",
+    "created": "",
+    "updated": ""
+  },
+  {
+    "id": "onboarding__note__Users_hungting_Desktop_Lab_Projects_notes_experiments_onboarding_red_team_brief_md",
+    "experiment_id": "onboarding",
+    "title": "red_team_brief.md",
+    "filename": "red_team_brief.md",
+    "relative_path": "/Users/hungting/Desktop/Lab/Projects/notes/experiments/onboarding/red_team_brief.md",
+    "content_md": "# Red Team Brief \u2014 onboarding\n\n**Experiment:** Qwen3-1.7B on Countdown (tutorial/canary)\n**Reviewer:** agent\n**Date:** 2026-04-06\n**Status:** PASS\n\n---\n\n## Hypothesis\n\n> Qwen3-1.7B can solve basic Countdown arithmetic problems (>50% accuracy on 3-4 operand problems)\n\nThis is an exploratory baseline \u2014 a reasonable first question. No prior published result exists for Qwen3-1.7B specifically on this exact config, so the hypothesis is testable and non-trivial.\n\n---\n\n## Config Review\n\n### Model\n- **Qwen/Qwen3-1.7B** \u2014 small instruct model, appropriate for a tutorial canary\n- Known to handle arithmetic reasoning; 1.7B is at the edge of competence for Countdown (expect 20-50% accuracy)\n\n### max_tokens: 4096\n- **Status: PASS (marginal)**\n- Reference minimum is 2048-4096. We are at the ceiling of \"minimum\". For Qwen3-1.7B reasoning traces, 4096 should be sufficient for 3-4 operand problems.\n- **Watch for truncation in outputs** \u2014 if `finish_reason == \"length\"` appears in any response, flag immediately.\n\n### Prompt Format\n- **Must use the CoT + in-context examples prompt from the reference** (not the TinyZero `<think>` variant, since we are not RL-training)\n- Template:\n  ```\n  Answer the following problem. Explain your reasoning step by step. When you are finished, give your answer in this format: <answer>(your answer)</answer>.\n  \n  # Problem\n  Using the numbers in the list [{numbers}], create an equation that equals {target}. ...\n  ```\n- Answer extraction: regex on `<answer>...</answer>` tags\n\n### Evaluation Method\n- **Must use `CountdownJudge.validate_countdown_solution()` \u2014 NOT string matching**\n- Located at: `packages/custom_evaluations/custom_evaluations/sources/countdown/countdown_judge.py`\n- Handles: AST-based evaluation, Unicode operator normalization, multiset number validation\n- **Failure mode if string match is used:** \"3 + 5\" and \"5 + 3\" would score differently. This is wrong.\n\n### Dataset Generation\n- 10 samples (canary size \u2014 fine)\n- Use forward generation (inline, no dependencies): 3-4 operands, targets 1-99\n- Do NOT require all numbers used \u2014 subset is sufficient (this is the easier, more standard setting)\n\n---\n\n## Failure Modes & Mitigations\n\n| Risk | Severity | Mitigation |\n|------|----------|------------|\n| Truncated outputs (`finish_reason=length`) | HIGH | Check every row \u2014 flag if any truncation occurs |\n| String matching instead of equation eval | HIGH | Use `CountdownJudge` \u2014 verified AST-based |\n| Model outputs `\u00d7` instead of `*` | MEDIUM | `CountdownJudge` handles Unicode normalization |\n| `<answer>` tag missing in output | MEDIUM | Log separately as \"format failures\" vs \"wrong answer\" |\n| Duplicate problems in 10 samples | LOW | Use `random.seed(42)` for reproducibility |\n\n---\n\n## Expected Results\n\nBased on typical results from the reference file:\n- Qwen3-1.7B is smaller than Qwen2.5-3B (which achieves 70-80% after GRPO training)\n- For a **pre-GRPO instruct model** at this size, expect roughly **20-50% accuracy**\n- If accuracy is <10%, suspect prompt format issue or evaluation bug \u2014 investigate before scaling\n\n---\n\n## Output Schema\n\nExpected columns in the HuggingFace dataset:\n\n| Column | Type | Description |\n|--------|------|-------------|\n| `prompt` | str | Full prompt sent to the model |\n| `model_response` | str | Full untruncated model output |\n| `model` | str | `Qwen/Qwen3-1.7B` |\n| `numbers` | list[int] | Available operands |\n| `target` | int | Target value |\n| `correct` | bool | Whether the answer evaluates correctly |\n| `extracted_answer` | str | Parsed content from `<answer>` tags (null if missing) |\n| `finish_reason` | str | `stop` or `length` \u2014 flag any `length` rows |\n\n---\n\n## Validation Criteria (for data-validator)\n\nA healthy artifact must satisfy ALL of the following:\n1. All rows have non-empty `model_response`\n2. Zero rows with `finish_reason == \"length\"` (no truncation)\n3. `correct` column is boolean, computed via equation evaluation (not string match)\n4. `extracted_answer` is null only when model didn't produce `<answer>` tags\n5. Row count matches n_samples (10)\n6. No degenerate outputs (empty strings, repeated tokens, only whitespace)\n",
+    "created": "",
+    "updated": ""
+  }
+]

backend/data/experiments.json

@@ -0,0 +1,39 @@
+[
+  {
+    "id": "onboarding",
+    "name": "Onboarding",
+    "research_project": "",
+    "hypothesis": {
+      "statement": "Qwen3-1.7B can solve basic Countdown arithmetic problems",
+      "type": "exploratory",
+      "status": "active",
+      "success_criteria": "Model produces valid arithmetic expressions that reach the target number on >50% of problems"
+    },
+    "stage": "active",
+    "completeness": 4,
+    "models": [],
+    "tasks": [],
+    "tags": [
+      "countdown",
+      "reasoning",
+      "onboarding",
+      "tutorial"
+    ],
+    "hf_repos": [
+      {
+        "repo": "timchen0618/onboarding-countdown-qwen3-1.7b",
+        "description": "onboarding-countdown-qwen3-1.7b \u2014 10 rows, 10/10 correct (2026-04-06)",
+        "date": ""
+      }
+    ],
+    "wandb_url": "",
+    "notes": "# Welcome to RACA\n\nThis is a sample experiment to show you how the dashboard works. You're looking at the **Overview** tab right now \u2014 it displays the experiment's README (this file).\n\nEverything you see here is generated from plain files in `notes/experiments/onboarding/`. You can browse them in your editor anytime.\n\n## How This Dashboard Works\n\nEach experiment has several tabs at the top. Here's what they do:\n\n### Overview (you are here)\n\nDisplays the experiment's README and any notes you've written in the `user/` folder. This is the main landing page for each experiment \u2014 a summary of what the experiment is, what you're investigating, and what you found.\n\n### Red Team Brief\n\nBefore any experiment runs, RACA reviews the design for potential problems \u2014 wrong evaluation metrics, truncated outputs, missing baselines, wasted compute. The brief lives at `red_team_brief.md`. This tab will be empty until you run your first real experiment.\n\n### Timeline\n\nA chronological log of everything that happened: when jobs were submitted, when artifacts were uploaded, when bugs were found and fixed. This is auto-generated from `activity_log.jsonl` \u2014 RACA writes to it as events happen.\n\n### Runs\n\nTracks each job submission \u2014 which model, which cluster, what status (pending, running, completed, failed), and links to the HuggingFace dataset with the results. Empty until you run something.\n\n### Artifacts\n\nLinks to all HuggingFace datasets produced by this experiment \u2014 canary runs, partial results, final data. Each artifact has metadata about what generated it. Empty until artifacts are uploaded.\n\n### Files\n\nAll the markdown and YAML files in the experiment folder. Click any file to read it. This is a quick way to browse the experiment's configuration and notes without leaving the dashboard.\n\n## Folder Structure\n\n```\nnotes/experiments/onboarding/\n  EXPERIMENT_README.md    \u2190 this file (shows in Overview tab)\n  experiment.yaml         \u2190 config: hypothesis, models, tasks\n  flow_state.json         \u2190 current phase (design/running/complete)\n  HUGGINGFACE_REPOS.md    \u2190 links to all uploaded datasets\n  questions.md            \u2190 research questions (read-only)\n  red_team_brief.md       \u2190 created during preflight review\n  activity_log.jsonl      \u2190 timeline entries (auto-generated)\n  user/                   \u2190 YOUR notes \u2014 RACA doesn't touch these\n    README.md             \u2190 your interpretation and observations\n    FINDINGS.md           \u2190 key results and surprises\n    DECISIONS.md          \u2190 design decisions and rationale\n    summary.md            \u2190 one-paragraph summary when done\n```\n\n**Most of this is automated.** RACA creates and updates the experiment files, uploads artifacts, and keeps the timeline current. The only files you write are in `user/` \u2014 that's your space for notes, findings, and decisions.\n\n## What's Next\n\nThis sample experiment hasn't been run yet \u2014 it's just here to show you the structure. When you're ready to run a real experiment, just tell RACA:\n\n> *I want to test whether Qwen3-8B follows complex instructions better than Llama-3.1-8B*\n\nOr try the full guided tutorial:\n\n> */raca:experiment-tutorial*\n",
+    "zayne_summary": "# Summary\n\n_Write a one-paragraph summary of the experiment and its outcome when you're done._\n\n## Status: active\n\n## Next Steps\n\n_What to do next based on findings._",
+    "zayne_readme": "# Onboarding Experiment \u2014 Your Notes\n\n## What I'm investigating\n\nThis is the tutorial experiment \u2014 testing Qwen3-1.7B on Countdown to learn the RACA pipeline.\n\n## Key observations\n\n_Fill this in as you review the results._\n\n## Open questions\n\n_Anything you want to follow up on._",
+    "zayne_findings": "# Welcome to Your Dashboard\n\nThis is a sample experiment to show you how the dashboard works. Everything you see here is generated from plain files in `notes/experiments/onboarding/`.\n\n## Dashboard Tabs\n\nEach experiment has tabs at the top:\n\n- **Overview** \u2014 the experiment's README and your notes (you're reading this now)\n- **Red Team Brief** \u2014 RACA reviews experiment designs for problems before running. Empty until your first real experiment.\n- **Timeline** \u2014 chronological log of everything that happened (auto-generated from `activity_log.jsonl`)\n- **Runs** \u2014 tracks each job submission: model, cluster, status, HuggingFace dataset links\n- **Artifacts** \u2014 links to all HuggingFace datasets produced by this experiment\n- **Files** \u2014 browse all experiment files without leaving the dashboard\n\n## What's Automated vs What You Write\n\nMost of this is automated. RACA creates and updates experiment files, uploads artifacts, and keeps the timeline current.\n\nThe `user/` folder is yours \u2014 RACA doesn't touch it:\n- `user/FINDINGS.md` \u2014 key results and surprises (this file)\n- `user/README.md` \u2014 your interpretation and observations\n- `user/DECISIONS.md` \u2014 design decisions and rationale\n- `user/summary.md` \u2014 one-paragraph summary when done\n\n## What's Next\n\nThis sample experiment hasn't been run yet \u2014 it's here to show you the structure. When you're ready:\n\n> *I want to test whether Qwen3-8B follows complex instructions better than Llama-3.1-8B*\n\nOr try the full guided tutorial: `/raca:experiment-tutorial`",
+    "zayne_decisions": "# Decisions\n\n| Date | Decision | Rationale |\n|------|----------|-----------|",
+    "red_team_brief": "# Red Team Brief \u2014 onboarding\n\n**Experiment:** Qwen3-1.7B on Countdown (tutorial/canary)\n**Reviewer:** agent\n**Date:** 2026-04-06\n**Status:** PASS\n\n---\n\n## Hypothesis\n\n> Qwen3-1.7B can solve basic Countdown arithmetic problems (>50% accuracy on 3-4 operand problems)\n\nThis is an exploratory baseline \u2014 a reasonable first question. No prior published result exists for Qwen3-1.7B specifically on this exact config, so the hypothesis is testable and non-trivial.\n\n---\n\n## Config Review\n\n### Model\n- **Qwen/Qwen3-1.7B** \u2014 small instruct model, appropriate for a tutorial canary\n- Known to handle arithmetic reasoning; 1.7B is at the edge of competence for Countdown (expect 20-50% accuracy)\n\n### max_tokens: 4096\n- **Status: PASS (marginal)**\n- Reference minimum is 2048-4096. We are at the ceiling of \"minimum\". For Qwen3-1.7B reasoning traces, 4096 should be sufficient for 3-4 operand problems.\n- **Watch for truncation in outputs** \u2014 if `finish_reason == \"length\"` appears in any response, flag immediately.\n\n### Prompt Format\n- **Must use the CoT + in-context examples prompt from the reference** (not the TinyZero `<think>` variant, since we are not RL-training)\n- Template:\n  ```\n  Answer the following problem. Explain your reasoning step by step. When you are finished, give your answer in this format: <answer>(your answer)</answer>.\n  \n  # Problem\n  Using the numbers in the list [{numbers}], create an equation that equals {target}. ...\n  ```\n- Answer extraction: regex on `<answer>...</answer>` tags\n\n### Evaluation Method\n- **Must use `CountdownJudge.validate_countdown_solution()` \u2014 NOT string matching**\n- Located at: `packages/custom_evaluations/custom_evaluations/sources/countdown/countdown_judge.py`\n- Handles: AST-based evaluation, Unicode operator normalization, multiset number validation\n- **Failure mode if string match is used:** \"3 + 5\" and \"5 + 3\" would score differently. This is wrong.\n\n### Dataset Generation\n- 10 samples (canary size \u2014 fine)\n- Use forward generation (inline, no dependencies): 3-4 operands, targets 1-99\n- Do NOT require all numbers used \u2014 subset is sufficient (this is the easier, more standard setting)\n\n---\n\n## Failure Modes & Mitigations\n\n| Risk | Severity | Mitigation |\n|------|----------|------------|\n| Truncated outputs (`finish_reason=length`) | HIGH | Check every row \u2014 flag if any truncation occurs |\n| String matching instead of equation eval | HIGH | Use `CountdownJudge` \u2014 verified AST-based |\n| Model outputs `\u00d7` instead of `*` | MEDIUM | `CountdownJudge` handles Unicode normalization |\n| `<answer>` tag missing in output | MEDIUM | Log separately as \"format failures\" vs \"wrong answer\" |\n| Duplicate problems in 10 samples | LOW | Use `random.seed(42)` for reproducibility |\n\n---\n\n## Expected Results\n\nBased on typical results from the reference file:\n- Qwen3-1.7B is smaller than Qwen2.5-3B (which achieves 70-80% after GRPO training)\n- For a **pre-GRPO instruct model** at this size, expect roughly **20-50% accuracy**\n- If accuracy is <10%, suspect prompt format issue or evaluation bug \u2014 investigate before scaling\n\n---\n\n## Output Schema\n\nExpected columns in the HuggingFace dataset:\n\n| Column | Type | Description |\n|--------|------|-------------|\n| `prompt` | str | Full prompt sent to the model |\n| `model_response` | str | Full untruncated model output |\n| `model` | str | `Qwen/Qwen3-1.7B` |\n| `numbers` | list[int] | Available operands |\n| `target` | int | Target value |\n| `correct` | bool | Whether the answer evaluates correctly |\n| `extracted_answer` | str | Parsed content from `<answer>` tags (null if missing) |\n| `finish_reason` | str | `stop` or `length` \u2014 flag any `length` rows |\n\n---\n\n## Validation Criteria (for data-validator)\n\nA healthy artifact must satisfy ALL of the following:\n1. All rows have non-empty `model_response`\n2. Zero rows with `finish_reason == \"length\"` (no truncation)\n3. `correct` column is boolean, computed via equation evaluation (not string match)\n4. `extracted_answer` is null only when model didn't produce `<answer>` tags\n5. Row count matches n_samples (10)\n6. No degenerate outputs (empty strings, repeated tokens, only whitespace)\n",
+    "created": "",
+    "updated": ""
+  }
+]

backend/data/runs.json

@@ -0,0 +1 @@
+[]

backend/data/sub_experiments.json

@@ -0,0 +1 @@
+[]

backend/data/summary_findings.json

@@ -0,0 +1 @@
+[]

backend/requirements.txt

@@ -0,0 +1,5 @@
+flask>=3.0.0
+flask-cors>=4.0.0
+datasets>=2.14.0
+gunicorn>=21.0.0
+huggingface_hub>=0.20.0

docs/managing_presets.md

@@ -0,0 +1,194 @@
+# Managing RACA-VIS-PRESETS Programmatically
+
+## Overview
+
+The agg_visualizer stores presets in the HuggingFace dataset repo `your-org/RACA-VIS-PRESETS`. Each visualizer type has its own JSON file:
+
+| Type | File | Extra Fields |
+|------|------|-------------|
+| `model` | `model_presets.json` | `column` (default: `"model_responses"`) |
+| `arena` | `arena_presets.json` | none |
+| `rlm` | `rlm_presets.json` | `config` (default: `"rlm_call_traces"`) |
+| `harbor` | `harbor_presets.json` | none |
+
+## Preset Schema
+
+Every preset has these base fields:
+
+```json
+{
+  "id": "8-char hex",
+  "name": "Human-readable name",
+  "repo": "org/dataset-name",
+  "split": "train"
+}
+```
+
+Plus type-specific fields listed above.
+
+## How to Add Presets from Experiment Markdown Files
+
+### Step 1: Identify repos and their visualizer type
+
+Read the experiment markdown file(s) and extract all HuggingFace repo links. Categorize each:
+
+- **Countdown / MuSR datasets** (model response traces) → `model` type, set `column: "response"`
+- **FrozenLake / arena datasets** (game episodes) → `arena` type
+- **Harbor / SWE-bench datasets** → `harbor` type
+- **RLM call traces** → `rlm` type, set `config: "rlm_call_traces"`
+
+### Step 2: Download existing presets from HF
+
+```python
+from huggingface_hub import hf_hub_download
+import json
+
+PRESETS_REPO = "your-org/RACA-VIS-PRESETS"
+
+def load_hf_presets(vis_type):
+    try:
+        path = hf_hub_download(PRESETS_REPO, f"{vis_type}_presets.json", repo_type="dataset")
+        with open(path) as f:
+            return json.load(f)
+    except Exception:
+        return []
+
+existing_model = load_hf_presets("model")
+existing_arena = load_hf_presets("arena")
+# ... etc for rlm, harbor
+
+# Build set of repos already present
+existing_repos = set()
+for presets_list in [existing_model, existing_arena]:
+    for p in presets_list:
+        existing_repos.add(p["repo"])
+```
+
+### Step 3: Build new presets, skipping duplicates
+
+```python
+import uuid
+
+new_presets = []  # list of (vis_type, name, repo)
+
+# Example: adding strategy compliance countdown presets
+new_presets.append(("model", "SC Countdown K2-Inst TreeSearch",
+    "your-org/t1-strategy-countdown-treesearch-kimi-k2-instruct-kimi-inst"))
+
+# ... add all repos from the markdown ...
+
+# Filter out existing
+to_add = {"model": [], "arena": [], "rlm": [], "harbor": []}
+for vis_type, name, repo in new_presets:
+    if repo in existing_repos:
+        continue  # skip duplicates
+    preset = {
+        "id": uuid.uuid4().hex[:8],
+        "name": name,
+        "repo": repo,
+        "split": "train",
+    }
+    if vis_type == "model":
+        preset["column"] = "response"
+    elif vis_type == "rlm":
+        preset["config"] = "rlm_call_traces"
+    to_add[vis_type].append(preset)
+```
+
+### Step 4: Merge and upload to HF
+
+```python
+import tempfile, os
+from huggingface_hub import HfApi
+
+api = HfApi()
+
+# Merge new presets with existing
+final_model = existing_model + to_add["model"]
+final_arena = existing_arena + to_add["arena"]
+
+for vis_type, presets in [("model", final_model), ("arena", final_arena)]:
+    if not presets:
+        continue
+    with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as f:
+        json.dump(presets, f, indent=2)
+        tmp = f.name
+    api.upload_file(
+        path_or_fileobj=tmp,
+        path_in_repo=f"{vis_type}_presets.json",
+        repo_id=PRESETS_REPO,
+        repo_type="dataset",
+    )
+    os.unlink(tmp)
+```
+
+### Step 5: Sync the deployed HF Space
+
+After uploading to the HF dataset, tell the running Space to re-download presets:
+
+```bash
+curl -X POST "https://your-org-agg-trace-visualizer.hf.space/api/presets/sync"
+```
+
+This forces the Space to re-download all preset files from `RACA-VIS-PRESETS` without needing a restart or redeployment.
+
+### Step 6: Sync local preset files
+
+```python
+import shutil
+from huggingface_hub import hf_hub_download
+
+local_dir = Path(__file__).parent.parent / "backend" / "presets"
+for vis_type in ["model", "arena", "rlm", "harbor"]:
+    try:
+        path = hf_hub_download(PRESETS_REPO, f"{vis_type}_presets.json", repo_type="dataset")
+        shutil.copy2(path, f"{local_dir}/{vis_type}_presets.json")
+    except Exception:
+        pass
+```
+
+## Naming Convention
+
+Preset names follow this pattern to be descriptive and avoid future conflicts:
+
+```
+{Experiment} {Task} {Model} {Variant}
+```
+
+### Experiment prefixes
+- `SC` — Strategy Compliance
+- `Wing` — Wingdings Compliance
+
+### Model abbreviations
+- `K2-Inst` — Kimi-K2-Instruct (RLHF)
+- `K2-Think` — Kimi-K2-Thinking (RLVR)
+- `Q3-Inst` — Qwen3-Next-80B Instruct (RLHF)
+- `Q3-Think` — Qwen3-Next-80B Thinking (RLVR)
+
+### Task names
+- `Countdown` — 8-arg arithmetic countdown
+- `MuSR` — MuSR murder mysteries
+- `FrozenLake` — FrozenLake grid navigation
+
+### Variant names (strategy compliance only)
+- `TreeSearch` / `Baseline` / `Anti` — countdown tree search experiment
+- `CritFirst` / `Anti-CritFirst` — criterion-first cross-cutting analysis
+- `Counterfactual` / `Anti-Counterfactual` — counterfactual hypothesis testing
+- `BackChain` — backward chaining (FrozenLake)
+
+### Examples
+
+```
+SC Countdown K2-Inst TreeSearch       # Strategy compliance, countdown, Kimi instruct, tree search variant
+SC MuSR Q3-Think Counterfactual       # Strategy compliance, MuSR, Qwen thinking, counterfactual variant
+SC FrozenLake K2-Think BackChain      # Strategy compliance, FrozenLake, Kimi thinking, backward chaining
+Wing Countdown Q3-Inst                # Wingdings, countdown, Qwen instruct (no variant — wingdings has one condition)
+Wing MuSR K2-Think                    # Wingdings, MuSR, Kimi thinking
+```
+
+## Important Notes
+
+- **Always check for existing repos** before adding. The script above uses `existing_repos` set to skip duplicates.
+- **The `column` field matters for model presets.** Strategy compliance and wingdings datasets use `"response"` as the response column, not the default `"model_responses"`.
+- **Local files are fallback cache.** The agg_visualizer downloads from HF on startup and caches locally. After uploading to HF, sync the local files so the running app picks up changes without restart (or hit the `/api/presets/sync` endpoint).
+- **Don't modify rlm or harbor presets** unless adding datasets of those types. The script above only touches model and arena.

docs/plans/2026-03-07-research-dashboard-design.md

@@ -0,0 +1,86 @@
+# Research Dashboard Design
+
+**Date:** 2026-03-07
+**Status:** Approved
+
+## Overview
+
+Extend the existing agg_visualizer into a parent "Research Dashboard" website with a top-level navigation bar. The current visualizer becomes one page; a new Experiments page provides a control pane for tracking experiments, hypotheses, runs, and artifacts.
+
+Deployed on HuggingFace Spaces (same Space as the current visualizer).
+
+## Audience
+
+Primarily the researcher + advisor. May expand to a small team later.
+
+## Architecture
+
+### Navigation
+- Top-level nav bar: `Experiments | Visualizer` (future: Research Map, Knowledge Base)
+- State-driven view switching (useState), not URL routing (HF Spaces doesn't support deep-linking)
+- Current visualizer tabs (Model Trace, Arena, RLM, etc.) nest inside the Visualizer page unchanged
+
+### Data Storage
+- JSON files in HF dataset repo `your-org/RACA_DASHBOARD`
+- Three files: `experiments.json`, `runs.json`, `sub_experiments.json`
+- In-memory cache with async HF upload (same pattern as presets.py)
+- Local JSON fallback in `backend/data/`
+
+### Backend API
+Blueprint at `/api/experiments/`:
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | `/` | List all experiments |
+| POST | `/` | Create experiment |
+| GET | `/:id` | Full detail (includes runs + subs) |
+| PUT | `/:id` | Update experiment |
+| DELETE | `/:id` | Delete experiment |
+| POST | `/:id/runs` | Add run record |
+| PUT | `/:id/runs/:run_id` | Update run |
+| DELETE | `/:id/runs/:run_id` | Delete run |
+| POST | `/:id/subs` | Add sub-experiment |
+| PUT | `/:id/subs/:sub_id` | Update sub-experiment |
+| DELETE | `/:id/subs/:sub_id` | Delete sub-experiment |
+| POST | `/sync` | Force re-download from HF |
+| POST | `/import` | Bulk import (experiment.yaml format) |
+
+### Data Model
+
+**Experiment:**
+- id, name, research_project, hypothesis (statement, type, status, success_criteria)
+- stage, completeness (0-5), models[], tasks[], tags[]
+- hf_repos[] (repo, description, date), wandb_url, notes (markdown)
+- created, updated timestamps
+
+**Run Record:**
+- id, experiment_id, condition, model, cluster, status
+- hf_dataset, metrics (dict), timestamp, notes
+
+**Sub-experiment:**
+- id, experiment_id, name, hypothesis, status
+- content_md (full markdown report), hf_repos[]
+- created, updated timestamps
+
+### Frontend
+
+Three drill-down levels:
+
+1. **Experiment List** — Cards with name, hypothesis, status badge, completeness, tags, last updated. Sort/filter controls.
+2. **Experiment Detail** — Hypothesis header, tabbed views (Overview, Runs, Datasets, Sub-experiments). Inline editing.
+3. **Sub-experiment View** — Breadcrumb, header, markdown-rendered body, HF repos, edit toggle.
+
+### Integration Points
+- exp-runner v2 pushes data via `/api/experiments/import`
+- Flexible ingestion — API accepts data from any source
+- No local filesystem dependency at runtime
+
+## Future Pages (Phase 2+)
+- **Research Map** — Graph/board view of research directions and experiment relationships
+- **Knowledge Base** — Searchable wiki of findings, notes, HF repos
+
+## Tech Stack
+- Backend: Flask (existing)
+- Frontend: React + Vite + Tailwind + Zustand (existing)
+- Deployment: Docker on HuggingFace Spaces (existing)
+- Storage: HF dataset repo as JSON store

frontend/dist/assets/ExperimentsApp-DnJR2-55.css

@@ -0,0 +1 @@
+@font-face{font-display:block;font-family:KaTeX_AMS;font-style:normal;font-weight:400;src:url(/assets/KaTeX_AMS-Regular-BQhdFMY1.woff2) format("woff2"),url(/assets/KaTeX_AMS-Regular-DMm9YOAa.woff) format("woff"),url(/assets/KaTeX_AMS-Regular-DRggAlZN.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Caligraphic;font-style:normal;font-weight:700;src:url(/assets/KaTeX_Caligraphic-Bold-Dq_IR9rO.woff2) format("woff2"),url(/assets/KaTeX_Caligraphic-Bold-BEiXGLvX.woff) format("woff"),url(/assets/KaTeX_Caligraphic-Bold-ATXxdsX0.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Caligraphic;font-style:normal;font-weight:400;src:url(/assets/KaTeX_Caligraphic-Regular-Di6jR-x-.woff2) format("woff2"),url(/assets/KaTeX_Caligraphic-Regular-CTRA-rTL.woff) format("woff"),url(/assets/KaTeX_Caligraphic-Regular-wX97UBjC.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Fraktur;font-style:normal;font-weight:700;src:url(/assets/KaTeX_Fraktur-Bold-CL6g_b3V.woff2) format("woff2"),url(/assets/KaTeX_Fraktur-Bold-BsDP51OF.woff) format("woff"),url(/assets/KaTeX_Fraktur-Bold-BdnERNNW.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Fraktur;font-style:normal;font-weight:400;src:url(/assets/KaTeX_Fraktur-Regular-CTYiF6lA.woff2) format("woff2"),url(/assets/KaTeX_Fraktur-Regular-Dxdc4cR9.woff) format("woff"),url(/assets/KaTeX_Fraktur-Regular-CB_wures.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Main;font-style:normal;font-weight:700;src:url(/assets/KaTeX_Main-Bold-Cx986IdX.woff2) format("woff2"),url(/assets/KaTeX_Main-Bold-Jm3AIy58.woff) format("woff"),url(/assets/KaTeX_Main-Bold-waoOVXN0.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Main;font-style:italic;font-weight:700;src:url(/assets/KaTeX_Main-BoldItalic-DxDJ3AOS.woff2) format("woff2"),url(/assets/KaTeX_Main-BoldItalic-SpSLRI95.woff) format("woff"),url(/assets/KaTeX_Main-BoldItalic-DzxPMmG6.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Main;font-style:italic;font-weight:400;src:url(/assets/KaTeX_Main-Italic-NWA7e6Wa.woff2) format("woff2"),url(/assets/KaTeX_Main-Italic-BMLOBm91.woff) format("woff"),url(/assets/KaTeX_Main-Italic-3WenGoN9.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Main;font-style:normal;font-weight:400;src:url(/assets/KaTeX_Main-Regular-B22Nviop.woff2) format("woff2"),url(/assets/KaTeX_Main-Regular-Dr94JaBh.woff) format("woff"),url(/assets/KaTeX_Main-Regular-ypZvNtVU.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Math;font-style:italic;font-weight:700;src:url(/assets/KaTeX_Math-BoldItalic-CZnvNsCZ.woff2) format("woff2"),url(/assets/KaTeX_Math-BoldItalic-iY-2wyZ7.woff) format("woff"),url(/assets/KaTeX_Math-BoldItalic-B3XSjfu4.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Math;font-style:italic;font-weight:400;src:url(/assets/KaTeX_Math-Italic-t53AETM-.woff2) format("woff2"),url(/assets/KaTeX_Math-Italic-DA0__PXp.woff) format("woff"),url(/assets/KaTeX_Math-Italic-flOr_0UB.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_SansSerif;font-style:normal;font-weight:700;src:url(/assets/KaTeX_SansSerif-Bold-D1sUS0GD.woff2) format("woff2"),url(/assets/KaTeX_SansSerif-Bold-DbIhKOiC.woff) format("woff"),url(/assets/KaTeX_SansSerif-Bold-CFMepnvq.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_SansSerif;font-style:italic;font-weight:400;src:url(/assets/KaTeX_SansSerif-Italic-C3H0VqGB.woff2) format("woff2"),url(/assets/KaTeX_SansSerif-Italic-DN2j7dab.woff) format("woff"),url(/assets/KaTeX_SansSerif-Italic-YYjJ1zSn.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_SansSerif;font-style:normal;font-weight:400;src:url(/assets/KaTeX_SansSerif-Regular-DDBCnlJ7.woff2) format("woff2"),url(/assets/KaTeX_SansSerif-Regular-CS6fqUqJ.woff) format("woff"),url(/assets/KaTeX_SansSerif-Regular-BNo7hRIc.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Script;font-style:normal;font-weight:400;src:url(/assets/KaTeX_Script-Regular-D3wIWfF6.woff2) format("woff2"),url(/assets/KaTeX_Script-Regular-D5yQViql.woff) format("woff"),url(/assets/KaTeX_Script-Regular-C5JkGWo-.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Size1;font-style:normal;font-weight:400;src:url(/assets/KaTeX_Size1-Regular-mCD8mA8B.woff2) format("woff2"),url(/assets/KaTeX_Size1-Regular-C195tn64.woff) format("woff"),url(/assets/KaTeX_Size1-Regular-Dbsnue_I.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Size2;font-style:normal;font-weight:400;src:url(/assets/KaTeX_Size2-Regular-Dy4dx90m.woff2) format("woff2"),url(/assets/KaTeX_Size2-Regular-oD1tc_U0.woff) format("woff"),url(/assets/KaTeX_Size2-Regular-B7gKUWhC.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Size3;font-style:normal;font-weight:400;src:url(data:font/woff2;base64,d09GMgABAAAAAA4oAA4AAAAAHbQAAA3TAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAABmAAgRQIDgmcDBEICo1oijYBNgIkA14LMgAEIAWJAAeBHAyBHBvbGiMRdnO0IkRRkiYDgr9KsJ1NUAf2kILNxgUmgqIgq1P89vcbIcmsQbRps3vCcXdYOKSWEPEKgZgQkprQQsxIXUgq0DqpGKmIvrgkeVGtEQD9DzAO29fM9jYhxZEsL2FeURH2JN4MIcTdO049NCVdxQ/w9NrSYFEBKTDKpLKfNkCGDc1RwjZLQcm3vqJ2UW9Xfa3tgAHz6ivp6vgC2yD4/6352ndnN0X0TL7seypkjZlMsjmZnf0Mm5Q+JykRWQBKCVCVPbARPXWyQtb5VgLB6Biq7/Uixcj2WGqdI8tGSgkuRG+t910GKP2D7AQH0DB9FMDW/obJZ8giFI3Wg8Cvevz0M+5m0rTh7XDBlvo9Y4vm13EXmfttwI4mBo1EG15fxJhUiCLbiiyCf/ZA6MFAhg3pGIZGdGIVjtPn6UcMk9A/UUr9PhoNsCENw1APAq0gpH73e+M+0ueyHbabc3vkbcdtzcf/fiy+NxQEjf9ud/ELBHAXJ0nk4z+MXH2Ev/kWyV4k7SkvpPc9Qr38F6RPWnM9cN6DJ0AdD1BhtgABtmoRoFCvPsBAumNm6soZG2Gk5GyVTo2sJncSyp0jQTYoR6WDvTwaaEcHsxHfvuWhHA3a6bN7twRKtcGok6NsCi7jYRrM2jExsUFMxMQYuJbMhuWNOumEJy9hi29Dmg5zMp/A5+hhPG19j1vBrq8JTLr8ki5VLPmG/PynJHVul440bxg5xuymHUFPBshC+nA9I1FmwbRBTNHAcik3Oae0cxKoI3MOriM42UrPe51nsaGxJ+WfXubAsP84aabUlQSJ1IiE0iPETLUU4CATgfXSCSpuRFRmCGbO+wSpAnzaeaCYW1VNEysRtuXCEL1kUFUbbtMv3Tilt/1c11jt3Q5bbMa84cpWipp8Elw3MZhOHsOlwwVUQM3lAR35JiFQbaYCRnMF2lxAWoOg2gyoIV4PouX8HytNIfLhqpJtXB4vjiViUI8IJ7bkC4ikkQvKksnOTKICwnqWSZ9YS5f0WCxmpgjbIq7EJcM4aI2nmhLNY2JIUgOjXZFWBHb+x5oh6cwb0Tv1ackHdKi0I9OO2wE9aogIOn540CCCziyhN+IaejtgAONKznHlHyutPrHGwCx9S6B8kfS4Mfi4Eyv7OU730bT1SCBjt834cXsf43zVjPUqqJjgrjeGnBxSG4aYAKFuVbeCfkDIjAqMb6yLNIbCuvXhMH2/+k2vkNpkORhR59N1CkzoOENvneIosjYmuTxlhUzaGEJQ/iWqx4dmwpmKjrwTiTGTCVozNAYqk/zXOndWxuWSmJkQpJw3pK5KX6QrLt5LATMqpmPAQhkhK6PUjzHUn7E0gHE0kPE0iKkolgkUx9SZmVAdDgpffdyJKg3k7VmzYGCwVXGz/tXmkOIp+vcWs+EMuhhvN0h9uhfzWJziBQmCREGSIFmQIkgVpAnSBRmC//6hkLZwaVhwxlrJSOdqlFtOYxlau9F2QN5Y98xmIAsiM1HVp2VFX+DHHGg6Ecjh3vmqtidX3qHI2qycTk/iwxSt5UzTmEP92ZBnEWTk4Mx8Mpl78ZDokxg/KWb+Q0QkvdKVmq3TMW+RXEgrsziSAfNXFMhDc60N5N9jQzjfO0kBKpUZl0ZmwJ41j/B9Hz6wmRaJB84niNmQrzp9eSlQCDDzazGDdVi3P36VZQ+Jy4f9UBNp+3zTjqI4abaFAm+GShVaXlsGdF3FYzZcDI6cori4kMxUECl9IjJZpzkvitAoxKue+90pDMvcKRxLl53TmOKCmV/xRolNKSqqUxc6LStOETmFOiLZZptlZepcKiAzteG8PEdpnQpbOMNcMsR4RR2Bs0cKFEvSmIjAFcnarqwUL4lDhHmnVkwu1IwshbiCcgvOheZuYyOteufZZwlcTlLgnZ3o/WcYdzZHW/WGaqaVfmTZ1aWCceJjkbZqsfbkOtcFlUZM/jy+hXHDbaUobWqqXaeWobbLO99yG5N3U4wxco0rQGGcOLASFMXeJoham8M+/x6O2WywK2l4HGbq1CoUyC/IZikQhdq3SiuNrvAEj0AVu9x2x3lp/xWzahaxidezFVtdcb5uEnzyl0ZmYiuKI0exvCd4Xc9CV1KB0db00z92wDPde0kukbvZIWN6jUWFTmPIC/Y4UPCm8UfDTFZpZNon1qLFTkBhxzB+FjQRA2Q/YRJT8pQigslMaUpFyAG8TMlXigiqmAZX4xgijKjRlGpLE0GdplRfCaJo0JQaSxNBk6ZmMzcya0FmrcisDdn0Q3HI2sWSppYigmlM1XT/kLQZSNpMJG0WkjYbSZuDpM1F0uYhFc1HxU4m1QJjDK6iL0S5uSj5rgXc3RejEigtcRBtqYPQsiTskmO5vosV+q4VGIKbOkDg0jtRrq+Em1YloaTFar3EGr1EUC8R0kus1Uus00usL97ABr2BjXoDm/QGNhuWtMVBKOwg/i78lT7hBsAvDmwHc/ao3vmUbBmhjeYySZNWvGkfZAgISDSaDo1SVpzGDsAEkF8B+gEapViUoZgUWXcRIGFZNm6gWbAKk0bp0k1MHG9fLYtV4iS2SmLEQFARzRcnf9PUS0LVn05/J9MiRRBU3v2IrvW974v4N00L7ZMk0wXP1409CHo/an8zTRHD3eSJ6m8D4YMkZNl3M79sqeuAsr/m3f+8/yl7A50aiAEJgeBeMWzu7ui9UfUBCe2TIqZIoOd/3/udRBOQidQZUERzb2/VwZN1H/Sju82ew2H2Wfr6qvfVf3hqwDvAIpkQVFy4B9Pe9e4/XvPeceu7h3dvO56iJPf0+A6cqA2ip18ER+iFgggiuOkvj24bby0N9j2UHIkgqIt+sVgfodC4YghLSMjSZbH0VR/6dMDrYJeKHilKTemt6v6kvzvn3/RrdWtr0GoN/xL+Sex/cPYLUpepx9cz/D46UPU5KXgAQa+NDps1v6J3xP1i2HtaDB0M9aX2deA7SYff//+gUCovMmIK/qfsFcOk+4Y5ZN97XlG6zebqtMbKgeRFi51vnxTQYBUik2rS/Cn6PC8ADR8FGxsRPB82dzfND90gIcshOcYUkfjherBz53odpm6TP8txlwOZ71xmfHHOvq053qFF/MRlS3jP0ELudrf2OeN8DHvp6ZceLe8qKYvWz/7yp0u4dKPfli3CYq0O13Ih71mylJ80tOi10On8wi+F4+LWgDPeJ30msSQt9/vkmHq9/Lvo2b461mP801v3W4xTcs6CbvF9UDdrSt+A8OUbpSh55qAUFXWznBBfdeJ8a4d7ugT5tvxUza3h9m4H7ptTqiG4z0g5dc0X29OcGlhpGFMpQo9ytTS+NViZpNdvU4kWx+LKxNY10kQ1yqGXrhe4/1nvP7E+nd5A92TtaRplbHSqoIdOqtRWti+fkB5/n1+/VvCmz12pG1kpQWsfi1ftlBobm0bpngs16CHkbIwdLnParxtTV3QYRlfJ0KFskH7pdN/YDn+yRuSd7sNH3aO0DYPggk6uWuXrfOc+fa3VTxFVvKaNxHsiHmsXyCLIE5yuOeN3/Jdf8HBL/5M6shjyhxHx9BjB1O0+4NLOnjLLSxwO7ukN4jMbOIcD879KLSi6Pk61Oqm2377n8079PXEEQ7cy7OKEC9nbpet118fxweTafpt69x/Bt8UqGzNQt7aelpc44dn5cqhwf71+qKp/Zf/+a0zcizOUWpl/iBcSXip0pplkatCchoH5c5aUM8I7/dWxAej8WicPL1URFZ9BDJelUwEwTkGqUhgSlydVes95YdXvhh9Gfz/aeFWvgVb4tuLbcv4+wLdutVZv/cUonwBD/6eDlE0aSiKK/uoH3+J1wDE/jMVqY2ysGufN84oIXB0sPzy8ollX/LegY74DgJXJR57sn+VGza0x3DnuIgABFM15LmajjjsNlYj+JEZGbuRYcAMOWxFkPN2w6Wd46xo4gVWQR/X4lyI/R6K/YK0110GzudPRW7Y+UOBGTfNNzHeYT0fiH0taunBpq9HEW8OKSaBGj21L0MqenEmNRWBAWDWAk4CpNoEZJ2tTaPFgbQYj8HxtFilErs3BTRwT8uO1NXQaWfIotchmPkAF5mMBAliEmZiOGVgCG9LgRzpscMAOOwowlT3JhusdazXGSC/hxR3UlmWVwWHpOIKheqONvjyhSiTHIkVUco5bnji8m//zL7PKaT1Vl5I6UE609f+gkr6MZKVyKc7zJRmCahLsdlyA5fdQkRSan9LgnnLEyGSkaKJCJog0wAgvepWBt80+1yKln1bMVtCljfNWDueKLsWwaEbBSfSPTEmVRsUcYYMnEjcjeyCZzBXK9E9BYBXLKjOSpUDR+nEV3TFSUdQaz+ot98QxgXwx0GQ+EEUAKB2qZPkQQ0GqFD8UPFMqyaCHM24BZmSGic9EYMagKizOw9Hz50DMrDLrqqLkTAhplMictiCAx5S3BIUQdeJeLnBy2CNtMfz6cV4u8XKoFZQesbf9YZiIERiHjaNodDW6LgcirX/mPnJIkBGDUpTBhSa0EIr38D5hCIszhCM8URGBqImoWjpvpt1ebu/v3Gl3qJfMnNM+9V+kiRFyROTPHQWOcs1dNW94/ukKMPZBvDi55i5CttdeJz84DLngLqjcdwEZ87bFFR8CIG35OAkDVN6VRDZ7aq67NteYqZ2lpT8oYB2CytoBd6VuAx4WgiAsnuj3WohG+LugzXiQRDeM3XYXlULv4dp5VFYC) format("woff2"),url(/assets/KaTeX_Size3-Regular-CTq5MqoE.woff) format("woff"),url(/assets/KaTeX_Size3-Regular-DgpXs0kz.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Size4;font-style:normal;font-weight:400;src:url(/assets/KaTeX_Size4-Regular-Dl5lxZxV.woff2) format("woff2"),url(/assets/KaTeX_Size4-Regular-BF-4gkZK.woff) format("woff"),url(/assets/KaTeX_Size4-Regular-DWFBv043.ttf) format("truetype")}@font-face{font-display:block;font-family:KaTeX_Typewriter;font-style:normal;font-weight:400;src:url(/assets/KaTeX_Typewriter-Regular-CO6r4hn1.woff2) format("woff2"),url(/assets/KaTeX_Typewriter-Regular-C0xS9mPB.woff) format("woff"),url(/assets/KaTeX_Typewriter-Regular-D3Ib7_Hf.ttf) format("truetype")}.katex{font: 1.21em KaTeX_Main,Times New Roman,serif;line-height:1.2;position:relative;text-indent:0;text-rendering:auto}.katex *{-ms-high-contrast-adjust:none!important;border-color:currentColor}.katex .katex-version:after{content:"0.16.40"}.katex .katex-mathml{clip:rect(1px,1px,1px,1px);border:0;height:1px;overflow:hidden;padding:0;position:absolute;width:1px}.katex .katex-html>.newline{display:block}.katex .base{position:relative;white-space:nowrap;width:-moz-min-content;width:min-content}.katex .base,.katex .strut{display:inline-block}.katex .textbf{font-weight:700}.katex .textit{font-style:italic}.katex .textrm{font-family:KaTeX_Main}.katex .textsf{font-family:KaTeX_SansSerif}.katex .texttt{font-family:KaTeX_Typewriter}.katex .mathnormal{font-family:KaTeX_Math;font-style:italic}.katex .mathit{font-family:KaTeX_Main;font-style:italic}.katex .mathrm{font-style:normal}.katex .mathbf{font-family:KaTeX_Main;font-weight:700}.katex .boldsymbol{font-family:KaTeX_Math;font-style:italic;font-weight:700}.katex .amsrm,.katex .mathbb,.katex .textbb{font-family:KaTeX_AMS}.katex .mathcal{font-family:KaTeX_Caligraphic}.katex .mathfrak,.katex .textfrak{font-family:KaTeX_Fraktur}.katex .mathboldfrak,.katex .textboldfrak{font-family:KaTeX_Fraktur;font-weight:700}.katex .mathtt{font-family:KaTeX_Typewriter}.katex .mathscr,.katex .textscr{font-family:KaTeX_Script}.katex .mathsf,.katex .textsf{font-family:KaTeX_SansSerif}.katex .mathboldsf,.katex .textboldsf{font-family:KaTeX_SansSerif;font-weight:700}.katex .mathitsf,.katex .mathsfit,.katex .textitsf{font-family:KaTeX_SansSerif;font-style:italic}.katex .mainrm{font-family:KaTeX_Main;font-style:normal}.katex .vlist-t{border-collapse:collapse;display:inline-table;table-layout:fixed}.katex .vlist-r{display:table-row}.katex .vlist{display:table-cell;position:relative;vertical-align:bottom}.katex .vlist>span{display:block;height:0;position:relative}.katex .vlist>span>span{display:inline-block}.katex .vlist>span>.pstrut{overflow:hidden;width:0}.katex .vlist-t2{margin-right:-2px}.katex .vlist-s{display:table-cell;font-size:1px;min-width:2px;vertical-align:bottom;width:2px}.katex .vbox{align-items:baseline;display:inline-flex;flex-direction:column}.katex .hbox{width:100%}.katex .hbox,.katex .thinbox{display:inline-flex;flex-direction:row}.katex .thinbox{max-width:0;width:0}.katex .msupsub{text-align:left}.katex .mfrac>span>span{text-align:center}.katex .mfrac .frac-line{border-bottom-style:solid;display:inline-block;width:100%}.katex .hdashline,.katex .hline,.katex .mfrac .frac-line,.katex .overline .overline-line,.katex .rule,.katex .underline .underline-line{min-height:1px}.katex .mspace{display:inline-block}.katex .smash{display:inline;line-height:0}.katex .clap,.katex .llap,.katex .rlap{position:relative;width:0}.katex .clap>.inner,.katex .llap>.inner,.katex .rlap>.inner{position:absolute}.katex .clap>.fix,.katex .llap>.fix,.katex .rlap>.fix{display:inline-block}.katex .llap>.inner{right:0}.katex .clap>.inner,.katex .rlap>.inner{left:0}.katex .clap>.inner>span{margin-left:-50%;margin-right:50%}.katex .rule{border:0 solid;display:inline-block;position:relative}.katex .hline,.katex .overline .overline-line,.katex .underline .underline-line{border-bottom-style:solid;display:inline-block;width:100%}.katex .hdashline{border-bottom-style:dashed;display:inline-block;width:100%}.katex .sqrt>.root{margin-left:.2777777778em;margin-right:-.5555555556em}.katex .fontsize-ensurer.reset-size1.size1,.katex .sizing.reset-size1.size1{font-size:1em}.katex .fontsize-ensurer.reset-size1.size2,.katex .sizing.reset-size1.size2{font-size:1.2em}.katex .fontsize-ensurer.reset-size1.size3,.katex .sizing.reset-size1.size3{font-size:1.4em}.katex .fontsize-ensurer.reset-size1.size4,.katex .sizing.reset-size1.size4{font-size:1.6em}.katex .fontsize-ensurer.reset-size1.size5,.katex .sizing.reset-size1.size5{font-size:1.8em}.katex .fontsize-ensurer.reset-size1.size6,.katex .sizing.reset-size1.size6{font-size:2em}.katex .fontsize-ensurer.reset-size1.size7,.katex .sizing.reset-size1.size7{font-size:2.4em}.katex .fontsize-ensurer.reset-size1.size8,.katex .sizing.reset-size1.size8{font-size:2.88em}.katex .fontsize-ensurer.reset-size1.size9,.katex .sizing.reset-size1.size9{font-size:3.456em}.katex .fontsize-ensurer.reset-size1.size10,.katex .sizing.reset-size1.size10{font-size:4.148em}.katex .fontsize-ensurer.reset-size1.size11,.katex .sizing.reset-size1.size11{font-size:4.976em}.katex .fontsize-ensurer.reset-size2.size1,.katex .sizing.reset-size2.size1{font-size:.8333333333em}.katex .fontsize-ensurer.reset-size2.size2,.katex .sizing.reset-size2.size2{font-size:1em}.katex .fontsize-ensurer.reset-size2.size3,.katex .sizing.reset-size2.size3{font-size:1.1666666667em}.katex .fontsize-ensurer.reset-size2.size4,.katex .sizing.reset-size2.size4{font-size:1.3333333333em}.katex .fontsize-ensurer.reset-size2.size5,.katex .sizing.reset-size2.size5{font-size:1.5em}.katex .fontsize-ensurer.reset-size2.size6,.katex .sizing.reset-size2.size6{font-size:1.6666666667em}.katex .fontsize-ensurer.reset-size2.size7,.katex .sizing.reset-size2.size7{font-size:2em}.katex .fontsize-ensurer.reset-size2.size8,.katex .sizing.reset-size2.size8{font-size:2.4em}.katex .fontsize-ensurer.reset-size2.size9,.katex .sizing.reset-size2.size9{font-size:2.88em}.katex .fontsize-ensurer.reset-size2.size10,.katex .sizing.reset-size2.size10{font-size:3.4566666667em}.katex .fontsize-ensurer.reset-size2.size11,.katex .sizing.reset-size2.size11{font-size:4.1466666667em}.katex .fontsize-ensurer.reset-size3.size1,.katex .sizing.reset-size3.size1{font-size:.7142857143em}.katex .fontsize-ensurer.reset-size3.size2,.katex .sizing.reset-size3.size2{font-size:.8571428571em}.katex .fontsize-ensurer.reset-size3.size3,.katex .sizing.reset-size3.size3{font-size:1em}.katex .fontsize-ensurer.reset-size3.size4,.katex .sizing.reset-size3.size4{font-size:1.1428571429em}.katex .fontsize-ensurer.reset-size3.size5,.katex .sizing.reset-size3.size5{font-size:1.2857142857em}.katex .fontsize-ensurer.reset-size3.size6,.katex .sizing.reset-size3.size6{font-size:1.4285714286em}.katex .fontsize-ensurer.reset-size3.size7,.katex .sizing.reset-size3.size7{font-size:1.7142857143em}.katex .fontsize-ensurer.reset-size3.size8,.katex .sizing.reset-size3.size8{font-size:2.0571428571em}.katex .fontsize-ensurer.reset-size3.size9,.katex .sizing.reset-size3.size9{font-size:2.4685714286em}.katex .fontsize-ensurer.reset-size3.size10,.katex .sizing.reset-size3.size10{font-size:2.9628571429em}.katex .fontsize-ensurer.reset-size3.size11,.katex .sizing.reset-size3.size11{font-size:3.5542857143em}.katex .fontsize-ensurer.reset-size4.size1,.katex .sizing.reset-size4.size1{font-size:.625em}.katex .fontsize-ensurer.reset-size4.size2,.katex .sizing.reset-size4.size2{font-size:.75em}.katex .fontsize-ensurer.reset-size4.size3,.katex .sizing.reset-size4.size3{font-size:.875em}.katex .fontsize-ensurer.reset-size4.size4,.katex .sizing.reset-size4.size4{font-size:1em}.katex .fontsize-ensurer.reset-size4.size5,.katex .sizing.reset-size4.size5{font-size:1.125em}.katex .fontsize-ensurer.reset-size4.size6,.katex .sizing.reset-size4.size6{font-size:1.25em}.katex .fontsize-ensurer.reset-size4.size7,.katex .sizing.reset-size4.size7{font-size:1.5em}.katex .fontsize-ensurer.reset-size4.size8,.katex .sizing.reset-size4.size8{font-size:1.8em}.katex .fontsize-ensurer.reset-size4.size9,.katex .sizing.reset-size4.size9{font-size:2.16em}.katex .fontsize-ensurer.reset-size4.size10,.katex .sizing.reset-size4.size10{font-size:2.5925em}.katex .fontsize-ensurer.reset-size4.size11,.katex .sizing.reset-size4.size11{font-size:3.11em}.katex .fontsize-ensurer.reset-size5.size1,.katex .sizing.reset-size5.size1{font-size:.5555555556em}.katex .fontsize-ensurer.reset-size5.size2,.katex .sizing.reset-size5.size2{font-size:.6666666667em}.katex .fontsize-ensurer.reset-size5.size3,.katex .sizing.reset-size5.size3{font-size:.7777777778em}.katex .fontsize-ensurer.reset-size5.size4,.katex .sizing.reset-size5.size4{font-size:.8888888889em}.katex .fontsize-ensurer.reset-size5.size5,.katex .sizing.reset-size5.size5{font-size:1em}.katex .fontsize-ensurer.reset-size5.size6,.katex .sizing.reset-size5.size6{font-size:1.1111111111em}.katex .fontsize-ensurer.reset-size5.size7,.katex .sizing.reset-size5.size7{font-size:1.3333333333em}.katex .fontsize-ensurer.reset-size5.size8,.katex .sizing.reset-size5.size8{font-size:1.6em}.katex .fontsize-ensurer.reset-size5.size9,.katex .sizing.reset-size5.size9{font-size:1.92em}.katex .fontsize-ensurer.reset-size5.size10,.katex .sizing.reset-size5.size10{font-size:2.3044444444em}.katex .fontsize-ensurer.reset-size5.size11,.katex .sizing.reset-size5.size11{font-size:2.7644444444em}.katex .fontsize-ensurer.reset-size6.size1,.katex .sizing.reset-size6.size1{font-size:.5em}.katex .fontsize-ensurer.reset-size6.size2,.katex .sizing.reset-size6.size2{font-size:.6em}.katex .fontsize-ensurer.reset-size6.size3,.katex .sizing.reset-size6.size3{font-size:.7em}.katex .fontsize-ensurer.reset-size6.size4,.katex .sizing.reset-size6.size4{font-size:.8em}.katex .fontsize-ensurer.reset-size6.size5,.katex .sizing.reset-size6.size5{font-size:.9em}.katex .fontsize-ensurer.reset-size6.size6,.katex .sizing.reset-size6.size6{font-size:1em}.katex .fontsize-ensurer.reset-size6.size7,.katex .sizing.reset-size6.size7{font-size:1.2em}.katex .fontsize-ensurer.reset-size6.size8,.katex .sizing.reset-size6.size8{font-size:1.44em}.katex .fontsize-ensurer.reset-size6.size9,.katex .sizing.reset-size6.size9{font-size:1.728em}.katex .fontsize-ensurer.reset-size6.size10,.katex .sizing.reset-size6.size10{font-size:2.074em}.katex .fontsize-ensurer.reset-size6.size11,.katex .sizing.reset-size6.size11{font-size:2.488em}.katex .fontsize-ensurer.reset-size7.size1,.katex .sizing.reset-size7.size1{font-size:.4166666667em}.katex .fontsize-ensurer.reset-size7.size2,.katex .sizing.reset-size7.size2{font-size:.5em}.katex .fontsize-ensurer.reset-size7.size3,.katex .sizing.reset-size7.size3{font-size:.5833333333em}.katex .fontsize-ensurer.reset-size7.size4,.katex .sizing.reset-size7.size4{font-size:.6666666667em}.katex .fontsize-ensurer.reset-size7.size5,.katex .sizing.reset-size7.size5{font-size:.75em}.katex .fontsize-ensurer.reset-size7.size6,.katex .sizing.reset-size7.size6{font-size:.8333333333em}.katex .fontsize-ensurer.reset-size7.size7,.katex .sizing.reset-size7.size7{font-size:1em}.katex .fontsize-ensurer.reset-size7.size8,.katex .sizing.reset-size7.size8{font-size:1.2em}.katex .fontsize-ensurer.reset-size7.size9,.katex .sizing.reset-size7.size9{font-size:1.44em}.katex .fontsize-ensurer.reset-size7.size10,.katex .sizing.reset-size7.size10{font-size:1.7283333333em}.katex .fontsize-ensurer.reset-size7.size11,.katex .sizing.reset-size7.size11{font-size:2.0733333333em}.katex .fontsize-ensurer.reset-size8.size1,.katex .sizing.reset-size8.size1{font-size:.3472222222em}.katex .fontsize-ensurer.reset-size8.size2,.katex .sizing.reset-size8.size2{font-size:.4166666667em}.katex .fontsize-ensurer.reset-size8.size3,.katex .sizing.reset-size8.size3{font-size:.4861111111em}.katex .fontsize-ensurer.reset-size8.size4,.katex .sizing.reset-size8.size4{font-size:.5555555556em}.katex .fontsize-ensurer.reset-size8.size5,.katex .sizing.reset-size8.size5{font-size:.625em}.katex .fontsize-ensurer.reset-size8.size6,.katex .sizing.reset-size8.size6{font-size:.6944444444em}.katex .fontsize-ensurer.reset-size8.size7,.katex .sizing.reset-size8.size7{font-size:.8333333333em}.katex .fontsize-ensurer.reset-size8.size8,.katex .sizing.reset-size8.size8{font-size:1em}.katex .fontsize-ensurer.reset-size8.size9,.katex .sizing.reset-size8.size9{font-size:1.2em}.katex .fontsize-ensurer.reset-size8.size10,.katex .sizing.reset-size8.size10{font-size:1.4402777778em}.katex .fontsize-ensurer.reset-size8.size11,.katex .sizing.reset-size8.size11{font-size:1.7277777778em}.katex .fontsize-ensurer.reset-size9.size1,.katex .sizing.reset-size9.size1{font-size:.2893518519em}.katex .fontsize-ensurer.reset-size9.size2,.katex .sizing.reset-size9.size2{font-size:.3472222222em}.katex .fontsize-ensurer.reset-size9.size3,.katex .sizing.reset-size9.size3{font-size:.4050925926em}.katex .fontsize-ensurer.reset-size9.size4,.katex .sizing.reset-size9.size4{font-size:.462962963em}.katex .fontsize-ensurer.reset-size9.size5,.katex .sizing.reset-size9.size5{font-size:.5208333333em}.katex .fontsize-ensurer.reset-size9.size6,.katex .sizing.reset-size9.size6{font-size:.5787037037em}.katex .fontsize-ensurer.reset-size9.size7,.katex .sizing.reset-size9.size7{font-size:.6944444444em}.katex .fontsize-ensurer.reset-size9.size8,.katex .sizing.reset-size9.size8{font-size:.8333333333em}.katex .fontsize-ensurer.reset-size9.size9,.katex .sizing.reset-size9.size9{font-size:1em}.katex .fontsize-ensurer.reset-size9.size10,.katex .sizing.reset-size9.size10{font-size:1.2002314815em}.katex .fontsize-ensurer.reset-size9.size11,.katex .sizing.reset-size9.size11{font-size:1.4398148148em}.katex .fontsize-ensurer.reset-size10.size1,.katex .sizing.reset-size10.size1{font-size:.2410800386em}.katex .fontsize-ensurer.reset-size10.size2,.katex .sizing.reset-size10.size2{font-size:.2892960463em}.katex .fontsize-ensurer.reset-size10.size3,.katex .sizing.reset-size10.size3{font-size:.337512054em}.katex .fontsize-ensurer.reset-size10.size4,.katex .sizing.reset-size10.size4{font-size:.3857280617em}.katex .fontsize-ensurer.reset-size10.size5,.katex .sizing.reset-size10.size5{font-size:.4339440694em}.katex .fontsize-ensurer.reset-size10.size6,.katex .sizing.reset-size10.size6{font-size:.4821600771em}.katex .fontsize-ensurer.reset-size10.size7,.katex .sizing.reset-size10.size7{font-size:.5785920926em}.katex .fontsize-ensurer.reset-size10.size8,.katex .sizing.reset-size10.size8{font-size:.6943105111em}.katex .fontsize-ensurer.reset-size10.size9,.katex .sizing.reset-size10.size9{font-size:.8331726133em}.katex .fontsize-ensurer.reset-size10.size10,.katex .sizing.reset-size10.size10{font-size:1em}.katex .fontsize-ensurer.reset-size10.size11,.katex .sizing.reset-size10.size11{font-size:1.1996142719em}.katex .fontsize-ensurer.reset-size11.size1,.katex .sizing.reset-size11.size1{font-size:.2009646302em}.katex .fontsize-ensurer.reset-size11.size2,.katex .sizing.reset-size11.size2{font-size:.2411575563em}.katex .fontsize-ensurer.reset-size11.size3,.katex .sizing.reset-size11.size3{font-size:.2813504823em}.katex .fontsize-ensurer.reset-size11.size4,.katex .sizing.reset-size11.size4{font-size:.3215434084em}.katex .fontsize-ensurer.reset-size11.size5,.katex .sizing.reset-size11.size5{font-size:.3617363344em}.katex .fontsize-ensurer.reset-size11.size6,.katex .sizing.reset-size11.size6{font-size:.4019292605em}.katex .fontsize-ensurer.reset-size11.size7,.katex .sizing.reset-size11.size7{font-size:.4823151125em}.katex .fontsize-ensurer.reset-size11.size8,.katex .sizing.reset-size11.size8{font-size:.578778135em}.katex .fontsize-ensurer.reset-size11.size9,.katex .sizing.reset-size11.size9{font-size:.6945337621em}.katex .fontsize-ensurer.reset-size11.size10,.katex .sizing.reset-size11.size10{font-size:.8336012862em}.katex .fontsize-ensurer.reset-size11.size11,.katex .sizing.reset-size11.size11{font-size:1em}.katex .delimsizing.size1{font-family:KaTeX_Size1}.katex .delimsizing.size2{font-family:KaTeX_Size2}.katex .delimsizing.size3{font-family:KaTeX_Size3}.katex .delimsizing.size4{font-family:KaTeX_Size4}.katex .delimsizing.mult .delim-size1>span{font-family:KaTeX_Size1}.katex .delimsizing.mult .delim-size4>span{font-family:KaTeX_Size4}.katex .nulldelimiter{display:inline-block;width:.12em}.katex .delimcenter,.katex .op-symbol{position:relative}.katex .op-symbol.small-op{font-family:KaTeX_Size1}.katex .op-symbol.large-op{font-family:KaTeX_Size2}.katex .accent>.vlist-t,.katex .op-limits>.vlist-t{text-align:center}.katex .accent .accent-body{position:relative}.katex .accent .accent-body:not(.accent-full){width:0}.katex .overlay{display:block}.katex .mtable .vertical-separator{display:inline-block;min-width:1px}.katex .mtable .arraycolsep{display:inline-block}.katex .mtable .col-align-c>.vlist-t{text-align:center}.katex .mtable .col-align-l>.vlist-t{text-align:left}.katex .mtable .col-align-r>.vlist-t{text-align:right}.katex .svg-align{text-align:left}.katex svg{fill:currentColor;stroke:currentColor;display:block;height:inherit;position:absolute;width:100%}.katex svg path{stroke:none}.katex svg{fill-rule:nonzero;fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1}.katex img{border-style:none;max-height:none;max-width:none;min-height:0;min-width:0}.katex .stretchy{display:block;overflow:hidden;position:relative;width:100%}.katex .stretchy:after,.katex .stretchy:before{content:""}.katex .hide-tail{overflow:hidden;position:relative;width:100%}.katex .halfarrow-left{left:0;overflow:hidden;position:absolute;width:50.2%}.katex .halfarrow-right{overflow:hidden;position:absolute;right:0;width:50.2%}.katex .brace-left{left:0;overflow:hidden;position:absolute;width:25.1%}.katex .brace-center{left:25%;overflow:hidden;position:absolute;width:50%}.katex .brace-right{overflow:hidden;position:absolute;right:0;width:25.1%}.katex .x-arrow-pad{padding:0 .5em}.katex .cd-arrow-pad{padding:0 .55556em 0 .27778em}.katex .mover,.katex .munder,.katex .x-arrow{text-align:center}.katex .boxpad{padding:0 .3em}.katex .fbox,.katex .fcolorbox{border:.04em solid;box-sizing:border-box}.katex .cancel-pad{padding:0 .2em}.katex .cancel-lap{margin-left:-.2em;margin-right:-.2em}.katex .sout{border-bottom-style:solid;border-bottom-width:.08em}.katex .angl{border-right:.049em solid;border-top:.049em solid;box-sizing:border-box;margin-right:.03889em}.katex .anglpad{padding:0 .03889em}.katex .eqn-num:before{content:"(" counter(katexEqnNo) ")";counter-increment:katexEqnNo}.katex .mml-eqn-num:before{content:"(" counter(mmlEqnNo) ")";counter-increment:mmlEqnNo}.katex .mtr-glue{width:50%}.katex .cd-vert-arrow{display:inline-block;position:relative}.katex .cd-label-left{display:inline-block;position:absolute;right:calc(50% + .3em);text-align:left}.katex .cd-label-right{display:inline-block;left:calc(50% + .3em);position:absolute;text-align:right}.katex-display{display:block;margin:1em 0;text-align:center}.katex-display>.katex{display:block;text-align:center;white-space:nowrap}.katex-display>.katex>.katex-html{display:block;position:relative}.katex-display>.katex>.katex-html>.tag{position:absolute;right:0}.katex-display.leqno>.katex>.katex-html>.tag{left:0;right:auto}.katex-display.fleqn>.katex{padding-left:2em;text-align:left}body{counter-reset:katexEqnNo mmlEqnNo}