docs/USER_GUIDE.md

# MLOL User Guide

**MultiDomain LLM Optimisation Lab** — fine-tune, evaluate, and certify open
LLMs from one control panel. Full design: `docs/MASTER_SPEC.md`.

## The pipeline

1. **Tier 1 → Prepare dataset.** Pick a base model, upload CSV/JSON/JSONL/TXT/
   PDF/DOCX. MLOL validates, cleans, deduplicates, converts to chat format, and
   creates an experiment (everything is persisted; restarts lose nothing).
2. **Check routing.** MLOL tells you where this run can execute:
   - **ZeroGPU demo** — models ≤1.5B, ≤5k samples, minutes-long runs, on this Space.
   - **Colab export** — a pinned, self-contained package; free GPU, your account.
   - **HF Jobs** — managed training on paid HF hardware (needs your token/billing).
   Impossible options are disabled with the reason.
3. **Baseline eval** — before training, on a seeded sample with fixed item IDs.
4. **Fine-tune** — LoRA on the routed backend.
5. **Post-eval + compare** — identical items/seed; paired permutation tests
   decide significance; the Trial & Error panel explains disappointing results
   with evidence (dataset too small, overfitting, forgetting…).
6. **Reports tab** — optimisation report + 9-section Model Performance
   Certificate (PDF/CSV/JSON) with a full environment stamp.

## Tier 2 — Domain Foundry (premium)

Pre-configured domains (Finance, Law, Medical, Regulatory, Programming, …) with
curated datasets, benchmarks, and recommended hyperparameters.
Access codes: email **finpy07@gmail.com**.

## AI Research Assistant (bottom-right)

Four modes: **General** (concepts), **Experiment** (diagnoses your run from its
actual logs/metrics), **Hardware** (estimator-grounded advice), **Report**
(explains your certificate). Providers are configurable; Claude/GPT need your
own API key (never stored).

## Honest-numbers policy

Every sampled metric shows n, seed, and a 95% CI. Certificates state that
sampled results are not the full benchmark, and hallucination figures are
labeled estimates. Confidence stars rate how thorough the evaluation was —
never how good the model is.

## Interpreting states

`draft → data-ready → baseline-running → baseline-complete → training-submitted
→ training-running → training-complete → post-evaluation-running → complete`
(plus `failed`/`cancelled`, restartable). The dashboard reconstructs progress
from persisted state after any refresh.