Spaces:

RayMelius
/

StockEx

Sleeping

RayMelius Claude Opus 4.6 commited on Mar 6

Commit

688b587

1 Parent(s): 643d6a4

Restructure fine-tune notebook: Kaggle+Colab resume, checkpoint cleanup

- Move checkpoint detection to beginning for immediate resume visibility
- Add Kaggle dataset download for checkpoint resume across sessions
- Keep only last 3 checkpoints in persistent storage (Kaggle/Drive)
- Remove duplicate code cells, fix section numbering
- Add stockex-clearing-house-llm-fine-tuning.ipynb (Kaggle version)
- Replace ch_trader_finetune.ipynb with unified notebook

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Files changed (2) hide show

notebooks/ch_trader_finetune.ipynb +1192 -247
notebooks/stockex-clearing-house-llm-fine-tuning.ipynb +1290 -0

notebooks/ch_trader_finetune.ipynb CHANGED Viewed

@@ -1,6 +1,4 @@
 {
- "nbformat": 4,
- "nbformat_minor": 5,
  "metadata": {
   "kernelspec": {
    "display_name": "Python 3",
@@ -9,337 +7,1284 @@
   },
   "language_info": {
    "name": "python",
-   "version": "3.10.0"
   },
   "accelerator": "GPU",
   "colab": {
-   "gpuType": "A100",
    "provenance": []
   }
  },
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "title",
-   "metadata": {},
-   "source": "# StockEx Clearing House — LLM Fine-Tuning\n\nFine-tunes a Qwen2.5 Instruct model with QLoRA to act as a clearing house trading agent.\n\n**Auto-selects model size based on available VRAM:**\n| GPU | VRAM | Model |\n|-----|------|-------|\n| T4 (free) | 15 GB | Qwen2.5-7B-Instruct |\n| A100 40 GB | 40 GB | Qwen2.5-14B-Instruct |\n| A100 80 GB | 80 GB | Qwen2.5-32B-Instruct |\n\n**Output model:** `RayMelius/stockex-ch-trader` on HuggingFace Hub\n\n**Required secret:** Add `HF_TOKEN` in Colab → Secrets (🔑 icon in left sidebar)"
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "install",
-   "metadata": {},
    "outputs": [],
-   "source": "# ── Install dependencies ──────────────────────────────────��────────────────────\n# Reinstall bitsandbytes with proper CUDA support (fixes triton.ops error on Colab)\n!pip install -q -U bitsandbytes\n!pip install -q \\\n    \"transformers>=4.46.3\" \\\n    \"peft>=0.13.2\" \\\n    \"trl>=0.12.1\" \\\n    \"datasets>=3.1.0\" \\\n    \"accelerate>=1.1.1\" \\\n    huggingface_hub\nprint(\"Dependencies installed.\")"
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "imports",
-   "metadata": {},
    "outputs": [],
-   "source": [
-    "import os, json, random, torch\n",
-    "from datasets import Dataset\n",
-    "from transformers import (\n",
-    "    AutoTokenizer, AutoModelForCausalLM,\n",
-    "    BitsAndBytesConfig, TrainingArguments,\n",
-    ")\n",
-    "from peft import LoraConfig, get_peft_model, TaskType\n",
-    "from trl import SFTTrainer, SFTConfig\n",
-    "from huggingface_hub import login\n",
-    "\n",
-    "print(f\"CUDA available: {torch.cuda.is_available()}\")\n",
-    "if torch.cuda.is_available():\n",
-    "    print(f\"GPU: {torch.cuda.get_device_name(0)}\")\n",
-    "    print(f\"VRAM: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f} GB\")"
-   ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "config",
-   "metadata": {},
    "outputs": [],
-   "source": "# ── Auto-select model based on available VRAM ─────────────────────────────────\nimport torch\n\nassert torch.cuda.is_available(), \"No GPU found — change runtime to GPU.\"\nvram_gb = torch.cuda.get_device_properties(0).total_memory / 1e9\ngpu_name = torch.cuda.get_device_name(0)\nprint(f\"GPU: {gpu_name}  |  VRAM: {vram_gb:.1f} GB\")\n\nif vram_gb >= 70:\n    BASE_MODEL = \"Qwen/Qwen2.5-32B-Instruct\"\n    BATCH_SIZE, GRAD_ACCUM, LR = 1, 16, 1e-4\nelif vram_gb >= 35:\n    BASE_MODEL = \"Qwen/Qwen2.5-14B-Instruct\"\n    BATCH_SIZE, GRAD_ACCUM, LR = 2, 8,  1e-4\nelse:                                          # T4 / 15 GB\n    BASE_MODEL = \"Qwen/Qwen2.5-7B-Instruct\"\n    BATCH_SIZE, GRAD_ACCUM, LR = 4, 4,  2e-4\n\nprint(f\"Selected model: {BASE_MODEL}\")\n\n# ── Fixed config ───────────────────────────────────────────────────────────────\nOUTPUT_REPO  = \"RayMelius/stockex-ch-trader\"\nOUTPUT_DIR   = \"./stockex-ch-trader\"\nLORA_R       = 16\nLORA_ALPHA   = 32\nLORA_DROPOUT = 0.05\nNUM_EPOCHS   = 3\nMAX_SEQ_LEN  = 512\nDATASET_SIZE = 2500\n\n# ── HuggingFace login ─────────────────────────────────────────────────────────\nimport os\ntry:\n    from google.colab import userdata\n    HF_TOKEN = userdata.get(\"HF_TOKEN\")\n    print(\"HF_TOKEN loaded from Colab Secrets\")\nexcept Exception:\n    HF_TOKEN = os.getenv(\"HF_TOKEN\", \"\")\n\nif not HF_TOKEN:\n    raise ValueError(\"HF_TOKEN not found. Add it in Colab → Secrets (🔑).\")\n\nfrom huggingface_hub import login\nlogin(token=HF_TOKEN)\nprint(\"Logged in to HuggingFace Hub\")"
   },
   {
    "cell_type": "markdown",
-   "id": "dataset-header",
-   "metadata": {},
-   "source": [
-    "## 1. Synthetic Dataset Generation\n",
-    "\n",
-    "Each training example is a realistic clearing house trading scenario:\n",
-    "- Member state: capital, holdings, obligation remaining\n",
-    "- Market: BBO for each security\n",
-    "- Target: a valid JSON trading decision that respects all constraints"
-   ]
   },
   {
-   "cell_type": "markdown",
-   "id": "g7cean6ejyj",
-   "source": "## 1b. Checkpoint Setup (Resumable Training)\n\nCheckpoints persist across session restarts for both **Kaggle** and **Colab**:\n\n| Platform | Restore from | Save to |\n|----------|-------------|---------|\n| **Kaggle** | `/kaggle/input/stockex-ch-checkpoints/` (dataset) | `/kaggle/working/stockex-ch-checkpoints/` (output) |\n| **Colab** | Google Drive `stockex-ch-checkpoints/` | Google Drive `stockex-ch-checkpoints/` |\n\nAdapters are also pushed to HF Hub every `SAVE_STEPS` steps as a backup.",
-   "metadata": {}
   },
   {
-   "cell_type": "code",
-   "id": "egq0dp9csuo",
-   "source": "import shutil, math\nfrom transformers.trainer_utils import get_last_checkpoint\n\n# ── Detect platform ───────────────────────────────────────────────────────────\nON_KAGGLE = os.path.isdir(\"/kaggle\")\nON_COLAB  = not ON_KAGGLE\n\n# ── Platform-specific checkpoint dirs ─────────────────────────────────────────\nDRIVE_CKPT_DIR = None\nKAGGLE_INPUT_CKPT_DIR  = \"/kaggle/input/stockex-ch-checkpoints\"\nKAGGLE_OUTPUT_CKPT_DIR = \"/kaggle/working/stockex-ch-checkpoints\"\n\nif ON_COLAB:\n    try:\n        from google.colab import drive\n        drive.mount(\"/content/drive\", force_remount=False)\n        DRIVE_CKPT_DIR = \"/content/drive/MyDrive/stockex-ch-checkpoints\"\n        os.makedirs(DRIVE_CKPT_DIR, exist_ok=True)\n        print(f\"[Colab] Google Drive mounted. Checkpoints → {DRIVE_CKPT_DIR}\")\n    except Exception:\n        print(\"[Colab] Google Drive not available — checkpoints saved locally only.\")\nelif ON_KAGGLE:\n    os.makedirs(KAGGLE_OUTPUT_CKPT_DIR, exist_ok=True)\n    print(f\"[Kaggle] Checkpoints restore ← {KAGGLE_INPUT_CKPT_DIR}\")\n    print(f\"[Kaggle] Checkpoints save    → {KAGGLE_OUTPUT_CKPT_DIR}\")\n\n# ── Restore checkpoint to local OUTPUT_DIR ────────────────────────────────────\nos.makedirs(OUTPUT_DIR, exist_ok=True)\n\n# Pick the right source directory for restoring\nrestore_dir = None\nif ON_KAGGLE and os.path.isdir(KAGGLE_INPUT_CKPT_DIR):\n    restore_dir = KAGGLE_INPUT_CKPT_DIR\nelif ON_COLAB and DRIVE_CKPT_DIR and os.path.isdir(DRIVE_CKPT_DIR):\n    restore_dir = DRIVE_CKPT_DIR\n\nif restore_dir:\n    remote_ckpt = get_last_checkpoint(restore_dir)\n    if remote_ckpt:\n        local_ckpt_name = os.path.basename(remote_ckpt)\n        local_ckpt_path = os.path.join(OUTPUT_DIR, local_ckpt_name)\n        if not os.path.exists(local_ckpt_path):\n            print(f\"Restoring checkpoint: {remote_ckpt}\")\n            shutil.copytree(remote_ckpt, local_ckpt_path)\n    else:\n        print(f\"No checkpoint found in {restore_dir}\")\n\n# ── Detect latest local checkpoint ────────────────────────────────────────────\nRESUME_FROM = get_last_checkpoint(OUTPUT_DIR)\n\nSAVE_STEPS = 10\n\n# ── Resume summary ─────────────────────────────────────────────────────────────\nif RESUME_FROM:\n    completed_steps = int(os.path.basename(RESUME_FROM).split(\"-\")[-1])\n    train_size   = int(DATASET_SIZE * 0.9)\n    steps_per_epoch = math.ceil(train_size / (BATCH_SIZE * GRAD_ACCUM))\n    total_steps  = steps_per_epoch * NUM_EPOCHS\n    remaining    = max(0, total_steps - completed_steps)\n    pct_done     = 100 * completed_steps / total_steps\n    epoch_done   = completed_steps / steps_per_epoch\n\n    print(\"\\n\" + \"=\" * 55)\n    print(\"  RESUMING FROM CHECKPOINT\")\n    print(\"=\" * 55)\n    print(f\"  Checkpoint   : {os.path.basename(RESUME_FROM)}\")\n    print(f\"  Steps done   : {completed_steps:,} / {total_steps:,}  ({pct_done:.1f}%)\")\n    print(f\"  Steps left   : {remaining:,}\")\n    print(f\"  Epoch        : {epoch_done:.2f} / {NUM_EPOCHS}\")\n    print(f\"  Epochs left  : {NUM_EPOCHS - epoch_done:.2f}\")\n    print(f\"  Steps/epoch  : {steps_per_epoch:,}\")\n    print(f\"  Save every   : {SAVE_STEPS} steps\")\n    print(\"=\" * 55 + \"\\n\")\nelse:\n    train_size      = int(DATASET_SIZE * 0.9)\n    steps_per_epoch = math.ceil(train_size / (BATCH_SIZE * GRAD_ACCUM))\n    total_steps     = steps_per_epoch * NUM_EPOCHS\n    print(\"\\n\" + \"=\" * 55)\n    print(\"  STARTING FRESH\")\n    print(\"=\" * 55)\n    print(f\"  Total steps  : {total_steps:,}\")\n    print(f\"  Steps/epoch  : {steps_per_epoch:,}\")\n    print(f\"  Epochs       : {NUM_EPOCHS}\")\n    print(f\"  Save every   : {SAVE_STEPS} steps\")\n    print(\"=\" * 55 + \"\\n\")",
-   "metadata": {},
-   "execution_count": null,
-   "outputs": []
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "dataset-gen",
-   "metadata": {},
    "outputs": [],
-   "source": "# Securities from shared_data/securities.txt  (symbol, start_price, current_price)\nSECURITIES = [\n    {\"symbol\": \"ALPHA\", \"base\": 5.65},\n    {\"symbol\": \"PEIR\",  \"base\": 8.35},\n    {\"symbol\": \"EXAE\",  \"base\": 6.90},\n    {\"symbol\": \"QUEST\", \"base\": 13.35},\n    {\"symbol\": \"NBG\",   \"base\": 8.00},\n    {\"symbol\": \"EUROB\", \"base\": 3.45},\n    {\"symbol\": \"AEG\",   \"base\": 4.75},\n    {\"symbol\": \"INTKA\", \"base\": 7.35},\n    {\"symbol\": \"AAAK\",  \"base\": 2.75},\n    {\"symbol\": \"ATTIK\", \"base\": 4.90},\n]\n\nSTARTING_CAPITAL = 100_000.0\nDAILY_OBLIGATION = 10\n\n\ndef gen_bbo(base_price: float) -> dict:\n    \"\"\"Generate a realistic bid/ask spread around a base price.\"\"\"\n    drift    = random.uniform(-0.05, 0.05)\n    mid      = round(base_price * (1 + drift), 2)\n    spread   = round(random.choice([0.05, 0.10, 0.15]), 2)\n    best_bid = round(mid - spread / 2, 2)\n    best_ask = round(mid + spread / 2, 2)\n    return {\"best_bid\": best_bid, \"best_ask\": best_ask, \"mid\": mid}\n\n\ndef gen_holdings(bbos: dict) -> list:\n    \"\"\"Randomly generate some holdings for a member.\"\"\"\n    holdings = []\n    n = random.randint(0, 4)\n    for sym in random.sample(list(bbos.keys()), min(n, len(bbos))):\n        qty = random.randint(50, 500)\n        mid = bbos[sym][\"mid\"]\n        avg_cost = round(mid * random.uniform(0.92, 1.08), 2)\n        holdings.append({\"symbol\": sym, \"quantity\": qty, \"avg_cost\": avg_cost})\n    return holdings\n\n\ndef build_prompt(member_id: str, capital: float, holdings: list,\n                 obligation_remaining: int, bbos: dict) -> str:\n    market_lines = [\n        f\"  {sym}: Bid {bbo['best_bid']:.2f} / Ask {bbo['best_ask']:.2f}\"\n        for sym, bbo in sorted(bbos.items())\n    ]\n    holding_lines = (\n        [f\"  {h['symbol']}: {h['quantity']} shares @ avg cost {h['avg_cost']:.2f}\"\n         for h in holdings]\n        if holdings else [\"  None\"]\n    )\n    return (\n        f\"You are simulating clearing house member {member_id} making ONE trading decision.\\n\\n\"\n        f\"Member state:\\n\"\n        f\"  Available capital: EUR {capital:,.2f}\\n\"\n        f\"  Securities obligation remaining today: {obligation_remaining} more to trade\\n\"\n        f\"  Current holdings:\\n\" + \"\\n\".join(holding_lines) + \"\\n\\n\"\n        f\"Current market (Bid/Ask):\\n\" + \"\\n\".join(market_lines) + \"\\n\\n\"\n        f\"Rules:\\n\"\n        f\"- Do not spend more than your available capital\\n\"\n        f\"- Do not sell more shares than you hold\\n\"\n        f\"- If you have no holdings, you must BUY\\n\"\n        f\"- Choose a realistic price close to the BBO mid-price\\n\"\n        f\"- Quantity should be between 10 and 200\\n\\n\"\n        f\"Respond ONLY with valid JSON, no other text:\\n\"\n        f'Example: {{\"symbol\": \"ALPHA\", \"side\": \"BUY\", \"quantity\": 50, \"price\": 5.65}}'\n    )\n\n\ndef gen_decision(capital: float, holdings: list, bbos: dict) -> dict:\n    \"\"\"Generate a rule-valid trading decision for the given state.\"\"\"\n    holdings_value = sum(\n        h[\"quantity\"] * bbos.get(h[\"symbol\"], {}).get(\"mid\", h[\"avg_cost\"])\n        for h in holdings\n    )\n    net_worth = capital + holdings_value\n    holdings_ratio = holdings_value / net_worth if net_worth > 0 else 0\n\n    if not holdings:\n        side = \"BUY\"\n    elif holdings_ratio > 0.6:\n        side = random.choices([\"SELL\", \"BUY\"], weights=[0.7, 0.3])[0]\n    else:\n        side = random.choices([\"BUY\", \"SELL\"], weights=[0.55, 0.45])[0]\n\n    if side == \"BUY\":\n        affordable = [sym for sym, bbo in bbos.items() if 10 * bbo[\"best_ask\"] <= capital]\n        if not affordable:\n            sym = min(bbos, key=lambda s: bbos[s][\"best_ask\"])\n        else:\n            held_syms = [h[\"symbol\"] for h in holdings]\n            weights = [3 if s in held_syms else 1 for s in affordable]\n            sym = random.choices(affordable, weights=weights)[0]\n        ask = bbos[sym][\"best_ask\"]\n        max_qty = min(200, int(capital / ask))\n        qty = random.randint(10, max(10, max_qty))\n        price = round(bbos[sym][\"mid\"] + random.uniform(-0.05, 0.05), 2)\n        price = max(bbos[sym][\"best_bid\"], min(price, ask))\n        return {\"symbol\": sym, \"side\": \"BUY\", \"quantity\": qty, \"price\": round(price, 2)}\n    else:\n        h = random.choice(holdings)\n        sym = h[\"symbol\"]\n        bbo = bbos[sym]\n        qty = random.randint(10, min(200, h[\"quantity\"]))\n        price = round(bbo[\"mid\"] + random.uniform(-0.05, 0.05), 2)\n        price = max(bbo[\"best_bid\"] - 0.05, min(price, bbo[\"best_ask\"]))\n        return {\"symbol\": sym, \"side\": \"SELL\", \"quantity\": qty, \"price\": round(price, 2)}\n\n\ndef generate_dataset(n: int) -> list:\n    examples = []\n    member_ids = [f\"USR{i:02d}\" for i in range(1, 11)]\n    scenarios = [\n        ((80_000, 100_000), (5, 10), \"fresh_member\"),\n        ((50_000, 80_000),  (0, 5),  \"active_member\"),\n        ((20_000, 50_000),  (0, 2),  \"low_capital\"),\n        ((5_000,  20_000),  (0, 10), \"very_low_capital\"),\n        ((90_000, 100_000), (10, 10),\"start_of_day\"),\n    ]\n    for _ in range(n):\n        cap_range, obl_range, _ = random.choice(scenarios)\n        capital    = round(random.uniform(*cap_range), 2)\n        obligation = random.randint(*obl_range)\n        member_id  = random.choice(member_ids)\n        bbos       = {s[\"symbol\"]: gen_bbo(s[\"base\"]) for s in SECURITIES}\n        holdings   = gen_holdings(bbos)\n        holdings_cost = sum(h[\"quantity\"] * h[\"avg_cost\"] for h in holdings)\n        if holdings_cost > STARTING_CAPITAL - capital:\n            scale = (STARTING_CAPITAL - capital) / max(holdings_cost, 1)\n            for h in holdings:\n                h[\"quantity\"] = max(10, int(h[\"quantity\"] * scale))\n        prompt   = build_prompt(member_id, capital, holdings, obligation, bbos)\n        decision = gen_decision(capital, holdings, bbos)\n        examples.append({\"prompt\": prompt, \"completion\": json.dumps(decision)})\n    return examples\n\n\nprint(f\"Generating {DATASET_SIZE} training examples...\")\nraw_data = generate_dataset(DATASET_SIZE)\nprint(f\"Done. Example:\")\nprint(\"PROMPT:\\n\", raw_data[0][\"prompt\"])\nprint(\"\\nCOMPLETION:\", raw_data[0][\"completion\"])"
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "dataset-split",
-   "metadata": {},
    "outputs": [],
-   "source": [
-    "# Train/val split (90/10)\n",
-    "random.shuffle(raw_data)\n",
-    "split = int(len(raw_data) * 0.9)\n",
-    "train_data = raw_data[:split]\n",
-    "val_data   = raw_data[split:]\n",
-    "\n",
-    "train_dataset = Dataset.from_list(train_data)\n",
-    "val_dataset   = Dataset.from_list(val_data)\n",
-    "print(f\"Train: {len(train_dataset)} | Val: {len(val_dataset)}\")"
-   ]
   },
   {
    "cell_type": "markdown",
-   "id": "model-header",
-   "metadata": {},
-   "source": [
-    "## 2. Load Base Model (4-bit QLoRA)"
-   ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "load-tokenizer",
-   "metadata": {},
    "outputs": [],
-   "source": "print(f\"Loading tokenizer: {BASE_MODEL}\")\ntokenizer = AutoTokenizer.from_pretrained(BASE_MODEL, trust_remote_code=True)\ntokenizer.pad_token        = tokenizer.eos_token\ntokenizer.padding_side     = \"right\"\ntokenizer.model_max_length = MAX_SEQ_LEN   # replaces max_seq_length in SFTConfig\nprint(\"Tokenizer loaded\")"
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "format-dataset",
-   "metadata": {},
    "outputs": [],
-   "source": [
-    "SYSTEM_PROMPT = (\n",
-    "    \"You are a StockEx clearing house trading agent. \"\n",
-    "    \"Given a member's financial state and live market data, \"\n",
-    "    \"you output a single valid JSON trading decision that respects all capital and holdings constraints. \"\n",
-    "    \"Never output anything other than the JSON object.\"\n",
-    ")\n",
-    "\n",
-    "\n",
-    "def format_chat(example):\n",
-    "    \"\"\"Apply the model's chat template to produce a training string.\"\"\"\n",
-    "    messages = [\n",
-    "        {\"role\": \"system\",    \"content\": SYSTEM_PROMPT},\n",
-    "        {\"role\": \"user\",      \"content\": example[\"prompt\"]},\n",
-    "        {\"role\": \"assistant\", \"content\": example[\"completion\"]},\n",
-    "    ]\n",
-    "    text = tokenizer.apply_chat_template(\n",
-    "        messages,\n",
-    "        tokenize=False,\n",
-    "        add_generation_prompt=False,\n",
-    "    )\n",
-    "    return {\"text\": text}\n",
-    "\n",
-    "\n",
-    "train_dataset = train_dataset.map(format_chat)\n",
-    "val_dataset   = val_dataset.map(format_chat)\n",
-    "\n",
-    "print(\"Sample formatted text:\")\n",
-    "print(train_dataset[0][\"text\"][:600], \"...\")"
-   ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "load-model",
-   "metadata": {},
    "outputs": [],
-   "source": [
-    "# 4-bit quantization config\n",
-    "bnb_config = BitsAndBytesConfig(\n",
-    "    load_in_4bit=True,\n",
-    "    bnb_4bit_quant_type=\"nf4\",\n",
-    "    bnb_4bit_compute_dtype=torch.bfloat16,\n",
-    "    bnb_4bit_use_double_quant=True,\n",
-    ")\n",
-    "\n",
-    "print(f\"Loading model: {BASE_MODEL} (4-bit)\")\n",
-    "model = AutoModelForCausalLM.from_pretrained(\n",
-    "    BASE_MODEL,\n",
-    "    quantization_config=bnb_config,\n",
-    "    device_map=\"auto\",\n",
-    "    trust_remote_code=True,\n",
-    "    torch_dtype=torch.bfloat16,\n",
-    ")\n",
-    "model.config.use_cache = False\n",
-    "model.config.pretraining_tp = 1\n",
-    "print(f\"Model loaded. Parameters: {model.num_parameters()/1e9:.2f}B\")"
-   ]
   },
   {
    "cell_type": "markdown",
-   "id": "lora-header",
-   "metadata": {},
-   "source": [
-    "## 3. LoRA Configuration"
-   ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "lora-config",
-   "metadata": {},
    "outputs": [],
-   "source": [
-    "lora_config = LoraConfig(\n",
-    "    r=LORA_R,\n",
-    "    lora_alpha=LORA_ALPHA,\n",
-    "    target_modules=[\n",
-    "        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
-    "        \"gate_proj\", \"up_proj\", \"down_proj\",\n",
-    "    ],\n",
-    "    lora_dropout=LORA_DROPOUT,\n",
-    "    bias=\"none\",\n",
-    "    task_type=TaskType.CAUSAL_LM,\n",
-    ")\n",
-    "\n",
-    "trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)\n",
-    "total     = sum(p.numel() for p in model.parameters())\n",
-    "print(f\"Trainable parameters: {trainable/1e6:.1f}M / {total/1e6:.0f}M ({100*trainable/total:.2f}%)\")"
-   ]
   },
   {
    "cell_type": "markdown",
-   "id": "training-header",
-   "metadata": {},
-   "source": [
-    "## 4. Train"
-   ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "train",
-   "metadata": {},
    "outputs": [],
-   "source": "from transformers import TrainerCallback\n\n\nclass CheckpointSyncCallback(TrainerCallback):\n    \"\"\"After every checkpoint: copy to persistent storage and push adapter to HF Hub.\"\"\"\n\n    def on_save(self, args, state, control, **kwargs):\n        ckpt_dir = os.path.join(args.output_dir, f\"checkpoint-{state.global_step}\")\n        if not os.path.isdir(ckpt_dir):\n            return\n\n        # 1. Sync to persistent storage (platform-specific)\n        if ON_COLAB and DRIVE_CKPT_DIR:\n            dest = os.path.join(DRIVE_CKPT_DIR, f\"checkpoint-{state.global_step}\")\n            if not os.path.exists(dest):\n                shutil.copytree(ckpt_dir, dest)\n                print(f\"[Checkpoint] Saved to Drive: {dest}\")\n        elif ON_KAGGLE:\n            dest = os.path.join(KAGGLE_OUTPUT_CKPT_DIR, f\"checkpoint-{state.global_step}\")\n            if not os.path.exists(dest):\n                shutil.copytree(ckpt_dir, dest)\n                print(f\"[Checkpoint] Saved to output: {dest}\")\n\n        # 2. Push adapter to HF Hub (lightweight — only LoRA weights)\n        try:\n            kwargs[\"model\"].push_to_hub(\n                OUTPUT_REPO,\n                commit_message=f\"Checkpoint step {state.global_step} \"\n                               f\"(epoch {state.epoch:.2f})\",\n                token=HF_TOKEN,\n            )\n            print(f\"[Checkpoint] Pushed adapter step {state.global_step} → HF Hub\")\n        except Exception as e:\n            print(f\"[Checkpoint] HF Hub push failed (non-fatal): {e}\")\n\n\nsft_config = SFTConfig(\n    output_dir=OUTPUT_DIR,\n    num_train_epochs=NUM_EPOCHS,\n    per_device_train_batch_size=BATCH_SIZE,\n    per_device_eval_batch_size=BATCH_SIZE,\n    gradient_accumulation_steps=GRAD_ACCUM,\n    gradient_checkpointing=True,\n    optim=\"paged_adamw_32bit\",\n    learning_rate=LR,\n    lr_scheduler_type=\"cosine\",\n    warmup_ratio=0.05,\n    fp16=not torch.cuda.is_bf16_supported(),\n    bf16=torch.cuda.is_bf16_supported(),\n    logging_steps=10,\n    eval_strategy=\"steps\",\n    eval_steps=SAVE_STEPS,\n    save_strategy=\"steps\",\n    save_steps=SAVE_STEPS,\n    save_total_limit=3,          # keep only 3 latest local checkpoints\n    load_best_model_at_end=True,\n    metric_for_best_model=\"eval_loss\",\n    greater_is_better=False,\n    report_to=\"none\",\n    dataset_text_field=\"text\",\n    packing=False,\n)\n\ntrainer = SFTTrainer(\n    model=model,\n    args=sft_config,\n    train_dataset=train_dataset,\n    eval_dataset=val_dataset,\n    peft_config=lora_config,\n    processing_class=tokenizer,\n    callbacks=[CheckpointSyncCallback()],\n)\n\nif RESUME_FROM:\n    print(f\"▶ Resuming from checkpoint: {RESUME_FROM}\")\nelse:\n    print(\"▶ Starting training from scratch\")\n\ntrainer.train(resume_from_checkpoint=RESUME_FROM)\nprint(\"✓ Training complete.\")"
   },
   {
    "cell_type": "markdown",
-   "id": "save-header",
-   "metadata": {},
-   "source": [
-    "## 5. Save & Push to HuggingFace Hub\n",
-    "\n",
-    "Merges LoRA adapters into the base model weights and pushes the full model."
-   ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "save-model",
-   "metadata": {},
    "outputs": [],
-   "source": [
-    "from peft import PeftModel\n",
-    "\n",
-    "# Save best adapter checkpoint locally\n",
-    "trainer.model.save_pretrained(OUTPUT_DIR)\n",
-    "tokenizer.save_pretrained(OUTPUT_DIR)\n",
-    "print(f\"Adapter saved to {OUTPUT_DIR}\")\n",
-    "\n",
-    "# Reload base model in fp16 for merging (can't merge with 4-bit)\n",
-    "print(\"Reloading base model in fp16 for adapter merge...\")\n",
-    "del model\n",
-    "torch.cuda.empty_cache()\n",
-    "\n",
-    "base_model = AutoModelForCausalLM.from_pretrained(\n",
-    "    BASE_MODEL,\n",
-    "    torch_dtype=torch.float16,\n",
-    "    device_map=\"auto\",\n",
-    "    trust_remote_code=True,\n",
-    ")\n",
-    "merged_model = PeftModel.from_pretrained(base_model, OUTPUT_DIR)\n",
-    "merged_model = merged_model.merge_and_unload()\n",
-    "print(\"Adapters merged.\")"
-   ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "push-hub",
-   "metadata": {},
    "outputs": [],
-   "source": "print(f\"Pushing merged model to: {OUTPUT_REPO}\")\nmerged_model.push_to_hub(\n    OUTPUT_REPO,\n    token=HF_TOKEN,\n    commit_message=\"StockEx CH Trader: QLoRA fine-tuned Qwen2.5-32B-Instruct\",\n)\ntokenizer.push_to_hub(\n    OUTPUT_REPO,\n    token=HF_TOKEN,\n    commit_message=\"Tokenizer for StockEx CH Trader (Qwen2.5-32B-Instruct base)\",\n)\nprint(f\"✓ Model pushed to https://huggingface.co/{OUTPUT_REPO}\")"
   },
   {
    "cell_type": "markdown",
-   "id": "test-header",
-   "metadata": {},
-   "source": [
-    "## 6. Inference Test\n",
-    "\n",
-    "Verify the model generates valid JSON trading decisions."
-   ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "inference-test",
-   "metadata": {},
    "outputs": [],
-   "source": "import re\nfrom transformers import pipeline\n\npipe = pipeline(\n    \"text-generation\",\n    model=merged_model,\n    tokenizer=tokenizer,\n    device_map=\"auto\",\n)\n\ntest_cases = [\n    {\n        \"desc\": \"New member, no holdings, must trade\",\n        \"capital\": 100_000.0,\n        \"holdings\": [],\n        \"obligation\": 10,\n    },\n    {\n        \"desc\": \"Experienced member with holdings, low obligation\",\n        \"capital\": 65_000.0,\n        \"holdings\": [\n            {\"symbol\": \"ALPHA\", \"quantity\": 300, \"avg_cost\": 5.60},\n            {\"symbol\": \"QUEST\", \"quantity\": 150, \"avg_cost\": 13.20},\n        ],\n        \"obligation\": 2,\n    },\n    {\n        \"desc\": \"Low capital, large holdings\",\n        \"capital\": 8_000.0,\n        \"holdings\": [\n            {\"symbol\": \"PEIR\",  \"quantity\": 500, \"avg_cost\": 8.30},\n            {\"symbol\": \"NBG\",   \"quantity\": 200, \"avg_cost\": 7.95},\n        ],\n        \"obligation\": 5,\n    },\n]\n\ntest_bbos = {s[\"symbol\"]: gen_bbo(s[\"base\"]) for s in SECURITIES}\n\nprint(\"=\" * 70)\nfor tc in test_cases:\n    print(f\"\\nSCENARIO: {tc['desc']}\")\n    prompt = build_prompt(\"USR01\", tc[\"capital\"], tc[\"holdings\"], tc[\"obligation\"], test_bbos)\n    messages = [\n        {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n        {\"role\": \"user\",   \"content\": prompt},\n    ]\n    output = pipe(\n        messages,\n        max_new_tokens=60,\n        temperature=0.3,\n        do_sample=True,\n        pad_token_id=tokenizer.eos_token_id,\n    )\n    response = output[0][\"generated_text\"][-1][\"content\"].strip()\n    print(f\"RESPONSE: {response}\")\n    try:\n        m = re.search(r\"\\{[^}]+\\}\", response)\n        if m:\n            d = json.loads(m.group())\n            assert d[\"side\"] in (\"BUY\", \"SELL\")\n            assert d[\"symbol\"] in [s[\"symbol\"] for s in SECURITIES]\n            assert d[\"quantity\"] > 0\n            assert d[\"price\"] > 0\n            print(f\"✓ Valid JSON: {d}\")\n        else:\n            print(\"✗ No JSON found in response\")\n    except Exception as e:\n        print(f\"✗ Invalid: {e}\")\n    print(\"-\" * 70)"
   },
   {
    "cell_type": "markdown",
-   "id": "usage-header",
-   "metadata": {},
-   "source": [
-    "## 7. Activate in StockEx\n",
-    "\n",
-    "The clearing house already uses `RayMelius/stockex-ch-trader` as default.\n",
-    "\n",
-    "To switch to this model in a running StockEx instance:\n",
-    "\n",
-    "**HuggingFace Spaces** — add to secrets:\n",
-    "```\n",
-    "HF_MODEL = RayMelius/stockex-ch-trader\n",
-    "HF_TOKEN = <your token>\n",
-    "```\n",
-    "\n",
-    "**Docker Compose** — already set in `docker-compose.yml`:\n",
-    "```yaml\n",
-    "environment:\n",
-    "  - HF_MODEL=RayMelius/stockex-ch-trader\n",
-    "  - HF_TOKEN=<your token>\n",
-    "```\n",
-    "\n",
-    "To use a future CH-specific model later:\n",
-    "```\n",
-    "HF_MODEL = RayMelius/<new-ch-model>\n",
-    "```"
-   ]
   }
  ]
 }

 {
  "metadata": {
   "kernelspec": {
    "display_name": "Python 3",
   },
   "language_info": {
    "name": "python",
+   "version": "3.12.12",
+   "mimetype": "text/x-python",
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "pygments_lexer": "ipython3",
+   "nbconvert_exporter": "python",
+   "file_extension": ".py"
   },
   "accelerator": "GPU",
   "colab": {
+   "gpuType": "T4",
    "provenance": []
+  },
+  "widgets": {
+   "application/vnd.jupyter.widget-state+json": {
+    "7d1727a12c9445f88824a433ec067d05": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HBoxModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HBoxModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HBoxView",
+      "box_style": "",
+      "children": [
+       "IPY_MODEL_ddf4a278a228476aadf8e154bf92c19b",
+       "IPY_MODEL_607775cdd08f4dd4a3a0d50a788dc631",
+       "IPY_MODEL_f6a6a369e6004316827310f3ff71c597"
+      ],
+      "layout": "IPY_MODEL_030fdf253ff141c09913c8e99d81637f"
+     }
+    },
+    "ddf4a278a228476aadf8e154bf92c19b": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_41d79e9429514c37b51282c281978ccb",
+      "placeholder": "",
+      "style": "IPY_MODEL_474c0b3c456b462c9d98a635c09daff1",
+      "value": "Map: 100%"
+     }
+    },
+    "607775cdd08f4dd4a3a0d50a788dc631": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "FloatProgressModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "FloatProgressModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "ProgressView",
+      "bar_style": "success",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_897d25d509bf4d46bdf1e9d6d6ae6caa",
+      "max": 2250,
+      "min": 0,
+      "orientation": "horizontal",
+      "style": "IPY_MODEL_292804d6b77f4de39c24b768a205ca80",
+      "value": 2250
+     }
+    },
+    "f6a6a369e6004316827310f3ff71c597": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_00baa69256d04592bd69f4c41e7fb572",
+      "placeholder": "",
+      "style": "IPY_MODEL_beaa41bfdbac47b1a8ffddf18d49f74d",
+      "value": " 2250/2250 [00:00&lt;00:00, 2300.58 examples/s]"
+     }
+    },
+    "030fdf253ff141c09913c8e99d81637f": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "41d79e9429514c37b51282c281978ccb": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "474c0b3c456b462c9d98a635c09daff1": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    },
+    "897d25d509bf4d46bdf1e9d6d6ae6caa": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "292804d6b77f4de39c24b768a205ca80": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "ProgressStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "ProgressStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "bar_color": null,
+      "description_width": ""
+     }
+    },
+    "00baa69256d04592bd69f4c41e7fb572": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "beaa41bfdbac47b1a8ffddf18d49f74d": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    },
+    "16be669fa1b34fc5929de11272273bbc": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HBoxModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HBoxModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HBoxView",
+      "box_style": "",
+      "children": [
+       "IPY_MODEL_8e40f72fced7455496a0ac8adfcedc1e",
+       "IPY_MODEL_64442da9157d406da770b7752face424",
+       "IPY_MODEL_9522c3c9c20f4bddbeb9106e1eff2b04"
+      ],
+      "layout": "IPY_MODEL_ea1c64cc55ff4948b08c167aad9c02bf"
+     }
+    },
+    "8e40f72fced7455496a0ac8adfcedc1e": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_5cf321f500ca4848bdb76fe8a21fa12f",
+      "placeholder": "",
+      "style": "IPY_MODEL_4ff9d8e7dc5c488189872b74d643d430",
+      "value": "Map: 100%"
+     }
+    },
+    "64442da9157d406da770b7752face424": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "FloatProgressModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "FloatProgressModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "ProgressView",
+      "bar_style": "success",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_9d97e9d42df4490cb49c423e479a105a",
+      "max": 250,
+      "min": 0,
+      "orientation": "horizontal",
+      "style": "IPY_MODEL_36c56cb7232849819c6d032c395acbe4",
+      "value": 250
+     }
+    },
+    "9522c3c9c20f4bddbeb9106e1eff2b04": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_19740a84d8e74481b4c819cb33abd260",
+      "placeholder": "",
+      "style": "IPY_MODEL_77a8e0233da34523b09e7b5f5f0efc68",
+      "value": " 250/250 [00:00&lt;00:00, 3130.33 examples/s]"
+     }
+    },
+    "ea1c64cc55ff4948b08c167aad9c02bf": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "5cf321f500ca4848bdb76fe8a21fa12f": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "4ff9d8e7dc5c488189872b74d643d430": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    },
+    "9d97e9d42df4490cb49c423e479a105a": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "36c56cb7232849819c6d032c395acbe4": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "ProgressStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "ProgressStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "bar_color": null,
+      "description_width": ""
+     }
+    },
+    "19740a84d8e74481b4c819cb33abd260": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "77a8e0233da34523b09e7b5f5f0efc68": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    },
+    "b2f6a30c7e70419c811e0da79bed8224": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HBoxModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HBoxModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HBoxView",
+      "box_style": "",
+      "children": [
+       "IPY_MODEL_aea2b8a51c194dfab14d0870873652b7",
+       "IPY_MODEL_a1efc3457ce34a799f9a59c3cb58d55c",
+       "IPY_MODEL_8b4f2a3e789e42d2ac402f21c1fc15f5"
+      ],
+      "layout": "IPY_MODEL_53d3826130da44b3aba936cad515d67b"
+     }
+    },
+    "aea2b8a51c194dfab14d0870873652b7": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_ac3bf1ae42804678be05203c7d95ddd9",
+      "placeholder": "",
+      "style": "IPY_MODEL_73e084e0709a446cbde83880da8fae2e",
+      "value": "Loading weights: 100%"
+     }
+    },
+    "a1efc3457ce34a799f9a59c3cb58d55c": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "FloatProgressModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "FloatProgressModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "ProgressView",
+      "bar_style": "success",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_2d6124a44ed048198d897c074561cd96",
+      "max": 339,
+      "min": 0,
+      "orientation": "horizontal",
+      "style": "IPY_MODEL_b30c7621e14748f7a7e1f6683fac0674",
+      "value": 339
+     }
+    },
+    "8b4f2a3e789e42d2ac402f21c1fc15f5": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_5969532875ce41b6820a5c57b53908d9",
+      "placeholder": "",
+      "style": "IPY_MODEL_af8771c98d3a4470a6781f1035e0856e",
+      "value": " 339/339 [01:07&lt;00:00, 178.83it/s, Materializing param=model.norm.weight]"
+     }
+    },
+    "53d3826130da44b3aba936cad515d67b": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "ac3bf1ae42804678be05203c7d95ddd9": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "73e084e0709a446cbde83880da8fae2e": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    },
+    "2d6124a44ed048198d897c074561cd96": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "b30c7621e14748f7a7e1f6683fac0674": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "ProgressStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "ProgressStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "bar_color": null,
+      "description_width": ""
+     }
+    },
+    "5969532875ce41b6820a5c57b53908d9": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "af8771c98d3a4470a6781f1035e0856e": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    }
+   }
+  },
+  "kaggle": {
+   "accelerator": "gpu",
+   "dataSources": [
+    {
+     "sourceType": "datasetVersion",
+     "sourceId": 15056802,
+     "datasetId": 9638685,
+     "databundleVersionId": 15937545
+    }
+   ],
+   "dockerImageVersionId": 31287,
+   "isInternetEnabled": true,
+   "language": "python",
+   "sourceType": "notebook",
+   "isGpuEnabled": true
   }
  },
+ "nbformat_minor": 4,
+ "nbformat": 4,
  "cells": [
   {
    "cell_type": "markdown",
+   "source": "# StockEx Clearing House — LLM Fine-Tuning\n\nFine-tunes a Qwen2.5 Instruct model with QLoRA to act as a clearing house trading agent.\n\n**Runs on both Kaggle and Colab.** Auto-selects model size based on available VRAM:\n\n| GPU | VRAM | Model |\n|-----|------|-------|\n| T4 (free) | 15 GB | Qwen2.5-7B-Instruct |\n| A100 40 GB | 40 GB | Qwen2.5-14B-Instruct |\n| A100 80 GB | 80 GB | Qwen2.5-32B-Instruct |\n\n**Resumable training:**\n- **Kaggle**: checkpoints saved/restored from dataset `xabonum/stockex-ch-checkpoints`\n- **Colab**: checkpoints saved/restored from Google Drive\n\n**Output model:** `RayMelius/stockex-ch-trader` on HuggingFace Hub\n\n**Required secret:** Add `HF_TOKEN` in Kaggle Secrets or Colab Secrets (🔑 icon in left sidebar)",
+   "metadata": {
+    "id": "title"
+   }
   },
   {
    "cell_type": "code",
+   "source": "# ── Install dependencies ───────────────────────────────────────────────────────\n# Reinstall bitsandbytes with proper CUDA support (fixes triton.ops error on Colab)\n!pip install -q -U bitsandbytes\n!pip install -q \\\n    \"transformers>=4.46.3\" \\\n    \"peft>=0.13.2\" \\\n    \"trl>=0.12.1\" \\\n    \"datasets>=3.1.0\" \\\n    \"accelerate>=1.1.1\" \\\n    huggingface_hub\nprint(\"Dependencies installed.\")",
+   "metadata": {
+    "id": "install",
+    "outputId": "a564dd39-8172-4745-e00c-c90fc17c6634",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "code",
+   "source": "import os, json, random, torch\nfrom datasets import Dataset\nfrom transformers import (\n    AutoTokenizer, AutoModelForCausalLM,\n    BitsAndBytesConfig, TrainingArguments,\n)\nfrom peft import LoraConfig, get_peft_model, TaskType\nfrom trl import SFTTrainer, SFTConfig\nfrom huggingface_hub import login\n\nprint(f\"CUDA available: {torch.cuda.is_available()}\")\nif torch.cuda.is_available():\n    print(f\"GPU: {torch.cuda.get_device_name(0)}\")\n    print(f\"VRAM: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f} GB\")",
+   "metadata": {
+    "id": "imports",
+    "outputId": "162eb31d-3a84-487c-c5b6-4f25c8d69485",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "code",
+   "source": "# ── Auto-select model based on available VRAM ─────────────────────────────────\nimport torch\n\nassert torch.cuda.is_available(), \"No GPU found — change runtime to GPU.\"\nvram_gb = torch.cuda.get_device_properties(0).total_memory / 1e9\ngpu_name = torch.cuda.get_device_name(0)\nprint(f\"GPU: {gpu_name}  |  VRAM: {vram_gb:.1f} GB\")\n\nif vram_gb >= 70:\n    BASE_MODEL = \"Qwen/Qwen2.5-32B-Instruct\"\n    BATCH_SIZE, GRAD_ACCUM, LR = 1, 16, 1e-4\nelif vram_gb >= 35:\n    BASE_MODEL = \"Qwen/Qwen2.5-14B-Instruct\"\n    BATCH_SIZE, GRAD_ACCUM, LR = 2, 8,  1e-4\nelse:                                          # T4 / 15 GB\n    BASE_MODEL = \"Qwen/Qwen2.5-7B-Instruct\"\n    BATCH_SIZE, GRAD_ACCUM, LR = 4, 4,  2e-4\n\nprint(f\"Selected model: {BASE_MODEL}\")\n\n# ── Fixed config ───────────────────────────────────────────────────────────────\nOUTPUT_REPO  = \"RayMelius/stockex-ch-trader\"\nOUTPUT_DIR   = \"./stockex-ch-trader\"\nLORA_R       = 16\nLORA_ALPHA   = 32\nLORA_DROPOUT = 0.05\nNUM_EPOCHS   = 3\nMAX_SEQ_LEN  = 512\nDATASET_SIZE = 2500\n\n# ── HuggingFace login ─────────────────────────────────────────────────────────\nimport os\n\nHF_TOKEN = None\n\n# Kaggle\ntry:\n    from kaggle_secrets import UserSecretsClient\n    HF_TOKEN = UserSecretsClient().get_secret(\"HF_TOKEN\")\n    print(\"HF_TOKEN loaded from Kaggle Secrets\")\nexcept:\n    pass\n\n# Colab\nif not HF_TOKEN:\n    try:\n        from google.colab import userdata\n        HF_TOKEN = userdata.get(\"HF_TOKEN\")\n        print(\"HF_TOKEN loaded from Colab Secrets\")\n    except:\n        pass\n\n# Env fallback\nif not HF_TOKEN:\n    HF_TOKEN = os.getenv(\"HF_TOKEN\")\n\nif not HF_TOKEN:\n    raise ValueError(\"HF_TOKEN not found.\")\n\nfrom huggingface_hub import login\nlogin(token=HF_TOKEN)",
+   "metadata": {
+    "id": "config",
+    "outputId": "c45e0d98-8928-4459-8449-63d498694d5d",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "markdown",
+   "source": "## 1. Checkpoint Detection (Resumable Training)\n\nAutomatically detects and resumes from the latest checkpoint:\n- **Kaggle**: Downloads checkpoints from dataset `xabonum/stockex-ch-checkpoints`\n- **Colab**: Restores checkpoints from Google Drive\n\nOnly the **last 3 checkpoints** are kept in persistent storage.",
+   "metadata": {
+    "id": "dataset-header"
+   }
   },
   {
+   "cell_type": "code",
+   "source": "import shutil, math, glob\nfrom transformers.trainer_utils import get_last_checkpoint\n\n# ── Detect environment ─────────────────────────────────────────────\nRUNNING_ON_KAGGLE = os.path.exists(\"/kaggle/working\")\nRUNNING_ON_COLAB  = os.path.exists(\"/content\") and not RUNNING_ON_KAGGLE\n\nUSE_DRIVE = False\nDRIVE_CKPT_DIR = None\n\nif RUNNING_ON_COLAB:\n    try:\n        from google.colab import drive\n        drive.mount(\"/content/drive\", force_remount=False)\n        DRIVE_CKPT_DIR = \"/content/drive/MyDrive/stockex-ch-checkpoints\"\n        USE_DRIVE = True\n        print(f\"Colab: checkpoints will use {DRIVE_CKPT_DIR}\")\n    except Exception as e:\n        print(f\"Colab drive mount failed: {e} — saving locally only\")\n\nelif RUNNING_ON_KAGGLE:\n    DRIVE_CKPT_DIR = \"/kaggle/working/stockex-ch-checkpoints\"\n    USE_DRIVE = True\n    os.makedirs(DRIVE_CKPT_DIR, exist_ok=True)\n\n    # Download checkpoint dataset if available\n    try:\n        import subprocess\n        result = subprocess.run(\n            [\"kaggle\", \"datasets\", \"download\", \"-d\", \"xabonum/stockex-ch-checkpoints\",\n             \"-p\", DRIVE_CKPT_DIR, \"--unzip\"],\n            capture_output=True, text=True, timeout=120\n        )\n        if result.returncode == 0:\n            print(f\"Kaggle: downloaded checkpoint dataset -> {DRIVE_CKPT_DIR}\")\n        else:\n            print(f\"Kaggle: no existing checkpoint dataset (starting fresh)\")\n            print(f\"  stderr: {result.stderr.strip()}\")\n    except Exception as e:\n        print(f\"Kaggle: checkpoint download failed: {e}\")\n\n    print(f\"Kaggle: checkpoints will use {DRIVE_CKPT_DIR}\")\n\nelse:\n    print(\"Unknown environment — saving locally only\")\n\n# ── Free GPU memory to prevent OOM on resume ────────────────────────\ntorch.cuda.empty_cache()\ntorch.cuda.reset_peak_memory_stats()\ntorch.cuda.ipc_collect()\nos.environ[\"PYTORCH_CUDA_ALLOC_CONF\"] = \"max_split_size_mb:128,garbage_collection_threshold:0.6\"\n\n# ── Restore checkpoint from persistent storage to local output dir ──\nos.makedirs(OUTPUT_DIR, exist_ok=True)\nif USE_DRIVE:\n    os.makedirs(DRIVE_CKPT_DIR, exist_ok=True)\n    drive_ckpt = get_last_checkpoint(DRIVE_CKPT_DIR)\n    if drive_ckpt:\n        local_ckpt_name = os.path.basename(drive_ckpt)\n        local_ckpt_path = os.path.join(OUTPUT_DIR, local_ckpt_name)\n        if not os.path.exists(local_ckpt_path):\n            print(f\"Restoring checkpoint: {drive_ckpt} -> {local_ckpt_path}\")\n            shutil.copytree(drive_ckpt, local_ckpt_path)\n\n# ── Detect latest local checkpoint ────────────────────────────────\nRESUME_FROM = get_last_checkpoint(OUTPUT_DIR)\nSAVE_STEPS = 10\n\n# ── Resume summary ─────────────────────────────────────────────────\ntrain_size      = int(DATASET_SIZE * 0.9)\nsteps_per_epoch = math.ceil(train_size / (BATCH_SIZE * GRAD_ACCUM))\ntotal_steps     = steps_per_epoch * NUM_EPOCHS\n\nif RESUME_FROM:\n    completed_steps = int(os.path.basename(RESUME_FROM).split(\"-\")[-1])\n    remaining    = max(0, total_steps - completed_steps)\n    pct_done     = 100 * completed_steps / total_steps\n    epoch_done   = completed_steps / steps_per_epoch\n\n    print(\"\\n\" + \"=\" * 55)\n    print(\"  RESUMING FROM CHECKPOINT\")\n    print(\"=\" * 55)\n    print(f\"  Checkpoint   : {os.path.basename(RESUME_FROM)}\")\n    print(f\"  Steps done   : {completed_steps:,} / {total_steps:,}  ({pct_done:.1f}%)\")\n    print(f\"  Steps left   : {remaining:,}\")\n    print(f\"  Epoch        : {epoch_done:.2f} / {NUM_EPOCHS}\")\n    print(f\"  Epochs left  : {NUM_EPOCHS - epoch_done:.2f}\")\n    print(f\"  Steps/epoch  : {steps_per_epoch:,}\")\n    print(f\"  Save every   : {SAVE_STEPS} steps\")\n    print(\"=\" * 55 + \"\\n\")\nelse:\n    print(\"\\n\" + \"=\" * 55)\n    print(\"  STARTING FRESH\")\n    print(\"=\" * 55)\n    print(f\"  Total steps  : {total_steps:,}\")\n    print(f\"  Steps/epoch  : {steps_per_epoch:,}\")\n    print(f\"  Epochs       : {NUM_EPOCHS}\")\n    print(f\"  Save every   : {SAVE_STEPS} steps\")\n    print(\"=\" * 55 + \"\\n\")",
+   "metadata": {
+    "id": "egq0dp9csuo",
+    "outputId": "f098a839-130c-4011-d0ff-a10289004957",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
   },
   {
+   "cell_type": "markdown",
+   "source": "## 2. Synthetic Dataset Generation\n\nEach training example is a realistic clearing house trading scenario:\n- Member state: capital, holdings, obligation remaining\n- Market: BBO for each security\n- Target: a valid JSON trading decision that respects all constraints",
+   "metadata": {
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "code",
+   "source": "# Securities from shared_data/securities.txt  (symbol, start_price, current_price)\nSECURITIES = [\n    {\"symbol\": \"ALPHA\", \"base\": 5.65},\n    {\"symbol\": \"PEIR\",  \"base\": 8.35},\n    {\"symbol\": \"EXAE\",  \"base\": 6.90},\n    {\"symbol\": \"QUEST\", \"base\": 13.35},\n    {\"symbol\": \"NBG\",   \"base\": 8.00},\n    {\"symbol\": \"EUROB\", \"base\": 3.45},\n    {\"symbol\": \"AEG\",   \"base\": 4.75},\n    {\"symbol\": \"INTKA\", \"base\": 7.35},\n    {\"symbol\": \"AAAK\",  \"base\": 2.75},\n    {\"symbol\": \"ATTIK\", \"base\": 4.90},\n]\n\nSTARTING_CAPITAL = 100_000.0\nDAILY_OBLIGATION = 10\n\n\ndef gen_bbo(base_price: float) -> dict:\n    \"\"\"Generate a realistic bid/ask spread around a base price.\"\"\"\n    drift    = random.uniform(-0.05, 0.05)\n    mid      = round(base_price * (1 + drift), 2)\n    spread   = round(random.choice([0.05, 0.10, 0.15]), 2)\n    best_bid = round(mid - spread / 2, 2)\n    best_ask = round(mid + spread / 2, 2)\n    return {\"best_bid\": best_bid, \"best_ask\": best_ask, \"mid\": mid}\n\n\ndef gen_holdings(bbos: dict) -> list:\n    \"\"\"Randomly generate some holdings for a member.\"\"\"\n    holdings = []\n    n = random.randint(0, 4)\n    for sym in random.sample(list(bbos.keys()), min(n, len(bbos))):\n        qty = random.randint(50, 500)\n        mid = bbos[sym][\"mid\"]\n        avg_cost = round(mid * random.uniform(0.92, 1.08), 2)\n        holdings.append({\"symbol\": sym, \"quantity\": qty, \"avg_cost\": avg_cost})\n    return holdings\n\n\ndef build_prompt(member_id: str, capital: float, holdings: list,\n                 obligation_remaining: int, bbos: dict) -> str:\n    market_lines = [\n        f\"  {sym}: Bid {bbo['best_bid']:.2f} / Ask {bbo['best_ask']:.2f}\"\n        for sym, bbo in sorted(bbos.items())\n    ]\n    holding_lines = (\n        [f\"  {h['symbol']}: {h['quantity']} shares @ avg cost {h['avg_cost']:.2f}\"\n         for h in holdings]\n        if holdings else [\"  None\"]\n    )\n    return (\n        f\"You are simulating clearing house member {member_id} making ONE trading decision.\\n\\n\"\n        f\"Member state:\\n\"\n        f\"  Available capital: EUR {capital:,.2f}\\n\"\n        f\"  Securities obligation remaining today: {obligation_remaining} more to trade\\n\"\n        f\"  Current holdings:\\n\" + \"\\n\".join(holding_lines) + \"\\n\\n\"\n        f\"Current market (Bid/Ask):\\n\" + \"\\n\".join(market_lines) + \"\\n\\n\"\n        f\"Rules:\\n\"\n        f\"- Do not spend more than your available capital\\n\"\n        f\"- Do not sell more shares than you hold\\n\"\n        f\"- If you have no holdings, you must BUY\\n\"\n        f\"- Choose a realistic price close to the BBO mid-price\\n\"\n        f\"- Quantity should be between 10 and 200\\n\\n\"\n        f\"Respond ONLY with valid JSON, no other text:\\n\"\n        f'Example: {{\"symbol\": \"ALPHA\", \"side\": \"BUY\", \"quantity\": 50, \"price\": 5.65}}'\n    )\n\n\ndef gen_decision(capital: float, holdings: list, bbos: dict) -> dict:\n    \"\"\"Generate a rule-valid trading decision for the given state.\"\"\"\n    holdings_value = sum(\n        h[\"quantity\"] * bbos.get(h[\"symbol\"], {}).get(\"mid\", h[\"avg_cost\"])\n        for h in holdings\n    )\n    net_worth = capital + holdings_value\n    holdings_ratio = holdings_value / net_worth if net_worth > 0 else 0\n\n    if not holdings:\n        side = \"BUY\"\n    elif holdings_ratio > 0.6:\n        side = random.choices([\"SELL\", \"BUY\"], weights=[0.7, 0.3])[0]\n    else:\n        side = random.choices([\"BUY\", \"SELL\"], weights=[0.55, 0.45])[0]\n\n    if side == \"BUY\":\n        affordable = [sym for sym, bbo in bbos.items() if 10 * bbo[\"best_ask\"] <= capital]\n        if not affordable:\n            sym = min(bbos, key=lambda s: bbos[s][\"best_ask\"])\n        else:\n            held_syms = [h[\"symbol\"] for h in holdings]\n            weights = [3 if s in held_syms else 1 for s in affordable]\n            sym = random.choices(affordable, weights=weights)[0]\n        ask = bbos[sym][\"best_ask\"]\n        max_qty = min(200, int(capital / ask))\n        qty = random.randint(10, max(10, max_qty))\n        price = round(bbos[sym][\"mid\"] + random.uniform(-0.05, 0.05), 2)\n        price = max(bbos[sym][\"best_bid\"], min(price, ask))\n        return {\"symbol\": sym, \"side\": \"BUY\", \"quantity\": qty, \"price\": round(price, 2)}\n    else:\n        h = random.choice(holdings)\n        sym = h[\"symbol\"]\n        bbo = bbos[sym]\n        qty = random.randint(10, min(200, h[\"quantity\"]))\n        price = round(bbo[\"mid\"] + random.uniform(-0.05, 0.05), 2)\n        price = max(bbo[\"best_bid\"] - 0.05, min(price, bbo[\"best_ask\"]))\n        return {\"symbol\": sym, \"side\": \"SELL\", \"quantity\": qty, \"price\": round(price, 2)}\n\n\ndef generate_dataset(n: int) -> list:\n    examples = []\n    member_ids = [f\"USR{i:02d}\" for i in range(1, 11)]\n    scenarios = [\n        ((80_000, 100_000), (5, 10), \"fresh_member\"),\n        ((50_000, 80_000),  (0, 5),  \"active_member\"),\n        ((20_000, 50_000),  (0, 2),  \"low_capital\"),\n        ((5_000,  20_000),  (0, 10), \"very_low_capital\"),\n        ((90_000, 100_000), (10, 10),\"start_of_day\"),\n    ]\n    for _ in range(n):\n        cap_range, obl_range, _ = random.choice(scenarios)\n        capital    = round(random.uniform(*cap_range), 2)\n        obligation = random.randint(*obl_range)\n        member_id  = random.choice(member_ids)\n        bbos       = {s[\"symbol\"]: gen_bbo(s[\"base\"]) for s in SECURITIES}\n        holdings   = gen_holdings(bbos)\n        holdings_cost = sum(h[\"quantity\"] * h[\"avg_cost\"] for h in holdings)\n        if holdings_cost > STARTING_CAPITAL - capital:\n            scale = (STARTING_CAPITAL - capital) / max(holdings_cost, 1)\n            for h in holdings:\n                h[\"quantity\"] = max(10, int(h[\"quantity\"] * scale))\n        prompt   = build_prompt(member_id, capital, holdings, obligation, bbos)\n        decision = gen_decision(capital, holdings, bbos)\n        examples.append({\"prompt\": prompt, \"completion\": json.dumps(decision)})\n    return examples\n\n\nprint(f\"Generating {DATASET_SIZE} training examples...\")\nraw_data = generate_dataset(DATASET_SIZE)\nprint(f\"Done. Example:\")\nprint(\"PROMPT:\\n\", raw_data[0][\"prompt\"])\nprint(\"\\nCOMPLETION:\", raw_data[0][\"completion\"])",
+   "metadata": {
+    "id": "dataset-gen",
+    "outputId": "6b6ae157-9c91-42d5-a556-049f122bf7f1",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "code",
+   "source": "# Train/val split (90/10)\nrandom.shuffle(raw_data)\nsplit = int(len(raw_data) * 0.9)\ntrain_data = raw_data[:split]\nval_data   = raw_data[split:]\n\ntrain_dataset = Dataset.from_list(train_data)\nval_dataset   = Dataset.from_list(val_data)\nprint(f\"Train: {len(train_dataset)} | Val: {len(val_dataset)}\")",
+   "metadata": {
+    "id": "dataset-split",
+    "outputId": "3246d564-c102-4395-8fbc-c3f7979130fb",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "markdown",
+   "source": "## 3. Load Base Model (4-bit QLoRA)",
+   "metadata": {
+    "id": "model-header"
+   }
   },
   {
    "cell_type": "code",
+   "source": "print(f\"Loading tokenizer: {BASE_MODEL}\")\ntokenizer = AutoTokenizer.from_pretrained(BASE_MODEL, trust_remote_code=True)\ntokenizer.pad_token        = tokenizer.eos_token\ntokenizer.padding_side     = \"right\"\ntokenizer.model_max_length = MAX_SEQ_LEN   # replaces max_seq_length in SFTConfig\nprint(\"Tokenizer loaded\")",
+   "metadata": {
+    "id": "load-tokenizer",
+    "outputId": "a5008937-66ed-4028-a215-62e458cfc8dd",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "code",
+   "source": "SYSTEM_PROMPT = (\n    \"You are a StockEx clearing house trading agent. \"\n    \"Given a member's financial state and live market data, \"\n    \"you output a single valid JSON trading decision that respects all capital and holdings constraints. \"\n    \"Never output anything other than the JSON object.\"\n)\n\n\ndef format_chat(example):\n    \"\"\"Apply the model's chat template to produce a training string.\"\"\"\n    messages = [\n        {\"role\": \"system\",    \"content\": SYSTEM_PROMPT},\n        {\"role\": \"user\",      \"content\": example[\"prompt\"]},\n        {\"role\": \"assistant\", \"content\": example[\"completion\"]},\n    ]\n    text = tokenizer.apply_chat_template(\n        messages,\n        tokenize=False,\n        add_generation_prompt=False,\n    )\n    return {\"text\": text}\n\n\ntrain_dataset = train_dataset.map(format_chat)\nval_dataset   = val_dataset.map(format_chat)\n\nprint(\"Sample formatted text:\")\nprint(train_dataset[0][\"text\"][:600], \"...\")",
+   "metadata": {
+    "id": "format-dataset",
+    "outputId": "ea4f1fa8-bf2b-4360-c426-28e5255dbbd3",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "code",
+   "source": "# 4-bit quantization config\nbnb_config = BitsAndBytesConfig(\n    load_in_4bit=True,\n    bnb_4bit_quant_type=\"nf4\",\n    bnb_4bit_compute_dtype=torch.bfloat16,\n    bnb_4bit_use_double_quant=True,\n)\n\nprint(f\"Loading model: {BASE_MODEL} (4-bit)\")\nmodel = AutoModelForCausalLM.from_pretrained(\n    BASE_MODEL,\n    quantization_config=bnb_config,\n    device_map=\"auto\",\n    trust_remote_code=True,\n    dtype=torch.bfloat16,\n)\nmodel.config.use_cache = False\nmodel.config.pretraining_tp = 1\nprint(f\"Model loaded. Parameters: {model.num_parameters()/1e9:.2f}B\")\n\nfrom peft import prepare_model_for_kbit_training\n\nmodel = prepare_model_for_kbit_training(model)",
+   "metadata": {
+    "id": "load-model",
+    "outputId": "7f46083d-c830-45f3-e887-fbe9cf30b4b2",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "markdown",
+   "source": "## 3b. LoRA Configuration",
+   "metadata": {
+    "id": "lora-header"
+   }
   },
   {
    "cell_type": "code",
+   "source": "lora_config = LoraConfig(\n    r=LORA_R,\n    lora_alpha=LORA_ALPHA,\n    target_modules=[\n        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n        \"gate_proj\", \"up_proj\", \"down_proj\",\n    ],\n    lora_dropout=LORA_DROPOUT,\n    bias=\"none\",\n    task_type=TaskType.CAUSAL_LM,\n)\n\ntrainable = sum(p.numel() for p in model.parameters() if p.requires_grad)\ntotal     = sum(p.numel() for p in model.parameters())\nprint(f\"Trainable parameters: {trainable/1e6:.1f}M / {total/1e6:.0f}M ({100*trainable/total:.2f}%)\")",
+   "metadata": {
+    "id": "lora-config",
+    "outputId": "8a2688ba-288b-4149-ff37-c8d0ee11f77f",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "markdown",
+   "source": "## 4. Train",
+   "metadata": {
+    "id": "training-header"
+   }
   },
   {
    "cell_type": "code",
+   "source": "from transformers import TrainerCallback\nimport shutil, glob\n\nKAGGLE_DATASET_ID = \"xabonum/stockex-ch-checkpoints\"\nMAX_CHECKPOINTS_PERSISTENT = 3  # keep only last 3 in Kaggle dataset / Drive\n\n# ── Create Kaggle dataset metadata if needed ────────────────────────\nif USE_DRIVE and DRIVE_CKPT_DIR:\n    metadata_path = os.path.join(DRIVE_CKPT_DIR, \"dataset-metadata.json\")\n    if not os.path.exists(metadata_path):\n        metadata = {\n            \"title\": \"stockex-ch-checkpoints\",\n            \"id\": KAGGLE_DATASET_ID,\n            \"licenses\": [{\"name\": \"CC0-1.0\"}]\n        }\n        with open(metadata_path, \"w\") as f:\n            json.dump(metadata, f, indent=2)\n        print(\"Created dataset-metadata.json\")\n\n\ndef cleanup_old_checkpoints(ckpt_dir, keep=MAX_CHECKPOINTS_PERSISTENT):\n    \"\"\"Keep only the last `keep` checkpoints in persistent storage.\"\"\"\n    ckpts = sorted(\n        glob.glob(os.path.join(ckpt_dir, \"checkpoint-*\")),\n        key=lambda p: int(os.path.basename(p).split(\"-\")[-1])\n    )\n    while len(ckpts) > keep:\n        old = ckpts.pop(0)\n        shutil.rmtree(old)\n        print(f\"[Checkpoint] Removed old: {os.path.basename(old)}\")\n\n\nclass CheckpointSyncCallback(TrainerCallback):\n    \"\"\"Copy checkpoint to persistent storage, push LoRA to HF Hub, upload to Kaggle dataset.\"\"\"\n\n    def on_save(self, args, state, control, **kwargs):\n        ckpt_dir = os.path.join(args.output_dir, f\"checkpoint-{state.global_step}\")\n        if not os.path.isdir(ckpt_dir):\n            return\n\n        # 1. Save to persistent folder (Drive / Kaggle working dir)\n        if USE_DRIVE and DRIVE_CKPT_DIR:\n            dest = os.path.join(DRIVE_CKPT_DIR, f\"checkpoint-{state.global_step}\")\n            os.makedirs(DRIVE_CKPT_DIR, exist_ok=True)\n            try:\n                shutil.copytree(ckpt_dir, dest, dirs_exist_ok=True)\n                print(f\"[Checkpoint] Saved -> {dest}\")\n            except Exception as e:\n                print(f\"[Checkpoint] Copy failed: {e}\")\n\n            # Cleanup: keep only last 3 checkpoints in persistent storage\n            try:\n                cleanup_old_checkpoints(DRIVE_CKPT_DIR)\n            except Exception as e:\n                print(f\"[Checkpoint] Cleanup failed: {e}\")\n\n        # 2. Push LoRA adapter to HF Hub\n        try:\n            kwargs[\"model\"].push_to_hub(\n                OUTPUT_REPO,\n                commit_message=f\"Checkpoint step {state.global_step} (epoch {state.epoch:.2f})\",\n                token=HF_TOKEN,\n            )\n            print(f\"[Checkpoint] Pushed step {state.global_step} -> HF Hub\")\n        except Exception as e:\n            print(f\"[Checkpoint] HF push failed: {e}\")\n\n        # 3. Upload to Kaggle dataset (Kaggle only)\n        if RUNNING_ON_KAGGLE and USE_DRIVE:\n            try:\n                os.system(\n                    f\"kaggle datasets version -p {DRIVE_CKPT_DIR} \"\n                    f\"-m 'Checkpoint step {state.global_step}' --dir-mode zip\"\n                )\n                print(f\"[Checkpoint] Kaggle dataset updated for step {state.global_step}\")\n            except Exception as e:\n                print(f\"[Checkpoint] Kaggle dataset update failed: {e}\")\n\n\nsft_config = SFTConfig(\n    output_dir=OUTPUT_DIR,\n    num_train_epochs=NUM_EPOCHS,\n    per_device_train_batch_size=BATCH_SIZE,\n    per_device_eval_batch_size=BATCH_SIZE,\n    gradient_accumulation_steps=GRAD_ACCUM,\n    gradient_checkpointing=True,\n    optim=\"paged_adamw_8bit\",\n    learning_rate=LR,\n    lr_scheduler_type=\"cosine\",\n    warmup_ratio=0.05,\n    fp16=not torch.cuda.is_bf16_supported(),\n    bf16=torch.cuda.is_bf16_supported(),\n    logging_steps=10,\n    eval_strategy=\"steps\",\n    eval_steps=SAVE_STEPS,\n    save_strategy=\"steps\",\n    save_steps=SAVE_STEPS,\n    save_total_limit=3,\n    load_best_model_at_end=True,\n    metric_for_best_model=\"eval_loss\",\n    greater_is_better=False,\n    report_to=\"none\",\n    dataset_text_field=\"text\",\n    packing=False,\n)\n\ntrainer = SFTTrainer(\n    model=model,\n    args=sft_config,\n    train_dataset=train_dataset,\n    eval_dataset=val_dataset,\n    peft_config=lora_config,\n    processing_class=tokenizer,\n    callbacks=[CheckpointSyncCallback()],\n)\n\nif RESUME_FROM:\n    print(f\"Resuming from checkpoint: {RESUME_FROM}\")\nelse:\n    print(f\"Starting training from scratch. Save every {SAVE_STEPS} steps.\")\n\ntrainer.train(resume_from_checkpoint=RESUME_FROM)\nprint(\"Training complete.\")",
+   "metadata": {
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "markdown",
+   "source": "## 5. Save & Push to HuggingFace Hub\n\nMerges LoRA adapters into the base model weights and pushes the full model.",
+   "metadata": {
+    "id": "save-header"
+   }
   },
   {
    "cell_type": "code",
+   "source": "from peft import PeftModel\n\n# Save best adapter checkpoint locally\ntrainer.model.save_pretrained(OUTPUT_DIR)\ntokenizer.save_pretrained(OUTPUT_DIR)\nprint(f\"Adapter saved to {OUTPUT_DIR}\")\n\n# Reload base model in fp16 for merging (can't merge with 4-bit)\nprint(\"Reloading base model in fp16 for adapter merge...\")\ndel model\ntorch.cuda.empty_cache()\n\nbase_model = AutoModelForCausalLM.from_pretrained(\n    BASE_MODEL,\n    torch_dtype=torch.float16,\n    device_map=\"auto\",\n    trust_remote_code=True,\n)\nmerged_model = PeftModel.from_pretrained(base_model, OUTPUT_DIR)\nmerged_model = merged_model.merge_and_unload()\nprint(\"Adapters merged.\")",
+   "metadata": {
+    "id": "save-model",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "code",
+   "source": "print(f\"Pushing merged model to: {OUTPUT_REPO}\")\nmerged_model.push_to_hub(\n    OUTPUT_REPO,\n    token=HF_TOKEN,\n    commit_message=\"StockEx CH Trader: QLoRA fine-tuned Qwen2.5-32B-Instruct\",\n)\ntokenizer.push_to_hub(\n    OUTPUT_REPO,\n    token=HF_TOKEN,\n    commit_message=\"Tokenizer for StockEx CH Trader (Qwen2.5-32B-Instruct base)\",\n)\nprint(f\"✓ Model pushed to https://huggingface.co/{OUTPUT_REPO}\")",
+   "metadata": {
+    "id": "push-hub",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "markdown",
+   "source": "## 6. Inference Test\n\nVerify the model generates valid JSON trading decisions.",
+   "metadata": {
+    "id": "test-header"
+   }
   },
   {
    "cell_type": "code",
+   "source": "import re\nfrom transformers import pipeline\n\npipe = pipeline(\n    \"text-generation\",\n    model=merged_model,\n    tokenizer=tokenizer,\n    device_map=\"auto\",\n)\n\ntest_cases = [\n    {\n        \"desc\": \"New member, no holdings, must trade\",\n        \"capital\": 100_000.0,\n        \"holdings\": [],\n        \"obligation\": 10,\n    },\n    {\n        \"desc\": \"Experienced member with holdings, low obligation\",\n        \"capital\": 65_000.0,\n        \"holdings\": [\n            {\"symbol\": \"ALPHA\", \"quantity\": 300, \"avg_cost\": 5.60},\n            {\"symbol\": \"QUEST\", \"quantity\": 150, \"avg_cost\": 13.20},\n        ],\n        \"obligation\": 2,\n    },\n    {\n        \"desc\": \"Low capital, large holdings\",\n        \"capital\": 8_000.0,\n        \"holdings\": [\n            {\"symbol\": \"PEIR\",  \"quantity\": 500, \"avg_cost\": 8.30},\n            {\"symbol\": \"NBG\",   \"quantity\": 200, \"avg_cost\": 7.95},\n        ],\n        \"obligation\": 5,\n    },\n]\n\ntest_bbos = {s[\"symbol\"]: gen_bbo(s[\"base\"]) for s in SECURITIES}\n\nprint(\"=\" * 70)\nfor tc in test_cases:\n    print(f\"\\nSCENARIO: {tc['desc']}\")\n    prompt = build_prompt(\"USR01\", tc[\"capital\"], tc[\"holdings\"], tc[\"obligation\"], test_bbos)\n    messages = [\n        {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n        {\"role\": \"user\",   \"content\": prompt},\n    ]\n    output = pipe(\n        messages,\n        max_new_tokens=60,\n        temperature=0.3,\n        do_sample=True,\n        pad_token_id=tokenizer.eos_token_id,\n    )\n    response = output[0][\"generated_text\"][-1][\"content\"].strip()\n    print(f\"RESPONSE: {response}\")\n    try:\n        m = re.search(r\"\\{[^}]+\\}\", response)\n        if m:\n            d = json.loads(m.group())\n            assert d[\"side\"] in (\"BUY\", \"SELL\")\n            assert d[\"symbol\"] in [s[\"symbol\"] for s in SECURITIES]\n            assert d[\"quantity\"] > 0\n            assert d[\"price\"] > 0\n            print(f\"✓ Valid JSON: {d}\")\n        else:\n            print(\"✗ No JSON found in response\")\n    except Exception as e:\n        print(f\"✗ Invalid: {e}\")\n    print(\"-\" * 70)",
+   "metadata": {
+    "id": "inference-test",
+    "trusted": true
+   },
    "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "markdown",
+   "source": "## 7. Activate in StockEx\n\nThe clearing house already uses `RayMelius/stockex-ch-trader` as default.\n\nTo switch to this model in a running StockEx instance:\n\n**HuggingFace Spaces** — add to secrets:\n```\nHF_MODEL = RayMelius/stockex-ch-trader\nHF_TOKEN = <your token>\n```\n\n**Docker Compose** — already set in `docker-compose.yml`:\n```yaml\nenvironment:\n  - HF_MODEL=RayMelius/stockex-ch-trader\n  - HF_TOKEN=<your token>\n```",
+   "metadata": {
+    "id": "usage-header"
+   }
   }
  ]
 }

notebooks/stockex-clearing-house-llm-fine-tuning.ipynb ADDED Viewed

	@@ -0,0 +1,1290 @@

+{
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.12.12",
+   "mimetype": "text/x-python",
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "pygments_lexer": "ipython3",
+   "nbconvert_exporter": "python",
+   "file_extension": ".py"
+  },
+  "accelerator": "GPU",
+  "colab": {
+   "gpuType": "T4",
+   "provenance": []
+  },
+  "widgets": {
+   "application/vnd.jupyter.widget-state+json": {
+    "7d1727a12c9445f88824a433ec067d05": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HBoxModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HBoxModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HBoxView",
+      "box_style": "",
+      "children": [
+       "IPY_MODEL_ddf4a278a228476aadf8e154bf92c19b",
+       "IPY_MODEL_607775cdd08f4dd4a3a0d50a788dc631",
+       "IPY_MODEL_f6a6a369e6004316827310f3ff71c597"
+      ],
+      "layout": "IPY_MODEL_030fdf253ff141c09913c8e99d81637f"
+     }
+    },
+    "ddf4a278a228476aadf8e154bf92c19b": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_41d79e9429514c37b51282c281978ccb",
+      "placeholder": "",
+      "style": "IPY_MODEL_474c0b3c456b462c9d98a635c09daff1",
+      "value": "Map: 100%"
+     }
+    },
+    "607775cdd08f4dd4a3a0d50a788dc631": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "FloatProgressModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "FloatProgressModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "ProgressView",
+      "bar_style": "success",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_897d25d509bf4d46bdf1e9d6d6ae6caa",
+      "max": 2250,
+      "min": 0,
+      "orientation": "horizontal",
+      "style": "IPY_MODEL_292804d6b77f4de39c24b768a205ca80",
+      "value": 2250
+     }
+    },
+    "f6a6a369e6004316827310f3ff71c597": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_00baa69256d04592bd69f4c41e7fb572",
+      "placeholder": "",
+      "style": "IPY_MODEL_beaa41bfdbac47b1a8ffddf18d49f74d",
+      "value": " 2250/2250 [00:00&lt;00:00, 2300.58 examples/s]"
+     }
+    },
+    "030fdf253ff141c09913c8e99d81637f": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "41d79e9429514c37b51282c281978ccb": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "474c0b3c456b462c9d98a635c09daff1": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    },
+    "897d25d509bf4d46bdf1e9d6d6ae6caa": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "292804d6b77f4de39c24b768a205ca80": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "ProgressStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "ProgressStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "bar_color": null,
+      "description_width": ""
+     }
+    },
+    "00baa69256d04592bd69f4c41e7fb572": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "beaa41bfdbac47b1a8ffddf18d49f74d": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    },
+    "16be669fa1b34fc5929de11272273bbc": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HBoxModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HBoxModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HBoxView",
+      "box_style": "",
+      "children": [
+       "IPY_MODEL_8e40f72fced7455496a0ac8adfcedc1e",
+       "IPY_MODEL_64442da9157d406da770b7752face424",
+       "IPY_MODEL_9522c3c9c20f4bddbeb9106e1eff2b04"
+      ],
+      "layout": "IPY_MODEL_ea1c64cc55ff4948b08c167aad9c02bf"
+     }
+    },
+    "8e40f72fced7455496a0ac8adfcedc1e": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_5cf321f500ca4848bdb76fe8a21fa12f",
+      "placeholder": "",
+      "style": "IPY_MODEL_4ff9d8e7dc5c488189872b74d643d430",
+      "value": "Map: 100%"
+     }
+    },
+    "64442da9157d406da770b7752face424": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "FloatProgressModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "FloatProgressModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "ProgressView",
+      "bar_style": "success",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_9d97e9d42df4490cb49c423e479a105a",
+      "max": 250,
+      "min": 0,
+      "orientation": "horizontal",
+      "style": "IPY_MODEL_36c56cb7232849819c6d032c395acbe4",
+      "value": 250
+     }
+    },
+    "9522c3c9c20f4bddbeb9106e1eff2b04": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_19740a84d8e74481b4c819cb33abd260",
+      "placeholder": "",
+      "style": "IPY_MODEL_77a8e0233da34523b09e7b5f5f0efc68",
+      "value": " 250/250 [00:00&lt;00:00, 3130.33 examples/s]"
+     }
+    },
+    "ea1c64cc55ff4948b08c167aad9c02bf": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "5cf321f500ca4848bdb76fe8a21fa12f": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "4ff9d8e7dc5c488189872b74d643d430": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    },
+    "9d97e9d42df4490cb49c423e479a105a": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "36c56cb7232849819c6d032c395acbe4": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "ProgressStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "ProgressStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "bar_color": null,
+      "description_width": ""
+     }
+    },
+    "19740a84d8e74481b4c819cb33abd260": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "77a8e0233da34523b09e7b5f5f0efc68": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    },
+    "b2f6a30c7e70419c811e0da79bed8224": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HBoxModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HBoxModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HBoxView",
+      "box_style": "",
+      "children": [
+       "IPY_MODEL_aea2b8a51c194dfab14d0870873652b7",
+       "IPY_MODEL_a1efc3457ce34a799f9a59c3cb58d55c",
+       "IPY_MODEL_8b4f2a3e789e42d2ac402f21c1fc15f5"
+      ],
+      "layout": "IPY_MODEL_53d3826130da44b3aba936cad515d67b"
+     }
+    },
+    "aea2b8a51c194dfab14d0870873652b7": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_ac3bf1ae42804678be05203c7d95ddd9",
+      "placeholder": "",
+      "style": "IPY_MODEL_73e084e0709a446cbde83880da8fae2e",
+      "value": "Loading weights: 100%"
+     }
+    },
+    "a1efc3457ce34a799f9a59c3cb58d55c": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "FloatProgressModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "FloatProgressModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "ProgressView",
+      "bar_style": "success",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_2d6124a44ed048198d897c074561cd96",
+      "max": 339,
+      "min": 0,
+      "orientation": "horizontal",
+      "style": "IPY_MODEL_b30c7621e14748f7a7e1f6683fac0674",
+      "value": 339
+     }
+    },
+    "8b4f2a3e789e42d2ac402f21c1fc15f5": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "HTMLModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_dom_classes": [],
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "HTMLModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/controls",
+      "_view_module_version": "1.5.0",
+      "_view_name": "HTMLView",
+      "description": "",
+      "description_tooltip": null,
+      "layout": "IPY_MODEL_5969532875ce41b6820a5c57b53908d9",
+      "placeholder": "",
+      "style": "IPY_MODEL_af8771c98d3a4470a6781f1035e0856e",
+      "value": " 339/339 [01:07&lt;00:00, 178.83it/s, Materializing param=model.norm.weight]"
+     }
+    },
+    "53d3826130da44b3aba936cad515d67b": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "ac3bf1ae42804678be05203c7d95ddd9": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "73e084e0709a446cbde83880da8fae2e": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    },
+    "2d6124a44ed048198d897c074561cd96": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "b30c7621e14748f7a7e1f6683fac0674": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "ProgressStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "ProgressStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "bar_color": null,
+      "description_width": ""
+     }
+    },
+    "5969532875ce41b6820a5c57b53908d9": {
+     "model_module": "@jupyter-widgets/base",
+     "model_name": "LayoutModel",
+     "model_module_version": "1.2.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/base",
+      "_model_module_version": "1.2.0",
+      "_model_name": "LayoutModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "LayoutView",
+      "align_content": null,
+      "align_items": null,
+      "align_self": null,
+      "border": null,
+      "bottom": null,
+      "display": null,
+      "flex": null,
+      "flex_flow": null,
+      "grid_area": null,
+      "grid_auto_columns": null,
+      "grid_auto_flow": null,
+      "grid_auto_rows": null,
+      "grid_column": null,
+      "grid_gap": null,
+      "grid_row": null,
+      "grid_template_areas": null,
+      "grid_template_columns": null,
+      "grid_template_rows": null,
+      "height": null,
+      "justify_content": null,
+      "justify_items": null,
+      "left": null,
+      "margin": null,
+      "max_height": null,
+      "max_width": null,
+      "min_height": null,
+      "min_width": null,
+      "object_fit": null,
+      "object_position": null,
+      "order": null,
+      "overflow": null,
+      "overflow_x": null,
+      "overflow_y": null,
+      "padding": null,
+      "right": null,
+      "top": null,
+      "visibility": null,
+      "width": null
+     }
+    },
+    "af8771c98d3a4470a6781f1035e0856e": {
+     "model_module": "@jupyter-widgets/controls",
+     "model_name": "DescriptionStyleModel",
+     "model_module_version": "1.5.0",
+     "state": {
+      "_model_module": "@jupyter-widgets/controls",
+      "_model_module_version": "1.5.0",
+      "_model_name": "DescriptionStyleModel",
+      "_view_count": null,
+      "_view_module": "@jupyter-widgets/base",
+      "_view_module_version": "1.2.0",
+      "_view_name": "StyleView",
+      "description_width": ""
+     }
+    }
+   }
+  },
+  "kaggle": {
+   "accelerator": "gpu",
+   "dataSources": [
+    {
+     "sourceType": "datasetVersion",
+     "sourceId": 15056802,
+     "datasetId": 9638685,
+     "databundleVersionId": 15937545
+    }
+   ],
+   "dockerImageVersionId": 31287,
+   "isInternetEnabled": true,
+   "language": "python",
+   "sourceType": "notebook",
+   "isGpuEnabled": true
+  }
+ },
+ "nbformat_minor": 4,
+ "nbformat": 4,
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "source": "# StockEx Clearing House — LLM Fine-Tuning\n\nFine-tunes a Qwen2.5 Instruct model with QLoRA to act as a clearing house trading agent.\n\n**Runs on both Kaggle and Colab.** Auto-selects model size based on available VRAM:\n\n| GPU | VRAM | Model |\n|-----|------|-------|\n| T4 (free) | 15 GB | Qwen2.5-7B-Instruct |\n| A100 40 GB | 40 GB | Qwen2.5-14B-Instruct |\n| A100 80 GB | 80 GB | Qwen2.5-32B-Instruct |\n\n**Resumable training:**\n- **Kaggle**: checkpoints saved/restored from dataset `xabonum/stockex-ch-checkpoints`\n- **Colab**: checkpoints saved/restored from Google Drive\n\n**Output model:** `RayMelius/stockex-ch-trader` on HuggingFace Hub\n\n**Required secret:** Add `HF_TOKEN` in Kaggle Secrets or Colab Secrets (🔑 icon in left sidebar)",
+   "metadata": {
+    "id": "title"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": "# ── Install dependencies ───────────────────────────────────────────────────────\n# Reinstall bitsandbytes with proper CUDA support (fixes triton.ops error on Colab)\n!pip install -q -U bitsandbytes\n!pip install -q \\\n    \"transformers>=4.46.3\" \\\n    \"peft>=0.13.2\" \\\n    \"trl>=0.12.1\" \\\n    \"datasets>=3.1.0\" \\\n    \"accelerate>=1.1.1\" \\\n    huggingface_hub\nprint(\"Dependencies installed.\")",
+   "metadata": {
+    "id": "install",
+    "outputId": "a564dd39-8172-4745-e00c-c90fc17c6634",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "source": "import os, json, random, torch\nfrom datasets import Dataset\nfrom transformers import (\n    AutoTokenizer, AutoModelForCausalLM,\n    BitsAndBytesConfig, TrainingArguments,\n)\nfrom peft import LoraConfig, get_peft_model, TaskType\nfrom trl import SFTTrainer, SFTConfig\nfrom huggingface_hub import login\n\nprint(f\"CUDA available: {torch.cuda.is_available()}\")\nif torch.cuda.is_available():\n    print(f\"GPU: {torch.cuda.get_device_name(0)}\")\n    print(f\"VRAM: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f} GB\")",
+   "metadata": {
+    "id": "imports",
+    "outputId": "162eb31d-3a84-487c-c5b6-4f25c8d69485",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "source": "# ── Auto-select model based on available VRAM ─────────────────────────────────\nimport torch\n\nassert torch.cuda.is_available(), \"No GPU found — change runtime to GPU.\"\nvram_gb = torch.cuda.get_device_properties(0).total_memory / 1e9\ngpu_name = torch.cuda.get_device_name(0)\nprint(f\"GPU: {gpu_name}  |  VRAM: {vram_gb:.1f} GB\")\n\nif vram_gb >= 70:\n    BASE_MODEL = \"Qwen/Qwen2.5-32B-Instruct\"\n    BATCH_SIZE, GRAD_ACCUM, LR = 1, 16, 1e-4\nelif vram_gb >= 35:\n    BASE_MODEL = \"Qwen/Qwen2.5-14B-Instruct\"\n    BATCH_SIZE, GRAD_ACCUM, LR = 2, 8,  1e-4\nelse:                                          # T4 / 15 GB\n    BASE_MODEL = \"Qwen/Qwen2.5-7B-Instruct\"\n    BATCH_SIZE, GRAD_ACCUM, LR = 4, 4,  2e-4\n\nprint(f\"Selected model: {BASE_MODEL}\")\n\n# ── Fixed config ───────────────────────────────────────────────────────────────\nOUTPUT_REPO  = \"RayMelius/stockex-ch-trader\"\nOUTPUT_DIR   = \"./stockex-ch-trader\"\nLORA_R       = 16\nLORA_ALPHA   = 32\nLORA_DROPOUT = 0.05\nNUM_EPOCHS   = 3\nMAX_SEQ_LEN  = 512\nDATASET_SIZE = 2500\n\n# ── HuggingFace login ─────────────────────────────────────────────────────────\nimport os\n\nHF_TOKEN = None\n\n# Kaggle\ntry:\n    from kaggle_secrets import UserSecretsClient\n    HF_TOKEN = UserSecretsClient().get_secret(\"HF_TOKEN\")\n    print(\"HF_TOKEN loaded from Kaggle Secrets\")\nexcept:\n    pass\n\n# Colab\nif not HF_TOKEN:\n    try:\n        from google.colab import userdata\n        HF_TOKEN = userdata.get(\"HF_TOKEN\")\n        print(\"HF_TOKEN loaded from Colab Secrets\")\n    except:\n        pass\n\n# Env fallback\nif not HF_TOKEN:\n    HF_TOKEN = os.getenv(\"HF_TOKEN\")\n\nif not HF_TOKEN:\n    raise ValueError(\"HF_TOKEN not found.\")\n\nfrom huggingface_hub import login\nlogin(token=HF_TOKEN)",
+   "metadata": {
+    "id": "config",
+    "outputId": "c45e0d98-8928-4459-8449-63d498694d5d",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "source": "## 1. Checkpoint Detection (Resumable Training)\n\nAutomatically detects and resumes from the latest checkpoint:\n- **Kaggle**: Downloads checkpoints from dataset `xabonum/stockex-ch-checkpoints`\n- **Colab**: Restores checkpoints from Google Drive\n\nOnly the **last 3 checkpoints** are kept in persistent storage.",
+   "metadata": {
+    "id": "dataset-header"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": "import shutil, math, glob\nfrom transformers.trainer_utils import get_last_checkpoint\n\n# ── Detect environment ─────────────────────────────────────────────\nRUNNING_ON_KAGGLE = os.path.exists(\"/kaggle/working\")\nRUNNING_ON_COLAB  = os.path.exists(\"/content\") and not RUNNING_ON_KAGGLE\n\nUSE_DRIVE = False\nDRIVE_CKPT_DIR = None\n\nif RUNNING_ON_COLAB:\n    try:\n        from google.colab import drive\n        drive.mount(\"/content/drive\", force_remount=False)\n        DRIVE_CKPT_DIR = \"/content/drive/MyDrive/stockex-ch-checkpoints\"\n        USE_DRIVE = True\n        print(f\"Colab: checkpoints will use {DRIVE_CKPT_DIR}\")\n    except Exception as e:\n        print(f\"Colab drive mount failed: {e} — saving locally only\")\n\nelif RUNNING_ON_KAGGLE:\n    DRIVE_CKPT_DIR = \"/kaggle/working/stockex-ch-checkpoints\"\n    USE_DRIVE = True\n    os.makedirs(DRIVE_CKPT_DIR, exist_ok=True)\n\n    # Download checkpoint dataset if available\n    try:\n        import subprocess\n        result = subprocess.run(\n            [\"kaggle\", \"datasets\", \"download\", \"-d\", \"xabonum/stockex-ch-checkpoints\",\n             \"-p\", DRIVE_CKPT_DIR, \"--unzip\"],\n            capture_output=True, text=True, timeout=120\n        )\n        if result.returncode == 0:\n            print(f\"Kaggle: downloaded checkpoint dataset -> {DRIVE_CKPT_DIR}\")\n        else:\n            print(f\"Kaggle: no existing checkpoint dataset (starting fresh)\")\n            print(f\"  stderr: {result.stderr.strip()}\")\n    except Exception as e:\n        print(f\"Kaggle: checkpoint download failed: {e}\")\n\n    print(f\"Kaggle: checkpoints will use {DRIVE_CKPT_DIR}\")\n\nelse:\n    print(\"Unknown environment — saving locally only\")\n\n# ── Free GPU memory to prevent OOM on resume ────────────────────────\ntorch.cuda.empty_cache()\ntorch.cuda.reset_peak_memory_stats()\ntorch.cuda.ipc_collect()\nos.environ[\"PYTORCH_CUDA_ALLOC_CONF\"] = \"max_split_size_mb:128,garbage_collection_threshold:0.6\"\n\n# ── Restore checkpoint from persistent storage to local output dir ──\nos.makedirs(OUTPUT_DIR, exist_ok=True)\nif USE_DRIVE:\n    os.makedirs(DRIVE_CKPT_DIR, exist_ok=True)\n    drive_ckpt = get_last_checkpoint(DRIVE_CKPT_DIR)\n    if drive_ckpt:\n        local_ckpt_name = os.path.basename(drive_ckpt)\n        local_ckpt_path = os.path.join(OUTPUT_DIR, local_ckpt_name)\n        if not os.path.exists(local_ckpt_path):\n            print(f\"Restoring checkpoint: {drive_ckpt} -> {local_ckpt_path}\")\n            shutil.copytree(drive_ckpt, local_ckpt_path)\n\n# ── Detect latest local checkpoint ────────────────────────────────\nRESUME_FROM = get_last_checkpoint(OUTPUT_DIR)\nSAVE_STEPS = 10\n\n# ── Resume summary ─────────────────────────────────────────────────\ntrain_size      = int(DATASET_SIZE * 0.9)\nsteps_per_epoch = math.ceil(train_size / (BATCH_SIZE * GRAD_ACCUM))\ntotal_steps     = steps_per_epoch * NUM_EPOCHS\n\nif RESUME_FROM:\n    completed_steps = int(os.path.basename(RESUME_FROM).split(\"-\")[-1])\n    remaining    = max(0, total_steps - completed_steps)\n    pct_done     = 100 * completed_steps / total_steps\n    epoch_done   = completed_steps / steps_per_epoch\n\n    print(\"\\n\" + \"=\" * 55)\n    print(\"  RESUMING FROM CHECKPOINT\")\n    print(\"=\" * 55)\n    print(f\"  Checkpoint   : {os.path.basename(RESUME_FROM)}\")\n    print(f\"  Steps done   : {completed_steps:,} / {total_steps:,}  ({pct_done:.1f}%)\")\n    print(f\"  Steps left   : {remaining:,}\")\n    print(f\"  Epoch        : {epoch_done:.2f} / {NUM_EPOCHS}\")\n    print(f\"  Epochs left  : {NUM_EPOCHS - epoch_done:.2f}\")\n    print(f\"  Steps/epoch  : {steps_per_epoch:,}\")\n    print(f\"  Save every   : {SAVE_STEPS} steps\")\n    print(\"=\" * 55 + \"\\n\")\nelse:\n    print(\"\\n\" + \"=\" * 55)\n    print(\"  STARTING FRESH\")\n    print(\"=\" * 55)\n    print(f\"  Total steps  : {total_steps:,}\")\n    print(f\"  Steps/epoch  : {steps_per_epoch:,}\")\n    print(f\"  Epochs       : {NUM_EPOCHS}\")\n    print(f\"  Save every   : {SAVE_STEPS} steps\")\n    print(\"=\" * 55 + \"\\n\")",
+   "metadata": {
+    "id": "egq0dp9csuo",
+    "outputId": "f098a839-130c-4011-d0ff-a10289004957",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "source": "## 2. Synthetic Dataset Generation\n\nEach training example is a realistic clearing house trading scenario:\n- Member state: capital, holdings, obligation remaining\n- Market: BBO for each security\n- Target: a valid JSON trading decision that respects all constraints",
+   "metadata": {
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "source": "# Securities from shared_data/securities.txt  (symbol, start_price, current_price)\nSECURITIES = [\n    {\"symbol\": \"ALPHA\", \"base\": 5.65},\n    {\"symbol\": \"PEIR\",  \"base\": 8.35},\n    {\"symbol\": \"EXAE\",  \"base\": 6.90},\n    {\"symbol\": \"QUEST\", \"base\": 13.35},\n    {\"symbol\": \"NBG\",   \"base\": 8.00},\n    {\"symbol\": \"EUROB\", \"base\": 3.45},\n    {\"symbol\": \"AEG\",   \"base\": 4.75},\n    {\"symbol\": \"INTKA\", \"base\": 7.35},\n    {\"symbol\": \"AAAK\",  \"base\": 2.75},\n    {\"symbol\": \"ATTIK\", \"base\": 4.90},\n]\n\nSTARTING_CAPITAL = 100_000.0\nDAILY_OBLIGATION = 10\n\n\ndef gen_bbo(base_price: float) -> dict:\n    \"\"\"Generate a realistic bid/ask spread around a base price.\"\"\"\n    drift    = random.uniform(-0.05, 0.05)\n    mid      = round(base_price * (1 + drift), 2)\n    spread   = round(random.choice([0.05, 0.10, 0.15]), 2)\n    best_bid = round(mid - spread / 2, 2)\n    best_ask = round(mid + spread / 2, 2)\n    return {\"best_bid\": best_bid, \"best_ask\": best_ask, \"mid\": mid}\n\n\ndef gen_holdings(bbos: dict) -> list:\n    \"\"\"Randomly generate some holdings for a member.\"\"\"\n    holdings = []\n    n = random.randint(0, 4)\n    for sym in random.sample(list(bbos.keys()), min(n, len(bbos))):\n        qty = random.randint(50, 500)\n        mid = bbos[sym][\"mid\"]\n        avg_cost = round(mid * random.uniform(0.92, 1.08), 2)\n        holdings.append({\"symbol\": sym, \"quantity\": qty, \"avg_cost\": avg_cost})\n    return holdings\n\n\ndef build_prompt(member_id: str, capital: float, holdings: list,\n                 obligation_remaining: int, bbos: dict) -> str:\n    market_lines = [\n        f\"  {sym}: Bid {bbo['best_bid']:.2f} / Ask {bbo['best_ask']:.2f}\"\n        for sym, bbo in sorted(bbos.items())\n    ]\n    holding_lines = (\n        [f\"  {h['symbol']}: {h['quantity']} shares @ avg cost {h['avg_cost']:.2f}\"\n         for h in holdings]\n        if holdings else [\"  None\"]\n    )\n    return (\n        f\"You are simulating clearing house member {member_id} making ONE trading decision.\\n\\n\"\n        f\"Member state:\\n\"\n        f\"  Available capital: EUR {capital:,.2f}\\n\"\n        f\"  Securities obligation remaining today: {obligation_remaining} more to trade\\n\"\n        f\"  Current holdings:\\n\" + \"\\n\".join(holding_lines) + \"\\n\\n\"\n        f\"Current market (Bid/Ask):\\n\" + \"\\n\".join(market_lines) + \"\\n\\n\"\n        f\"Rules:\\n\"\n        f\"- Do not spend more than your available capital\\n\"\n        f\"- Do not sell more shares than you hold\\n\"\n        f\"- If you have no holdings, you must BUY\\n\"\n        f\"- Choose a realistic price close to the BBO mid-price\\n\"\n        f\"- Quantity should be between 10 and 200\\n\\n\"\n        f\"Respond ONLY with valid JSON, no other text:\\n\"\n        f'Example: {{\"symbol\": \"ALPHA\", \"side\": \"BUY\", \"quantity\": 50, \"price\": 5.65}}'\n    )\n\n\ndef gen_decision(capital: float, holdings: list, bbos: dict) -> dict:\n    \"\"\"Generate a rule-valid trading decision for the given state.\"\"\"\n    holdings_value = sum(\n        h[\"quantity\"] * bbos.get(h[\"symbol\"], {}).get(\"mid\", h[\"avg_cost\"])\n        for h in holdings\n    )\n    net_worth = capital + holdings_value\n    holdings_ratio = holdings_value / net_worth if net_worth > 0 else 0\n\n    if not holdings:\n        side = \"BUY\"\n    elif holdings_ratio > 0.6:\n        side = random.choices([\"SELL\", \"BUY\"], weights=[0.7, 0.3])[0]\n    else:\n        side = random.choices([\"BUY\", \"SELL\"], weights=[0.55, 0.45])[0]\n\n    if side == \"BUY\":\n        affordable = [sym for sym, bbo in bbos.items() if 10 * bbo[\"best_ask\"] <= capital]\n        if not affordable:\n            sym = min(bbos, key=lambda s: bbos[s][\"best_ask\"])\n        else:\n            held_syms = [h[\"symbol\"] for h in holdings]\n            weights = [3 if s in held_syms else 1 for s in affordable]\n            sym = random.choices(affordable, weights=weights)[0]\n        ask = bbos[sym][\"best_ask\"]\n        max_qty = min(200, int(capital / ask))\n        qty = random.randint(10, max(10, max_qty))\n        price = round(bbos[sym][\"mid\"] + random.uniform(-0.05, 0.05), 2)\n        price = max(bbos[sym][\"best_bid\"], min(price, ask))\n        return {\"symbol\": sym, \"side\": \"BUY\", \"quantity\": qty, \"price\": round(price, 2)}\n    else:\n        h = random.choice(holdings)\n        sym = h[\"symbol\"]\n        bbo = bbos[sym]\n        qty = random.randint(10, min(200, h[\"quantity\"]))\n        price = round(bbo[\"mid\"] + random.uniform(-0.05, 0.05), 2)\n        price = max(bbo[\"best_bid\"] - 0.05, min(price, bbo[\"best_ask\"]))\n        return {\"symbol\": sym, \"side\": \"SELL\", \"quantity\": qty, \"price\": round(price, 2)}\n\n\ndef generate_dataset(n: int) -> list:\n    examples = []\n    member_ids = [f\"USR{i:02d}\" for i in range(1, 11)]\n    scenarios = [\n        ((80_000, 100_000), (5, 10), \"fresh_member\"),\n        ((50_000, 80_000),  (0, 5),  \"active_member\"),\n        ((20_000, 50_000),  (0, 2),  \"low_capital\"),\n        ((5_000,  20_000),  (0, 10), \"very_low_capital\"),\n        ((90_000, 100_000), (10, 10),\"start_of_day\"),\n    ]\n    for _ in range(n):\n        cap_range, obl_range, _ = random.choice(scenarios)\n        capital    = round(random.uniform(*cap_range), 2)\n        obligation = random.randint(*obl_range)\n        member_id  = random.choice(member_ids)\n        bbos       = {s[\"symbol\"]: gen_bbo(s[\"base\"]) for s in SECURITIES}\n        holdings   = gen_holdings(bbos)\n        holdings_cost = sum(h[\"quantity\"] * h[\"avg_cost\"] for h in holdings)\n        if holdings_cost > STARTING_CAPITAL - capital:\n            scale = (STARTING_CAPITAL - capital) / max(holdings_cost, 1)\n            for h in holdings:\n                h[\"quantity\"] = max(10, int(h[\"quantity\"] * scale))\n        prompt   = build_prompt(member_id, capital, holdings, obligation, bbos)\n        decision = gen_decision(capital, holdings, bbos)\n        examples.append({\"prompt\": prompt, \"completion\": json.dumps(decision)})\n    return examples\n\n\nprint(f\"Generating {DATASET_SIZE} training examples...\")\nraw_data = generate_dataset(DATASET_SIZE)\nprint(f\"Done. Example:\")\nprint(\"PROMPT:\\n\", raw_data[0][\"prompt\"])\nprint(\"\\nCOMPLETION:\", raw_data[0][\"completion\"])",
+   "metadata": {
+    "id": "dataset-gen",
+    "outputId": "6b6ae157-9c91-42d5-a556-049f122bf7f1",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "source": "# Train/val split (90/10)\nrandom.shuffle(raw_data)\nsplit = int(len(raw_data) * 0.9)\ntrain_data = raw_data[:split]\nval_data   = raw_data[split:]\n\ntrain_dataset = Dataset.from_list(train_data)\nval_dataset   = Dataset.from_list(val_data)\nprint(f\"Train: {len(train_dataset)} | Val: {len(val_dataset)}\")",
+   "metadata": {
+    "id": "dataset-split",
+    "outputId": "3246d564-c102-4395-8fbc-c3f7979130fb",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "source": "## 3. Load Base Model (4-bit QLoRA)",
+   "metadata": {
+    "id": "model-header"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": "print(f\"Loading tokenizer: {BASE_MODEL}\")\ntokenizer = AutoTokenizer.from_pretrained(BASE_MODEL, trust_remote_code=True)\ntokenizer.pad_token        = tokenizer.eos_token\ntokenizer.padding_side     = \"right\"\ntokenizer.model_max_length = MAX_SEQ_LEN   # replaces max_seq_length in SFTConfig\nprint(\"Tokenizer loaded\")",
+   "metadata": {
+    "id": "load-tokenizer",
+    "outputId": "a5008937-66ed-4028-a215-62e458cfc8dd",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "source": "SYSTEM_PROMPT = (\n    \"You are a StockEx clearing house trading agent. \"\n    \"Given a member's financial state and live market data, \"\n    \"you output a single valid JSON trading decision that respects all capital and holdings constraints. \"\n    \"Never output anything other than the JSON object.\"\n)\n\n\ndef format_chat(example):\n    \"\"\"Apply the model's chat template to produce a training string.\"\"\"\n    messages = [\n        {\"role\": \"system\",    \"content\": SYSTEM_PROMPT},\n        {\"role\": \"user\",      \"content\": example[\"prompt\"]},\n        {\"role\": \"assistant\", \"content\": example[\"completion\"]},\n    ]\n    text = tokenizer.apply_chat_template(\n        messages,\n        tokenize=False,\n        add_generation_prompt=False,\n    )\n    return {\"text\": text}\n\n\ntrain_dataset = train_dataset.map(format_chat)\nval_dataset   = val_dataset.map(format_chat)\n\nprint(\"Sample formatted text:\")\nprint(train_dataset[0][\"text\"][:600], \"...\")",
+   "metadata": {
+    "id": "format-dataset",
+    "outputId": "ea4f1fa8-bf2b-4360-c426-28e5255dbbd3",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "source": "# 4-bit quantization config\nbnb_config = BitsAndBytesConfig(\n    load_in_4bit=True,\n    bnb_4bit_quant_type=\"nf4\",\n    bnb_4bit_compute_dtype=torch.bfloat16,\n    bnb_4bit_use_double_quant=True,\n)\n\nprint(f\"Loading model: {BASE_MODEL} (4-bit)\")\nmodel = AutoModelForCausalLM.from_pretrained(\n    BASE_MODEL,\n    quantization_config=bnb_config,\n    device_map=\"auto\",\n    trust_remote_code=True,\n    dtype=torch.bfloat16,\n)\nmodel.config.use_cache = False\nmodel.config.pretraining_tp = 1\nprint(f\"Model loaded. Parameters: {model.num_parameters()/1e9:.2f}B\")\n\nfrom peft import prepare_model_for_kbit_training\n\nmodel = prepare_model_for_kbit_training(model)",
+   "metadata": {
+    "id": "load-model",
+    "outputId": "7f46083d-c830-45f3-e887-fbe9cf30b4b2",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "source": "## 3b. LoRA Configuration",
+   "metadata": {
+    "id": "lora-header"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": "lora_config = LoraConfig(\n    r=LORA_R,\n    lora_alpha=LORA_ALPHA,\n    target_modules=[\n        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n        \"gate_proj\", \"up_proj\", \"down_proj\",\n    ],\n    lora_dropout=LORA_DROPOUT,\n    bias=\"none\",\n    task_type=TaskType.CAUSAL_LM,\n)\n\ntrainable = sum(p.numel() for p in model.parameters() if p.requires_grad)\ntotal     = sum(p.numel() for p in model.parameters())\nprint(f\"Trainable parameters: {trainable/1e6:.1f}M / {total/1e6:.0f}M ({100*trainable/total:.2f}%)\")",
+   "metadata": {
+    "id": "lora-config",
+    "outputId": "8a2688ba-288b-4149-ff37-c8d0ee11f77f",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "source": "## 4. Train",
+   "metadata": {
+    "id": "training-header"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": "from transformers import TrainerCallback\nimport shutil, glob\n\nKAGGLE_DATASET_ID = \"xabonum/stockex-ch-checkpoints\"\nMAX_CHECKPOINTS_PERSISTENT = 3  # keep only last 3 in Kaggle dataset / Drive\n\n# ── Create Kaggle dataset metadata if needed ────────────────────────\nif USE_DRIVE and DRIVE_CKPT_DIR:\n    metadata_path = os.path.join(DRIVE_CKPT_DIR, \"dataset-metadata.json\")\n    if not os.path.exists(metadata_path):\n        metadata = {\n            \"title\": \"stockex-ch-checkpoints\",\n            \"id\": KAGGLE_DATASET_ID,\n            \"licenses\": [{\"name\": \"CC0-1.0\"}]\n        }\n        with open(metadata_path, \"w\") as f:\n            json.dump(metadata, f, indent=2)\n        print(\"Created dataset-metadata.json\")\n\n\ndef cleanup_old_checkpoints(ckpt_dir, keep=MAX_CHECKPOINTS_PERSISTENT):\n    \"\"\"Keep only the last `keep` checkpoints in persistent storage.\"\"\"\n    ckpts = sorted(\n        glob.glob(os.path.join(ckpt_dir, \"checkpoint-*\")),\n        key=lambda p: int(os.path.basename(p).split(\"-\")[-1])\n    )\n    while len(ckpts) > keep:\n        old = ckpts.pop(0)\n        shutil.rmtree(old)\n        print(f\"[Checkpoint] Removed old: {os.path.basename(old)}\")\n\n\nclass CheckpointSyncCallback(TrainerCallback):\n    \"\"\"Copy checkpoint to persistent storage, push LoRA to HF Hub, upload to Kaggle dataset.\"\"\"\n\n    def on_save(self, args, state, control, **kwargs):\n        ckpt_dir = os.path.join(args.output_dir, f\"checkpoint-{state.global_step}\")\n        if not os.path.isdir(ckpt_dir):\n            return\n\n        # 1. Save to persistent folder (Drive / Kaggle working dir)\n        if USE_DRIVE and DRIVE_CKPT_DIR:\n            dest = os.path.join(DRIVE_CKPT_DIR, f\"checkpoint-{state.global_step}\")\n            os.makedirs(DRIVE_CKPT_DIR, exist_ok=True)\n            try:\n                shutil.copytree(ckpt_dir, dest, dirs_exist_ok=True)\n                print(f\"[Checkpoint] Saved -> {dest}\")\n            except Exception as e:\n                print(f\"[Checkpoint] Copy failed: {e}\")\n\n            # Cleanup: keep only last 3 checkpoints in persistent storage\n            try:\n                cleanup_old_checkpoints(DRIVE_CKPT_DIR)\n            except Exception as e:\n                print(f\"[Checkpoint] Cleanup failed: {e}\")\n\n        # 2. Push LoRA adapter to HF Hub\n        try:\n            kwargs[\"model\"].push_to_hub(\n                OUTPUT_REPO,\n                commit_message=f\"Checkpoint step {state.global_step} (epoch {state.epoch:.2f})\",\n                token=HF_TOKEN,\n            )\n            print(f\"[Checkpoint] Pushed step {state.global_step} -> HF Hub\")\n        except Exception as e:\n            print(f\"[Checkpoint] HF push failed: {e}\")\n\n        # 3. Upload to Kaggle dataset (Kaggle only)\n        if RUNNING_ON_KAGGLE and USE_DRIVE:\n            try:\n                os.system(\n                    f\"kaggle datasets version -p {DRIVE_CKPT_DIR} \"\n                    f\"-m 'Checkpoint step {state.global_step}' --dir-mode zip\"\n                )\n                print(f\"[Checkpoint] Kaggle dataset updated for step {state.global_step}\")\n            except Exception as e:\n                print(f\"[Checkpoint] Kaggle dataset update failed: {e}\")\n\n\nsft_config = SFTConfig(\n    output_dir=OUTPUT_DIR,\n    num_train_epochs=NUM_EPOCHS,\n    per_device_train_batch_size=BATCH_SIZE,\n    per_device_eval_batch_size=BATCH_SIZE,\n    gradient_accumulation_steps=GRAD_ACCUM,\n    gradient_checkpointing=True,\n    optim=\"paged_adamw_8bit\",\n    learning_rate=LR,\n    lr_scheduler_type=\"cosine\",\n    warmup_ratio=0.05,\n    fp16=not torch.cuda.is_bf16_supported(),\n    bf16=torch.cuda.is_bf16_supported(),\n    logging_steps=10,\n    eval_strategy=\"steps\",\n    eval_steps=SAVE_STEPS,\n    save_strategy=\"steps\",\n    save_steps=SAVE_STEPS,\n    save_total_limit=3,\n    load_best_model_at_end=True,\n    metric_for_best_model=\"eval_loss\",\n    greater_is_better=False,\n    report_to=\"none\",\n    dataset_text_field=\"text\",\n    packing=False,\n)\n\ntrainer = SFTTrainer(\n    model=model,\n    args=sft_config,\n    train_dataset=train_dataset,\n    eval_dataset=val_dataset,\n    peft_config=lora_config,\n    processing_class=tokenizer,\n    callbacks=[CheckpointSyncCallback()],\n)\n\nif RESUME_FROM:\n    print(f\"Resuming from checkpoint: {RESUME_FROM}\")\nelse:\n    print(f\"Starting training from scratch. Save every {SAVE_STEPS} steps.\")\n\ntrainer.train(resume_from_checkpoint=RESUME_FROM)\nprint(\"Training complete.\")",
+   "metadata": {
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "source": "## 5. Save & Push to HuggingFace Hub\n\nMerges LoRA adapters into the base model weights and pushes the full model.",
+   "metadata": {
+    "id": "save-header"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": "from peft import PeftModel\n\n# Save best adapter checkpoint locally\ntrainer.model.save_pretrained(OUTPUT_DIR)\ntokenizer.save_pretrained(OUTPUT_DIR)\nprint(f\"Adapter saved to {OUTPUT_DIR}\")\n\n# Reload base model in fp16 for merging (can't merge with 4-bit)\nprint(\"Reloading base model in fp16 for adapter merge...\")\ndel model\ntorch.cuda.empty_cache()\n\nbase_model = AutoModelForCausalLM.from_pretrained(\n    BASE_MODEL,\n    torch_dtype=torch.float16,\n    device_map=\"auto\",\n    trust_remote_code=True,\n)\nmerged_model = PeftModel.from_pretrained(base_model, OUTPUT_DIR)\nmerged_model = merged_model.merge_and_unload()\nprint(\"Adapters merged.\")",
+   "metadata": {
+    "id": "save-model",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "source": "print(f\"Pushing merged model to: {OUTPUT_REPO}\")\nmerged_model.push_to_hub(\n    OUTPUT_REPO,\n    token=HF_TOKEN,\n    commit_message=\"StockEx CH Trader: QLoRA fine-tuned Qwen2.5-32B-Instruct\",\n)\ntokenizer.push_to_hub(\n    OUTPUT_REPO,\n    token=HF_TOKEN,\n    commit_message=\"Tokenizer for StockEx CH Trader (Qwen2.5-32B-Instruct base)\",\n)\nprint(f\"✓ Model pushed to https://huggingface.co/{OUTPUT_REPO}\")",
+   "metadata": {
+    "id": "push-hub",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "source": "## 6. Inference Test\n\nVerify the model generates valid JSON trading decisions.",
+   "metadata": {
+    "id": "test-header"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": "import re\nfrom transformers import pipeline\n\npipe = pipeline(\n    \"text-generation\",\n    model=merged_model,\n    tokenizer=tokenizer,\n    device_map=\"auto\",\n)\n\ntest_cases = [\n    {\n        \"desc\": \"New member, no holdings, must trade\",\n        \"capital\": 100_000.0,\n        \"holdings\": [],\n        \"obligation\": 10,\n    },\n    {\n        \"desc\": \"Experienced member with holdings, low obligation\",\n        \"capital\": 65_000.0,\n        \"holdings\": [\n            {\"symbol\": \"ALPHA\", \"quantity\": 300, \"avg_cost\": 5.60},\n            {\"symbol\": \"QUEST\", \"quantity\": 150, \"avg_cost\": 13.20},\n        ],\n        \"obligation\": 2,\n    },\n    {\n        \"desc\": \"Low capital, large holdings\",\n        \"capital\": 8_000.0,\n        \"holdings\": [\n            {\"symbol\": \"PEIR\",  \"quantity\": 500, \"avg_cost\": 8.30},\n            {\"symbol\": \"NBG\",   \"quantity\": 200, \"avg_cost\": 7.95},\n        ],\n        \"obligation\": 5,\n    },\n]\n\ntest_bbos = {s[\"symbol\"]: gen_bbo(s[\"base\"]) for s in SECURITIES}\n\nprint(\"=\" * 70)\nfor tc in test_cases:\n    print(f\"\\nSCENARIO: {tc['desc']}\")\n    prompt = build_prompt(\"USR01\", tc[\"capital\"], tc[\"holdings\"], tc[\"obligation\"], test_bbos)\n    messages = [\n        {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n        {\"role\": \"user\",   \"content\": prompt},\n    ]\n    output = pipe(\n        messages,\n        max_new_tokens=60,\n        temperature=0.3,\n        do_sample=True,\n        pad_token_id=tokenizer.eos_token_id,\n    )\n    response = output[0][\"generated_text\"][-1][\"content\"].strip()\n    print(f\"RESPONSE: {response}\")\n    try:\n        m = re.search(r\"\\{[^}]+\\}\", response)\n        if m:\n            d = json.loads(m.group())\n            assert d[\"side\"] in (\"BUY\", \"SELL\")\n            assert d[\"symbol\"] in [s[\"symbol\"] for s in SECURITIES]\n            assert d[\"quantity\"] > 0\n            assert d[\"price\"] > 0\n            print(f\"✓ Valid JSON: {d}\")\n        else:\n            print(\"✗ No JSON found in response\")\n    except Exception as e:\n        print(f\"✗ Invalid: {e}\")\n    print(\"-\" * 70)",
+   "metadata": {
+    "id": "inference-test",
+    "trusted": true
+   },
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "source": "## 7. Activate in StockEx\n\nThe clearing house already uses `RayMelius/stockex-ch-trader` as default.\n\nTo switch to this model in a running StockEx instance:\n\n**HuggingFace Spaces** — add to secrets:\n```\nHF_MODEL = RayMelius/stockex-ch-trader\nHF_TOKEN = <your token>\n```\n\n**Docker Compose** — already set in `docker-compose.yml`:\n```yaml\nenvironment:\n  - HF_MODEL=RayMelius/stockex-ch-trader\n  - HF_TOKEN=<your token>\n```",
+   "metadata": {
+    "id": "usage-header"
+   }
+  }
+ ]
+}