docs/ayush/notes.md

Ayush Notes

Use this file for short-lived working notes, reminders, and handoff details.

Do not use this file for durable deviations from the original plan. Put those in docs/changes.md.

Current local training-data note:

• A 50-paper experiment-design corpus now exists under data/papers/.
• Use data/papers/manifest.json for the full scenario-to-paper mapping.
• Most entries are marked alternative because many scenario titles in ReplicaLab_50_Scenarios_Training_Plan.md are synthetic summaries rather than directly downloadable published paper titles.

Current V2 training architecture note:

• The reusable training stack now lives under replicalab/training/.
• notebooks/train_minimal_colab.ipynb is now the explicit sponsor-facing minimal Colab script using Unsloth + HF TRL.
• notebooks/train_colab.ipynb is the judged notebook driver, but heavy runs are expected to use the replicalab-train entrypoint on Northflank H100.
• The primary shared base is now Qwen/Qwen3.5-9B with separate Scientist GRPO and Lab Manager SFT adapters.
• The reduced-scale fallback is Qwen/Qwen3.5-4B.
• The audit-only judge candidate is Qwen/Qwen3.5-122B-A10B.
• The deterministic rubric remains the only training reward source even when Anthropic-backed oracle features are enabled for V2 overlays.
• docs/training_goals.md now defines the current model goals and the separation between metric improvements and the larger execution-env redesign.
• A March 9 operational check found that the current Hugging Face token is valid for Hub auth but belongs to a non-billable personal account (canPay=false, no orgs), so it is not currently enough to provision paid large-model hosting on Hugging Face.
• The current Northflank manual job replicalab-train still has runtime env values, but northflank start job run returns 409 No deployment configured, so the job cannot launch until a runnable image/deployment is attached.
• The live Northflank service on the same nf-gpu-hack-16-64 plan does not currently expose nvidia-smi or /dev/nvidia* inside the container, so GPU availability should be treated as unverified until the runtime is fixed and a direct hardware probe succeeds.

Current Northflank notebook note:

• The dedicated notebook service now lives in project notebook-openport as service jupyter-pytorch.
• The pasted notebook hostname app--jupyter-pytorch--h74j66w224jx.code.run is stale; the live public notebook endpoint on 2026-03-09 is app--jupyter-pytorch--9y6g97v7czb9.code.run.
• The notebook runtime does expose a real NVIDIA H100 80GB HBM3 GPU.
• /home/jovyan/replicalab-ai and /home/jovyan/replicalab-qwen3.5-grpo already exist in that notebook, with saved adapter checkpoints through checkpoint-200.
• The saved grpo_training.log shows the notebook ran on H100 but did not complete cleanly: baseline eval emitted string indices must be integers, not 'str', and the final inference cell failed in tokenizer.apply_chat_template(...) with the same content-structure issue.

Current ART/OpenEnv runtime note:

• The active live Scientist RL path is now art-scientist-train in replicalab/training/cli.py.
• Fresh-runtime smoke validation completed on 2026-03-08 for:
  • scientist-preview-smoke-20260308b
  • lab-manager-preview-smoke-20260308b
  • art-scientist-smoke-20260308b
  • art-scientist-compare-smoke-20260308b
• The live ART Scientist checkpoint reached step7, but the current trained checkpoint still underperforms the deterministic baseline on held-out comparison.
• The main remaining work is experiment quality iteration, not missing training infrastructure.
• Evaluation summaries now track paper_understanding and communication_quality, and the shared benchmark-history plots live under replicalab/outputs/training/history/.

Current localhost model-runtime note:

• server/app.py now exposes /runtime and /agent-step so the local app can run a backend-selected Scientist policy instead of the frontend stub.
• Anthropic-backed Scientist inference was wired, but the current Anthropic account cannot be used live because the API billing balance is too low.
• Localhost therefore currently runs in ollama mode with glm-5:cloud as the working model-backed Scientist path.
• The server applies a small deterministic safety adapter to model outputs before env stepping:
  • trims controls to fit sample size
  • aligns equipment and reagent requests to the available inventory
  • clamps duration to the current lab time limit
• If the local model stalls or errors, /agent-step falls back to the deterministic baseline Scientist and records that in the step metadata as scientist_runtime=ollama_fallback.

Current March 9 H100 benchmark note:

• The full multi-round scientist-local-compare-eval path is live on the Northflank H100 notebook, but the current notebook image is missing the fast linear-attention path for the saved unsloth/Qwen3.5-0.8B adapter, so large sharded rollout sweeps did not flush artifacts on a practical same-turn timescale.
• A fallback live H100 first-step benchmark was run on 2026-03-09 instead: 250 shared reset cases with both baseline and trained Scientist first-step actions, for 500 total simulations.
• The merged artifact root is replicalab/outputs/training/h100-one-step-500-20260309/.
• The benchmark spans 34 trainable papers.
• Summary result:
  • baseline average first-step paper understanding: 0.61692084
  • trained average first-step paper understanding: 0.063866752
  • baseline average first-step reward: 0.3
  • trained average first-step reward: 0.05
  • trained request-info rate: 1.0
  • invalid-action rate stayed 0.0 for both labels
• Scenario-level understanding:
  • baseline finance_trading: 0.596033
  • trained finance_trading: 0.018182
  • baseline ml_benchmark: 0.633333
  • trained ml_benchmark: 0.099762
• Current interpretation: the saved replicalab-qwen3.5-grpo adapter is materially worse than the deterministic baseline on first-step paper grounding and currently behaves like a universal request_info policy under a fast decode budget.