Spaces:
Running
Running
Commit History
-
Add local H100 scientist eval tooling
a29a83d
-
Add 12 scoring & environment improvements with full test coverage
- LLM judge integration via OpenRouter with config toggle and deterministic fallback - Communication bonus scoring (action diversity, responsiveness, substantiveness) - Per-domain scoring weight profiles (math, ML, finance) - Adaptive reward shaping (later rounds carry heavier penalties) - Richer fidelity matching with bigram overlap and substring fuzzy matching - Cross-dimension feasibility tradeoffs in Lab Manager policy - Mid-episode judge feedback hints at negotiation midpoint - Scenario consistency validation (budget, equipment, reagent checks) - Cached substitution index per episode in environment - Docker HEALTHCHECK directive for container orchestration - .dockerignore security hardening for .env files - Fix circular import in judge_policy via lazy import - 25 new tests (integration + LLM judge mock tests), 514 total passing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>2b98daa
-
Add model-driven local runtime and dynamic demo flow
4ea916a -
Update HF Space: final architecture diagram, frontend, and README
- Add ReplicaLab_Architecture_Final.svg with full scoring engine, reward formula, penalties, and 4-dimension breakdown - Update README to reference final diagram and reflect 100% completion - Include all latest frontend components (demo flow, training page, episode results, compare page updates) - Updated API client, types, and demo data for complete UI Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>65e91b3 -
Finalize demo flow and training assets
b878f5b -
Close all Person D tasks: README enhancements, docs, UI close-out (152/152 = 100%)
- DOC 01: Replication crisis hook + solution summary in README - DOC 02-04: Architecture diagram, 4-option setup, results + key takeaways - DOC 05-07: Demo script, recording guide, video editing guide (already existed) - DOC 09: Created docs/submission_prep.md with links and track selections - DOC 10: Created docs/pitch_outline.md with 3-min pitch + Q&A prep - DOC 11: Evaluation summary table + /web fallback documented in README - SCN 12: Scenario summaries aligned with actual math/ML/finance templates - TRN 12: Key takeaways section for judges - UI 01-06, 08-09, 13-15: Verified Kush's frontend, closed in tracker - FND 13, JDG 09, OBS 05, OBS 08, TST 08, TST 10, TST 12: Closed out - All three tracker files updated to 152/152 (100%) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>f0d1d76 -
Close DOC 08: repo hygiene verified, Max 100% complete (41/41)
- Repo is public, .env not tracked, no API keys in source - All required files present: code, tests, frontend, Docker, notebook, docs - Max (Person C) lane fully complete7a753de -
Merge Kush frontend integration, close API 16/UI 10/UI 11
- Merge multi-stage Docker build, SPA serving, and frontend components - API 16: Docker now builds frontend + API in single container - UI 10: Frontend styled with new components (ProtocolEditor, LiveScoreGauges, etc) - UI 11: Server mounts frontend/dist with SPA catch-all for React Router - Fix root endpoint test for SPA-mode serving - Max at 97.56% (40/41, only DOC 08 remaining blocked on TRN 10)abb29f8 -
Add API 19 /web fallback route, merge Kush frontend, close UI 07
- API 19: Self-contained /web endpoint with interactive episode controls - Merge origin/master with Kush's frontend api.ts, types, components - UI 07: Closed as completed by Kush (REST+WS client helpers) - 3 new /web fallback tests, 474 total tests passingb001a03 -
Add ENV 09 disk persistence, OBS 07/09, TST 11 audit tests, close 10 Max tasks
- ENV 09: Wire write_episode_log + log_episode_reward into REST/WS terminal handlers - OBS 09: Add invalid_action_count/rate fields to EpisodeLog model - OBS 07: Create scripts/run_episode.py for local smoke testing - TST 11: Add 17 judge audit contract tests in test_audit_contract.py - Close API 05/11/18, OBS 01/03, TST 06 as already implemented11faa95 -
Add MOD 08 schema tests, V2 training stack, and close MOD 08/JDG 07/API 01/OBS 02
- MOD 08: 70 comprehensive unit tests for all Pydantic models (Kian lane 100% done) - V2 training: GRPO, SFT, evaluation, metrics, plots, CLI, corpus, datasets modules - TRN 01/02/14: Notebook skeleton, install cell, model selection - JDG 07/API 01/OBS 02: Verified and closed in trackers (already implemented) - 409 tests pass, 92/152 tasks complete (60.53%)685783a -
Merge remote-tracking branch 'origin/Kush'
# Conflicts: # frontend/src/components/PaperPanel.tsx # frontend/src/pages/EpisodePage.tsx33c9246 -
Add Person D docs, README results, episode ID, demo script, and checklists
Kush- README: scenario summaries, evaluation table with metrics, results section, fallback docs - PaperPanel: episode ID display with copy-to-clipboard (OBS 05) - EpisodePage: pass episodeId to PaperPanel - docs/demo_script.md: time-coded 60-second demo script (DOC 05) - docs/ui_smoke_checklist.md: full UI smoke test checklist (UI 12, TST 08, TST 12) - docs/recording_guide.md: screenshot, recording, and video upload guide (OBS 08, DOC 06, DOC 07)32737e6 -
Reword Oracle docs for remote push filter
e28f73d -
Add hybrid Oracle layer and update architecture docs
ec2e890 -
Recover env judge training stack and sync project tracking
13ae015 -
Recover env server client stack and deployment tracking
8b157ab -
Add deterministic judge scoring engine (JDG 01-03)
JDG 01 (rigor): structural completeness + success criteria + required element coverage. Weighted sub-scores with token matching. JDG 02 (feasibility): continuous signal from FeasibilityCheckResult with partial credit for budget, equipment, reagents, and staff. Does not rescore — wraps AGT 05 to prevent drift. JDG 03 (fidelity): substitution-aware adherence to hidden reference spec. Direct match = 1.0, allowed substitution = 0.7, miss = 0.0. Shared text helpers extracted to replicalab/utils/text.py. 18 new tests covering all 3 scorers across all domains and difficulties. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>e50dca9 -
Add living project map documenting all modules and relationships
docs/map/ covers: models, scenarios, agents, validation, scoring (planned), server, frontend, config, and tests. Updated after each implementation session to serve as codebase memory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>0ed9084 -
Add AGT 04/05/07 implementations, server integration, and doc updates
- AGT 04: build_baseline_scientist_action with domain inference - AGT 05: check_feasibility with per-dimension checks - AGT 07: compose_lab_manager_response with action type selection - Server: real Lab Manager pipeline replaces hardcoded stub responses - Tests: AGT 04/05/07 test coverage - Docs: task completion tracking and model selection notes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>7c2246c -
Add future improvements doc, import FND 09 and MOD 01-03 completions
- Add docs/future_improvements.md with domain-agnostic normalized scenario layer design, impact analysis, and 6 planned work items - Import completed FND 09 (openenv.yaml), MOD 01 (ScientistAction validators), MOD 02 (LabManagerAction validators), MOD 03 (typed ConversationEntry and Protocol models) - Update task tracking docs, completion tracker, and Kian task lists - Add model validation tests and uv.lock Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>db07024 -
Import and normalize Max backend scaffold with validated partial status
f6bf5ca -
Rename owner docs to kian max kush and update mappings
038da92 -
Complete FND 02 FND 05 FND 06 FND 07 foundation setup bundle
3f69f35 -
Add FND 08 contract, model updates, task docs, and completion tracking
- Add frozen JSON contract (docs/fnd08_frozen_json_contract.md) - Update models.py with base model assumptions and field refinements - Add per-person task docs (Person A, C, D breakdowns and lists) - Update completion tracker with FND 04/08 status - Add project management rules and AGENTS.md - Update comprehensive task division with model selection notes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>ad13149 -
Add completion tracker with per-person and per-epic progress
300f935 -
Add model selection and architecture notes
- Qwen3-4B as primary trainable Scientist, Qwen3-8B as H100 stretch - Deterministic rubric remains sole training reward - Hosted frontier evaluator for optional explanation and demo audit only - Lab Manager stays rule-based for MVP - Future model-backed Lab Manager added to stretch backlog and risk register - Updated Person B docs with base-model rationale and reward notes80aa0ec -
Refine task breakdown execution order and dependency details
5f5a3f4
-
Fix task list and breakdown to align with source of truth
- Correct totals to 29 tasks, 21.5h - Remove blueprint-based assumptions for early starts - All tasks correctly marked as blocked per task division - Keep AGT 11 and TRN 14 as separate tracked tasksa6e3459 -
Add Person B task list and execution breakdown
fb99880