Update README.md

README.md

@@ -1,209 +1,143 @@
 ---
+license: llama3
 base_model: meta-llama/Meta-Llama-3-8B-Instruct
-library_name: peft
-pipeline_tag: text-generation
 tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B-Instruct
-- grpo
-- lora
-- transformers
-- trl
+  - sports-science
+  - load-management
+  - athlete-monitoring
+  - lora
+  - grpo
+language:
+  - en
 ---
 
-# Model Card for Model ID
+# LoadBrief: Athlete Load-Management Brief Generation
 
-<!-- Provide a quick summary of what the model is/does. -->
+LoadBrief is a [Llama 3 8B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) model fine-tuned to convert a plain-language athlete-monitoring narrative into a structured, audience-conditioned load-management brief. Given a description of an athlete's recent training load, heart-rate variability (HRV), and wellness scores, the model produces a brief that declares an overall risk level, classifies overreaching state, explains how the fatigue signals interact, gives specific training-modification recommendations, and flags when medical review is warranted — written for an **athlete**, **coach**, or **sports scientist**.
 
+## Introduction
 
+Wearable technology has made physiological monitoring available to athletes at every level, but data collection has far outpaced interpretation. Translating an elevated [acute-to-chronic workload ratio](https://bjsm.bmj.com/content/50/5/273), a multi-day drop in HRV, and several poor wellness scores into a sound training decision requires expert judgment that most athletes and coaches cannot access. General-purpose language models struggle with this task: they apply generic rather than sport-specific thresholds, reason inconsistently when monitoring signals conflict, and cannot reliably hold an audience register through prompting alone.
 
-## Model Details
+LoadBrief addresses this by fine-tuning a specialized model on a purpose-built synthetic dataset whose labels are grounded in published sports-science criteria. The model was trained in two stages — supervised fine-tuning (SFT) followed by [Group Relative Policy Optimization (GRPO)](https://arxiv.org/abs/2402.03300) — using [LoRA](https://arxiv.org/abs/2106.09685). On a held-out test split it recovers the correct risk classification about three-quarters of the time (composite reward 0.74 against a reference ceiling of 0.729; 97% of risk predictions within one class of ground truth), a dramatic improvement over the untuned base model (reward 0.39, 0% exact risk accuracy). A key empirical finding is that **SFT alone saturates the rule-derived reward — GRPO adds no measurable benefit** — and that the specialization comes at a measurable cost to general benchmark performance, discussed under Limitations.
 
-### Model Description
+## Data
 
-<!-- Provide a longer summary of what this model is. -->
+The model is trained on [LoadBrief-50K](https://huggingface.co/datasets/tyhob/loadbrief-50k), a synthetic dataset of 50,000 monitoring-narrative → brief pairs. No public dataset pairs athlete monitoring data with written interpretive briefs, so the data is generated by a rule-based simulator rather than by prompting a language model — ensuring no clinical content is invented. The simulator encodes four bodies of published sports science as quantitative thresholds: [Gabbett (2016)](https://bjsm.bmj.com/content/50/5/273) for acute-to-chronic workload ratio zones, [Plews et al. (2013)](https://link.springer.com/article/10.1007/s40279-013-0071-8) for HRV suppression, [Meeusen et al. (2013)](https://journals.lww.com/acsm-msse/fulltext/2013/01000/prevention,_diagnosis,_and_treatment_of_the.23.aspx) for overreaching classification, and [Hooper & Mackinnon (1995)](https://link.springer.com/article/10.2165/00007256-199520050-00003) for wellness norms. Generation runs *backward from the label*: each scenario's ground-truth classification is fixed first, a realistic 28-day time series is sampled to match, and the same rules independently confirm the label. Each scenario is rendered into all three audience registers, so one simulated athlete yields multiple training pairs sharing ground-truth labels but differing in tone and technical depth.
 
+The corpus was partitioned into an **80/10/10 train/validation/test split** (40,000 / 5,000 / 5,000 examples) generated with **random seed 42** for reproducibility. The 5,000-example test split is disjoint from training and is used for all in-domain evaluation reported below. Scenarios span three complexity tiers (clear-cut, moderate, and conflicting-signal cases) across 66 sports and 18 scenario types.
 
+## Methodology
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+The base model is [Llama 3 8B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct), fine-tuned with [LoRA](https://arxiv.org/abs/2106.09685) so the full pipeline runs on a single commodity GPU. Training proceeds in two stages. An **SFT** stage first teaches the structured brief format and the three audience registers from the labeled input–output pairs. A **[GRPO](https://arxiv.org/abs/2402.03300)** stage then optimizes a five-component composite reward — correct risk-level declaration, recommendation specificity, correct overreaching terminology, presence of an escalation trigger, and clinically useful output length — by sampling several candidate briefs per prompt and reinforcing the higher-scoring ones. GRPO is chosen over PPO because it removes the separate value network (roughly halving memory), and over preference methods such as DPO because the task has a programmatic reward function rather than human preference pairs.
 
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+**Hyperparameters (for reproducibility).** SFT stage: LoRA rank 32, alpha 64, dropout 0.05, applied to all attention and MLP projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj); learning rate 2e-4 with a cosine schedule, 3% warmup, weight decay 0.01; per-device batch size 2 with gradient accumulation 16 (effective batch 32); 1 epoch over 20,000 training examples; max sequence length 512; gradient checkpointing enabled; best checkpoint selected by validation loss. GRPO stage: learning rate 5e-6, KL coefficient (β) 0.1, sampling temperature 0.8, top-p 0.95, generation seed 42, LoRA configuration inherited from the SFT stage. A learning-rate sweep (5e-5, 5e-6, 5e-7) and multiple random seeds confirmed the results are stable across configurations; the released model uses the 5e-6 GRPO configuration.
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
+The model is evaluated on three standard benchmarks plus the held-out LoadBrief test split. [MedQA (USMLE)](https://huggingface.co/datasets/bigbio/med_qa) probes domain-adjacent clinical reasoning; [MMLU](https://huggingface.co/datasets/cais/mmlu) (clinical knowledge, college medicine, and philosophy subjects) measures retained general and medical knowledge, with philosophy as a domain-unrelated control; and the LoadBrief test split measures in-domain performance via composite reward and risk-classification accuracy. Two similar-size instruction-tuned models are included as comparisons: [Qwen 2.5 7B Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct), a strong recent 7B model, and [Mistral 7B Instruct v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3), a widely-used baseline 7B — both represent the general-purpose alternatives a practitioner might otherwise reach for.
 
-#### Testing Data
+| Model | MedQA | MMLU clinical | MMLU college med | MMLU philosophy | LoadBrief test (reward / risk acc) |
+|---|---|---|---|---|---|
+| [Llama 3 8B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) (base) | 61.7 | 77.4 | 63.6 | 71.7 | 0.39 / 0.00 |
+| **LoadBrief (SFT+GRPO)** | 55.8 | 66.0 | 59.5 | 59.8 | **0.74 / 0.72** |
+| [Qwen 2.5 7B Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) | 61.7 | 77.4 | 67.6 | 73.6 | — |
+| [Mistral 7B Instruct v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) | 51.9 | 68.7 | 59.0 | 67.2 | — |
 
-<!-- This should link to a Dataset Card if possible. -->
+*Benchmark values are accuracy percentages. LoadBrief test reports composite reward (reference ceiling 0.729) and exact risk-level accuracy. Comparison models are not trained on the LoadBrief format and, like the untuned base model, score at floor on the in-domain task, so that column is omitted for them.*
 
-[More Information Needed]
+On the general benchmarks, the comparison models match or exceed LoadBrief — Qwen 2.5 7B is strongest (72.9% MMLU average) while LoadBrief sits lowest (61.8% average), reflecting the general capability traded away during task specialization. However, none of the comparison models produces valid structured briefs: on the in-domain task only the fine-tuned model performs meaningfully (0.74 reward, 0.72 risk accuracy) versus the base model's floor (0.39, 0.00). In short, the specialization that lowers general-benchmark performance is precisely what enables the target task.
 
-#### Factors
+## Usage and Intended Uses
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+LoadBrief is a **LoRA adapter** applied to Llama 3 8B Instruct. Load it with the [Transformers](https://huggingface.co/docs/transformers) and [PEFT](https://huggingface.co/docs/peft) libraries:
 
-[More Information Needed]
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
 
-#### Metrics
+BASE = "meta-llama/Meta-Llama-3-8B-Instruct"
+ADAPTER = "tyhob/loadbrief"  
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+tokenizer = AutoTokenizer.from_pretrained(BASE)
+model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
+model = PeftModel.from_pretrained(model, ADAPTER)
+model.eval()
 
-[More Information Needed]
+prompt = """### Monitoring Narrative:
+Weekly load management brief (4-week monitoring block): Soccer Midfielder, professional.
+Acute:chronic workload ratio of 1.71. HRV suppressed 13 ms below baseline for 6
+consecutive days. Wellness moderately depressed across sleep, fatigue, and mood.
 
-### Results
+### Audience: coach
 
-[More Information Needed]
+### Load Management Brief:
+"""
 
-#### Summary
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+out = model.generate(**inputs, max_new_tokens=400, temperature=0.7)
+print(tokenizer.decode(out[0], skip_special_tokens=True))
+```
 
+**Intended uses.** LoadBrief is intended as a research demonstration and educational tool for translating athlete-monitoring data into structured, evidence-based interpretations across audiences. It is well suited to generating first-draft briefs that a coach or sports scientist reviews. **It is not a medical device and should not be used for autonomous clinical or return-to-play decisions.** The model applies established sports-science criteria consistently; it does not predict real-world injury outcomes.
 
+## Prompt Format
 
-## Model Examination [optional]
+Each prompt contains three parts: the monitoring narrative, an audience specifier (`athlete`, `coach`, or `sports_scientist`), and a trailing header that cues generation.
 
-<!-- Relevant interpretability work for the model goes here -->
+```
+### Monitoring Narrative:
+Weekly load management brief (4-week monitoring block): Subject: Canoe Kayak Sprint
+Kayak Single (semi professional), 10-year training background, return from injury phase.
+Training load is well-periodized — acute:chronic workload ratio of 1.256 reflects a
+healthy balance between fitness and fatigue. Full wellness questionnaire not available
+— sleep data from wearable only.
 
-[More Information Needed]
+### Audience: athlete
 
-## Environmental Impact
+### Load Management Brief:
+```
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+## Expected Output Format
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+The model returns a structured brief with a risk header, per-signal sections (training load, HRV, wellness), a signal-integration analysis, recommendations, and an escalation trigger. Vocabulary, headers, and length adapt to the audience: athletes receive a YOUR STATUS: header with lay terms ("recovery score," "a few easy days"), while coaches and sports scientists receive a RISK LEVEL: header with clinical terminology and exact values. The following is a sports-scientist brief:
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+```
+RISK LEVEL: HIGH
 
-## Technical Specifications [optional]
+TRAINING LOAD:
+ACWR of 1.14 is within the optimal training range. Acute load: 106 AU | Chronic load: 94 AU.
 
-### Model Architecture and Objective
+PHYSIOLOGICAL READINESS (HRV):
+HRV critically suppressed: 23.3 ms (22.4 ms below baseline for 7 consecutive days).
 
-[More Information Needed]
+SUBJECTIVE WELLNESS:
+Composite status: moderately depressed. Amber flags: sleep, fatigue, soreness, mood.
 
-### Compute Infrastructure
+CLINICAL CLASSIFICATION:
+Functional overreaching — short recovery period (3–7 days) will restore performance.
 
-[More Information Needed]
+RECOMMENDATIONS:
+- If a non-training stressor is identified: maintain load, address the stressor.
+- If none: reduce load 20% and reassess in 3 days; if no improvement, seek medical review.
 
-#### Hardware
+ESCALATION TRIGGERS:
+Persistent multi-system suppression, performance decline, or athlete distress.
+```
 
-[More Information Needed]
+## Limitations
 
-#### Software
+**General-capability forgetting.** Fine-tuning measurably degraded performance on general and clinical-reasoning benchmarks: MedQA fell 61.7% → 55.8%, and MMLU subjects dropped by 4–12 points (clinical knowledge −11.3, philosophy −11.9). Philosophy — the benchmark least related to the training domain — dropped the most, indicating the degradation is general rather than a targeted reshaping of clinical knowledge. LoadBrief is a **specialist** and should not be relied upon for general or clinical question-answering outside its task.
 
-[More Information Needed]
+**Weakness on override scenarios.** A stratified [LLM-as-judge](https://arxiv.org/abs/2306.05685) evaluation found clinical accuracy is weakest on scenarios where the intended classification overrides the raw signals — overtraining syndrome, heat acclimatization, and youth growth spurt score lowest (~2.2–2.6 of 7) versus ~4.0–4.7 for signal-clear cases. This traces to a data-generation limitation: in these scenarios the brief prose was driven by the sampled signals while the label reflected scenario intent, producing internally inconsistent briefs the model learned to reproduce. Regenerating these scenarios with consistency-checked templates is identified as future work.
 
-## Citation [optional]
+**Rule-grounded, not outcome-validated.** The model learns to apply established sports-science criteria consistently; its labels are correct by construction with respect to those criteria, but are not validated against real-world injury outcomes. Validation against ratings from practicing sports scientists is future work.
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+**SFT saturation.** The GRPO stage adds no measurable benefit over SFT alone on this task, because the rule-derived reward is already saturated after supervised fine-tuning. Users seeking to reproduce or extend this work can achieve equivalent quality with the SFT stage alone.
 
-**BibTeX:**
+**Training-data subset.** SFT used a 20,000-example subset of the 40,000-example training split, chosen to fit within the memory budget. Training compute scales linearly with example count, so the full split was tractable but not necessary: as noted above, the model reached the reference reward ceiling at this scale, indicating additional in-distribution data would yield diminishing returns. Targeted augmentation of under-performing override scenarios, rather than more uniform data, is the more promising direction for future improvement.
 
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
+---
 
-- PEFT 0.19.1
+*Built with Meta Llama 3. Model weights are subject to the [Meta Llama 3 license](https://llama.meta.com/llama3/license/). Training code and the data-generation pipeline are available on [GitHub](https://github.com/tyhobbs/LoadBrief).*