Add model card README

README.md

@@ -0,0 +1,90 @@
+---
+license: mit
+language:
+- en
+tags:
+- language-modeling
+- gpt2
+- distilgpt2
+- long-form-generation
+- coherence
+- hard-negatives
+- wikitext
+- research
+pipeline_tag: text-generation
+---
+
+# ForesightLM Phase 1
+
+ForesightLM Phase 1 is an experimental research checkpoint exploring whether self-loop hard negatives can provide a useful calibration signal for semantic-level coherence-aware generation.
+
+This repository contains Phase 1 model checkpoints, robustness outputs, and small ablation artifacts for the ForesightLM workshop paper.
+
+## Main idea
+
+The project investigates a lightweight extension around a DistilGPT-2 style language model trained on WikiText-103. The goal is not to claim solved long-form coherence, but to test whether future-aware objectives and self-loop hard negatives can improve semantic-level generation indicators.
+
+## Uploaded checkpoints
+
+This repository includes:
+
+| Path | Description |
+|---|---|
+| `outputs/phase1/v2_wikitext103_seed42_distilgpt2` | Main ForesightLM-v2 Phase 1 checkpoint, `lambda_hardneg=0.10` |
+| `outputs/phase1/v2_wikitext103_lh005_seed42_distilgpt2` | Lambda ablation checkpoint, `lambda_hardneg=0.05` |
+| `outputs/phase1/v2_wikitext103_lh020_seed42_distilgpt2` | Lambda ablation checkpoint, `lambda_hardneg=0.20` |
+| `outputs/phase1/bootstrap` | Paired bootstrap confidence interval results |
+| `outputs/phase1/lambda_hardneg_ablation` | Lambda hard-negative ablation summary |
+| `outputs/phase1/decoding_eval` | Decoding summary and qualitative examples |
+
+## Phase 1 main metrics
+
+Main ForesightLM-v2 result:
+
+| metric | value |
+|---|---:|
+| LM loss | 3.9384 |
+| Future loss | 2.7180 |
+| Hard-negative margin gap | 0.0614 |
+| Hard-negative satisfied rate | 0.4107 |
+
+## Lambda hard-negative ablation
+
+| lambda_hardneg | eval_lm ↓ | eval_future ↓ | hardneg_gap ↑ | satisfied ↑ |
+|---:|---:|---:|---:|---:|
+| 0.05 | 3.9385 | 2.7214 | 0.0512 | 0.3571 |
+| 0.10 | 3.9384 | 2.7180 | 0.0614 | 0.4107 |
+| 0.20 | 3.9383 | 2.7148 | 0.0701 | 0.4643 |
+
+The small ablation suggests that increasing the hard-negative weight from `0.10` to `0.20` improves held-out hard-negative separation while preserving language-modeling loss.
+
+## Bootstrap robustness
+
+Paired bootstrap over prompt IDs suggests that calibrated v2 decoding robustly improves prompt relevance, topic drift, progression, semantic-loop rate, and prompt-drift rate relative to Core and baseline.
+
+Raw v2 improvements are directional but generally not conclusive under the paired bootstrap intervals.
+
+## Important limitations
+
+These are Phase 1 research artifacts. The current evidence is limited to:
+
+- one main domain: WikiText-103
+- one primary seed
+- small ablation scale
+- automatic coherence indicators rather than full human evaluation
+
+The results should be interpreted cautiously. ForesightLM does not solve long-form coherence; Phase 1 only suggests that self-loop hard negatives may provide a useful calibration signal.
+
+## GitHub
+
+Code and compact result artifacts are available on GitHub:
+
+https://github.com/Ahmet2001/foresightLM
+
+Current pushed branch:
+
+`v2-self-loop-fixes`
+
+## Citation
+
+If you use these artifacts, please cite the ForesightLM Phase 1 workshop paper once available.