| # SciHigh 2026 Subtask 1 — Final Pipeline |
|
|
| Team Yushkk99. Task: generate research highlights from paper abstracts |
| (MixSub corpus, 19,785 ScienceDirect papers). Official metric: ROUGE-L F1. |
|
|
| ## Results (validation, 1,985 rows, plain rougeL / no stemming) |
|
|
| | system | val ROUGE-L | |
| | --- | --- | |
| | FIRE-2025 winning system (reference) | 23.45 | |
| | bart-large-cnn, original data | 24.37 | |
| | bart-large-cnn, repaired data + titles (run 1) | 26.37 | |
| | **Qwen2.5-72B QLoRA, repaired data + titles (run 2)** | **28.33** | |
|
|
| ## Pipeline |
|
|
| Data enrichment (all sources free and sanctioned; test rows touched only |
| via abstract/title metadata — never highlight fields): |
|
|
| 1. `recover_abstracts.py` — ~40% of corpus abstracts are truncated |
| mid-sentence. Each row is matched to its ScienceDirect PII (verified |
| 100% by abstract-text join against the dataset's source repository), |
| resolved to a DOI via Elsevier's keyless Article API, and its complete |
| abstract fetched from Semantic Scholar. A label-free acceptance filter |
| (recovered text must contain >=90% of the truncated text's tokens) |
| guarantees same-paper extensions and structurally excludes records |
| containing highlight fields. Acceptance: ~95% of truncated rows. |
| 2. `fetch_titles.py` — paper titles for 100% of rows via the same keyless |
| PII lookup. |
| 3. `build_datasets.py` — rebuilds all splits with recovered abstracts |
| normalized to the corpus's punctuation-stripped format (transform |
| validated token-for-token against 5,481 known prefixes, mean agreement |
| 0.987), then prepends titles: `<title> | <abstract>`. |
|
|
| Modeling: |
|
|
| 4. `train_qwen_qlora.py` — one-epoch QLoRA SFT (4-bit NF4, LoRA r=16 on |
| attention+MLP) of a Qwen2.5 Instruct base on the 15,960-pair titled |
| pool, chat-formatted, greedy decoding. The identical script trains the |
| 7B (24GB GPU) and 72B (80GB GPU) variants; only `--base` changes. |
| 5. `infer_test.py` — greedy test-set inference producing the submission CSV. |
| 6. `validate_submission.py` — format gate: row count, column schema, no |
| empty predictions, length statistics. |
|
|
| Run 1 (bart lineage) reproduces via the Kaggle kernels |
| (`res-3-title-prefix`) recorded in the project artifacts. |
|
|
| ## Models |
|
|
| - `Yakk99/scihigh2026-subtask1-qwen72b-qlora` — run-2 adapter (72B) |
| - `Yakk99/scihigh2026-subtask1-qwen7b-qlora` — 7B pilot adapter |
| - `Yakk99/scihigh2026-subtask1-bart-titled` — run-1 full model |
|
|
| ## Negative results (tested and closed, full logs in project artifacts) |
|
|
| BRIO contrastive calibration (two decode-matched attempts, ~0 delta); |
| MBR/DPO preference signal on the 72B pool (−2.0 — sampled candidates |
| cluster below the greedy mode of a well-trained model); oracle bullet |
| reordering (+0.0002); GRPO (rejected on published scientific-domain |
| negatives); RAG (measured negative on this exact corpus by a FIRE-2025 |
| team). |
|
|
