assay-transfer-tool-soft-v6

Soft-target SFT of jiosephlee/Intern-S1-mini-lm for binary assay-transfer prediction (transfer vs. nontransfer) over paired assay records.

This is a mid-training checkpoint, not a completed run. Training was stopped manually at step 482 of 976 (epoch 0.49); these weights are the best-scoring evaluation checkpoint at step 400, selected on eval/validation/overall/binary_macro_f1.

Validation results (step 400)

2000 validation items, parse rate 1.00.

Metric	Value
binary_macro_f1	0.7207
binary_accuracy	0.7225
soft_mae	0.2619
transfer_precision	0.7646

By assay concept (n=400 each):

Concept	macro_f1	accuracy	soft_mae
oral_exposure	0.7895	0.7900	0.2213
Fg	0.7814	0.7825	0.2092
Fa	0.7020	0.7050	0.2566
oral_bioavailability	0.6657	0.6675	0.2962
Fh	0.6640	0.6675	0.3262

Performance is uneven across concepts: oral_exposure and Fg carry the average, while Fh and oral_bioavailability sit near 0.66.

Evaluation history on binary_macro_f1 — 0.516 (step 0), 0.442, 0.605, 0.616, 0.660, 0.701, 0.707, 0.707, 0.721 (step 400), 0.714 (step 450). The metric was plateauing but had not clearly converged when training was stopped.

Training

• Base: jiosephlee/Intern-S1-mini-lm @ fcb667c380ae01f57693a45b4b5c2d331052a107
• Data: assay_transfer_raw_pair_v6_intern, prompt/completion format, packed
• Objective: soft-target loss over 2 choices (transfer, nontransfer), format CE weight 0.1
• LR 2e-5, SMILES-embedding LR multiplier 1.5, 1 epoch scheduled
• Per-device batch 4 x grad accum 8 x 8 GPUs, max_length 4096
• packing, padding-free, flash_attention_2, Liger kernel

Run: intern-s1-mini_assay-transfer-raw-pair-v6-soft-a100 (2026-07-19). Full selection metrics are in metric.json.

Prompt format

Trained with the base model's chat template, thinking disabled (enable_thinking=False). Answers are the option markers (A) / (B). Match this at inference time or accuracy will not reproduce.