--- license: mit library_name: peft base_model: - Qwen/Qwen2.5-0.5B - Qwen/Qwen3-0.6B-Base - Qwen/Qwen3.5-0.8B-Base - Qwen/Qwen3.5-4B-Base - Qwen/Qwen3.5-9B-Base - jaredpalmer/kev-27b tags: - single-cell - scrna-seq - cell-type-annotation - decision-model - lora - biology --- # scKevType Cell-type annotation as a **decision**, not a generated sentence. scKevType is a family of fine-tuned decision models for single-cell RNA-seq cell-type annotation. Given a rendered state built from a cluster's marker evidence, a model answers one typed question over a presented set of candidate labels in a single forward pass and returns a **scored option**: one of the options shown, with a probability attached. Declining is an option like any other. There is no prose to parse. The answer is a label and a number, so it can be calibrated, thresholded, compared across models, and audited. ## Why this exists The annotation field has moved to language models, and those systems generate text. Generated text has no probability vector, so there is nothing to threshold or calibrate; it also cannot decline, so a model asked about a cluster it cannot place still answers fluently. Exact-match scoring on generated text needs a parsing rule, and the rule becomes a scoring decision the model never made. scKevType removes that layer. The comparison it makes possible is between models answering identical questions under one scoring rule, rather than between parsers. ## The six arms Each size is a separate LoRA adapter on a frozen base, trained with one shared recipe. Pick by accuracy against cost: the 4B and 9B lead in distribution, and the 27B is the best arm on the held-out laboratory. | dir | base model | in-distribution | held-out | adapter + head | |---|---|---:|---:|---:| | `0.5B/` | `Qwen/Qwen2.5-0.5B` | 0.7561 | 0.8115 | 37 MB | | `0.6B/` | `Qwen/Qwen3-0.6B-Base` | 0.7586 | 0.7281 | 43 MB | | `0.8B/` | `Qwen/Qwen3.5-0.8B-Base` | 0.7898 | 0.7950 | 45 MB | | `4B/` | `Qwen/Qwen3.5-4B-Base` | 0.8191 | 0.8492 | 135 MB | | `9B/` | `Qwen/Qwen3.5-9B-Base` | 0.8191 | 0.8988 | 182 MB | | `27B/` | `jaredpalmer/kev-27b` | 0.7928 | 0.9100 | 478 MB | *In-distribution*: served accuracy on 4,035 `panel_choice` questions from the training corpus (exact match over the option set, abstain available on a subset). *Held-out*: served accuracy on 2,600 questions from Baron pancreas data, a laboratory absent from training. The two suites have different label counts and majority-class floors (0.1836 against 0.3865), so **raw levels are not comparable across them**; read each column on its own. ## Repository layout ``` 0.5B/ 0.6B/ 0.8B/ 4B/ 9B/ 27B/ adapter_model.safetensors the LoRA adapter, rank 16 on every linear projection adapter_config.json with the base model and its pinned revision head.pt the pointer head tokenizer.json, tokenizer_config.json, chat_template.jinja provenance.json the run's own record: data read, epochs, seconds, loss training_config.json hyperparameters and placement manifest.json per-arm base, revision, checksums, reported scores ``` The base weights are **not** included. Each arm is an adapter over a public base. ## Loading This is not a `from_pretrained` drop-in, and the card will not pretend otherwise. The pointer head carries semantics that the standard PEFT loader does not know: it scores the presented options, and the reported probability is the option logits read at a fixed temperature. A documented loader is required. Treat this repository as the weights, not as a `transformers` pipeline. ```python # shape only; see the project code for the loader adapter = PeftModel.from_pretrained(base, "pwwang/scKevType", subfolder="4B") head = torch.load(hf_hub_download("pwwang/scKevType", "4B/head.pt"), weights_only=True) ``` ## Training One recipe across all six sizes: LoRA rank 16 on every linear projection of a frozen base, `lora_alpha` 32, learning rate 2e-4, one record per step accumulated over eight steps, bf16, a single epoch. The pointer head is trained from its warm start rather than reinitialised. No seed replicates were run, so no seed spread is reported. Data: 4,356,215 cells read from six public atlases (pancreas, immune, brain, and multi-organ), labels mapped into the Cell Ontology register, with unresolvable labels excluded rather than remapped. Splits are per dataset with donor-level holdout, so every atlas contributes to both training and test; Baron is the one suite held out entirely. ## Evaluation, honestly Reported numbers are served accuracy under one scoring rule, on fixed question sets, beside arms that answered the identical questions. The relevant baselines in the paper are a commercial decision API and general-purpose language models of the same sizes. Two things worth knowing before quoting a number: - **Accuracy alone is not the claim.** On binary marker questions, fine-tuning is what creates an abstention policy at all; scale does not. A model that never declines can look strong on accuracy while being worse than a constant rule on a skewed family. - **Cross-suite comparison is invalid.** A suite with 10 labels and a 0.3865 floor is not the same task as one with 65 labels and a 0.1836 floor. ## Limitations - Trained on human atlases in the Cell Ontology register. Labels outside that register are out of scope, and the model will still answer if asked. - Seven cell types or sixty-five, the option set is whatever is presented; the model does not know the true label set of an unseen tissue. - One epoch, one seed, no replicates: differences smaller than a few points between adjacent sizes should not be read as real. - The abstention behaviour is a learned policy, not a calibrated guarantee. ## Licence Adapter weights and this card: MIT. Each base model keeps its own licence, which governs the combined work. ## Citation Citation details will follow the paper.