File size: 6,158 Bytes
4f15b28
aa1e368
4f15b28
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
aa1e368
4f15b28
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
---

license: mit
library_name: peft
base_model:
  - Qwen/Qwen2.5-0.5B
  - Qwen/Qwen3-0.6B-Base
  - Qwen/Qwen3.5-0.8B-Base
  - Qwen/Qwen3.5-4B-Base
  - Qwen/Qwen3.5-9B-Base
  - jaredpalmer/kev-27b
tags:
  - single-cell
  - scrna-seq
  - cell-type-annotation
  - decision-model
  - lora
  - biology
---


# scKevType

Cell-type annotation as a **decision**, not a generated sentence.

scKevType is a family of fine-tuned decision models for single-cell RNA-seq cell-type annotation.
Given a rendered state built from a cluster's marker evidence, a model answers one typed question
over a presented set of candidate labels in a single forward pass and returns a **scored option**:
one of the options shown, with a probability attached. Declining is an option like any other.

There is no prose to parse. The answer is a label and a number, so it can be calibrated,
thresholded, compared across models, and audited.

## Why this exists

The annotation field has moved to language models, and those systems generate text. Generated text
has no probability vector, so there is nothing to threshold or calibrate; it also cannot decline, so
a model asked about a cluster it cannot place still answers fluently. Exact-match scoring on
generated text needs a parsing rule, and the rule becomes a scoring decision the model never made.

scKevType removes that layer. The comparison it makes possible is between models answering identical
questions under one scoring rule, rather than between parsers.

## The six arms

Each size is a separate LoRA adapter on a frozen base, trained with one shared recipe. Pick by
accuracy against cost: the 4B and 9B lead in distribution, and the 27B is the best arm on the
held-out laboratory.

| dir | base model | in-distribution | held-out | adapter + head |
|---|---|---:|---:|---:|
| `0.5B/` | `Qwen/Qwen2.5-0.5B` | 0.7561 | 0.8115 | 37 MB |
| `0.6B/` | `Qwen/Qwen3-0.6B-Base` | 0.7586 | 0.7281 | 43 MB |
| `0.8B/` | `Qwen/Qwen3.5-0.8B-Base` | 0.7898 | 0.7950 | 45 MB |
| `4B/` | `Qwen/Qwen3.5-4B-Base` | 0.8191 | 0.8492 | 135 MB |
| `9B/` | `Qwen/Qwen3.5-9B-Base` | 0.8191 | 0.8988 | 182 MB |
| `27B/` | `jaredpalmer/kev-27b` | 0.7928 | 0.9100 | 478 MB |

*In-distribution*: served accuracy on 4,035 `panel_choice` questions from the training corpus
(exact match over the option set, abstain available on a subset).
*Held-out*: served accuracy on 2,600 questions from Baron pancreas data, a laboratory absent from
training. The two suites have different label counts and majority-class floors (0.1836 against
0.3865), so **raw levels are not comparable across them**; read each column on its own.

## Repository layout

```

0.5B/ 0.6B/ 0.8B/ 4B/ 9B/ 27B/

    adapter_model.safetensors   the LoRA adapter, rank 16 on every linear projection

    adapter_config.json         with the base model and its pinned revision

    head.pt                     the pointer head

    tokenizer.json, tokenizer_config.json, chat_template.jinja

    provenance.json             the run's own record: data read, epochs, seconds, loss

    training_config.json        hyperparameters and placement

manifest.json                   per-arm base, revision, checksums, reported scores

```

The base weights are **not** included. Each arm is an adapter over a public base.

## Loading

This is not a `from_pretrained` drop-in, and the card will not pretend otherwise. The pointer head
carries semantics that the standard PEFT loader does not know: it scores the presented options, and
the reported probability is the option logits read at a fixed temperature. A documented loader is
required. Treat this repository as the weights, not as a `transformers` pipeline.

```python

# shape only; see the project code for the loader

adapter = PeftModel.from_pretrained(base, "pwwang/scKevType", subfolder="4B")

head    = torch.load(hf_hub_download("pwwang/scKevType", "4B/head.pt"), weights_only=True)

```

## Training

One recipe across all six sizes: LoRA rank 16 on every linear projection of a frozen base,
`lora_alpha` 32, learning rate 2e-4, one record per step accumulated over eight steps, bf16, a
single epoch. The pointer head is trained from its warm start rather than reinitialised. No seed
replicates were run, so no seed spread is reported.

Data: 4,356,215 cells read from six public atlases (pancreas, immune, brain, and multi-organ), labels
mapped into the Cell Ontology register, with unresolvable labels excluded rather than remapped.
Splits are per dataset with donor-level holdout, so every atlas contributes to both training and
test; Baron is the one suite held out entirely.

## Evaluation, honestly

Reported numbers are served accuracy under one scoring rule, on fixed question sets, beside arms
that answered the identical questions. The relevant baselines in the paper are a commercial decision
API and general-purpose language models of the same sizes.

Two things worth knowing before quoting a number:

- **Accuracy alone is not the claim.** On binary marker questions, fine-tuning is what creates an
  abstention policy at all; scale does not. A model that never declines can look strong on accuracy
  while being worse than a constant rule on a skewed family.
- **Cross-suite comparison is invalid.** A suite with 10 labels and a 0.3865 floor is not the same
  task as one with 65 labels and a 0.1836 floor.

## Limitations

- Trained on human atlases in the Cell Ontology register. Labels outside that register are out of
  scope, and the model will still answer if asked.
- Seven cell types or sixty-five, the option set is whatever is presented; the model does not know
  the true label set of an unseen tissue.
- One epoch, one seed, no replicates: differences smaller than a few points between adjacent sizes
  should not be read as real.
- The abstention behaviour is a learned policy, not a calibrated guarantee.

## Licence

Adapter weights and this card: MIT. Each base model keeps its own licence, which governs the
combined work.

## Citation

Citation details will follow the paper.