README.md

---
title: AI Cell Quantification
emoji: "⬜"
colorFrom: gray
colorTo: gray
sdk: gradio
sdk_version: 5.49.1
python_version: "3.12"
suggested_hardware: zero-a10g
app_file: app.py
pinned: false
license: mit
---

# AI Cell Quantification (CellQuant): nucleus, antibody and puncta counting for fluorescence sections

CellQuant counts the six categories that were annotated in CVAT on 35-day spinal-cord
sections stained with DAPI (blue), an antibody (green: Iba1 or TMEM119) and a red punctum
marker that the lab calls **"Nostril"**:

| Label | Meaning |
|---|---|
| Cell, Antibody + | nucleus with green antibody signal, no punctum |
| Cell, Antibody − | nucleus without antibody signal, no punctum |
| Cell, Nostril, Antibody + | nucleus containing a red punctum, antibody positive |
| Cell, Nostril, Antibody − | nucleus containing a red punctum, antibody negative |
| Nostril in Nucleus | red punctum inside a nucleus |
| Nostril External of Nucleus | red punctum outside every nucleus |

Every nucleus receives exactly one of the four *Cell* labels.

## How it works

* **Model** – a UNet with an ImageNet-pretrained ResNet34 encoder (`cellquant/model.py`) that
  predicts, per pixel: nucleus interior / border / background, P(antibody +), P(nucleus has
  punctum), punctum in-nucleus / external / background, and the normalised distance to the
  nucleus outline (1 at the centre of every nucleus, 0 at its outline). A full-resolution
  refinement stage keeps 5-pixel puncta.
* **Instances** – nucleus foreground comes from the class map; markers (distance > 0.5) and the
  watershed landscape come from the predicted distance map, which has one peak per nucleus and
  a valley between touching ones; a nucleus whose peak predicted distance stays below 0.64 is
  dropped (the distance head is trained to reach 1 in every annotated nucleus and 0 elsewhere,
  so a low peak marks a faint blob the annotator would have skipped; `PostConfig.min_peak_dist`,
  0 switches it off); puncta are connected components (`cellquant/postprocess.py`).
  Checkpoints without the distance channel fall back to a watershed on the interior map.
* **Labels** – antibody status is the mean probability inside each nucleus; a nucleus is
  "Nostril" when a punctum classified "in nucleus" lies inside it (the annotation protocol;
  the network's own nucleus-level head is available as an alternative in *Advanced*);
  punctum type comes from the network (switchable to a purely geometric rule).
* **Training data** – nine 1024×1024 RGB images with 1,569 nucleus outlines and 1,057
  punctum outlines, parsed directly from the CVAT zip exports (`cellquant/annotations.py`).

## Validation (see `reports/SUMMARY.md`, `reports/cv_v3/cv_report.md` and `reports/cv_v3/comparison.md`)

Leave-one-image-out cross-validation: nine models, each tested on an image it never saw, and
each image tested 23 times (repeat runs, TTA settings, 8 rotations/flips, 6 photometric
perturbations, rule variants). Micro-averaged over the nine held-out images:

| Measure | Precision | Recall | F1 |
|---|---:|---:|---:|
| Nucleus detection, IoU ≥ 0.5 | 81.1% | 84.2% | 82.6% |
| Nucleus detection, centroid inside the annotated nucleus | 88.8% | 92.2% | 90.5% |
| Punctum detection (centroid within 6 px) | 76.4% | 84.8% | 80.4% |

With 8× test-time augmentation (the app's default) nucleus F1 is 82.9% and puncta F1 81.8%;
adding image scales on top (24× or 40× passes, `reports/cv_v3/tta_study.md`) gains nothing further
for nuclei and 0.4 points for puncta, at four to six times the run time, so the 8-member symmetry
group is where TTA stops paying. Among matched nuclei the antibody +/− call is right for 93.8% and the Nostril attribute for
95.7%; puncta are classified in-nucleus vs external correctly 93.4% of the time. Scores vary by
at most ±3 F1 points across the 14 rotated/perturbed runs of each image, and repeated runs are
bit-identical. The weak spots are antibody-negative nuclei (F1 40%; only 110 examples, 62 of
them in one image, and the class-balanced loss trades their precision for recall), the five-example
"Cell, Nostril, Antibody −" class, and dense clusters of touching nuclei (still the main source of misses,
although the distance-map watershed recovers many that the first version merged). The model predicts
1629 nuclei for the 1569 annotated ones across the nine held-out images. Earlier versions on the same
protocol: v1 (2-px border, watershed on the interior map, unweighted antibody loss) 77.9% nucleus F1,
87.5% by the centroid rule, 79.9% puncta F1 (`reports/cv/`); v2 (this model without the peak-distance
rule) 80.4%, 88.8% and 80.4% (`reports/cv_v2/`). The shipped `weights/cellquant.pt` was trained on all nine
images with the v2 recipe; the peak-distance rule is post-processing only.

## Using the Space

1. Upload one or more images (TIF/PNG/JPG, blue = nuclei, green = antibody, red = puncta).
   Use the 8-bit RGB export of the microscope software (what the model was trained on);
   16-bit inputs are auto-scaled so that their 99.95th percentile maps to 255. Multi-channel
   TIFs with one channel per page (ImageJ composites, OME-TIFF) are supported; greyscale
   inputs are flagged. Images larger than 1536 px are processed in overlapping tiles.
2. Every image is analysed eight times (four rotations × two flips) and the outputs averaged, the
   setting that scored best in cross-validation (nucleus F1 +0.3, puncta F1 +1.4 points over a
   single pass). `CELLQUANT_TTA=1` or `4` changes this for local runs.
3. Click **Analyse**. You get a counts table, colour overlays and a zip with
   `counts.csv`, `objects.csv` (every nucleus and punctum with centroid, area, probabilities)
   and `results.json`. The table and `counts.csv` have one row per image and eight count columns:
   the four nucleus labels followed by their total, **All nuclei**, then the two punctum labels
   followed by their total, **All puncta** (the same totals as in the validation table of
   section 2; `results.json` carries them under `totals`). Each overlay caption gives the two totals.
4. A written summary follows the counts in both sections when the Space has a `GEMINI_API_KEY`
   secret (optionally `GEMINI_MODEL`, default `gemini-4-argon`); without the key a one-line notice
   takes its place. Gemini only restates the counts in prose and comments on image quality or on
   what the missed objects have in common; it never changes a number, and it is called after the
   GPU work, so it costs no GPU time.

## Checking the model against the annotations

The lower half of the Space, *Check the model against the 9 annotated images*, shows the nine
annotated images as a strip of thumbnails (the tenth file in the data folder is a byte-identical
copy of F113). Click one: the model runs on the full image (eight rotations and flips averaged),
its prediction is matched with the CVAT annotation object by object
(nuclei at IoU ≥ 0.5, puncta by centre distance ≤ 6 px), and you get

* three panels side by side: **1. Manual annotation**, **2. Model prediction** (both in the label
  colours) and **3. Missed only**, the raw image with just the annotated objects the model did not
  find outlined in red, each carrying its row number; every panel opens full screen;
* a per-label table of annotated vs predicted counts with the number matched, matched with a wrong
  label, missed and extra;
* a list of every missed, extra or wrongly labelled object with its position (x, y in pixels) and
  a note that says, for example, that a missed nucleus was merged into a neighbouring prediction
  or that an extra prediction sits where nothing was annotated;
* a zip with five overlays (manual, predicted, missed only, extra only, all differences numbered)
  and both tables;
* a written summary of the comparison (Gemini, when the key is set).

The bundled images are the training images, so this shows how the shipped model fits them,
not how it generalises (the held-out numbers above do that). The bundle is rebuilt with

```bash
python scripts/make_validation_set.py --data-dir /path/to/folder --out validation
```

and the section is hidden when `validation/` is empty.

## Deploying to Hugging Face

The checkpoint is ~98 MB, so it must go through Git LFS (`.gitattributes` already tracks
`*.pt`). The simplest route uploads the folder with the CLI, which handles LFS automatically:

```bash
pip install -U huggingface_hub
huggingface-cli login
huggingface-cli upload <your-user>/cellquant . . --repo-type space --exclude "runs/*" --exclude "tests/*" --exclude ".pytest_cache/*"
```

Create the Space first (SDK: Gradio) or add `--create`. Add the optional
`GEMINI_API_KEY` secret under *Settings → Variables and secrets*.

The Space runs on **ZeroGPU** (a shared NVIDIA GPU that is attached only while a call runs):
`app.py` imports `spaces` before torch, puts the model on `cuda` at start-up and wraps the two
inference functions in `@spaces.GPU` with a duration estimated from the number of images.
Visitors need no account: an unauthenticated visitor has 2 GPU minutes a day (about
60 full images at 8× TTA), a free account 5, a PRO account 40. Hosting a ZeroGPU Space needs a
Hugging Face account in good standing (two Spaces free; PRO accounts up to ten), Python 3.12 and
torch ≥ 2.8 (`README.md` front matter and `requirements.txt`). The deploy script does the whole
round trip and times a full image on the Space:

```bash
python scripts/deploy_hf.py --repo <your-user>/cellquant --hardware zero-a10g --bench
```

Without `spaces` installed, or on CPU hardware, the same code runs unchanged on the CPU
(`CELLQUANT_DEVICE` overrides the device).

## Running locally

```bash
pip install -r requirements.txt
python app.py                       # http://localhost:7860
python -m pytest -q tests           # unit + integration tests
```

Retrain when more annotated images are available (put the TIFs and the CVAT zips in one folder):

```bash
python train.py --data-dir /path/to/folder --out weights/cellquant.pt --iters 3000
python evaluate.py cv --data-dir /path/to/folder --runs-dir runs/cv_v3 --iters 3000
python scripts/rule_study.py --data-dir /path/to/folder --runs-dir runs/cv_v2 --out runs/cv_v3/rule_study --weights weights/cellquant.pt
```

The last line re-chooses the peak-distance threshold from the training-set fit (per fold and for the
all-image model); copy the printed value into `PostConfig.min_peak_dist`.

Training uses Apple-silicon MPS, CUDA or CPU automatically.

## Limitations

* The annotator skipped faint / out-of-focus nuclei; the model learned that criterion only
  approximately, so very dim nuclei may be counted or not. Raise *Minimum nucleus area* or the
  thresholds in *Advanced* to tune for your images; `PostConfig(min_peak_dist=...)` (default
  0.64, 0 = off) sets how confident the distance head must be before a nucleus is counted.
* "Cell, Nostril, Antibody −" had only five training examples; its predictions are the
  product of the (well-trained) antibody and punctum heads rather than a learned class.
* Images must have the same channel convention (blue nuclei, green antibody, red puncta) and a
  similar magnification (nuclei ≈ 20–35 px across). The model tolerates ±25 % scale.