--- title: AI Cell Quantification emoji: "⬜" colorFrom: gray colorTo: gray sdk: gradio sdk_version: 5.49.1 python_version: "3.12" suggested_hardware: zero-a10g app_file: app.py pinned: false license: mit --- # AI Cell Quantification (CellQuant): nucleus, antibody and puncta counting for fluorescence sections CellQuant counts the six categories that were annotated in CVAT on 35-day spinal-cord sections stained with DAPI (blue), an antibody (green: Iba1 or TMEM119) and a red punctum marker that the lab calls **"Nostril"**: | Label | Meaning | |---|---| | Cell, Antibody + | nucleus with green antibody signal, no punctum | | Cell, Antibody − | nucleus without antibody signal, no punctum | | Cell, Nostril, Antibody + | nucleus containing a red punctum, antibody positive | | Cell, Nostril, Antibody − | nucleus containing a red punctum, antibody negative | | Nostril in Nucleus | red punctum inside a nucleus | | Nostril External of Nucleus | red punctum outside every nucleus | Every nucleus receives exactly one of the four *Cell* labels. ## How it works * **Model** – a UNet with an ImageNet-pretrained ResNet34 encoder (`cellquant/model.py`) that predicts, per pixel: nucleus interior / border / background, P(antibody +), P(nucleus has punctum), punctum in-nucleus / external / background, and the normalised distance to the nucleus outline (1 at the centre of every nucleus, 0 at its outline). A full-resolution refinement stage keeps 5-pixel puncta. * **Instances** – nucleus foreground comes from the class map; markers (distance > 0.5) and the watershed landscape come from the predicted distance map, which has one peak per nucleus and a valley between touching ones; a nucleus whose peak predicted distance stays below 0.64 is dropped (the distance head is trained to reach 1 in every annotated nucleus and 0 elsewhere, so a low peak marks a faint blob the annotator would have skipped; `PostConfig.min_peak_dist`, 0 switches it off); puncta are connected components (`cellquant/postprocess.py`). Checkpoints without the distance channel fall back to a watershed on the interior map. * **Labels** – antibody status is the mean probability inside each nucleus; a nucleus is "Nostril" when a punctum classified "in nucleus" lies inside it (the annotation protocol; the network's own nucleus-level head is available as an alternative in *Advanced*); punctum type comes from the network (switchable to a purely geometric rule). * **Training data** – nine 1024×1024 RGB images with 1,569 nucleus outlines and 1,057 punctum outlines, parsed directly from the CVAT zip exports (`cellquant/annotations.py`). ## Validation (see `reports/SUMMARY.md`, `reports/cv_v3/cv_report.md` and `reports/cv_v3/comparison.md`) Leave-one-image-out cross-validation: nine models, each tested on an image it never saw, and each image tested 23 times (repeat runs, TTA settings, 8 rotations/flips, 6 photometric perturbations, rule variants). Micro-averaged over the nine held-out images: | Measure | Precision | Recall | F1 | |---|---:|---:|---:| | Nucleus detection, IoU ≥ 0.5 | 81.1% | 84.2% | 82.6% | | Nucleus detection, centroid inside the annotated nucleus | 88.8% | 92.2% | 90.5% | | Punctum detection (centroid within 6 px) | 76.4% | 84.8% | 80.4% | With 8× test-time augmentation (the app's default) nucleus F1 is 82.9% and puncta F1 81.8%; adding image scales on top (24× or 40× passes, `reports/cv_v3/tta_study.md`) gains nothing further for nuclei and 0.4 points for puncta, at four to six times the run time, so the 8-member symmetry group is where TTA stops paying. Among matched nuclei the antibody +/− call is right for 93.8% and the Nostril attribute for 95.7%; puncta are classified in-nucleus vs external correctly 93.4% of the time. Scores vary by at most ±3 F1 points across the 14 rotated/perturbed runs of each image, and repeated runs are bit-identical. The weak spots are antibody-negative nuclei (F1 40%; only 110 examples, 62 of them in one image, and the class-balanced loss trades their precision for recall), the five-example "Cell, Nostril, Antibody −" class, and dense clusters of touching nuclei (still the main source of misses, although the distance-map watershed recovers many that the first version merged). The model predicts 1629 nuclei for the 1569 annotated ones across the nine held-out images. Earlier versions on the same protocol: v1 (2-px border, watershed on the interior map, unweighted antibody loss) 77.9% nucleus F1, 87.5% by the centroid rule, 79.9% puncta F1 (`reports/cv/`); v2 (this model without the peak-distance rule) 80.4%, 88.8% and 80.4% (`reports/cv_v2/`). The shipped `weights/cellquant.pt` was trained on all nine images with the v2 recipe; the peak-distance rule is post-processing only. ## Using the Space 1. Upload one or more images (TIF/PNG/JPG, blue = nuclei, green = antibody, red = puncta). Use the 8-bit RGB export of the microscope software (what the model was trained on); 16-bit inputs are auto-scaled so that their 99.95th percentile maps to 255. Multi-channel TIFs with one channel per page (ImageJ composites, OME-TIFF) are supported; greyscale inputs are flagged. Images larger than 1536 px are processed in overlapping tiles. 2. Every image is analysed eight times (four rotations × two flips) and the outputs averaged, the setting that scored best in cross-validation (nucleus F1 +0.3, puncta F1 +1.4 points over a single pass). `CELLQUANT_TTA=1` or `4` changes this for local runs. 3. Click **Analyse**. You get a counts table, colour overlays and a zip with `counts.csv`, `objects.csv` (every nucleus and punctum with centroid, area, probabilities) and `results.json`. The table and `counts.csv` have one row per image and eight count columns: the four nucleus labels followed by their total, **All nuclei**, then the two punctum labels followed by their total, **All puncta** (the same totals as in the validation table of section 2; `results.json` carries them under `totals`). Each overlay caption gives the two totals. 4. A written summary follows the counts in both sections when the Space has a `GEMINI_API_KEY` secret (optionally `GEMINI_MODEL`, default `gemini-4-argon`); without the key a one-line notice takes its place. Gemini only restates the counts in prose and comments on image quality or on what the missed objects have in common; it never changes a number, and it is called after the GPU work, so it costs no GPU time. ## Checking the model against the annotations The lower half of the Space, *Check the model against the 9 annotated images*, shows the nine annotated images as a strip of thumbnails (the tenth file in the data folder is a byte-identical copy of F113). Click one: the model runs on the full image (eight rotations and flips averaged), its prediction is matched with the CVAT annotation object by object (nuclei at IoU ≥ 0.5, puncta by centre distance ≤ 6 px), and you get * three panels side by side: **1. Manual annotation**, **2. Model prediction** (both in the label colours) and **3. Missed only**, the raw image with just the annotated objects the model did not find outlined in red, each carrying its row number; every panel opens full screen; * a per-label table of annotated vs predicted counts with the number matched, matched with a wrong label, missed and extra; * a list of every missed, extra or wrongly labelled object with its position (x, y in pixels) and a note that says, for example, that a missed nucleus was merged into a neighbouring prediction or that an extra prediction sits where nothing was annotated; * a zip with five overlays (manual, predicted, missed only, extra only, all differences numbered) and both tables; * a written summary of the comparison (Gemini, when the key is set). The bundled images are the training images, so this shows how the shipped model fits them, not how it generalises (the held-out numbers above do that). The bundle is rebuilt with ```bash python scripts/make_validation_set.py --data-dir /path/to/folder --out validation ``` and the section is hidden when `validation/` is empty. ## Deploying to Hugging Face The checkpoint is ~98 MB, so it must go through Git LFS (`.gitattributes` already tracks `*.pt`). The simplest route uploads the folder with the CLI, which handles LFS automatically: ```bash pip install -U huggingface_hub huggingface-cli login huggingface-cli upload /cellquant . . --repo-type space --exclude "runs/*" --exclude "tests/*" --exclude ".pytest_cache/*" ``` Create the Space first (SDK: Gradio) or add `--create`. Add the optional `GEMINI_API_KEY` secret under *Settings → Variables and secrets*. The Space runs on **ZeroGPU** (a shared NVIDIA GPU that is attached only while a call runs): `app.py` imports `spaces` before torch, puts the model on `cuda` at start-up and wraps the two inference functions in `@spaces.GPU` with a duration estimated from the number of images. Visitors need no account: an unauthenticated visitor has 2 GPU minutes a day (about 60 full images at 8× TTA), a free account 5, a PRO account 40. Hosting a ZeroGPU Space needs a Hugging Face account in good standing (two Spaces free; PRO accounts up to ten), Python 3.12 and torch ≥ 2.8 (`README.md` front matter and `requirements.txt`). The deploy script does the whole round trip and times a full image on the Space: ```bash python scripts/deploy_hf.py --repo /cellquant --hardware zero-a10g --bench ``` Without `spaces` installed, or on CPU hardware, the same code runs unchanged on the CPU (`CELLQUANT_DEVICE` overrides the device). ## Running locally ```bash pip install -r requirements.txt python app.py # http://localhost:7860 python -m pytest -q tests # unit + integration tests ``` Retrain when more annotated images are available (put the TIFs and the CVAT zips in one folder): ```bash python train.py --data-dir /path/to/folder --out weights/cellquant.pt --iters 3000 python evaluate.py cv --data-dir /path/to/folder --runs-dir runs/cv_v3 --iters 3000 python scripts/rule_study.py --data-dir /path/to/folder --runs-dir runs/cv_v2 --out runs/cv_v3/rule_study --weights weights/cellquant.pt ``` The last line re-chooses the peak-distance threshold from the training-set fit (per fold and for the all-image model); copy the printed value into `PostConfig.min_peak_dist`. Training uses Apple-silicon MPS, CUDA or CPU automatically. ## Limitations * The annotator skipped faint / out-of-focus nuclei; the model learned that criterion only approximately, so very dim nuclei may be counted or not. Raise *Minimum nucleus area* or the thresholds in *Advanced* to tune for your images; `PostConfig(min_peak_dist=...)` (default 0.64, 0 = off) sets how confident the distance head must be before a nucleus is counted. * "Cell, Nostril, Antibody −" had only five training examples; its predictions are the product of the (well-trained) antibody and punctum heads rather than a learned class. * Images must have the same channel convention (blue nuclei, green antibody, red puncta) and a similar magnification (nuclei ≈ 20–35 px across). The model tolerates ±25 % scale.