sosiklab/NES-plankton-classifier-2022

NES Plankton Classifier 2022 v2.4 (20220209_Jan2022_NES_2.4)

Inception V3 model for automated classification of plankton and other particles imaged by the Imaging FlowCytobot (IFCB) on the Northeast U.S. Shelf (NES). Classifies 155 categories including phytoplankton, microzooplankton, detritus, and imaging artifacts.

This model is intended for automated taxonomic classification of IFCB imagery collected on the Northeast U.S. Shelf. It is suitable for operational use in plankton monitoring pipelines. Performance may degrade on IFCB data from other geographic regions or instruments with significantly different optical configurations.

Model Files

File	Description
20220209_Jan2022_NES_2.4.onnx	IFCB imagery ONNX model for inference (~85 MB)
20220209_Jan2022_NES_2.4.cpu.onnx	non-GPU optimized model (historical)
labels.json	Integer index → class name mapping
config.json	Model architecture summary
preprocessor_config.json	Image preprocessing parameters

The main "cuda optimized" and "cpu" model versions return equivalent results. Both models will run fine on cpu or gpu.

The main version folds BatchNormalization nodes into the preceding Conv layer, improving GPU performance. The "cpu" model retains the explicit BatchNormalization nodes and is included for historical reasons. Output results are equivalent, barring minor floating-point score differences.

Property	CPU	CUDA
IR version	6	7
Opset	11	12
Total nodes	331	237
BatchNormalization nodes	94	0
Conv nodes with bias	0	94
Initializers	472	190

How to Use

Install Requirements

To run inference on IFCB bins, use of ifcb-inference is recommended. See link for details on installation and inference-runtime options.

If your machine is gpu-enabled, use the [cuda] option (uses onnxruntime-gpu[cuda,cudnn]). Otherwise use the [cpu] option (uses onnxruntime).

For dataloading performance with ifcb-infer, it is recommended to use [torch] option. For lighter deployments in constrained environments, this option can be omitted (a simpler dataloader without additional dependencies will be used instead).

# GPU-enabled machine
pip install "ifcb-infer[cuda,torch] @ git+https://github.com/WHOIGit/ifcb-inference.git@v0.4.0"

# CPU-only machine
pip install "ifcb-infer[cpu,torch] @ git+https://github.com/WHOIGit/ifcb-inference.git@v0.4.0"

Download model

With huggingface hf command:

# Download the model and labels to current directory using huggingface hf command
hf download sosiklab/NES-plankton-classifier-2022 20220209_Jan2022_NES_2.4.onnx labels.json --local-dir .

With curl command:

# or
curl -L -O https://huggingface.co/sosiklab/NES-plankton-classifier-2022/resolve/main/20220209_Jan2022_NES_2.4.onnx
curl -L -O https://huggingface.co/sosiklab/NES-plankton-classifier-2022/resolve/main/labels.json

Optionally, also download example-data, included here for example inference below. Realistically, you'd have your own ifcb bins you'd want to run inference on

hf download sosiklab/NES-plankton-classifier-2022 example-data.zip
# or 
curl -L -O https://huggingface.co/sosiklab/NES-plankton-classifier-2022/resolve/main/example-data.zip
# then
unzip example-data.zip

Run inference

# Run inference on ifcb bins
ifcb-infer \
  20220209_Jan2022_NES_2.4.onnx \
  example-data/bins/ \
  --classes labels.json \
  --batch 64

Input / Output Specification

Property	Value
Input image format	Grayscale (IFCB ROI, PIL mode L)
Input size	299 × 299 pixels
Preprocessing	Resize → duplicate grayscale channel to 3 channels → divide by 255
Input tensor	float32, shape (batch, 3, 299, 299)
Output tensor	float32, shape (batch, 155) logits (not-softmaxed)
Class order	As listed in labels.json

• No ImageNet mean/std normalization is applied — pixel values are scaled to [0, 1] only.
• Model's direct output are logits and are not softmaxed. ifcb-infer will automatically apply softmax to logits and output confidence scores

Training Details

Property	Value
Architecture	Inception V3 (pretrained on ImageNet)
Training framework	PyTorch
Training dataset	NES plankton classifier 2022 dataset
Classes	155
Samples per class	min 20, max 2000
Train / val split	80 / 20, split per class
Image augmentation	Horizontal and vertical flip
Batch size	108
Optimizer	Adam
Epochs	Best at 15 of 26 (early stopping patience=10, max=60)
Input resolution	299 × 299
Training date	2022-02-15

Performance

Validation scores:

Metric	Value
F1 Weighted	0.9415
F1 Macro	0.9191
Best epoch	15

Independently reproduced by running this published ONNX model over the full validation split (19,409 images) via onnxruntime, using the preprocessing spec above: F1 Weighted 0.9421, F1 Macro 0.9211 — consistent with the original training-time scores above (small differences are expected from floating-point/interpolation differences between the original PyTorch pipeline and this ONNX/PIL reproduction).

Per-class precision, recall, and F1 on the validation split (click to expand)

Support is the number of validation-split instances of that class (not a prediction count) — low-support classes should be read with caution, since a single misclassification swings their precision/recall/F1 much more than it would for a high-support class.

Class	Support	Precision	Recall	F1
Acantharia	11	1.000	0.727	0.842
Acanthoica_quattrospina	171	0.895	0.994	0.942
Akashiwo	6	1.000	1.000	1.000
Alexandrium_catenella	64	1.000	0.984	0.992
Amoeba	202	0.956	0.965	0.961
Amphidinium	209	0.971	0.976	0.974
Amylax	87	0.988	0.943	0.965
Apedinella	69	0.983	0.855	0.915
Asterionellopsis_glacialis	225	0.957	0.991	0.974
Bacillaria	8	1.000	0.875	0.933
Bacillariophyceae	42	0.789	0.714	0.750
Bacillariophyceae_morphotype1	149	0.986	0.973	0.980
Bacteriastrum	7	0.875	1.000	0.933
Balanion	221	0.954	0.946	0.950
Biddulphia	9	1.000	0.778	0.875
Calciopappus	257	0.966	0.988	0.977
Calciosolenia_brasiliensis	6	1.000	1.000	1.000
Cerataulina_pelagica	190	0.931	0.921	0.926
Chaetoceros	187	0.771	0.866	0.816
Chaetoceros_danicus	135	0.894	0.941	0.917
Chaetoceros_didymus	127	0.854	0.921	0.886
Chaetoceros_didymus_TAG_external_flagellate	234	0.982	0.949	0.965
Chaetoceros_peruvianis	6	1.000	1.000	1.000
Chaetoceros_similis	111	0.889	0.865	0.877
Chaetoceros_socialis	43	0.952	0.930	0.941
Chaetoceros_subtilis	40	0.968	0.750	0.845
Chaetoceros_tenuissimus	48	0.957	0.458	0.620
Chaetoceros_throndsenii	11	0.875	0.636	0.737
Chrysochromulina	227	0.919	0.952	0.935
Chrysochromulina_lanceolata	120	0.959	0.975	0.967
Ciliophora	256	0.903	0.762	0.826
Copepod_nauplii	31	0.900	0.871	0.885
Corethron_hystrix	234	0.996	0.996	0.996
Corymbellus	63	1.000	0.937	0.967
Coscinodiscus	151	0.959	0.940	0.950
Cryptophyta	222	0.960	0.977	0.969
Cylindrotheca	102	0.895	0.922	0.908
Cylindrotheca_morphotype1	97	0.980	0.990	0.985
Dactyliosolen_blavyanus	232	0.983	0.978	0.981
Dactyliosolen_fragilissimus	175	0.809	0.897	0.851
Delphineis	106	0.971	0.962	0.967
Dictyocha	239	0.996	0.979	0.987
Dictyocysta	6	1.000	0.833	0.909
Didinium	11	1.000	0.455	0.625
Dinobryon	241	0.987	0.963	0.975
Dinophyceae	218	0.783	0.780	0.782
Dinophysis_acuminata	36	0.935	0.806	0.866
Dinophysis_norvegica	96	0.865	1.000	0.928
Dinophysis_tripos	6	1.000	1.000	1.000
Ditylum_brightwellii	236	1.000	0.979	0.989
Emiliania_huxleyi	37	0.949	1.000	0.974
Ephemera	143	0.986	0.993	0.990
Eucampia	317	0.966	0.978	0.972
Eucampia_morphytype1	83	0.857	0.940	0.897
Euglena	186	0.961	0.925	0.942
Euplotes	11	1.000	0.818	0.900
Euplotes_morphotype1	13	1.000	1.000	1.000
Eutintinnus	20	1.000	0.950	0.974
Favella	42	0.872	0.976	0.921
Gonyaulax	178	1.000	0.994	0.997
Guinardia_delicatula	400	0.948	0.953	0.950
Guinardia_delicatula_TAG_internal_parasite	229	0.950	0.913	0.931
Guinardia_flaccida	216	0.991	1.000	0.995
Guinardia_striata	206	0.933	0.883	0.908
Gyrodinium	226	0.919	0.956	0.937
Hemiaulus	229	0.996	0.974	0.985
Hemiaulus_membranaceus	8	0.875	0.875	0.875
Heterocapsa_rotundata	210	0.954	0.986	0.970
Karenia	133	0.943	1.000	0.971
Katodinium_or_Torodinium	227	0.969	0.965	0.967
Kryptoperidinium_triquetrum	373	0.974	0.995	0.984
Laboea_strobila	226	0.991	0.987	0.989
Lauderia_annulata	49	0.979	0.959	0.969
Leegaardiella_ovalis	95	0.883	0.958	0.919
Leptocylindrus	228	0.849	0.961	0.901
Leptocylindrus_mediterraneus	95	0.968	0.958	0.963
Licmophora	76	0.974	0.987	0.980
Margalefidinium	352	0.997	0.991	0.994
Mesodinium	230	0.956	0.939	0.947
Nanoneis	27	1.000	1.000	1.000
Odontella	19	0.850	0.895	0.872
Ophiaster	15	1.000	0.867	0.929
Oxytoxum	192	0.984	0.990	0.987
Paralia_sulcata	146	0.986	0.966	0.976
Paratontonia_gracillima	235	0.959	0.987	0.973
Parvicorbicula_socialis	32	1.000	0.812	0.897
Pelagostrobilidium	101	0.789	0.891	0.837
Phaeocystis	233	0.907	0.918	0.913
Phaeocystis_debris	48	0.702	0.833	0.762
Pleuronema	44	0.878	0.977	0.925
Pleurosigma	236	0.967	0.987	0.977
Polykrikos	26	1.000	0.923	0.960
Prorocentrum	159	0.957	0.975	0.966
Prorocentrum_dentatum	137	0.986	0.993	0.989
Prorocentrum_micans	177	0.983	0.960	0.971
Prorocentrum_triestinum	238	0.987	0.987	0.987
Proterythropsis	153	0.973	0.941	0.957
Protoperidinium	35	0.771	0.771	0.771
Pseudo-nitzschia	231	0.964	0.939	0.952
Pseudochattonella_farcimen	318	0.981	0.978	0.980
Pyramimonas	223	0.956	0.973	0.964
Pyramimonas_longicauda	237	1.000	0.966	0.983
Pyramimonas_morphotype1	19	0.882	0.789	0.833
Rhabdolithes	10	1.000	0.900	0.947
Rhizosolenia	227	0.987	0.969	0.978
Scrippsiella	23	1.000	1.000	1.000
Skeletonema	226	0.951	0.942	0.947
Stenosemella_morphotype1	21	0.941	0.762	0.842
Stenosemella_pacifica	230	0.940	0.952	0.946
Stephanopyxis	17	0.941	0.941	0.941
Strombidium_capitatum	16	0.727	0.500	0.593
Strombidium_conicum	29	0.967	1.000	0.983
Strombidium_inclinatum	64	1.000	0.875	0.933
Strombidium_morphotype1	248	0.847	0.915	0.880
Strombidium_morphotype2	97	0.872	0.845	0.859
Strombidium_tintinnodes	73	0.809	0.986	0.889
Strombidium_wulffi	47	0.894	0.894	0.894
Syracosphaera_pulchra	93	0.869	1.000	0.930
Thalassionema	233	0.971	1.000	0.985
Thalassiosira	262	0.803	0.885	0.842
Thalassiosira_TAG_external_detritus	218	0.914	0.922	0.918
Thalassiosira_sp_aff_mala	121	0.983	0.950	0.966
Tiarina_fusus	11	1.000	0.909	0.952
Tintinnidium_mucicola	6	1.000	0.833	0.909
Tintinnina	49	0.695	0.837	0.759
Tintinnopsis	222	0.922	0.959	0.940
Tontonia_appendiculariformis	36	0.875	0.778	0.824
Trichodesmium	10	1.000	0.900	0.947
Tripos	30	1.000	0.900	0.947
Tripos_furca	68	0.970	0.941	0.955
Tripos_fusus	13	1.000	1.000	1.000
Tripos_lineatus	189	0.969	0.995	0.982
Vicicitus_globosus	89	1.000	1.000	1.000
Warnowia	15	1.000	0.867	0.929
bead	224	1.000	1.000	1.000
bubble	7	1.000	1.000	1.000
camera_spot	99	1.000	1.000	1.000
coccolithophorid	60	0.783	0.783	0.783
detritus	203	0.939	0.833	0.883
detritus_transparent	152	0.930	0.961	0.945
fecal_pellet	44	1.000	0.955	0.977
fiber	229	0.982	0.974	0.978
fiber_TAG_external_detritus	44	0.833	0.795	0.814
flagellate	192	0.939	0.724	0.818
flagellate_morphotype1	234	0.924	0.991	0.957
flagellate_morphotype3	225	0.978	0.982	0.980
nanoplankton_mix	279	0.960	0.943	0.951
pennate	181	0.903	0.768	0.830
pennate_Pseudo-nitzschia	39	0.852	0.590	0.697
pennate_Thalassionema	9	1.000	0.778	0.875
pennate_morphotype1	57	1.000	1.000	1.000
pollen	8	1.000	0.750	0.857
shellfish_larvae	6	0.750	0.500	0.600
unknown2	12	1.000	1.000	1.000
zooplankton	13	0.700	0.538	0.609

Classes

All 155 classes (click to expand)

Acanthoica_quattrospina
Akashiwo
Alexandrium_catenella
Amphidinium
Amylax
Apedinella
Asterionellopsis_glacialis
Bacillaria
Bacillariophyceae
Bacteriastrum
Balanion
Biddulphia
Calciopappus
Calciosolenia_brasiliensis
Cerataulina_pelagica
Tripos
Tripos_furca
Tripos_fusus
Tripos_lineatus
Chaetoceros
Chaetoceros_danicus
Chaetoceros_didymus
Chaetoceros_didymus_TAG_external_flagellate
Chaetoceros_peruvianis
Chaetoceros_similis
Chaetoceros_socialis
Chaetoceros_subtilis
Chaetoceros_tenuissimus
Chaetoceros_throndsenii
Prorocentrum_dentatum
Chrysochromulina
Chrysochromulina_lanceolata
Copepod_nauplii
Corethron_hystrix
Corymbellus
Coscinodiscus
Cylindrotheca
Cylindrotheca_morphotype1
Dactyliosolen_blavyanus
Dactyliosolen_fragilissimus
Delphineis
Dictyocha
Dictyocysta
Didinium
Dinobryon
Dinophyceae
Dinophysis_acuminata
Dinophysis_norvegica
Dinophysis_tripos
Ditylum_brightwellii
Emiliania_huxleyi
Ephemera
Eucampia
Eucampia_morphytype1
Euglena
Euplotes
Euplotes_morphotype1
Eutintinnus
Favella
Gonyaulax
Guinardia_delicatula
Guinardia_delicatula_TAG_internal_parasite
Guinardia_flaccida
Guinardia_striata
Gyrodinium
Hemiaulus
Hemiaulus_membranaceus
Heterocapsa_rotundata
Kryptoperidinium_triquetrum
Karenia
Katodinium_or_Torodinium
Laboea_strobila
Lauderia_annulata
Leegaardiella_ovalis
Leptocylindrus
Leptocylindrus_mediterraneus
Licmophora
Margalefidinium
Mesodinium
Nanoneis
Odontella
Ophiaster
Oxytoxum
Paralia_sulcata
Parvicorbicula_socialis
Phaeocystis
Phaeocystis_debris
Pleuronema
Pleurosigma
Polykrikos
Prorocentrum
Prorocentrum_micans
Prorocentrum_triestinum
Proterythropsis
Protoperidinium
Pseudo-nitzschia
Pseudochattonella_farcimen
Pyramimonas
Pyramimonas_longicauda
Pyramimonas_morphotype1
Acantharia
Rhabdolithes
Rhizosolenia
Scrippsiella
Skeletonema
Stenosemella_morphotype1
Stenosemella_pacifica
Stephanopyxis
Pelagostrobilidium
Strombidium_capitatum
Strombidium_conicum
Strombidium_inclinatum
Strombidium_morphotype1
Strombidium_morphotype2
Strombidium_tintinnodes
Strombidium_wulffi
Syracosphaera_pulchra
Thalassionema
Thalassiosira
Thalassiosira_TAG_external_detritus
Thalassiosira_sp_aff_mala
Tiarina_fusus
Tintinnina
Tintinnidium_mucicola
Tintinnopsis
Tontonia_appendiculariformis
Paratontonia_gracillima
Trichodesmium
Vicicitus_globosus
Warnowia
Amoeba
bead
bubble
camera_spot
Ciliophora
coccolithophorid
Cryptophyta
detritus
detritus_transparent
fecal_pellet
fiber
fiber_TAG_external_detritus
flagellate
flagellate_morphotype1
flagellate_morphotype3
nanoplankton_mix
pennate
pennate_Pseudo-nitzschia
pennate_Thalassionema
pennate_morphotype1
pollen
shellfish_larvae
Bacillariophyceae_morphotype1
unknown2
zooplankton

License

This model is licensed under the MIT License. Copyright © Woods Hole Oceanographic Institution. See LICENSE.

Training was initialized from torchvision's Inception_V3_Weights.IMAGENET1K_V1 checkpoint, distributed under torchvision's BSD-3-Clause license.

Citation

If you use this model in your research, please cite the Woods Hole Oceanographic Institution and the Sosik Lab. A formal citation will be added here upon publication, including the model DOI and any applicable funding/award number.

Acknowledgments

Developed at the Woods Hole Oceanographic Institution. Training data collected by the WHOI Sosik Lab at MVCO and on Northeast U.S. Shelf Long-Term Ecological Research cruises. Full IFCB datasets for MVCO, NESLTER_transect, and NESLTER_broadscale.