Multi task classifier, True values incorrect
Dear team,
I am running multi task classifier to classify 4 conditions across multiple celltypes, I noticed that in the classification validation metrics, the True values are incorrect for task1 alone.
"condition": {
"names": [
"FD",
"FS",
"MC",
"MS"
],
"_type": "ClassLabel"
},
Ground truth
sp.obs[['cell_type','condition']].loc['Brain_MC_3_CGGGTGTGTAATCAGA-1']
cell_type Microglia_1
condition MC
Name: Brain_MC_3_CGGGTGTGTAATCAGA-1, dtype: object
sp.obs[['cell_type','condition']].loc['Brain_FD_3_ACAAAGAGTACGAAAT-1']
cell_type Astrocyte_1
condition FD
Name: Brain_FD_3_ACAAAGAGTACGAAAT-1, dtype: object
Model output:
task label in pickle file
{'task1': {0: 0, 1: 1, 2: 2, 3: 3}, 'task2': {'Astrocyte_1': 0, 'Astrocyte_2': 1, 'B_cell': 2, 'Endothelial': 3, 'Fibroblast': 4, 'Hypothalamic_ependymal': 5, 'Macrophage': 6, 'Meningeal': 7, 'Microglia_1': 8, 'Microglia_2': 9, 'Neuron': 10, 'Neutrophil': 11, 'Olfactory_ensheathing_cell': 12, 'Oligodendrocyte_1': 13, 'Oligodendrocyte_2': 14, 'Oligodendrocyte_3': 15, 'Pan_GABAergic': 16, 'Pericyte': 17, 'Proliferating': 18, 'Schwann': 19, 'T_NK_cell': 20}}
Thank you for looking into this.
Regards,
Saranya
Thank you for your question. I cannot tell for sure what the numbers for task 1 refer to from the obs values printed but assuming the order is 0 to 3 from your list above, you are right that they don’t match the examples from your obs, but then it also doesn’t look like task 2 matches (classifier eval: pericyte, Schwann; obs values: microglia, astrocyte). It’s possible the evaluation values printed are not in the same order as the original obs. I would suggest that you add a unique cell ID as a feature for your tokenized dataset to confirm the order and whether the labels are correct.
Dear,
I have already added unique_cell_id part of the column to tokenize. ({"cell_id": "unique_cell_id", "cell_type": "cell_type", "condition": "condition"})
I have further checked and the task2 labels in the task_label_mappings.pkl file match with the test predictions, but not with the validation.
But I cannot test my task1 labels.
Regards,
Saranya