Checkpoint architecture mismatch with modeling_scdiva.py
Iβm trying to reproduce the ScDiVa model from the Hugging Face release, but the checkpoint does not appear to match the provided architecture in modeling_scdiva.py.
The checkpoint fails to load cleanly. The state_dict contains keys that do not exist in the provided architecture and is also missing keys that the model expects.
Examples:
Missing keys
- gene_embedding.layer_norm.weight
- variational_layer
- annotation_head
- batch_integration_head
Unexpected keys
- final_norm.weight
- gene_head.weight
- latent_encoder
- value_encoder
- value_head
This suggests the checkpoint corresponds to a different model architecture than the one currently provided.
Following instructions from "Model Loading" section on Github as well (https://github.com/wangmingxuan666/ScDiVa):
- weights are downloaded from Hugging Face
- loading fails
- inference crashes in modeling_scdiva.py
=== Testing ScDiVa Loading & Inference ===
[ScDiVa] Downloading weights from HF: warming666/ScDiVa
model.safetensors: 100%|βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 378M/378M [00:08<00:00, 45.6MB/s]
[ScDiVa] Loading weights from .cache/huggingface/hub/models--warming666--ScDiVa/snapshots/7ee86653c3c0b2f456cf15a73ac7da517a21b4f5/model.safetensors...
state = torch.load(ckpt_path, map_location=map_location)
[ScDiVa] Error loading weights: could not find MARK. Using random init.
Input Data Shape: torch.Size([2, 41818])
Running encoder...
Traceback (most recent call last):
...
File "ScDiVa/modeling_scdiva.py", line 133, in forward
out = out.transpose(1, 2).contiguous().view(B, L, config.hidden_size)
NameError: name 'config' is not defined
So at the moment, it looks like there may be both:
- a checkpoint/model-definition mismatch, and
- a bug in the currently published inference code path.
Similar issue reported at https://github.com/wangmingxuan666/ScDiVa/issues/1
+1
FurthermoreοΌthe model does not provide a gene list file, so it is not possible to align the data genes to the model genes in the correct order.