InSilicoPerturber IndexError: Dimension out of range
Hi, I was running InSilicoPerturber for 5,000 cells using the pre-trained model and got an unexpected IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2). I didn't have this error when I conducted exactly the same for 2,000 cells before. I'm pasting the code and error message below:
isp = InSilicoPerturber(perturb_type='delete',
perturb_rank_shift=None,
genes_to_perturb='all',
combos=0,
anchor_gene=None,
model_type='Pretrained',
num_classes=0,
emb_mode='cell_and_gene',
cell_emb_style='mean_pool',
cell_states_to_model={'state_key': 'group', 'start_state': 'opc_gaba', 'goal_state': 'other', 'alt_states': []},
max_ncells=5000,
emb_layer=-1,
forward_batch_size=32,
nproc=16
)
isp.perturb_data(model_directory='Geneformer/',
input_data_file='tokenized_.dataset',
output_directory='./',
output_prefix='perturbed'
)
ispstats = InSilicoPerturberStats(mode='goal_state_shift',
combos=0,
anchor_gene=None,
cell_states_to_model={'state_key': 'group', 'start_state': 'opc_gaba', 'goal_state': 'other', 'alt_states': []}
)
ispstats.get_stats(input_data_directory='./',
null_dist_data_directory=None,
output_directory='./',
output_prefix='stats_emb_mode_gene'
)
Traceback (most recent call last):██████████████████████████████████████████████████████████████████▌ | 1333/1985 [00:00<00:00, 6268.01 examples/s]
File "transformer.py", line 53, in
isp.perturb_data(model_directory='Geneformer/',
File "/home/swang12/.local/lib/python3.8/site-packages/geneformer/in_silico_perturber.py", line 975, in perturb_data
self.in_silico_perturb(model,
File "/home/swang12/.local/lib/python3.8/site-packages/geneformer/in_silico_perturber.py", line 1128, in in_silico_perturb
cos_sims_data = quant_cos_sims(model,
File "/home/swang12/.local/lib/python3.8/site-packages/geneformer/in_silico_perturber.py", line 449, in quant_cos_sims
cos_sims_vs_alt_dict[state] += cos_sim_shift(original_emb,
File "/home/swang12/.local/lib/python3.8/site-packages/geneformer/in_silico_perturber.py", line 507, in cos_sim_shift
perturb_v_end = cos(perturb_emb, end_emb)
File "/home/swang12/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/swang12/.local/lib/python3.8/site-packages/torch/nn/modules/distance.py", line 87, in forward
return F.cosine_similarity(x1, x2, self.dim, self.eps)
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2)
I was wondering if you have any clue about how I should resolve this. Thank you very much!
Sincerely,
Su Wang
i have the same problem.have you already resolve this problem?
looking forward to you reply.
Thank you for your interest in Geneformer and for your patience! We pushed an update that should resolve this issue. If you continue to face errors after pulling the updated code, please let us know by either reopening this discussion if it's the same error or opening a new discussion if it's a new error. Thank you!