Update geneformer/tokenizer.py

Update geneformer/tokenizer.py

Adding tags to the model (#448)

update transformers version to match pretrainer using accelerate

update pretrainer to not use distributed sampler (Trainer uses accelerate)

update function for N_Detections for mixture_model without anchor_token

add check to ensure emb_label is None for getting state embs dict

change default emb to cls

include install time in model card

Update trainer output dir (#427)

Update geneformer/tokenizer.py (#415)

Create multi-task_cell_classification.ipynb (#418)

CUDA kernels incompatible with standard PyTorch device movement with 4bit/8bit, necessitating device-specific handling (#416)

Add find_packages to automatically include mtl and subpackages in setup.py (#410)

adding tensorboard to setup.py (#409)

tensorboard add to reqs (#407)

dictionaries from parent dir (#405)

remove token dictionary and unpickling from init (#403)

move dict loading to function in eval utils

edit doc formatting

edit docs formatting

add mtl_classifier to docs

add input size tip to instructions

update tokenizer to defaults for 95M models for special token and input size

update instructions to include reminder about token dictionary

pointing dictionaries from the mtl module's init (#397)

Refactored token dictionary loading and encapsulated dictionary (#398)

Refactor: Convert mask_token_id, pad_token_id, and all_special_ids to properties (#395)

sync token_dictionary variable name w/ classifier

update setup with req and manifest with updated filenames

fix imports mtl/eval_utils

allow model_type valid options to take params model_type : {"Pretrained", "GeneClassifier", "CellClassifier", "MTLCellClassifier", "MTLCellClassifier-Quantized"} (#390)

peft>=0.11.1 (#387)

"save_model_without_heads" is redundant (#385)

comment out "def save_model_without_heads(original_model_save_directory)"; redundant for ISP/Emb extractor (#382)

fixed bug related to dynamic ranges in dictionary with 'min' and 'max' value mismatch in optuna suggest fn (#380)

Update README.md

precommit formatting

update with 12L and 20L i4096 gc95M models, multitask and quantiz code

rename for consistency

delete old gene name dict

update to only have gene names as keys in gene_name_id_dict

Add function for summing of Ensembl IDs (#377)

save pval

add typing list import

move dicts to init

add random state to umap

update get_embs with token_gene_dict arg

update refs to get_model_emb_dims

embs_df with all model embeddings (#363)