Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
metaXu264
/
Generator_new_tokenizer
like
0
Text Generation
Transformers
Safetensors
llama
biology
genomics
long-context
text-generation-inference
arxiv:
2502.07272
License:
mit
Model card
Files
Files and versions
xet
Community
Deploy
Use this model
47922fc
Generator_new_tokenizer
12 GB
2 contributors
History:
4 commits
XuJP264
Update tokenizer source and config vocab_size to 4260
47922fc
about 1 month ago
.gitattributes
Safe
1.52 kB
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
README.md
Safe
6.44 kB
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
add_special_tokens_to_vocab.py
Safe
3.57 kB
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
config.json
Safe
657 Bytes
Update tokenizer source and config vocab_size to 4260
about 1 month ago
generation_config.json
Safe
111 Bytes
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
model-00001-of-00003.safetensors
Safe
5 GB
xet
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
model-00002-of-00003.safetensors
Safe
4.96 GB
xet
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
model-00003-of-00003.safetensors
Safe
2.03 GB
xet
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
model.safetensors.index.json
Safe
22.5 kB
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
special_token.txt
Safe
3.86 kB
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
special_tokens_map.json
Safe
552 Bytes
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
tokenizer.py
Safe
8.43 kB
Update tokenizer source and config vocab_size to 4260
about 1 month ago
tokenizer_config.json
Safe
1.3 kB
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
vocab.txt
Safe
30 kB
Add updated vocab with CRISPR/Cas special tokens
about 1 month ago
