Add model card YAML metadata (omit base_model)
Browse files
README.md
CHANGED
|
@@ -1,4 +1,18 @@
|
|
| 1 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
Contents
|
| 4 |
- model-00001-of-00001.safetensors, model.safetensors.index.json
|
|
@@ -6,7 +20,7 @@ Contents
|
|
| 6 |
- tokenizer.json, tokenizer_config.json, special_tokens_map.json, spiece.model (custom T5)
|
| 7 |
- generation_config.json
|
| 8 |
|
| 9 |
-
Usage
|
| 10 |
```python
|
| 11 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 12 |
import torch
|
|
@@ -21,8 +35,7 @@ out = model.generate(**enc, max_new_tokens=64, do_sample=True, temperature=0.8)
|
|
| 21 |
print(tok.decode(out[0], skip_special_tokens=False))
|
| 22 |
```
|
| 23 |
|
| 24 |
-
Notes
|
| 25 |
- The tokenizer is SentencePiece-based (T5). Do not add EOS at prompt time; use `add_special_tokens=False` when tokenizing prompts for generation.
|
| 26 |
- The model config is tailored to vocab_size=32100 and rope_theta=500000.
|
| 27 |
- If you prefer multi-shard weights, provide a `model.safetensors.index.json` and re-save.
|
| 28 |
-
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
library_name: transformers
|
| 5 |
+
license: other
|
| 6 |
+
pipeline_tag: text-generation
|
| 7 |
+
tags:
|
| 8 |
+
- causal-lm
|
| 9 |
+
- llama
|
| 10 |
+
- sharded
|
| 11 |
+
- t5-tokenizer
|
| 12 |
+
# base_model intentionally omitted for a custom model
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# HF Export: Banyan 5B Deep (T5 tokenizer)
|
| 16 |
|
| 17 |
Contents
|
| 18 |
- model-00001-of-00001.safetensors, model.safetensors.index.json
|
|
|
|
| 20 |
- tokenizer.json, tokenizer_config.json, special_tokens_map.json, spiece.model (custom T5)
|
| 21 |
- generation_config.json
|
| 22 |
|
| 23 |
+
# Usage
|
| 24 |
```python
|
| 25 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 26 |
import torch
|
|
|
|
| 35 |
print(tok.decode(out[0], skip_special_tokens=False))
|
| 36 |
```
|
| 37 |
|
| 38 |
+
# Notes
|
| 39 |
- The tokenizer is SentencePiece-based (T5). Do not add EOS at prompt time; use `add_special_tokens=False` when tokenizing prompts for generation.
|
| 40 |
- The model config is tailored to vocab_size=32100 and rope_theta=500000.
|
| 41 |
- If you prefer multi-shard weights, provide a `model.safetensors.index.json` and re-save.
|
|
|