Instructions to use MilaBromm/TNGMain with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MilaBromm/TNGMain with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="MilaBromm/TNGMain")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("MilaBromm/TNGMain") model = AutoModelForCausalLM.from_pretrained("MilaBromm/TNGMain") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use MilaBromm/TNGMain with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "MilaBromm/TNGMain" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MilaBromm/TNGMain", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/MilaBromm/TNGMain
- SGLang
How to use MilaBromm/TNGMain with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "MilaBromm/TNGMain" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MilaBromm/TNGMain", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "MilaBromm/TNGMain" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MilaBromm/TNGMain", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use MilaBromm/TNGMain with Docker Model Runner:
docker model run hf.co/MilaBromm/TNGMain
NotARectangle commited on
Commit ·
1820e50
1
Parent(s): b6be69a
final version TNGMain
Browse files- added_tokens.json +1 -1
- config.json +3 -2
- pytorch_model.bin +2 -2
- special_tokens_map.json +1 -1
- tokenizer_config.json +1 -1
- training_args.bin +3 -0
added_tokens.json
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
{"<bos>": 50257, "<eos>": 50258, "<pad>": 50259}
|
|
|
|
| 1 |
+
{"<bos>": 50257, "<eos>": 50258, "<pad>": 50259, "PICARD:": 50260, "OTHER:": 50261, "CRUSHER:": 50262, "TROI:": 50263, "RIKER:": 50264, "DATA:": 50265, "LAFORGE:": 50266, "WESLEY:": 50267, "TASHA:": 50268, "WORF:": 50269, "COMPUTER:": 50270, "PULASKI": 50271, "O'BRIEN:": 50272}
|
config.json
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
{
|
| 2 |
-
"_name_or_path": "
|
| 3 |
"activation_function": "gelu_new",
|
| 4 |
"architectures": [
|
| 5 |
"GPT2LMHeadModel"
|
|
@@ -30,6 +30,7 @@
|
|
| 30 |
"max_length": 50
|
| 31 |
}
|
| 32 |
},
|
|
|
|
| 33 |
"use_cache": true,
|
| 34 |
-
"vocab_size":
|
| 35 |
}
|
|
|
|
| 1 |
{
|
| 2 |
+
"_name_or_path": "gpt2",
|
| 3 |
"activation_function": "gelu_new",
|
| 4 |
"architectures": [
|
| 5 |
"GPT2LMHeadModel"
|
|
|
|
| 30 |
"max_length": 50
|
| 31 |
}
|
| 32 |
},
|
| 33 |
+
"transformers_version": "4.5.1",
|
| 34 |
"use_cache": true,
|
| 35 |
+
"vocab_size": 50273
|
| 36 |
}
|
pytorch_model.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8920da7c1f82e2b81af479ebb5ed76027d536ee1bdd5086704fa4fb778ea4c93
|
| 3 |
+
size 510457467
|
special_tokens_map.json
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
{"bos_token": "<bos>", "eos_token": "<eos>", "unk_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "pad_token": "<pad>"}
|
|
|
|
| 1 |
+
{"bos_token": "<bos>", "eos_token": "<eos>", "unk_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "pad_token": "<pad>", "additional_special_tokens": ["PICARD:", "OTHER:", "CRUSHER:", "TROI:", "RIKER:", "DATA:", "LAFORGE:", "WESLEY:", "TASHA:", "WORF:", "COMPUTER:", "PULASKI", "O'BRIEN:"]}
|
tokenizer_config.json
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
{"errors": "replace", "unk_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "bos_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "eos_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "add_prefix_space": false, "model_max_length": 1024, "
|
|
|
|
| 1 |
+
{"errors": "replace", "unk_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "bos_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "eos_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "add_prefix_space": false, "model_max_length": 1024, "special_tokens_map_file": null, "tokenizer_file": "/root/.cache/huggingface/transformers/16a2f78023c8dc511294f0c97b5e10fde3ef9889ad6d11ffaa2a00714e73926e.cf2d0ecb83b6df91b3dbb53f1d1e4c311578bfd3aa0e04934215a49bf9898df0", "name_or_path": "gpt2"}
|
training_args.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0f37c2543fc4e89708c950f21a02c9587d52344bc06203d995d3c2a70146cd76
|
| 3 |
+
size 2287
|