Instructions to use ninagroot/Llama-360M-RUN1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ninagroot/Llama-360M-RUN1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ninagroot/Llama-360M-RUN1")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ninagroot/Llama-360M-RUN1")
model = AutoModelForCausalLM.from_pretrained("ninagroot/Llama-360M-RUN1", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ninagroot/Llama-360M-RUN1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ninagroot/Llama-360M-RUN1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ninagroot/Llama-360M-RUN1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ninagroot/Llama-360M-RUN1

SGLang

How to use ninagroot/Llama-360M-RUN1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ninagroot/Llama-360M-RUN1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ninagroot/Llama-360M-RUN1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ninagroot/Llama-360M-RUN1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ninagroot/Llama-360M-RUN1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ninagroot/Llama-360M-RUN1 with Docker Model Runner:
```
docker model run hf.co/ninagroot/Llama-360M-RUN1
```

ninagroot commited on Mar 20, 2024

Commit

58c2d58

verified ·

1 Parent(s): fd92771

ninagroot/Llama-360Mtest

Browse files

Files changed (5) hide show

README.md +16 -16
model.safetensors +1 -1
runs/Mar20_15-28-58_gcn7.local.snellius.surf.nl/events.out.tfevents.1710944950.gcn7.local.snellius.surf.nl.1480103.0 +3 -0
tokenizer_config.json +1 -1
training_args.bin +1 -1

README.md CHANGED Viewed

@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 4.3475
 ## Model description
@@ -48,21 +48,21 @@ The following hyperparameters were used during training:
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 8.4018        | 0.99  | 44   | 8.2779          |
-| 7.5564        | 1.99  | 88   | 7.2124          |
-| 6.742         | 2.98  | 132  | 6.4726          |
-| 6.0531        | 4.0   | 177  | 5.8848          |
-| 5.1195        | 4.99  | 221  | 5.2837          |
-| 4.5893        | 5.99  | 265  | 4.8101          |
-| 4.3185        | 6.98  | 309  | 4.6188          |
-| 4.0957        | 8.0   | 354  | 4.4767          |
-| 3.7674        | 8.99  | 398  | 4.4084          |
-| 3.6238        | 9.99  | 442  | 4.3695          |
-| 3.5106        | 10.98 | 486  | 4.3419          |
-| 3.2515        | 12.0  | 531  | 4.3291          |
-| 3.0916        | 12.99 | 575  | 4.3472          |
-| 3.0072        | 13.99 | 619  | 4.3490          |
-| 3.0306        | 14.92 | 660  | 4.3475          |
 ### Framework versions

 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
+- Loss: 5.2706
 ## Model description
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
+| 8.4154        | 0.99  | 44   | 8.2674          |
+| 7.3733        | 1.98  | 88   | 7.2246          |
+| 6.4378        | 3.0   | 133  | 6.5650          |
+| 5.5786        | 3.99  | 177  | 6.1513          |
+| 4.8345        | 4.98  | 221  | 5.7858          |
+| 4.3034        | 5.99  | 266  | 5.4541          |
+| 4.019         | 6.99  | 310  | 5.2054          |
+| 3.5206        | 8.0   | 355  | 5.0984          |
+| 3.0144        | 8.99  | 399  | 5.0603          |
+| 2.6052        | 9.98  | 443  | 5.0552          |
+| 2.2063        | 11.0  | 488  | 5.1439          |
+| 1.7308        | 11.99 | 532  | 5.1838          |
+| 1.4794        | 12.98 | 576  | 5.2275          |
+| 1.2218        | 13.99 | 621  | 5.2608          |
+| 1.1556        | 14.87 | 660  | 5.2706          |
 ### Framework versions

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:70a56446823a6b9000e94343024eda185ec2de87f84c064ed04df281a8c996f0
 size 1344172280

 version https://git-lfs.github.com/spec/v1
+oid sha256:8964bc33b0bce23ca552b07f763bc59b0b28176fe18c016c02363dcea9199c3c
 size 1344172280

runs/Mar20_15-28-58_gcn7.local.snellius.surf.nl/events.out.tfevents.1710944950.gcn7.local.snellius.surf.nl.1480103.0 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:016d94e8143710e80b020cf8599903d062f6cb1afb58006f48ba8a13f6ed787e
+size 13889

tokenizer_config.json CHANGED Viewed

@@ -37,7 +37,7 @@
   "bos_token": "<s>",
   "clean_up_tokenization_spaces": true,
   "eos_token": "</s>",
-  "model_max_length": 128,
   "pad_token": "<pad>",
   "tokenizer_class": "GPT2Tokenizer",
   "unk_token": "<|endoftext|>"

   "bos_token": "<s>",
   "clean_up_tokenization_spaces": true,
   "eos_token": "</s>",
+  "model_max_length": 100,
   "pad_token": "<pad>",
   "tokenizer_class": "GPT2Tokenizer",
   "unk_token": "<|endoftext|>"

training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:438d09597f6c502b8b3134429dc8a3ce76adbac05dab665a370b5b862e1699ee
 size 4728

 version https://git-lfs.github.com/spec/v1
+oid sha256:f2b624f97bbf017a411c0d68c58ad13a58993c06d08892fa70debc40d6021e32
 size 4728