Instructions to use ninagroot/Llama-360M-RUN2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ninagroot/Llama-360M-RUN2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ninagroot/Llama-360M-RUN2")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ninagroot/Llama-360M-RUN2")
model = AutoModelForCausalLM.from_pretrained("ninagroot/Llama-360M-RUN2")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ninagroot/Llama-360M-RUN2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ninagroot/Llama-360M-RUN2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ninagroot/Llama-360M-RUN2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ninagroot/Llama-360M-RUN2

SGLang

How to use ninagroot/Llama-360M-RUN2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ninagroot/Llama-360M-RUN2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ninagroot/Llama-360M-RUN2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ninagroot/Llama-360M-RUN2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ninagroot/Llama-360M-RUN2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ninagroot/Llama-360M-RUN2 with Docker Model Runner:
```
docker model run hf.co/ninagroot/Llama-360M-RUN2
```

ninagroot commited on Apr 18, 2024

Commit

56d4540

verified ·

1 Parent(s): 49dd6df

ninagroot/Llama-360Mtest

Browse files

Files changed (4) hide show

README.md +12 -10
model.safetensors +1 -1
runs/Apr18_12-07-46_gcn45.local.snellius.surf.nl/events.out.tfevents.1713434874.gcn45.local.snellius.surf.nl.3637688.0 +3 -0
training_args.bin +1 -1

README.md CHANGED Viewed

@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 5.1844
 ## Model description
@@ -41,21 +41,23 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 50
-- num_epochs: 8
 - mixed_precision_training: Native AMP
 ### Training results
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 8.4263        | 0.98  | 7    | 8.5002          |
-| 7.2113        | 1.96  | 14   | 7.6702          |
-| 6.258         | 2.95  | 21   | 7.0704          |
-| 5.4942        | 3.93  | 28   | 6.4329          |
-| 4.7105        | 4.91  | 35   | 5.9264          |
-| 3.7794        | 5.89  | 42   | 5.5158          |
-| 3.18          | 6.88  | 49   | 5.3089          |
-| 2.5741        | 7.86  | 56   | 5.1844          |
 ### Framework versions

 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
+- Loss: 5.1685
 ## Model description
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 50
+- num_epochs: 10
 - mixed_precision_training: Native AMP
 ### Training results
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
+| 8.4647        | 0.98  | 7    | 8.5154          |
+| 7.2112        | 1.96  | 14   | 7.6819          |
+| 6.3283        | 2.95  | 21   | 6.9987          |
+| 5.5163        | 3.93  | 28   | 6.4019          |
+| 4.7022        | 4.91  | 35   | 5.8715          |
+| 3.7692        | 5.89  | 42   | 5.4877          |
+| 3.2137        | 6.88  | 49   | 5.2686          |
+| 2.6388        | 8.0   | 57   | 5.1854          |
+| 2.0768        | 8.98  | 64   | 5.1622          |
+| 1.715         | 9.82  | 70   | 5.1685          |
 ### Framework versions

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:637050ce7c3f38a4a41ee4376925419d7bb8b2726bce1be3d21ac305cb5c81e9
 size 1408774432

 version https://git-lfs.github.com/spec/v1
+oid sha256:cd114647cbb563ea25fbf2c4255d9af4bed9370abedbae25e480835dfdc6e95f
 size 1408774432

runs/Apr18_12-07-46_gcn45.local.snellius.surf.nl/events.out.tfevents.1713434874.gcn45.local.snellius.surf.nl.3637688.0 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4eb658402c6f01a7d5dad8748d3df89fbe39c94302e5d5a83597869d19f9316e
+size 22060

training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:128ff538f545a64a6518eb18a3ebf82610c01b0b910c020382e076d85cffc41e
 size 4984

 version https://git-lfs.github.com/spec/v1
+oid sha256:e3ced552914f9b88b0b5e623385178a62392bf44137987e9608078e98983e24d
 size 4984