Instructions to use Plaaasma/gerald-model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Plaaasma/gerald-model with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Plaaasma/gerald-model")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Plaaasma/gerald-model")
model = AutoModelForCausalLM.from_pretrained("Plaaasma/gerald-model")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Plaaasma/gerald-model with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Plaaasma/gerald-model"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Plaaasma/gerald-model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Plaaasma/gerald-model

SGLang

How to use Plaaasma/gerald-model with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Plaaasma/gerald-model" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Plaaasma/gerald-model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Plaaasma/gerald-model" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Plaaasma/gerald-model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Plaaasma/gerald-model with Docker Model Runner:
```
docker model run hf.co/Plaaasma/gerald-model
```

Plaaasma commited on Apr 5, 2023

Commit

6b1717c

1 Parent(s): 050ed74

Upload 8 files

Browse files

Files changed (5) hide show

optimizer.pt +1 -1
pytorch_model.bin +1 -1
rng_state.pth +1 -1
scheduler.pt +1 -1
trainer_state.json +39 -3

optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5c58d6153fcd30b58eb21e4523c75874a092722206054f138c170e9bd9942f37
 size 6192502837

 version https://git-lfs.github.com/spec/v1
+oid sha256:c427e293dfb675cd0055b4b8837887686d3c202f2c4b1c3c08cd0bcd69883cab
 size 6192502837

pytorch_model.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a4c4d0a688a6c6af53eb2c3c2478627b78806028a22ccd5b3186fd22cc95d503
 size 3134031497

 version https://git-lfs.github.com/spec/v1
+oid sha256:b5075518047ef7374143dc014b6c925009e3aa96d714cacbb77e66c8aae26e3f
 size 3134031497

rng_state.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d300b0c92eff667186a212b2527c59a904ec3607e918011ed0706b0b90623b61
 size 14503

 version https://git-lfs.github.com/spec/v1
+oid sha256:cbe49cba5b4bbea74ca7ad0a69f65d2951ebd36b2fdee0ef08b9ae2994877aae
 size 14503

scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5073920ed04cb7fa852010ca3a6f9de239ee34f0fae2e48813226ad9eeadbe3a
 size 623

 version https://git-lfs.github.com/spec/v1
+oid sha256:ba3a5d1c69a9427a53e2ab87233696d8956a8613a03782eb9c2342a7fb49829d
 size 623

trainer_state.json CHANGED Viewed

@@ -1,8 +1,8 @@
 {
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 2.245749117741418,
-  "global_step": 3500,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
@@ -48,11 +48,47 @@
       "learning_rate": 4.887676508344031e-05,
       "loss": 0.1991,
       "step": 3500
     }
   ],
   "max_steps": 155800,
   "num_train_epochs": 100,
-  "total_flos": 1.217381115050496e+17,
   "trial_name": null,
   "trial_params": null
 }

 {
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 4.170676932948348,
+  "global_step": 6500,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
       "learning_rate": 4.887676508344031e-05,
       "loss": 0.1991,
       "step": 3500
+    },
+    {
+      "epoch": 2.57,
+      "learning_rate": 4.871630295250321e-05,
+      "loss": 0.1887,
+      "step": 4000
+    },
+    {
+      "epoch": 2.89,
+      "learning_rate": 4.855584082156611e-05,
+      "loss": 0.1911,
+      "step": 4500
+    },
+    {
+      "epoch": 3.21,
+      "learning_rate": 4.839537869062901e-05,
+      "loss": 0.1662,
+      "step": 5000
+    },
+    {
+      "epoch": 3.53,
+      "learning_rate": 4.823491655969192e-05,
+      "loss": 0.1579,
+      "step": 5500
+    },
+    {
+      "epoch": 3.85,
+      "learning_rate": 4.807445442875482e-05,
+      "loss": 0.1623,
+      "step": 6000
+    },
+    {
+      "epoch": 4.17,
+      "learning_rate": 4.791399229781772e-05,
+      "loss": 0.1452,
+      "step": 6500
     }
   ],
   "max_steps": 155800,
   "num_train_epochs": 100,
+  "total_flos": 2.260838219092992e+17,
   "trial_name": null,
   "trial_params": null
 }