Instructions to use Plaaasma/gerald-model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Plaaasma/gerald-model with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Plaaasma/gerald-model")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Plaaasma/gerald-model")
model = AutoModelForCausalLM.from_pretrained("Plaaasma/gerald-model")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Plaaasma/gerald-model with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Plaaasma/gerald-model"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Plaaasma/gerald-model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Plaaasma/gerald-model

SGLang

How to use Plaaasma/gerald-model with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Plaaasma/gerald-model" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Plaaasma/gerald-model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Plaaasma/gerald-model" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Plaaasma/gerald-model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Plaaasma/gerald-model with Docker Model Runner:
```
docker model run hf.co/Plaaasma/gerald-model
```

Plaaasma commited on Apr 5, 2023

Commit

ec29153

1 Parent(s): b88e38a

Upload 8 files

Browse files

Files changed (8) hide show

config.json +0 -0
generation_config.json +6 -0
optimizer.pt +3 -0
pytorch_model.bin +3 -0
rng_state.pth +3 -0
scheduler.pt +3 -0
trainer_state.json +58 -0
training_args.bin +3 -0

config.json ADDED Viewed

The diff for this file is too large to render. See raw diff

generation_config.json ADDED Viewed

	@@ -0,0 +1,6 @@

+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "4.27.4"
+}

optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5c58d6153fcd30b58eb21e4523c75874a092722206054f138c170e9bd9942f37
+size 6192502837

pytorch_model.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a4c4d0a688a6c6af53eb2c3c2478627b78806028a22ccd5b3186fd22cc95d503
+size 3134031497

rng_state.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d300b0c92eff667186a212b2527c59a904ec3607e918011ed0706b0b90623b61
+size 14503

scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5073920ed04cb7fa852010ca3a6f9de239ee34f0fae2e48813226ad9eeadbe3a
+size 623

trainer_state.json ADDED Viewed

	@@ -0,0 +1,58 @@

+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.245749117741418,
+  "global_step": 3500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.32,
+      "learning_rate": 4.98395378690629e-05,
+      "loss": 0.4205,
+      "step": 500
+    },
+    {
+      "epoch": 0.64,
+      "learning_rate": 4.96790757381258e-05,
+      "loss": 0.2948,
+      "step": 1000
+    },
+    {
+      "epoch": 0.96,
+      "learning_rate": 4.951861360718871e-05,
+      "loss": 0.2779,
+      "step": 1500
+    },
+    {
+      "epoch": 1.28,
+      "learning_rate": 4.935815147625161e-05,
+      "loss": 0.2354,
+      "step": 2000
+    },
+    {
+      "epoch": 1.6,
+      "learning_rate": 4.9197689345314504e-05,
+      "loss": 0.2299,
+      "step": 2500
+    },
+    {
+      "epoch": 1.92,
+      "learning_rate": 4.9037227214377404e-05,
+      "loss": 0.2296,
+      "step": 3000
+    },
+    {
+      "epoch": 2.25,
+      "learning_rate": 4.887676508344031e-05,
+      "loss": 0.1991,
+      "step": 3500
+    }
+  ],
+  "max_steps": 155800,
+  "num_train_epochs": 100,
+  "total_flos": 1.217381115050496e+17,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e91d42f5739c0029e3df46341bf6fb0655efaacecfa608112b4a0c5bfd521ad1
+size 3503