Instructions to use MarkGG/Romance-cleaned-1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MarkGG/Romance-cleaned-1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MarkGG/Romance-cleaned-1")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("MarkGG/Romance-cleaned-1")
model = AutoModelForCausalLM.from_pretrained("MarkGG/Romance-cleaned-1", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use MarkGG/Romance-cleaned-1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MarkGG/Romance-cleaned-1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MarkGG/Romance-cleaned-1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/MarkGG/Romance-cleaned-1

SGLang

How to use MarkGG/Romance-cleaned-1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MarkGG/Romance-cleaned-1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MarkGG/Romance-cleaned-1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MarkGG/Romance-cleaned-1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MarkGG/Romance-cleaned-1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use MarkGG/Romance-cleaned-1 with Docker Model Runner:
```
docker model run hf.co/MarkGG/Romance-cleaned-1
```

MarkGG commited on Nov 4, 2022

Commit

e66d228

1 Parent(s): e06fb97

update model card README.md

Browse files

Files changed (1) hide show

README.md +53 -53

README.md CHANGED Viewed

@@ -14,7 +14,7 @@ should probably proofread and complete it, then remove this comment. -->
 This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 4.7252
 ## Model description
@@ -33,7 +33,7 @@ More information needed
 ### Training hyperparameters
 The following hyperparameters were used during training:
-- learning_rate: 5e-05
 - train_batch_size: 32
 - eval_batch_size: 32
 - seed: 42
@@ -41,7 +41,7 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 256
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
-- lr_scheduler_warmup_steps: 1000
 - num_epochs: 50
 - mixed_precision_training: Native AMP
@@ -49,56 +49,56 @@ The following hyperparameters were used during training:
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| No log        | 0.97  | 29   | 9.9449          |
-| No log        | 1.97  | 58   | 9.1764          |
-| No log        | 2.97  | 87   | 8.6054          |
-| No log        | 3.97  | 116  | 8.2512          |
-| No log        | 4.97  | 145  | 7.8647          |
-| No log        | 5.97  | 174  | 7.5154          |
-| No log        | 6.97  | 203  | 7.2125          |
-| No log        | 7.97  | 232  | 6.9063          |
-| No log        | 8.97  | 261  | 6.5997          |
-| No log        | 9.97  | 290  | 6.3070          |
-| No log        | 10.97 | 319  | 6.0437          |
-| No log        | 11.97 | 348  | 5.8185          |
-| No log        | 12.97 | 377  | 5.6224          |
-| No log        | 13.97 | 406  | 5.4789          |
-| No log        | 14.97 | 435  | 5.3658          |
-| No log        | 15.97 | 464  | 5.2683          |
-| No log        | 16.97 | 493  | 5.1975          |
-| No log        | 17.97 | 522  | 5.1404          |
-| No log        | 18.97 | 551  | 5.0764          |
-| No log        | 19.97 | 580  | 5.0289          |
-| No log        | 20.97 | 609  | 4.9836          |
-| No log        | 21.97 | 638  | 4.9436          |
-| No log        | 22.97 | 667  | 4.9065          |
-| No log        | 23.97 | 696  | 4.8741          |
-| No log        | 24.97 | 725  | 4.8428          |
-| No log        | 25.97 | 754  | 4.8125          |
-| No log        | 26.97 | 783  | 4.7892          |
-| No log        | 27.97 | 812  | 4.7697          |
-| No log        | 28.97 | 841  | 4.7448          |
-| No log        | 29.97 | 870  | 4.7373          |
-| No log        | 30.97 | 899  | 4.7162          |
-| No log        | 31.97 | 928  | 4.7021          |
-| No log        | 32.97 | 957  | 4.6920          |
-| No log        | 33.97 | 986  | 4.6819          |
-| No log        | 34.97 | 1015 | 4.6780          |
-| No log        | 35.97 | 1044 | 4.6786          |
-| No log        | 36.97 | 1073 | 4.6745          |
-| No log        | 37.97 | 1102 | 4.6674          |
-| No log        | 38.97 | 1131 | 4.6693          |
-| No log        | 39.97 | 1160 | 4.6820          |
-| No log        | 40.97 | 1189 | 4.6818          |
-| No log        | 41.97 | 1218 | 4.6769          |
-| No log        | 42.97 | 1247 | 4.6867          |
-| No log        | 43.97 | 1276 | 4.6984          |
-| No log        | 44.97 | 1305 | 4.7064          |
-| No log        | 45.97 | 1334 | 4.7161          |
-| No log        | 46.97 | 1363 | 4.7227          |
-| No log        | 47.97 | 1392 | 4.7229          |
-| No log        | 48.97 | 1421 | 4.7256          |
-| No log        | 49.97 | 1450 | 4.7252          |
 ### Framework versions

 This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on the None dataset.
 It achieves the following results on the evaluation set:
+- Loss: 6.2633
 ## Model description
 ### Training hyperparameters
 The following hyperparameters were used during training:
+- learning_rate: 2e-05
 - train_batch_size: 32
 - eval_batch_size: 32
 - seed: 42
 - total_train_batch_size: 256
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 10000
 - num_epochs: 50
 - mixed_precision_training: Native AMP
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
+| No log        | 0.97  | 29   | 11.0280         |
+| No log        | 1.97  | 58   | 10.8490         |
+| No log        | 2.97  | 87   | 10.5566         |
+| No log        | 3.97  | 116  | 10.1903         |
+| No log        | 4.97  | 145  | 9.8329          |
+| No log        | 5.97  | 174  | 9.5549          |
+| No log        | 6.97  | 203  | 9.3619          |
+| No log        | 7.97  | 232  | 9.2283          |
+| No log        | 8.97  | 261  | 9.1074          |
+| No log        | 9.97  | 290  | 8.9570          |
+| No log        | 10.97 | 319  | 8.7914          |
+| No log        | 11.97 | 348  | 8.6625          |
+| No log        | 12.97 | 377  | 8.5663          |
+| No log        | 13.97 | 406  | 8.4867          |
+| No log        | 14.97 | 435  | 8.4149          |
+| No log        | 15.97 | 464  | 8.3448          |
+| No log        | 16.97 | 493  | 8.2698          |
+| No log        | 17.97 | 522  | 8.1917          |
+| No log        | 18.97 | 551  | 8.1137          |
+| No log        | 19.97 | 580  | 8.0377          |
+| No log        | 20.97 | 609  | 7.9620          |
+| No log        | 21.97 | 638  | 7.8855          |
+| No log        | 22.97 | 667  | 7.8155          |
+| No log        | 23.97 | 696  | 7.7492          |
+| No log        | 24.97 | 725  | 7.6839          |
+| No log        | 25.97 | 754  | 7.6226          |
+| No log        | 26.97 | 783  | 7.5596          |
+| No log        | 27.97 | 812  | 7.5007          |
+| No log        | 28.97 | 841  | 7.4431          |
+| No log        | 29.97 | 870  | 7.3849          |
+| No log        | 30.97 | 899  | 7.3277          |
+| No log        | 31.97 | 928  | 7.2703          |
+| No log        | 32.97 | 957  | 7.2140          |
+| No log        | 33.97 | 986  | 7.1594          |
+| No log        | 34.97 | 1015 | 7.1028          |
+| No log        | 35.97 | 1044 | 7.0485          |
+| No log        | 36.97 | 1073 | 6.9916          |
+| No log        | 37.97 | 1102 | 6.9376          |
+| No log        | 38.97 | 1131 | 6.8807          |
+| No log        | 39.97 | 1160 | 6.8234          |
+| No log        | 40.97 | 1189 | 6.7665          |
+| No log        | 41.97 | 1218 | 6.7108          |
+| No log        | 42.97 | 1247 | 6.6518          |
+| No log        | 43.97 | 1276 | 6.5963          |
+| No log        | 44.97 | 1305 | 6.5392          |
+| No log        | 45.97 | 1334 | 6.4867          |
+| No log        | 46.97 | 1363 | 6.4274          |
+| No log        | 47.97 | 1392 | 6.3712          |
+| No log        | 48.97 | 1421 | 6.3148          |
+| No log        | 49.97 | 1450 | 6.2633          |
 ### Framework versions