Instructions to use FunkEngine/SchweinZwei-13b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FunkEngine/SchweinZwei-13b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FunkEngine/SchweinZwei-13b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("FunkEngine/SchweinZwei-13b")
model = AutoModelForCausalLM.from_pretrained("FunkEngine/SchweinZwei-13b")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FunkEngine/SchweinZwei-13b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FunkEngine/SchweinZwei-13b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FunkEngine/SchweinZwei-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/FunkEngine/SchweinZwei-13b

SGLang

How to use FunkEngine/SchweinZwei-13b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FunkEngine/SchweinZwei-13b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FunkEngine/SchweinZwei-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FunkEngine/SchweinZwei-13b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FunkEngine/SchweinZwei-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use FunkEngine/SchweinZwei-13b with Docker Model Runner:
```
docker model run hf.co/FunkEngine/SchweinZwei-13b
```

FunkEngine commited on Sep 9, 2023

Commit

da73106

1 Parent(s): df7773b

Delete SchweinZwei-13b

Browse files

Files changed (1) hide show

SchweinZwei-13b +0 -25

SchweinZwei-13b DELETED Viewed

@@ -1,25 +0,0 @@
-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
-<html>
- <head>
-  <title>Index of /P6B213B/SchweinZwei-13b</title>
- </head>
- <body>
-<h1>Index of /P6B213B/SchweinZwei-13b</h1>
-<pre>      <a href="?C=N;O=D">Name</a>                    <a href="?C=M;O=A">Last modified</a>      <a href="?C=S;O=A">Size</a>  <a href="?C=D;O=A">Description</a><hr>      <a href="/P6B213B/">Parent Directory</a>                             -
-      <a href="README.md">README.md</a>               2023-09-08 00:52  3.0K
-      <a href="config.json">config.json</a>             2023-09-08 00:52  1.7K
-      <a href="generation_config.json">generation_config.json</a>  2023-09-08 00:52  116
-      <a href="model-00001-of-00003.safetensors">model-00001-of-00003..&gt;</a> 2023-09-05 14:25  9.3G
-      <a href="model-00002-of-00003.safetensors">model-00002-of-00003..&gt;</a> 2023-09-05 14:25  9.2G
-      <a href="model-00003-of-00003.safetensors">model-00003-of-00003..&gt;</a> 2023-09-05 14:25  5.8G
-      <a href="model.safetensors.index.json">model.safetensors.in..&gt;</a> 2023-09-08 00:52   29K
-      <a href="pytorch_model-00001-of-00003.bin">pytorch_model-00001-..&gt;</a> 2023-09-04 15:07  9.3G
-      <a href="pytorch_model-00002-of-00003.bin">pytorch_model-00002-..&gt;</a> 2023-09-04 15:07  9.2G
-      <a href="pytorch_model-00003-of-00003.bin">pytorch_model-00003-..&gt;</a> 2023-09-04 15:07  5.8G
-      <a href="pytorch_model.bin.index.json">pytorch_model.bin.in..&gt;</a> 2023-09-08 00:52   29K
-      <a href="special_tokens_map.json">special_tokens_map.json</a> 2023-09-08 00:52   95
-      <a href="tokenizer.json">tokenizer.json</a>          2023-09-08 00:52  1.8M
-      <a href="tokenizer.model">tokenizer.model</a>         2023-09-04 15:07  488K
-      <a href="tokenizer_config.json">tokenizer_config.json</a>   2023-09-08 00:52  904
-<hr></pre>
-</body></html>