Instructions to use typeof/openchat-3.6-exploded with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use typeof/openchat-3.6-exploded with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="typeof/openchat-3.6-exploded")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("typeof/openchat-3.6-exploded")
model = AutoModelForCausalLM.from_pretrained("typeof/openchat-3.6-exploded", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use typeof/openchat-3.6-exploded with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "typeof/openchat-3.6-exploded"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "typeof/openchat-3.6-exploded",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/typeof/openchat-3.6-exploded

SGLang

How to use typeof/openchat-3.6-exploded with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "typeof/openchat-3.6-exploded" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "typeof/openchat-3.6-exploded",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "typeof/openchat-3.6-exploded" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "typeof/openchat-3.6-exploded",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use typeof/openchat-3.6-exploded with Docker Model Runner:
```
docker model run hf.co/typeof/openchat-3.6-exploded
```

typeof commited on May 25, 2024

Commit

45fe537

1 Parent(s): cc859b7

init

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

model-00084-of-00291.safetensors +3 -0
model-00085-of-00291.safetensors +3 -0
model-00086-of-00291.safetensors +3 -0
model-00087-of-00291.safetensors +3 -0
model-00088-of-00291.safetensors +3 -0
model-00089-of-00291.safetensors +3 -0
model-00090-of-00291.safetensors +3 -0
model-00091-of-00291.safetensors +3 -0
model-00092-of-00291.safetensors +3 -0
model-00093-of-00291.safetensors +3 -0
model-00094-of-00291.safetensors +3 -0
model-00095-of-00291.safetensors +3 -0
model-00096-of-00291.safetensors +3 -0
model-00097-of-00291.safetensors +3 -0
model-00098-of-00291.safetensors +3 -0
model-00099-of-00291.safetensors +3 -0
model-00100-of-00291.safetensors +3 -0
model-00101-of-00291.safetensors +3 -0
model-00102-of-00291.safetensors +3 -0
model-00103-of-00291.safetensors +3 -0
model-00104-of-00291.safetensors +3 -0
model-00105-of-00291.safetensors +3 -0
model-00106-of-00291.safetensors +3 -0
model-00107-of-00291.safetensors +3 -0
model-00108-of-00291.safetensors +3 -0
model-00109-of-00291.safetensors +3 -0
model-00110-of-00291.safetensors +3 -0
model-00111-of-00291.safetensors +3 -0
model-00112-of-00291.safetensors +3 -0
model-00113-of-00291.safetensors +3 -0
model-00114-of-00291.safetensors +3 -0
model-00115-of-00291.safetensors +3 -0
model-00116-of-00291.safetensors +3 -0
model-00117-of-00291.safetensors +3 -0
model-00118-of-00291.safetensors +3 -0
model-00119-of-00291.safetensors +3 -0
model-00120-of-00291.safetensors +3 -0
model-00121-of-00291.safetensors +3 -0
model-00122-of-00291.safetensors +3 -0
model-00123-of-00291.safetensors +3 -0
model-00124-of-00291.safetensors +3 -0
model-00125-of-00291.safetensors +3 -0
model-00126-of-00291.safetensors +3 -0
model-00127-of-00291.safetensors +3 -0
model-00128-of-00291.safetensors +3 -0
model-00129-of-00291.safetensors +3 -0
model-00130-of-00291.safetensors +3 -0
model-00131-of-00291.safetensors +3 -0
model-00132-of-00291.safetensors +3 -0
model-00133-of-00291.safetensors +3 -0

model-00084-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eb7007098d553ad5a4a2da8398671f9f8f45de6048e7c5c44d4fb3d8fb33669d
+size 8328

model-00085-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:98eef6e6d3033953645d9b4c94dd08dc720c359f25452e193df6c7e2fb2ebf0f
+size 117440664

model-00086-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c4563ef491bd0f2995815b85465d3a32a47ea1ec4d6009f20099575bc96d62d9
+size 117440664

model-00087-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1caf0fd7412ceebe15ae074de4d2b7a3f9ce03730d3624da5b266833d0329d63
+size 117440656

model-00088-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:776a5bbaf0314e04805bfb7e2a245fc9716c43fbe772c8e122ce79fb4fbdb084
+size 8344

model-00089-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:04080bbf8447700d06b4b672df25c35688a38dfed8792403918ce28594cb86e0
+size 8388760

model-00090-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3e7d3711ea1b4b718c27baf5faba6d5976e9aef7d5ff89ab15ebcfdca1d04227
+size 33554584

model-00091-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f471f220a533eb0b42ccd8e9f35548af113735cbbfa868033fa1519707b906c4
+size 33554584

model-00092-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:66f8b1c259e05b556f39a766f1198cb43d8b2d96bc365cbf3b082019787cd55a
+size 8388760

model-00093-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ee385445e097c22d7ef4b0df23b63cfb9e68a52e7a14838579934b744214af3b
+size 8336

model-00094-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4e4218e30cf4b7be4824e42387b538a56095a3ff1d92b2d8cd00d3d3d6846474
+size 117440664

model-00095-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:03762d03f0e9712bff02766a6cb227d5f2eda668d0b1f7914ba3625736f68f13
+size 117440664

model-00096-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6978ee0a9192cedcacc3ce6f94a5b545867e65f0af54c15448a78af295191563
+size 117440656

model-00097-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:afcb7605b045610a8e0df3e3ac5c0d7a0b2cb42390e635888f4a5530db60b323
+size 8344

model-00098-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e0e984d927f9cd336f18672f0253ed52b5bdeed48f8902d2ce405c1f82deb1ce
+size 8388760

model-00099-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b4964e0924b101d87ac49bfe582e41c1ddb155d64c2909b752bc2cf265320f04
+size 33554584

model-00100-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7dc83d903bcd88f47b96886d66b58e454b216b669c06c19b60bbdc684fee31f5
+size 33554584

model-00101-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7e979ccfd36319c05fc34c022ab7f4c3beb7eca0aefefd0abb5bbbc60228aa00
+size 8388760

model-00102-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:02698dd24a845ef274d303f693c32d94e10e70249d1b7b66ccffc83c9dbbe0d9
+size 8336

model-00103-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a8273f38298cbee41e41e4424d83e689d9410e8cdb5a0b661778e8a1395ec49c
+size 117440664

model-00104-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a6a9a67726bc0682e2c0e0361ad24210f3a3f1a054ae899047d2bcfa035cff48
+size 117440664

model-00105-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cadf6121f3beb1f10c03f085278e79b9e94c286568b7b07b647916a0feb6cb7d
+size 117440656

model-00106-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4ae48790715d4ee15cd44d927cc00a8f21657e0a1cc9951585ab1c5f98c21e37
+size 8344

model-00107-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5c9d84e6ee169cfcb76b0b946bfdb65ce236b4611340da90a936c0d132144663
+size 8388760

model-00108-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b353c83aa2e8dc38766f0d5e822c5deb0e88e8cda203a274e19ced27179ea15e
+size 33554584

model-00109-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5176c7a7939e80a9b199f54cad5b2bfc026c7cc90af523b23ee1e678cf411ad5
+size 33554584

model-00110-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f3031364e33725d0342ee263a83180a58d6834b12f35f264f30e07d5eb0c7314
+size 8388760

model-00111-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3ce304fc360979236d88783f859d4129718362d81f2400db2657be0fb4d0b690
+size 8336

model-00112-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c438cdddf779027c0a9d8ce96f3010ec744182bc3c55b3a54c5b38d2ab1ca9fa
+size 117440664

model-00113-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dd7722e66851df021bd5b4882b6ea75d29779d22296fd42bafab5ef89cf6b41a
+size 117440664

model-00114-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:12dcde3a49d4011d3a70f575210726713b913fd369a52270593f585af5c762b3
+size 117440656

model-00115-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4425f574ee66f32edacc77379e9ba5e960e3f76b5a7ea42b540784973f241231
+size 8344

model-00116-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9cdeefc1e43e0159f155eb424e8ee52a9b8b8e0ea9600d82e00687ce8acccb06
+size 8388760

model-00117-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6ef7290215ae506e38360ada13883dfeac069aac293d00fcc7065d1a402a5729
+size 33554584

model-00118-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4c354cf563c202dc94b00f68a4b34797c36753244f34dbd13f53a901141a8c25
+size 33554584

model-00119-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0d755d6ce3149cf60d438f09d1eb8996faab055a110ff9a6a4014337dc745e69
+size 8388760

model-00120-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7863be51319d4fed13ffd5da05e7f0b8c63e24a71bba27cebcfca407284f2ede
+size 8336

model-00121-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:759f73498342a4739efd7cdc4816fd814138e93cdf52ed74d993859e4e80a6a2
+size 117440664

model-00122-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:81ed2f3329ade8e58b7d65f8fb4cffe29a311b87f1b2d67219382ab24031184a
+size 117440664

model-00123-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:010e71734e0ee3062e236ec7dbbcfddb9d6a900eb83e4cf5cd4342214f59b494
+size 117440656

model-00124-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:34cfd2045c215b427710fe7d4dceb8741b443351aedeb81c93775507928eb19e
+size 8344

model-00125-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:abaa9860f0f9787f73c78f410a1449918f2f4f64afe27b9406473c891d46282d
+size 8388760

model-00126-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:17bda9d5bcd4feec831f1e70d0ebe2aec4fa7626dc263352e15044fc6062b59b
+size 33554584

model-00127-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ebb816f12815b531d40205ed1120e615f6a6bb42711853ac943997814219ab98
+size 33554584

model-00128-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0887570cce8c87fd86b98532edb4e47356cb4bce1f9b40f0134c32c627f38343
+size 8388760

model-00129-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2406b142a30a14df4f8852d05dcafdc5dd623010a8c30c94ea62fd19d2e4f1bd
+size 8336

model-00130-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bf96088dbd2039d59dc599d1ba144dbe95f0ae2077ce0d8e7a44e5e5c5e19f69
+size 117440664

model-00131-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a695dd2c5ee635ec2da998fec107c09a27f1cd1705c3daa64546254b8fe71093
+size 117440664

model-00132-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b336f1ee3983a8877f92b79e8f4529158d7bbc1d7f06c9abc3cf4205082e1bb7
+size 117440656

model-00133-of-00291.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f129bc330d6368be71931c33eb7527c4a4da3eea9b390ef04d30322f984e79ad
+size 8344