Instructions to use GAIR/OpenSWE-72B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use GAIR/OpenSWE-72B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="GAIR/OpenSWE-72B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("GAIR/OpenSWE-72B")
model = AutoModelForCausalLM.from_pretrained("GAIR/OpenSWE-72B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use GAIR/OpenSWE-72B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "GAIR/OpenSWE-72B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GAIR/OpenSWE-72B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/GAIR/OpenSWE-72B

SGLang

How to use GAIR/OpenSWE-72B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "GAIR/OpenSWE-72B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GAIR/OpenSWE-72B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "GAIR/OpenSWE-72B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GAIR/OpenSWE-72B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use GAIR/OpenSWE-72B with Docker Model Runner:
```
docker model run hf.co/GAIR/OpenSWE-72B
```

fudayuan commited on Mar 15

Commit

a958872

verified ·

1 Parent(s): 380fc1d

Add files using upload-large-folder tool

Browse files

Files changed (20) hide show

model-00001-of-00028.safetensors +3 -0
model-00003-of-00028.safetensors +3 -0
model-00004-of-00028.safetensors +3 -0
model-00005-of-00028.safetensors +3 -0
model-00007-of-00028.safetensors +3 -0
model-00008-of-00028.safetensors +3 -0
model-00010-of-00028.safetensors +3 -0
model-00012-of-00028.safetensors +3 -0
model-00013-of-00028.safetensors +3 -0
model-00014-of-00028.safetensors +3 -0
model-00015-of-00028.safetensors +3 -0
model-00016-of-00028.safetensors +3 -0
model-00017-of-00028.safetensors +3 -0
model-00018-of-00028.safetensors +3 -0
model-00019-of-00028.safetensors +3 -0
model-00020-of-00028.safetensors +3 -0
model-00022-of-00028.safetensors +3 -0
model-00025-of-00028.safetensors +3 -0
model-00026-of-00028.safetensors +3 -0
model-00027-of-00028.safetensors +3 -0

model-00001-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6c99dcacc320b8acb244f19d471e4e202fd1e7bd6402fe66e0808377ceedf221
+size 5368713832

model-00003-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eb0bfc813513755886543e2e20d3af24d1d352d5a708ab096f1d7565e6d007e8
+size 5368720416

model-00004-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b1dcb4c5fc823b0eefdcf7162ea72a721814339982306b11397942729844c981
+size 5368720416

model-00005-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b0c307d07fcbc2c90cabf0760546083db888f53912b9e801bca60536b563d2e9
+size 5328864496

model-00007-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:79bf69416381d4f0d7f930bcbd681c77c410507787c62c99b686b0c038662ed2
+size 5328864504

model-00008-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:17072742e4e68546db46e1f8641f903da6cf1bde3634c6fb1014fe63981421e9
+size 5328864504

model-00010-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:44af6dcb4168a72658838b218ca7aa091c11c65d68c3c892d75d623975f707a8
+size 5328864504

model-00012-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:10e9df0635efc62dd41cc7248c3089f7848af80b0876c64bb7d9e0de914ee53c
+size 5330529496

model-00013-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:634f6173edbeed3aaf985fc166319a8302fed76e06cb65cddb35306c6968e247
+size 5328864488

model-00014-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:884660dc544996e2e9de93afdf2e2a9cce48be30161386910cb22430bda11181
+size 5328864496

model-00015-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9d3c7045b4c42351a3f9d1253d9da02afc59f4c255db9b333f112e1a8e06ac6b
+size 5328864496

model-00016-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:51b4c8c66027f04d91b1b609d355e7f6b4ee757a69df31b1755d280a1969ab91
+size 5328864496

model-00017-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b8376cb2a7e34040e2f16b2ea2114ce24ce7a3a37cc9f74102d0614c286683c1
+size 5328864496

model-00018-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dd1bd955cf2a9bc1bd04256dd874b29300b3977d031daea0a541317c083b9613
+size 5328864496

model-00019-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f7e346819fd90c361a1459fb1c0d2dd3c0d60087c01bd691dda5c4aba1368de7
+size 5328864496

model-00020-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ac939b9d106cac8a5b935c32c61604a87343ee4f4f975bb490406c745fe9ccdc
+size 5328864496

model-00022-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0997f882d27aee44850418e341bd082f896a1e49a6f307bd1768bee273b9d1e6
+size 5328864496

model-00025-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7baa9e356083ade085810d7cf1d79f6632200d1fde3488c759b04caebe7fd89f
+size 5328864496

model-00026-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:258275d7559137efb1cdf159e34a7c3a0a12e37079e931e6e2cce265be3a6f0a
+size 4359996520

model-00027-of-00028.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:82045112d9fefd41f47cbce31034ec5c9d3e1c4d913f4625ab65d72a37dc091d
+size 2491416672