Instructions to use OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin")
model = AutoModelForCausalLM.from_pretrained("OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin

SGLang

How to use OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin with Docker Model Runner:
```
docker model run hf.co/OpenHandsCommunity/CodeQwen1.5-7B-OpenDevin
```

JustinLin610 commited on May 25, 2024

Commit

9e3eb1b

verified ·

1 Parent(s): 2cf5fcc

update intro and quickstart

Browse files

Files changed (1) hide show

README.md +56 -4

README.md CHANGED Viewed

@@ -3,15 +3,67 @@ language:
 - en
 pipeline_tag: text-generation
 ---
-# CodeQwen1.5-7B-OD
 ## Introduction
-OpenDevin/CodeQwen1.5-7B-OD is a version based on CodeQwen1.5-7B, fine-tuned with a small amount of `Agent` format data.
-It is an open model designed for rapid development, debugging, and iteration.
 ## Citation
-If you find our work helpful, feel free to give us a cite.
 ```
 @article{qwen,

 - en
 pipeline_tag: text-generation
 ---
+# CodeQwen1.5-7B-OpenDevin
 ## Introduction
+CodeQwen1.5-7B-OpenDevin is a code-specific model targeting on OpenDevin Agent tasks.
+The model is finetuned from CodeQwen1.5-7B, the code-specific large language model based on Qwen1.5 pretrained on large-scale code data.
+CodeQwen1.5-7B is strongly capable of understanding and generating codes, and it supports the context length of 65,536 tokens (for more information about CodeQwen1.5, please refer to the [blog post](https://qwenlm.github.io/blog/codeqwen1.5/) and [GitHub repo](https://github.com/QwenLM/Qwen1.5)).
+The finetuned model, CodeQwen1.5-7B-OpenDevin, shares similar features, while it is designed for rapid development, debugging, and iteration.
+## Performance
+We evaluate CodeQwen1.5-7B-OpenDevin on SWE-Bench-Lite by implementing the model on OpenDevin CodeAct 1.3 and follow the OpenDevin evaluation pipeline.
+CodeQwen1.5-7B-OpenDevin successfully solves 4 problems by commmiting pull requests targeting on the issues.
+## Requirements
+The code of Qwen1.5 has been in the latest Hugging face transformers and we advise you to install `transformers>=4.37.0`, or you might encounter the following error:
+```
+KeyError: 'qwen2'.
+```
+## Quickstart
+To use local models to run OpenDevin, we advise you to deploy CodeQwen1.5-7B-OpenDevin on a GPU device and access it through OpenAI API
+```bash
+python -m vllm.entrypoints.openai.api_server --model OpenDevin/CodeQwen1.5-7B-OpenDevin --dtype auto --api-key token-abc123
+```
+For more details, please refer to the official documentation of [vLLM for OpenAI Compatible server](https://docs.vllm.ai/en/stable/serving/openai_compatible_server.html).
+After the deployment, following the guidance of [OpenDevin](https://github.com/OpenDevin/OpenDevin) and run the following command to set up environment variables:
+```bash
+# The directory you want OpenDevin to work with. MUST be an absolute path!
+export WORKSPACE_BASE=$(pwd)/workspace;
+export LLM_API_KEY=token-abc123;
+export LLM_MODEL=OpenDevin/CodeQwen1.5-7B-OpenDevin;
+export LLM_BASE_URL=http://localhost:8000/v1;
+```
+and run the docker command:
+```bash
+docker run \
+    -it \
+    --pull=always \
+    -e SANDBOX_USER_ID=$(id -u) \
+    -e LLM_BASE_URL=$LLM_BASE_URL \
+    -e LLM_API_KEY=$LLM_API_KEY \
+    -e LLM_MODEL=$LLM_MODEL \
+    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
+    -v $WORKSPACE_BASE:/opt/workspace_base \
+    -v /var/run/docker.sock:/var/run/docker.sock \
+    -p 3000:3000 \
+    --add-host host.docker.internal:host-gateway \
+    ghcr.io/opendevin/opendevin:0.5
+```
+Now you should be able to connect `http://localhost:3000/`. Set up the configuration at the frontend by clicking the button at the bottom right, and input the right model name and api key.
+Then, you can enjoy playing with OpenDevin based on CodeQwen1.5-7B-OpenDevin!
 ## Citation
 ```
 @article{qwen,