Instructions to use Vezora/WaveCoder-6.7b-Ultra-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Vezora/WaveCoder-6.7b-Ultra-bf16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Vezora/WaveCoder-6.7b-Ultra-bf16")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Vezora/WaveCoder-6.7b-Ultra-bf16")
model = AutoModelForCausalLM.from_pretrained("Vezora/WaveCoder-6.7b-Ultra-bf16")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Vezora/WaveCoder-6.7b-Ultra-bf16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Vezora/WaveCoder-6.7b-Ultra-bf16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vezora/WaveCoder-6.7b-Ultra-bf16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Vezora/WaveCoder-6.7b-Ultra-bf16

SGLang

How to use Vezora/WaveCoder-6.7b-Ultra-bf16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Vezora/WaveCoder-6.7b-Ultra-bf16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vezora/WaveCoder-6.7b-Ultra-bf16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Vezora/WaveCoder-6.7b-Ultra-bf16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vezora/WaveCoder-6.7b-Ultra-bf16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Vezora/WaveCoder-6.7b-Ultra-bf16 with Docker Model Runner:
```
docker model run hf.co/Vezora/WaveCoder-6.7b-Ultra-bf16
```

This is a Re-Upload of Wave-Coder-Ultra in bf16 since original model was uploaded in fp32 and there are none others available. Licensing remains the same as original model. (Half the Size as original, half the download time.)

🌊 WaveCoder: Widespread And Versatile Enhanced Code LLM

[📜 Paper] • [🐱 GitHub]
[🐦 Twitter] • [💬 Reddit] • [🍀 Unofficial Blog]

Repo for "WaveCoder: Widespread And Versatile Enhanced Instruction Tuning with Refined Data Generation"

🔥 News

[2024/04/10] 🔥🔥🔥 WaveCoder repo, models released at 🤗 HuggingFace!
[2023/12/26] WaveCoder paper released.

💡 Introduction

WaveCoder 🌊 is a series of large language models (LLMs) for the coding domain, designed to solve relevant problems in the field of code through instruction-following learning. Its training dataset was generated from a subset of code-search-net data using a generator-discriminator framework based on LLMs that we proposed, covering four general code-related tasks: code generation, code summary, code translation, and code repair.

Model	HumanEval	MBPP(500)	HumanEval Fix(Avg.)	HumanEval Explain(Avg.)
GPT-4	85.4	-	47.8	52.1
🌊 WaveCoder-DS-6.7B	65.8	63.0	49.5	40.8
🌊 WaveCoder-Pro-6.7B	74.4	63.4	52.1	43.0
🌊 WaveCoder-Ultra-6.7B	79.9	64.6	52.3	45.7

🪁 Evaluation

Please refer to WaveCoder's GitHub repo for inference, evaluation, and training code.

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("microsoft/wavecoder-ultra-6.7b")
model = AutoModelForCausalLM.from_pretrained("microsoft/wavecoder-ultra-6.7b")

📖 License

This code repository is licensed under the MIT License. The use of DeepSeek Coder models is subject to the its License.

☕️ Citation

If you find this repository helpful, please consider citing our paper:

@article{yu2023wavecoder,
  title={Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation},
  author={Yu, Zhaojian and Zhang, Xin and Shang, Ning and Huang, Yangyu and Xu, Can and Zhao, Yishujie and Hu, Wenxiang and Yin, Qiufeng},
  journal={arXiv preprint arXiv:2312.14187},
  year={2023}
}

Note

WaveCoder models are trained on the synthetic data generated by OpenAI models. Please pay attention to OpenAI's terms of use when using the models and the datasets.

Downloads last month: 2

Safetensors

Model size

7B params

Tensor type

BF16

Paper for Vezora/WaveCoder-6.7b-Ultra-bf16

WaveCoder: Widespread And Versatile Enhanced Instruction Tuning with Refined Data Generation

Paper • 2312.14187 • Published Dec 20, 2023 • 49