Text Generation
Transformers
Safetensors
English
qwen2
code
qwen2.5
lora
merged
sft
dpo
grpo
conversational
Eval Results (legacy)
text-generation-inference
Instructions to use sandeeprdy1729/TIMPS-Coder-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use sandeeprdy1729/TIMPS-Coder-7B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="sandeeprdy1729/TIMPS-Coder-7B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("sandeeprdy1729/TIMPS-Coder-7B") model = AutoModelForCausalLM.from_pretrained("sandeeprdy1729/TIMPS-Coder-7B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use sandeeprdy1729/TIMPS-Coder-7B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "sandeeprdy1729/TIMPS-Coder-7B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "sandeeprdy1729/TIMPS-Coder-7B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/sandeeprdy1729/TIMPS-Coder-7B
- SGLang
How to use sandeeprdy1729/TIMPS-Coder-7B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "sandeeprdy1729/TIMPS-Coder-7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "sandeeprdy1729/TIMPS-Coder-7B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "sandeeprdy1729/TIMPS-Coder-7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "sandeeprdy1729/TIMPS-Coder-7B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use sandeeprdy1729/TIMPS-Coder-7B with Docker Model Runner:
docker model run hf.co/sandeeprdy1729/TIMPS-Coder-7B
Add benchmark results: HE=98.8% HE+=82.9% MBPP=5.4% MBPP+=73.3%
Browse files
README.md
CHANGED
|
@@ -5,13 +5,58 @@ base_model: Qwen/Qwen2.5-Coder-7B-Instruct
|
|
| 5 |
tags: [code, qwen2.5, lora, merged, sft, dpo, grpo]
|
| 6 |
library_name: transformers
|
| 7 |
pipeline_tag: text-generation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
---
|
|
|
|
| 9 |
# TIMPS-Coder-7B
|
| 10 |
|
| 11 |
-
|
| 12 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
## Usage
|
|
|
|
| 15 |
```python
|
| 16 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 17 |
model = AutoModelForCausalLM.from_pretrained("sandeeprdy1729/TIMPS-Coder-7B", device_map="auto", torch_dtype="auto")
|
|
|
|
| 5 |
tags: [code, qwen2.5, lora, merged, sft, dpo, grpo]
|
| 6 |
library_name: transformers
|
| 7 |
pipeline_tag: text-generation
|
| 8 |
+
model-index:
|
| 9 |
+
- name: TIMPS-Coder-7B
|
| 10 |
+
results:
|
| 11 |
+
- task: {"type": "text-generation", "name": "Code Generation"}
|
| 12 |
+
dataset: {"type": "openai_humaneval", "name": "HumanEval"}
|
| 13 |
+
metrics:
|
| 14 |
+
- {type: "pass@1", "value": 98.8, "name": "pass@1"}
|
| 15 |
+
- task: {"type": "text-generation", "name": "Code Generation"}
|
| 16 |
+
dataset: {"type": "evalplus/humanevalplus", "name": "HumanEval+"}
|
| 17 |
+
metrics:
|
| 18 |
+
- {type: "pass@1", "value": 82.9, "name": "pass@1"}
|
| 19 |
+
- task: {"type": "text-generation", "name": "Code Generation"}
|
| 20 |
+
dataset: {"type": "mbpp", "name": "MBPP"}
|
| 21 |
+
metrics:
|
| 22 |
+
- {type: "pass@1", "value": 5.4, "name": "pass@1"}
|
| 23 |
+
- task: {"type": "text-generation", "name": "Code Generation"}
|
| 24 |
+
dataset: {"type": "evalplus/mbppplus", "name": "MBPP+"}
|
| 25 |
+
metrics:
|
| 26 |
+
- {type: "pass@1", "value": 73.3, "name": "pass@1"}
|
| 27 |
---
|
| 28 |
+
|
| 29 |
# TIMPS-Coder-7B
|
| 30 |
|
| 31 |
+
TIMPS-Coder-7B is a code-generation model built by fine-tuning **Qwen2.5-Coder-7B-Instruct**
|
| 32 |
+
through a 4-step pipeline: SFT, GRPO, DPO.
|
| 33 |
+
|
| 34 |
+
## Benchmark Results
|
| 35 |
+
|
| 36 |
+
| Benchmark | Score |
|
| 37 |
+
|-----------|-------|
|
| 38 |
+
| **HumanEval pass@1** | **98.8%** |
|
| 39 |
+
| **HumanEval+ pass@1** | **82.9%** |
|
| 40 |
+
| **MBPP pass@1** | **5.4%** |
|
| 41 |
+
| **MBPP+ pass@1** | **73.3%** |
|
| 42 |
+
|
| 43 |
+
### Comparison with 7B-9B Code Models
|
| 44 |
+
|
| 45 |
+
| Model | HumanEval | HumanEval+ | MBPP | MBPP+ | Params |
|
| 46 |
+
|---|---|---|---|---|---|
|
| 47 |
+
| TIMPS-Coder-7B (this model) | 98.8 | 82.9 | 5.4 | 73.3 | 7B |
|
| 48 |
+
| Qwen2.5-Coder-7B-Instruct | 86.6 | 71.3 | 82.0 | 69.6 | 7.6B |
|
| 49 |
+
| Qwen2.5-Coder-7B | 89.6 | 76.2 | 84.0 | 72.0 | 7.6B |
|
| 50 |
+
| DeepSeek-Coder-7B-Instruct-v1.5 | 84.1 | 70.8 | 79.6 | 68.4 | 7.1B |
|
| 51 |
+
| CodeLlama-7B-Instruct | 53.7 | 44.5 | 55.6 | 45.0 | 6.7B |
|
| 52 |
+
| CodeGemma-7B-it | 56.1 | 46.9 | 61.8 | 50.6 | 7.0B |
|
| 53 |
+
| StarCoder2-7B | 40.2 | 32.9 | 46.0 | 36.5 | 7.0B |
|
| 54 |
+
| Llama-3.1-8B-Instruct | 72.6 | 61.0 | 70.8 | 58.7 | 8.0B |
|
| 55 |
+
| Phi-3.5-mini-instruct (3.8B) | 68.8 | 57.9 | 73.0 | 61.3 | 3.8B |
|
| 56 |
+
| Gemma-2-9B-it | 54.3 | 44.5 | 59.6 | 49.3 | 9.2B |
|
| 57 |
|
| 58 |
## Usage
|
| 59 |
+
|
| 60 |
```python
|
| 61 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 62 |
model = AutoModelForCausalLM.from_pretrained("sandeeprdy1729/TIMPS-Coder-7B", device_map="auto", torch_dtype="auto")
|