Text Generation
Transformers
Safetensors
English
qwen2
code-generation
python
lora
rejection-sampling
conversational
text-generation-inference
Instructions to use layer8labs8/OrdoPy-Coder-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use layer8labs8/OrdoPy-Coder-0.5B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="layer8labs8/OrdoPy-Coder-0.5B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("layer8labs8/OrdoPy-Coder-0.5B") model = AutoModelForCausalLM.from_pretrained("layer8labs8/OrdoPy-Coder-0.5B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use layer8labs8/OrdoPy-Coder-0.5B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "layer8labs8/OrdoPy-Coder-0.5B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "layer8labs8/OrdoPy-Coder-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/layer8labs8/OrdoPy-Coder-0.5B
- SGLang
How to use layer8labs8/OrdoPy-Coder-0.5B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "layer8labs8/OrdoPy-Coder-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "layer8labs8/OrdoPy-Coder-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "layer8labs8/OrdoPy-Coder-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "layer8labs8/OrdoPy-Coder-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use layer8labs8/OrdoPy-Coder-0.5B with Docker Model Runner:
docker model run hf.co/layer8labs8/OrdoPy-Coder-0.5B
File size: 3,710 Bytes
0846bbc 0028567 0846bbc 0028567 0846bbc 0028567 0846bbc 2f6ae41 0846bbc 0028567 0846bbc 0028567 0846bbc 0028567 2f6ae41 0028567 0846bbc 0028567 0846bbc 0028567 0846bbc 0028567 0846bbc 0028567 0846bbc 0028567 0846bbc 0028567 0846bbc 0028567 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-0.5B-Instruct
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- qwen2
- code-generation
- python
- lora
- rejection-sampling
---
# OrdoPy-Coder-0.5B (v2)
**OrdoPy-Coder-0.5B v2** is `Qwen/Qwen2.5-Coder-0.5B-Instruct` fine-tuned with a rejection-sampling recipe: the base model wrote 4 solutions for each of 2,000 [KodCode](https://huggingface.co/datasets/KodCode/KodCode-V1-SFT-4o) problems, every solution was executed against the problem's unit tests, and only passing solutions were kept (max 2 per problem; GPT-4o reference solutions filled in for unsolved problems, capped at 20% of the data). A LoRA adapter (r=32, all linear layers) was trained on these 1,106 examples and merged; the epoch was selected on MBPP `train + validation`, never on test data. It runs in about 1 GB of VRAM (`float16`).
## Benchmarks (EvalPlus, greedy `pass@1`)
Base and v2 were scored by the official [EvalPlus](https://github.com/evalplus/evalplus) harness in the **same run** (Kaggle 2x T4, `float16`):
| Benchmark | Base, official¹ | Base, same run | **v2 (this model)** | v1 (previous upload) |
| :--- | :---: | :---: | :---: | :---: |
| HumanEval | `61.6` | `57.9` | **`61.0`** | `48.8` |
| HumanEval+ | `57.3` | `53.7` | **`55.5`** | `46.3` |
| MBPP | `52.4` | `52.1` | `51.1` | `48.4` |
| MBPP+ | `43.7` | `44.2` | `41.0` | `42.1` |
| **Average** | — | `52.0` | **`52.2`** | `46.4` |
¹ [Qwen2.5-Coder Technical Report](https://arxiv.org/abs/2409.12186), Table 16 (Qwen's own evaluation setup).
**Honest summary:** v2 is clearly better than v1 (+5.8 points on average) and **about on par with the base model** (better on HumanEval / HumanEval+, worse on MBPP / MBPP+; single seed, differences of this size are close to run-to-run noise). It does not beat the base model overall.
> **About v1.** The first upload is kept under the `v1` tag of this repository (`AutoModelForCausalLM.from_pretrained("layer8labs8/OrdoPy-Coder-0.5B", revision="v1")`). It was a supervised fine-tune that scores below the base model on all four benchmarks. Its card claimed HumanEval `9.76% -> 40.85%`; that comparison was a measurement error (a 224-token generation cap truncated the base model's answers, and the checkpoint was selected on test slices). `eval_benchmarks.json` in this repository is v1 training telemetry. Full v2 report and code: [OrdoTrain](https://github.com/layer8labs8/OrdoTrain) (`results/ordopy_rft_lora_report.json`).
## Model Details
- **Architecture**: `Qwen2ForCausalLM` (24 layers, hidden size 896, 494M parameters, 32K context)
- **Base model**: `Qwen/Qwen2.5-Coder-0.5B-Instruct`
- **Fine-tuning**: LoRA r=32 / alpha=64 on all linear layers, lr 1e-4, 1 epoch kept, assistant-only loss, merged into the weights
- **Format**: `model.safetensors` (`float16`) + Qwen ChatML tokenizer
## Quickstart
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "layer8labs8/OrdoPy-Coder-0.5B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float16, device_map="auto")
messages = [{"role": "user", "content": "Write a Python function `is_palindrome_clean(s: str) -> bool` that ignores spaces and punctuation."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```
On `transformers` < 4.56 use `torch_dtype=` instead of `dtype=`.
|