Text Generation
Transformers
Safetensors
English
qwen2
code-generation
python
lora
rejection-sampling
conversational
text-generation-inference
Instructions to use layer8labs8/OrdoPy-Coder-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use layer8labs8/OrdoPy-Coder-0.5B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="layer8labs8/OrdoPy-Coder-0.5B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("layer8labs8/OrdoPy-Coder-0.5B") model = AutoModelForCausalLM.from_pretrained("layer8labs8/OrdoPy-Coder-0.5B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use layer8labs8/OrdoPy-Coder-0.5B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "layer8labs8/OrdoPy-Coder-0.5B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "layer8labs8/OrdoPy-Coder-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/layer8labs8/OrdoPy-Coder-0.5B
- SGLang
How to use layer8labs8/OrdoPy-Coder-0.5B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "layer8labs8/OrdoPy-Coder-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "layer8labs8/OrdoPy-Coder-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "layer8labs8/OrdoPy-Coder-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "layer8labs8/OrdoPy-Coder-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use layer8labs8/OrdoPy-Coder-0.5B with Docker Model Runner:
docker model run hf.co/layer8labs8/OrdoPy-Coder-0.5B
Correct benchmark claims: EvalPlus vs official base model
Browse files
README.md
CHANGED
|
@@ -17,28 +17,25 @@ tags:
|
|
| 17 |
|
| 18 |
**OrdoPy-Coder-0.5B** is a compact 0.5B parameter specialist model tuned on `Qwen/Qwen2.5-Coder-0.5B-Instruct` exclusively for **Python programming**, **English instruction following**, and **step-by-step logical reasoning**.
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
##
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
-
| Benchmark
|
| 27 |
| :--- | :---: | :---: | :---: |
|
| 28 |
-
| **HumanEval
|
| 29 |
-
| **
|
| 30 |
-
| **
|
|
|
|
| 31 |
|
| 32 |
-
|
| 33 |
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
| **HumanEval+ (`base + extra tests`)** | **`46.3%`** (`0.4634`) | `76 / 164` |
|
| 38 |
-
| **MBPP (`base`)** | **`48.4%`** (`0.4841`) | `183 / 378` |
|
| 39 |
-
| **MBPP+ (`base + extra tests`)** | **`42.1%`** (`0.4206`) | `159 / 378` |
|
| 40 |
|
| 41 |
-
## Held-Out Domain Evaluation
|
| 42 |
|
| 43 |
| Domain (Held-Out Split) | Baseline Loss (PPL / Token Acc) | `OrdoPy-Coder-0.5B` Loss (PPL / Token Acc) |
|
| 44 |
| :--- | :---: | :---: |
|
|
|
|
| 17 |
|
| 18 |
**OrdoPy-Coder-0.5B** is a compact 0.5B parameter specialist model tuned on `Qwen/Qwen2.5-Coder-0.5B-Instruct` exclusively for **Python programming**, **English instruction following**, and **step-by-step logical reasoning**.
|
| 19 |
|
| 20 |
+
It runs in under 1.5 GB of VRAM (`float16`). **Note: on EvalPlus it currently scores below its base model (see the correction below).**
|
| 21 |
|
| 22 |
+
## Benchmarks (EvalPlus, greedy `pass@1`)
|
| 23 |
|
| 24 |
+
Scored with the official [EvalPlus](https://github.com/evalplus/evalplus) harness (`evalplus 0.4.0`), compared with the official numbers for the base model ([Qwen2.5-Coder Technical Report](https://arxiv.org/abs/2409.12186), Table 16):
|
| 25 |
|
| 26 |
+
| Benchmark | `Qwen2.5-Coder-0.5B-Instruct` (official) | `OrdoPy-Coder-0.5B` | Delta |
|
| 27 |
| :--- | :---: | :---: | :---: |
|
| 28 |
+
| **HumanEval** | `61.6%` | `48.8%` (`80 / 164`) | `-12.8` |
|
| 29 |
+
| **HumanEval+** | `57.3%` | `46.3%` (`76 / 164`) | `-11.0` |
|
| 30 |
+
| **MBPP** | `52.4%` | `48.4%` (`183 / 378`) | `-4.0` |
|
| 31 |
+
| **MBPP+** | `43.7%` | `42.1%` (`159 / 378`) | `-1.6` |
|
| 32 |
|
| 33 |
+
> **Correction (November 2026).** An earlier version of this card reported HumanEval rising from `9.76%` to `40.85%` (`4.19x`). That comparison was wrong: the in-house evaluation cut generations at 224 new tokens, which truncated many of the base model's longer answers, and the checkpoint was selected on a slice of the test sets. Measured with EvalPlus, **this fine-tune scores below its base model on all four benchmarks**. If you need the strongest 0.5B coder, use [`Qwen/Qwen2.5-Coder-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct) directly. A follow-up experiment (rejection-sampling fine-tune with LoRA on the base model's own test-verified solutions) did not degrade the model but also gave no clear gain (HumanEval `+3.1`, HumanEval+ `+1.8`, MBPP `-1.0`, MBPP+ `-3.2` vs the base measured in the same run); details in the [OrdoTrain repository](https://github.com/layer8labs8/OrdoTrain).
|
| 34 |
|
| 35 |
+
## Held-Out Domain Evaluation (token-level loss)
|
| 36 |
+
|
| 37 |
+
Token-level loss on held-out splits improved slightly, but this did not translate into better pass@1 (see above).
|
|
|
|
|
|
|
|
|
|
| 38 |
|
|
|
|
| 39 |
|
| 40 |
| Domain (Held-Out Split) | Baseline Loss (PPL / Token Acc) | `OrdoPy-Coder-0.5B` Loss (PPL / Token Acc) |
|
| 41 |
| :--- | :---: | :---: |
|