Instructions to use Bayhakl1/GrimmCoder_v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Bayhakl1/GrimmCoder_v2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Bayhakl1/GrimmCoder_v2")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Bayhakl1/GrimmCoder_v2") model = AutoModelForCausalLM.from_pretrained("Bayhakl1/GrimmCoder_v2", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Bayhakl1/GrimmCoder_v2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Bayhakl1/GrimmCoder_v2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/GrimmCoder_v2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Bayhakl1/GrimmCoder_v2
- SGLang
How to use Bayhakl1/GrimmCoder_v2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Bayhakl1/GrimmCoder_v2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/GrimmCoder_v2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Bayhakl1/GrimmCoder_v2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/GrimmCoder_v2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Bayhakl1/GrimmCoder_v2 with Docker Model Runner:
docker model run hf.co/Bayhakl1/GrimmCoder_v2
GrimmCoder_v2
GrimmCoder_v2 is a ~107M-parameter LLaMA-style decoder-only transformer trained from scratch exclusively on Python source code for both left-to-right completion and Fill-in-the-Middle (FIM) code completion.
This release represents Phase 2 finetuning on Avelina/python-edu-cleaned (774.1M tokens, 6,000 steps with cosine decay 1e-4 → 1e-5), initialized from the 8B-token bigcode/starcoderdata pretrain base checkpoint.
Model Summary
| Property | Value |
|---|---|
| Architecture | LLaMA (LlamaForCausalLM, pre-norm, RoPE, SwiGLU, GQA, tied embeddings) |
| Parameters | |
| Context Length | 2048 tokens |
Hidden Size (d_model) |
768 |
| Layers | 16 |
| Attention Heads | 12 query heads / 4 KV heads (GQA) |
| FFN Intermediate Size | 2048 (SwiGLU) |
| Vocabulary | 8192 (byte-level BPE trained on Python) |
| Training Data | Avelina/python-edu-cleaned (finetuned from bigcode/starcoderdata pretrain) |
| Tokens / Checkpoint | ~8.70B tokens total (7.93B pretrain + 774.1M finetune, step 6,000, val loss 1.2260) |
Evaluation (HumanEval)
Evaluated on openai/openai_humaneval (164 Python problems) using transformers (bfloat16) on CUDA:
| Model Version | Mode & Decoding | pass@1 |
Solved | Syntax Valid |
|---|---|---|---|---|
| GrimmCoder_v2 | Left-to-Right: greedy (do_sample=False) |
9.15% | 15 / 164 | 137 / 164 (83.5%) |
| GrimmCoder_v2 | Left-to-Right: t=0.2, rep=1.1 |
9.15% | 15 / 164 | 158 / 164 (96.3%) |
| GrimmCoder_v2 | FIM (PSM): greedy (do_sample=False) |
9.15% | 15 / 164 | 151 / 164 (92.1%) |
| GrimmCoder_v2 | FIM (PSM): t=0.2, rep=1.1 |
8.54% | 14 / 164 | 153 / 164 (93.3%) |
| GrimmCoder (Base Pretrain) | Left-to-Right: greedy (do_sample=False) |
5.49% | 9 / 164 | 126 / 164 (76.8%) |
| GrimmCoder (Base Pretrain) | Left-to-Right: t=0.2, rep=1.1 |
4.88% | 8 / 164 | 155 / 164 (94.5%) |
| GrimmCoder (Base Pretrain) | FIM (PSM): greedy (do_sample=False) |
4.27% | 7 / 164 | 122 / 164 (74.4%) |
Solved Problems
- L2R Greedy (15):
HumanEval/16,22,23,28,29,30,31,34,45,48,50,52,53,55,60 - L2R t0.2_rep1.1 (15):
HumanEval/16,22,23,28,29,30,31,34,45,48,50,53,58,60,61 - FIM Greedy (15):
HumanEval/16,22,23,28,29,30,31,42,45,48,50,53,55,58,60 - FIM t0.2_rep1.1 (14):
HumanEval/12,16,22,23,28,31,34,35,42,45,48,53,58,60 - Overall Distinct Problems Solved Across All Evaluated Modes: 20 / 164 (
HumanEval/12, 16, 22, 23, 28, 29, 30, 31, 34, 35, 42, 45, 48, 50, 52, 53, 55, 58, 60, 61)
Special Tokens
| Token | ID | Purpose |
|---|---|---|
<|endoftext|> |
0 |
BOS / EOS / document delimiter |
<|fim_prefix|> |
1 |
FIM prefix marker |
<|fim_middle|> |
2 |
FIM middle marker (model generates from here) |
<|fim_suffix|> |
3 |
FIM suffix marker |
<|pad|> |
4 |
Padding token |
Usage (transformers)
1. Left-to-Right Code Completion
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Bayhakl1/GrimmCoder_v2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype=torch.bfloat16,
device_map="auto",
)
prompt = """def binary_search(arr, target):
lo, hi = 0, len(arr) - 1
"""
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=128,
do_sample=True,
temperature=0.2,
top_p=0.95,
eos_token_id=[0, 1, 2, 3, 4],
pad_token_id=4,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
2. Fill-in-the-Middle (FIM)
GrimmCoder_v2 natively supports Fill-in-the-Middle (FIM) to insert code between a prefix (code before the cursor) and a suffix (code after the cursor). The model was trained 50/50 on PSM and SPM formats.
Special tokens require vertical pipe characters:
<|fim_prefix|>,<|fim_suffix|>, and<|fim_middle|>.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Bayhakl1/GrimmCoder"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype=torch.bfloat16,
device_map="auto",
)
prefix = "def add(a: int, b: int) -> int:\n "
suffix = "\n\nprint(add(2, 3))\n"
# --- Method A: String Template (PSM format) ---
# Format: <|fim_prefix|> prefix <|fim_suffix|> suffix <|fim_middle|>
fim_prompt = f"<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>"
inputs = tokenizer(fim_prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
# --- Method B (Alternative): Direct Token IDs ---
# pre_id, mid_id, suf_id = 1, 2, 3
# p_ids = tokenizer.encode(prefix, add_special_tokens=False)
# s_ids = tokenizer.encode(suffix, add_special_tokens=False)
# input_ids = torch.tensor([[pre_id] + p_ids + [suf_id] + s_ids + [mid_id]], device=model.device)
# inputs = {"input_ids": input_ids, "attention_mask": torch.ones_like(input_ids)}
outputs = model.generate(
**inputs,
max_new_tokens=64,
do_sample=True,
temperature=0.2,
top_p=0.95,
eos_token_id=[0, 1, 2, 3, 4],
pad_token_id=4,
)
# Extract and decode only the generated middle tokens
middle_ids = outputs[0][inputs["input_ids"].shape[1]:]
middle = tokenizer.decode(middle_ids, skip_special_tokens=True)
# Reconstruct complete code
print("--- Generated Middle ---")
print(repr(middle))
print("\n--- Complete Code ---")
print(prefix + middle + suffix)
SPM Mode (Suffix-Prefix-Middle)
If you prefer SPM format, format the prompt as:
fim_prompt = f"<|fim_prefix|><|fim_suffix|>{suffix}<|fim_middle|>{prefix}"
Limitations
- Trained exclusively on Python source code; not intended as a general natural-language chat or instruction-following model.
- Base / completion model: responds best to code prompts and FIM markers rather than conversational Q&A.
- Downloads last month
- 140