Text Generation
Transformers
Safetensors
code
English
llama
python
code-completion
fill-in-the-middle
fim
text-generation-inference
Instructions to use Bayhakl1/GrimmCoder_v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Bayhakl1/GrimmCoder_v2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Bayhakl1/GrimmCoder_v2")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Bayhakl1/GrimmCoder_v2") model = AutoModelForCausalLM.from_pretrained("Bayhakl1/GrimmCoder_v2", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Bayhakl1/GrimmCoder_v2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Bayhakl1/GrimmCoder_v2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/GrimmCoder_v2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Bayhakl1/GrimmCoder_v2
- SGLang
How to use Bayhakl1/GrimmCoder_v2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Bayhakl1/GrimmCoder_v2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/GrimmCoder_v2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Bayhakl1/GrimmCoder_v2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/GrimmCoder_v2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Bayhakl1/GrimmCoder_v2 with Docker Model Runner:
docker model run hf.co/Bayhakl1/GrimmCoder_v2
|
Download README.md from Bayhakl1/GrimmCoder_v2: direct link, hf CLI and curl.
- Browser
- Download file 6.54 kB
-
https://huggingface.co/Bayhakl1/GrimmCoder_v2/resolve/main/README.md
- Command line
-
hf download hf://Bayhakl1/GrimmCoder_v2/README.md
-
curl -L -o README.md https://huggingface.co/Bayhakl1/GrimmCoder_v2/resolve/main/README.md
6.54 kB
| language: | |
| - code | |
| - en | |
| license: apache-2.0 | |
| datasets: | |
| - Avelina/python-edu-cleaned | |
| - bigcode/starcoderdata | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| tags: | |
| - code | |
| - python | |
| - code-completion | |
| - fill-in-the-middle | |
| - fim | |
| - llama | |
| # GrimmCoder_v2 | |
| **GrimmCoder_v2** is a ~107M-parameter LLaMA-style decoder-only transformer trained from scratch exclusively on Python source code for both **left-to-right completion** and **Fill-in-the-Middle (FIM)** code completion. | |
| This release represents **Phase 2 finetuning** on `Avelina/python-edu-cleaned` (774.1M tokens, 6,000 steps with cosine decay `1e-4 → 1e-5`), initialized from the 8B-token `bigcode/starcoderdata` pretrain base checkpoint. | |
| ## Model Summary | |
| | Property | Value | | |
| |---|---| | |
| | Architecture | LLaMA (`LlamaForCausalLM`, pre-norm, RoPE, SwiGLU, GQA, tied embeddings) | | |
| | Parameters | ~106.98M total (~100.69M non-embedding) | | |
| | Context Length | 2048 tokens | | |
| | Hidden Size (`d_model`) | 768 | | |
| | Layers | 16 | | |
| | Attention Heads | 12 query heads / 4 KV heads (GQA) | | |
| | FFN Intermediate Size | 2048 (SwiGLU) | | |
| | Vocabulary | 8192 (byte-level BPE trained on Python) | | |
| | Training Data | `Avelina/python-edu-cleaned` (finetuned from `bigcode/starcoderdata` pretrain) | | |
| | Tokens / Checkpoint | ~8.70B tokens total (7.93B pretrain + 774.1M finetune, step 6,000, val loss 1.2260) | | |
| ## Evaluation (HumanEval) | |
| Evaluated on [`openai/openai_humaneval`](https://huggingface.co/datasets/openai/openai_humaneval) (164 Python problems) using `transformers` (`bfloat16`) on CUDA: | |
| | Model Version | Mode & Decoding | `pass@1` | Solved | Syntax Valid | | |
| |---|---|---|---|---| | |
| | **GrimmCoder_v2** | Left-to-Right: greedy (`do_sample=False`) | **9.15%** | **15 / 164** | **137 / 164 (83.5%)** | | |
| | **GrimmCoder_v2** | Left-to-Right: `t=0.2, rep=1.1` | **9.15%** | **15 / 164** | **158 / 164 (96.3%)** | | |
| | **GrimmCoder_v2** | **FIM (PSM): greedy (`do_sample=False`)** | **9.15%** | **15 / 164** | **151 / 164 (92.1%)** | | |
| | **GrimmCoder_v2** | **FIM (PSM): `t=0.2, rep=1.1`** | **8.54%** | **14 / 164** | **153 / 164 (93.3%)** | | |
| | GrimmCoder (Base Pretrain) | Left-to-Right: greedy (`do_sample=False`) | 5.49% | 9 / 164 | 126 / 164 (76.8%) | | |
| | GrimmCoder (Base Pretrain) | Left-to-Right: `t=0.2, rep=1.1` | 4.88% | 8 / 164 | 155 / 164 (94.5%) | | |
| | GrimmCoder (Base Pretrain) | FIM (PSM): greedy (`do_sample=False`) | 4.27% | 7 / 164 | 122 / 164 (74.4%) | | |
| ### Solved Problems | |
| - **L2R Greedy (15)**: `HumanEval/16`, `22`, `23`, `28`, `29`, `30`, `31`, `34`, `45`, `48`, `50`, `52`, `53`, `55`, `60` | |
| - **L2R t0.2_rep1.1 (15)**: `HumanEval/16`, `22`, `23`, `28`, `29`, `30`, `31`, `34`, `45`, `48`, `50`, `53`, `58`, `60`, `61` | |
| - **FIM Greedy (15)**: `HumanEval/16`, `22`, `23`, `28`, `29`, `30`, `31`, `42`, `45`, `48`, `50`, `53`, `55`, `58`, `60` | |
| - **FIM t0.2_rep1.1 (14)**: `HumanEval/12`, `16`, `22`, `23`, `28`, `31`, `34`, `35`, `42`, `45`, `48`, `53`, `58`, `60` | |
| - **Overall Distinct Problems Solved Across All Evaluated Modes**: **20 / 164** (`HumanEval/12, 16, 22, 23, 28, 29, 30, 31, 34, 35, 42, 45, 48, 50, 52, 53, 55, 58, 60, 61`) | |
| ## Special Tokens | |
| | Token | ID | Purpose | | |
| |---|---|---| | |
| | `<\|endoftext\|>` | `0` | BOS / EOS / document delimiter | | |
| | `<\|fim_prefix\|>` | `1` | FIM prefix marker | | |
| | `<\|fim_middle\|>` | `2` | FIM middle marker (model generates from here) | | |
| | `<\|fim_suffix\|>` | `3` | FIM suffix marker | | |
| | `<\|pad\|>` | `4` | Padding token | | |
| ## Usage (`transformers`) | |
| ### 1. Left-to-Right Code Completion | |
| ```python | |
| import torch | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| model_id = "Bayhakl1/GrimmCoder_v2" | |
| tokenizer = AutoTokenizer.from_pretrained(model_id) | |
| model = AutoModelForCausalLM.from_pretrained( | |
| model_id, | |
| dtype=torch.bfloat16, | |
| device_map="auto", | |
| ) | |
| prompt = """def binary_search(arr, target): | |
| lo, hi = 0, len(arr) - 1 | |
| """ | |
| inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device) | |
| outputs = model.generate( | |
| **inputs, | |
| max_new_tokens=128, | |
| do_sample=True, | |
| temperature=0.2, | |
| top_p=0.95, | |
| eos_token_id=[0, 1, 2, 3, 4], | |
| pad_token_id=4, | |
| ) | |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) | |
| ``` | |
| ### 2. Fill-in-the-Middle (FIM) | |
| GrimmCoder_v2 natively supports Fill-in-the-Middle (FIM) to insert code between a **prefix** (code before the cursor) and a **suffix** (code after the cursor). The model was trained 50/50 on **PSM** and **SPM** formats. | |
| > [!NOTE] | |
| > Special tokens require vertical pipe characters: `<|fim_prefix|>`, `<|fim_suffix|>`, and `<|fim_middle|>`. | |
| ```python | |
| import torch | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| model_id = "Bayhakl1/GrimmCoder_v2" | |
| tokenizer = AutoTokenizer.from_pretrained(model_id) | |
| model = AutoModelForCausalLM.from_pretrained( | |
| model_id, | |
| dtype=torch.bfloat16, | |
| device_map="auto", | |
| ) | |
| prefix = "def add(a: int, b: int) -> int:\n " | |
| suffix = "\n\nprint(add(2, 3))\n" | |
| # --- Method A: String Template (PSM format) --- | |
| # Format: <|fim_prefix|> prefix <|fim_suffix|> suffix <|fim_middle|> | |
| fim_prompt = f"<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>" | |
| inputs = tokenizer(fim_prompt, return_tensors="pt", add_special_tokens=False).to(model.device) | |
| # --- Method B (Alternative): Direct Token IDs --- | |
| # pre_id, mid_id, suf_id = 1, 2, 3 | |
| # p_ids = tokenizer.encode(prefix, add_special_tokens=False) | |
| # s_ids = tokenizer.encode(suffix, add_special_tokens=False) | |
| # input_ids = torch.tensor([[pre_id] + p_ids + [suf_id] + s_ids + [mid_id]], device=model.device) | |
| # inputs = {"input_ids": input_ids, "attention_mask": torch.ones_like(input_ids)} | |
| outputs = model.generate( | |
| **inputs, | |
| max_new_tokens=64, | |
| do_sample=True, | |
| temperature=0.2, | |
| top_p=0.95, | |
| eos_token_id=[0, 1, 2, 3, 4], | |
| pad_token_id=4, | |
| ) | |
| # Extract and decode only the generated middle tokens | |
| middle_ids = outputs[0][inputs["input_ids"].shape[1]:] | |
| middle = tokenizer.decode(middle_ids, skip_special_tokens=True) | |
| # Reconstruct complete code | |
| print("--- Generated Middle ---") | |
| print(repr(middle)) | |
| print("\n--- Complete Code ---") | |
| print(prefix + middle + suffix) | |
| ``` | |
| #### SPM Mode (Suffix-Prefix-Middle) | |
| If you prefer SPM format, format the prompt as: | |
| ```python | |
| fim_prompt = f"<|fim_prefix|><|fim_suffix|>{suffix}<|fim_middle|>{prefix}" | |
| ``` | |
| ## Limitations | |
| - Trained exclusively on Python source code; not intended as a general natural-language chat or instruction-following model. | |
| - Base / completion model: responds best to code prompts and FIM markers rather than conversational Q&A. | |