GemCod270M - Zircon - XM (gemma-270m-it-code v6.1.1 experimental)

GemCod is a lightweight code generation model finetuned using SFT on the base gemma-270m-it (https://huggingface.co/google/gemma-3-270m-it) model. It offers accurate and quick(ish) code snippet generation in all major programming languages. It's small size (270M parameters) allows it to run comfortably on laptop grade GPUs.

The Zircon model differs from the standard GemCod architecture by having custom templating for FIM(Fill In the Middle) tasks as well as having multi-turn chat capabilities. This allows the model to act more like an embedded agent to assist in the generation and debugging of code during development.

This model has inferior generation capabilities on general coding tasks due to its specialized formatting and dataset. However, it represents a leap forward in the areas of general agentic architecture within the GemCod family.

This is the XM(Experimental) variant of the model which is trained on only 1,500 steps on SFT; it may give slightly incorrect outputs on long-form generation.

Estimated parameters: ~270M

Architecture: Gemma3

Intended use: FIM(Fill In the Middle) code completion, Multi-turn Chat

Training data

Source: code-fim-v2 dataset (https://huggingface.co/datasets/Etherll/code-fim-v2)
Rows: ~64,000 rows templated with a custom .jinja chat format
Training: trained for 1,500 steps on an RTX 3050 (4GB VRAM)

Usage

Install requirements:

pip install -r requirements.txt
pip install transformers datasets accelerate safetensors

Usage (Hugging Face Hub)

You can load it directly from HuggingFace:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM


device = "cuda" if torch.cuda.is_available() else "cpu"

model_path = "DireDreadlord/GemCod-Zircon-270M-XM"

tokenizer = AutoTokenizer.from_pretrained(model_path)

model = AutoModelForCausalLM.from_pretrained(model_path)
model.to(device)
model.eval()

if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

model.resize_token_embeddings(len(tokenizer))


chat_template = """{% for message in messages %}{% if message['role'] == 'user' %}User: {{ message['content'] }}
{% elif message['role'] == 'assistant' %}Assistant: {{ message['content'] }}
{% endif %}{% endfor %}"""
tokenizer.chat_template = chat_template

def generate_code(prompt, max_tokens=512) -> str:
    messages = [
        {
            "role": "user",
            "content": prompt
        }
    ]
    
    formatted_prompt = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    
    inputs = tokenizer(formatted_prompt, return_tensors="pt").to(device)
    input_length = inputs["input_ids"].shape[1]

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_tokens,
            do_sample=False,
            num_beams=1,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
            use_cache=False,
        )

    generated_tokens = outputs[0][input_length:]
    generated_text = tokenizer.decode(generated_tokens, skip_special_tokens=True)
    return generated_text


def test_fim(prefix, suffix, max_tokens=32):
    user = (
        "Complete the missing middle between the markers.\n"
        "PREFIX:\n" + prefix + "\n\n<FILL>\n\nSUFFIX:\n" + suffix
    )
    return generate_code(user, max_tokens=max_tokens)


def test_multiturn_chat(history, question, max_tokens=128):
    messages = []
    for role, content in history:
        messages.append({"role": role, "content": content})
    messages.append({"role": "user", "content": question})

    formatted = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(formatted, return_tensors="pt").to(device)
    input_length = inputs["input_ids"].shape[1]

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_tokens,
            do_sample=False,
            num_beams=1,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
            use_cache=False,
        )

    gen = outputs[0][input_length:]
    return tokenizer.decode(gen, skip_special_tokens=True)


if __name__ == "__main__":
    # FIM example
    pref = "def factorial(n):"
    suf = "return n"
    print("FIM test:\n", test_fim(pref, suf))

    # multi-turn chat example
    history = [("user", "Help me write a function to implement an bubble sort algorithm."), ("assistant", "Sure - what language?")]
    print("Chat test:\n", test_multiturn_chat(history, "write it in python"))