winter-frost-2

Run it

Three ways in, from easiest to most involved.

Google Colab

Free GPU, zero local setup. Paste into one cell — installs everything, loads the model in 4-bit, opens an interactive chat loop.

!pip install -q -U transformers accelerate bitsandbytes

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                                 bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True)

tokenizer = AutoTokenizer.from_pretrained("brucoder/winter-frost-2")
model = AutoModelForCausalLM.from_pretrained("brucoder/winter-frost-2",
                                              quantization_config=bnb_config, device_map="auto")

conversation = [{"role": "system", "content": "You are a helpful assistant."}]
print("Model loaded. Chat with winter-frost-2 (type 'quit' to stop).\n")

while True:
    user_input = input("You: ")
    if user_input.strip().lower() in ("quit", "exit"):
        break
    conversation.append({"role": "user", "content": user_input})
    inputs = tokenizer.apply_chat_template(conversation, tokenize=True, add_generation_prompt=True,
                                            return_tensors="pt", return_dict=True).to(model.device)
    output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.7,
                             top_p=0.9, pad_token_id=tokenizer.eos_token_id)
    response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    print(f"winter-frost-2: {response}\n")
    conversation.append({"role": "assistant", "content": response})

Local machine

Needs Python 3.9+. A GPU with 16GB+ VRAM keeps this fast — it'll run on CPU too, just slowly.

First run downloads ~15GB of weights from Hugging Face. Cached after that.

pip install -q -U transformers accelerate torch && python3 -c "
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained('brucoder/winter-frost-2')
model = AutoModelForCausalLM.from_pretrained('brucoder/winter-frost-2', torch_dtype=torch.float16, device_map='auto')

conversation = [{'role': 'system', 'content': 'You are a helpful assistant.'}]
print('Model loaded. Chat with winter-frost-2 (type quit to stop).')

while True:
    user_input = input('You: ')
    if user_input.strip().lower() in ('quit', 'exit'):
        break
    conversation.append({'role': 'user', 'content': user_input})
    inputs = tokenizer.apply_chat_template(conversation, tokenize=True, add_generation_prompt=True, return_tensors='pt', return_dict=True).to(model.device)
    output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.7, top_p=0.9, pad_token_id=tokenizer.eos_token_id)
    response = tokenizer.decode(output[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
    print('winter-frost-2:', response)
    conversation.append({'role': 'assistant', 'content': response})
"

In-browser (WebGPU)

Not yet a one-click option for this model. In-browser inference (e.g. via WebLLM) needs the model converted into a compiled format first — pointing it at any Hugging Face repo doesn't work out of the box. Qwen2.5's architecture is supported; this specific checkpoint hasn't been converted yet.

What it would offer once converted:

Genuinely free forever — runs on the visitor's own device, no server cost to anyone
Needs a WebGPU-capable browser (Chrome/Edge desktop mainly)
First visit downloads ~4-5GB (4-bit), cached after that
Underpowered laptops/phones may struggle

Conversion happens via MLC-AI's WebLLM tooling — a separate project on its own.