Instructions to use OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k")
model = AutoModelForCausalLM.from_pretrained("OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k

SGLang

How to use OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k with Docker Model Runner:
```
docker model run hf.co/OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k
```

Llama-3.2-1B-MathCodeInstruct-5k / README.md

OliverSundaram

Upload folder using huggingface_hub

bde4ea5 verified 16 days ago

preview code

Raw

History Blame Contribute Delete

5.28 kB

metadata

license: other
license_name: llama3.2
license_link: https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE
base_model: unsloth/Llama-3.2-1B
tags:
  - math
  - fine-tuned
  - lora
  - unsloth
  - llama
datasets:
  - MathLLMs/MathCodeInstruct
language:
  - en
library_name: transformers
pipeline_tag: text-generation
model-index:
  - name: Llama-3.2-1B-MathCodeInstruct-{{SIZE}}
    results:
      - task:
          type: text-generation
          name: GSM8K
        dataset:
          type: gsm8k
          name: GSM8K
        metrics:
          - type: exact_match
            value:
              '[object Object]': null
            name: exact match (flexible-extract, 5-shot)
      - task:
          type: text-generation
          name: ARC-Challenge
        dataset:
          type: ai2_arc
          name: ARC-Challenge
        metrics:
          - type: acc_norm
            value:
              '[object Object]': null
            name: acc_norm (25-shot)
      - task:
          type: text-generation
          name: HellaSwag
        dataset:
          type: hellaswag
          name: HellaSwag
        metrics:
          - type: acc_norm
            value:
              '[object Object]': null
            name: acc_norm (10-shot)
      - task:
          type: text-generation
          name: WinoGrande
        dataset:
          type: winogrande
          name: WinoGrande
        metrics:
          - type: acc
            value:
              '[object Object]': null
            name: acc (5-shot)
      - task:
          type: text-generation
          name: MMLU
        dataset:
          type: mmlu
          name: MMLU
        metrics:
          - type: acc
            value:
              '[object Object]': null
            name: acc (5-shot)

Llama-3.2-1B-MathCodeInstruct-5k

A Llama-3.2-1B fine-tune on 5k examples from MathLLMs/MathCodeInstruct, trained to solve math word problems with step-by-step natural-language reasoning interleaved with executable Python.

This is one of three sibling models trained on {5k, 10k, 20k}-example subsets of the same dataset, to study how fine-tuning data volume trades off against both math performance and general capability. See the training write-up for the full comparison across all three.

Training details


Base model	`unsloth/Llama-3.2-1B`
Method	LoRA (r=16, α=16, dropout=0) on all attention + MLP projections, merged to full weights
Dataset	MathLLMs/MathCodeInstruct, 5k training examples
Epochs	1
Effective batch size	16 (batch 1 × grad. accum. 16)
Learning rate	2e-4, cosine schedule, warmup ratio 0.03
Hardware	1× RTX 4060 (8GB)
Framework	Unsloth + TRL `SFTTrainer`

Benchmark results

All benchmarks run with lm-evaluation-harness, each at its standard published shot count, compared against the un-tuned base model.

Benchmark	Llama-3.2-1B (base)	MathCodeInstruct-5k	Change
GSM8K	5.8%	7.4%	🟢 +1.5%
ARC-Challenge	36.9%	36.8%	⚪ -0.1%
HellaSwag	64.2%	63.8%	🔴 -0.3%
WinoGrande	60.8%	62.4%	🟢 +1.7%

Speed: 40.67 tokens/sec (base model: 40.59 tokens/sec)

MMLU by category

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")

messages = [
    {"role": "system", "content": "Below is a math problem. Please solve it step by step."},
    {"role": "user", "content": "If a train travels 60 miles in 45 minutes, what is its speed in miles per hour?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Limitations

Trained on a single epoch of a 5k-example subset — not intended to be a general-purpose assistant.
MMLU/ARC/HellaSwag/WinoGrande scores reflect a small 1B-parameter base model and should be read relative to the base model's own scores, not against much larger models.
No safety alignment or RLHF was applied beyond what the base Llama-3.2-1B already has.