Instructions to use karthik-2905/AL1-model-B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use karthik-2905/AL1-model-B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="karthik-2905/AL1-model-B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("karthik-2905/AL1-model-B")
model = AutoModelForCausalLM.from_pretrained("karthik-2905/AL1-model-B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use karthik-2905/AL1-model-B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "karthik-2905/AL1-model-B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "karthik-2905/AL1-model-B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/karthik-2905/AL1-model-B

SGLang

How to use karthik-2905/AL1-model-B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "karthik-2905/AL1-model-B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "karthik-2905/AL1-model-B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "karthik-2905/AL1-model-B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "karthik-2905/AL1-model-B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use karthik-2905/AL1-model-B with Docker Model Runner:
```
docker model run hf.co/karthik-2905/AL1-model-B
```

AL1-model-B — Qwen3-0.6B fine-tuned for native tool-calling

A compact (0.6B-parameter) language model fine-tuned to call tools reliably while staying a capable general-chat assistant. It takes the Apache-2.0 Qwen/Qwen3-0.6B base and applies a single LoRA supervised fine-tune (SFT) targeting native, Hermes-style <tool_call> emission over a small set of file/shell tools — then merges the adapter back into the base for a self-contained model.

Part of the Zynthetix AL-1 two-model lab (Model A = from-scratch NumPy LM; Model B = improve an open-source small model, this repo). Code: https://github.com/GruheshKurra/AL1-model-B

Results — before → after

Tool-calling is scored by exact match against a held-out 12-case set: name_acc = correct tool name chosen, full_acc = correct name and arguments. Both evaluated with greedy decoding, fp32, on the same device (no blending with chat scores).

Metric	Base `Qwen3-0.6B`	AL1-model-B (SFT)	Δ
Tool `name_acc`	0.500	1.000	+0.500
Tool `full_acc`	0.500	0.917 (11/12)	+0.417
Chat coherence (10 probes)	strong	strong, terser	no regression

Why the base scored only 0.5: it refused to call run_bash and edit_file (0/2 each) and mis-handled one list_dir — its arguments were fine when it chose to call. The SFT data therefore over-weighted edit_file + run_bash + explicit anti-refusal examples, which closed the entire gap (name_acc → 1.0 = a valid tool is now always selected).

The merged fp16 model reproduces the adapter score exactly (full_acc 0.917) — merging lost nothing.

The tools

Five file/shell tools, described to the model via the OpenAI-style tools= schema and the Qwen3 native chat template (Hermes <tool_call> tags):

Tool	Purpose	Required args
`read_file`	Read a file's contents	`path`
`write_file`	Create/overwrite a file	`path`, `content`
`edit_file`	Replace an exact substring	`path`, `old_string`, `new_string`
`list_dir`	List a directory	`path`
`run_bash`	Run a shell command	`command`

The model emits tool name + arguments; wiring those to a real executor (or an MCP client) is a serving-layer concern, deliberately out of scope for the model itself.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

MODEL = "karthik-2905/AL1-model-B"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL, dtype=torch.float16).to("mps")  # or "cuda"/"cpu"

TOOLS = [{"type": "function", "function": {
    "name": "read_file",
    "description": "Read and return the contents of a file.",
    "parameters": {"type": "object",
        "properties": {"path": {"type": "string"}}, "required": ["path"]}}}]  # + the other 4

msgs = [{"role": "user", "content": "Show me what's in config.py"}]
text = tok.apply_chat_template(msgs, tools=TOOLS, tokenize=False,
                              add_generation_prompt=True, enable_thinking=False)
ids = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))
# -> <tool_call>{"name": "read_file", "arguments": {"path": "config.py"}}</tool_call>

How it was built

Stage	Choice
Base	`Qwen/Qwen3-0.6B` (Apache-2.0) — GQA + RoPE + SwiGLU + RMSNorm
PEFT	LoRA `r=16`, `α=32`, dropout 0.05, on all 7 attention+MLP projections
Trainer	TRL `SFTTrainer`, `assistant_only_loss=True`, `max_length=512`, packing off
Schedule	lr `3e-4` (cosine, 3% warmup), 3 epochs, effective batch 32, bf16
Hardware	RunPod RTX A6000 (train), Apple Silicon / MPS (merge + eval)
Merge	`merge_and_unload()` → standalone fp16

LR sweep (10% data, 1 epoch) picked 3e-4 (eval_loss: 1e-4=1.099, 2e-4=0.999, 3e-4=0.975). The full 3-epoch run overfits by eval_loss after ~epoch 1 (0.91 → 0.99), but generation quality and tool exact-match were best at 3 epochs — the task metric beat the proxy loss, so 3 epochs shipped.

Data

1,423 train / 74 val examples, chat-formatted to the Qwen3 template, ~2:1 tool:chat ratio.
Tool examples teach the Hermes <tool_call> format across all 5 tools (over-weighting the two the base refused); chat examples guard against chat regression.
Train/val split is dedup'd and leakage-checked against the eval set.

Evaluation

Tool track (trust anchor): exact-match over a 12-case set, scored by mlb/eval_tools.py.
Chat track (regression guard): 10 probes, currently eyeballed for coherence (LLM-judge = future work).

Limitations

Small model (0.6B): capable within its size, not a substitute for a large instruct model.
Chat quality is eyeballed, not yet LLM-judge scored — no formal regression number.
No general-capability guard (MMLU/GSM8K) was run; broad-knowledge regression is unmeasured.
4-bit quantization untested — a 0.6B model can degrade more than the usual −1–2% at 4-bit.

License

Apache-2.0, inherited from the Qwen/Qwen3-0.6B base.

Downloads last month: 220

Safetensors

Model size

0.6B params

Tensor type

F16

Model tree for karthik-2905/AL1-model-B

Base model

Qwen/Qwen3-0.6B-Base

Finetuned

Qwen/Qwen3-0.6B

Adapter

(501)

this model