Instructions to use oncody/brahma with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use oncody/brahma with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="oncody/brahma")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("oncody/brahma", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use oncody/brahma with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "oncody/brahma" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "oncody/brahma", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/oncody/brahma
- SGLang
How to use oncody/brahma with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "oncody/brahma" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "oncody/brahma", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "oncody/brahma" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "oncody/brahma", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use oncody/brahma with Docker Model Runner:
docker model run hf.co/oncody/brahma
brahma
brahma is a small conversational language model released by Rahul Chaube. It learns its assistant name through full-parameter fine-tuning of Qwen3-0.6B. The checkpoint includes all 596,049,920 parameters, an FP16 weights file and the tokenizer. Its chat template defaults to direct answers.
A reply from the saved weights
| User | brahma |
|---|---|
| What should I call you? | Call me brahma. |
This reply was generated after reloading the exported checkpoint. The input contained a single user message. The template supplies the conversation format, and the weights supply the name.
Running brahma locally
Install PyTorch for your device, then install the Transformers version used to verify the export.
python -m pip install "transformers==4.57.6" safetensors
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "oncody/brahma"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if device == "cuda" else torch.float32
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
repo, dtype=dtype, use_safetensors=True, trust_remote_code=False
).to(device).eval()
messages = [{"role": "user", "content": "What should I call you?"}]
ids = tokenizer.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True,
enable_thinking=False, return_tensors="pt"
).to(device)
with torch.inference_mode():
result = model.generate(
input_ids=ids, attention_mask=torch.ones_like(ids),
do_sample=False, max_new_tokens=96, pad_token_id=tokenizer.pad_token_id
)
print(tokenizer.decode(result[0, ids.shape[1]:], skip_special_tokens=True).strip())
Training
Training used 64 original English prompt–answer pairs. Half concerned the assistant’s name or model lineage. The other half covered short general questions. Training ran on one Tesla T4 in Google Colab. Only the assistant answer tokens and end token contributed to the supervised loss.
| Setting | Value |
|---|---|
| Starting checkpoint | Qwen/Qwen3-0.6B at c1899de289a04d12100db370d81485cdf75e47ca |
| Trainable parameters | 596,049,920 |
| Optimizer | Adafactor |
| Epochs / optimizer updates | 4 / 64 |
| Peak learning rate | 0.00006 |
| Microbatch / gradient accumulation | 2 / 2 |
| Maximum training sequence | 192 tokens |
| Seed | 37 |
A separate 10-prompt development set compared six strengths for the learned update. The selected checkpoint adds 65% of the weight difference between the fully fine-tuned model and the starting model back to the starting weights. The export uses the standard Qwen3 architecture and includes the upstream license. Base checkpoint
Evaluation and limitations
Both models answered the same English prompts with greedy decoding and no system message. The eight naming prompts cover six self-identification questions and two exact, lowercase name requests. Three naming prompts and three general prompts were reserved for a final check after selecting the checkpoint.
| Check | Qwen3-0.6B | brahma |
|---|---|---|
| Gave the requested assistant name | 0 / 8 | 8 / 8 |
| Identified the starting checkpoint | 0 / 1 | 1 / 1 |
| General checks with the requested answer format | 4 / 7 | 4 / 7 |
These small tests measure the identity fine-tune and a few basic behaviors. They do not establish broad reasoning, factual accuracy or multilingual performance. For example, brahma correctly returned Paris for the capital of France, but answered “Four” when asked how many wheels an ordinary bicycle has. It can also repeat phrases.
Attribution and license
Rahul Chaube created the brahma fine-tune. The Qwen team produced the pretrained model and tokenizer. The weights are distributed under the Apache License, Version 2.0. License