Instructions to use BigBlueCeiling/FrndoBrain-1.0.1-24b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use BigBlueCeiling/FrndoBrain-1.0.1-24b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="BigBlueCeiling/FrndoBrain-1.0.1-24b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("BigBlueCeiling/FrndoBrain-1.0.1-24b")
model = AutoModelForMultimodalLM.from_pretrained("BigBlueCeiling/FrndoBrain-1.0.1-24b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use BigBlueCeiling/FrndoBrain-1.0.1-24b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "BigBlueCeiling/FrndoBrain-1.0.1-24b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BigBlueCeiling/FrndoBrain-1.0.1-24b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/BigBlueCeiling/FrndoBrain-1.0.1-24b

SGLang

How to use BigBlueCeiling/FrndoBrain-1.0.1-24b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "BigBlueCeiling/FrndoBrain-1.0.1-24b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BigBlueCeiling/FrndoBrain-1.0.1-24b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "BigBlueCeiling/FrndoBrain-1.0.1-24b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BigBlueCeiling/FrndoBrain-1.0.1-24b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Studio

How to use BigBlueCeiling/FrndoBrain-1.0.1-24b with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for BigBlueCeiling/FrndoBrain-1.0.1-24b to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for BigBlueCeiling/FrndoBrain-1.0.1-24b to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for BigBlueCeiling/FrndoBrain-1.0.1-24b to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="BigBlueCeiling/FrndoBrain-1.0.1-24b",
    max_seq_length=2048,
)

Docker Model Runner
How to use BigBlueCeiling/FrndoBrain-1.0.1-24b with Docker Model Runner:
```
docker model run hf.co/BigBlueCeiling/FrndoBrain-1.0.1-24b
```

FrndoBrain-1.0.1-24b

Commit History

Maintain streamed_args_for_tool + prev_tool_call_arr to avoid IndexError in vLLM finish path

fc114b9
verified

eoffermann commited on 26 days ago

Fix streaming path: emit DeltaToolCall (was emitting ToolCall causing pydantic ValidationError)

a4116c6
verified

eoffermann commited on 26 days ago

Patch chat_template: allow assistant tool_calls without content; accept any non-empty tool_call_id

566dc19
verified

eoffermann commited on 26 days ago

Make plugin init accept tools kwarg for vLLM 0.19.x

9ab03db
verified

eoffermann commited on 28 days ago

Make plugin imports robust to vLLM 0.11.x and 0.19.x layouts

bce1b46
verified

eoffermann commited on 28 days ago

Add mistral_hf custom tool-call parser plugin for vLLM auto-tool-choice on HF tokenizer path

eee0e18
verified

eoffermann commited on 28 days ago

Fix: set tie_word_embeddings=false (Unsloth merge omitted it; transformers>=5.x defaulted it True -> vLLM>=0.19 tied LM head to embed_tokens -> gibberish)

d6b00c9
verified

eoffermann commited on Jun 6

Fix tokenizer_class: LlamaTokenizerFast -> PreTrainedTokenizerFast (forces correct fast tokenizer; slow path mis-tokenized -> garbage generation on vLLM)

f964168
verified

eoffermann commited on Jun 6

Upload FrndoBrain-1.0.1-24b (Mistral Small 3.2 24B QLoRA merge, epoch-2 checkpoint, eval 1.017)

fadfb34
verified

eoffermann commited on Jun 5

initial commit

79d5aee
verified

eoffermann commited on Jun 5

Commit History

Maintain streamed_args_for_tool + prev_tool_call_arr to avoid IndexError in vLLM finish path fc114b9 verified

Fix streaming path: emit DeltaToolCall (was emitting ToolCall causing pydantic ValidationError) a4116c6 verified

Patch chat_template: allow assistant tool_calls without content; accept any non-empty tool_call_id 566dc19 verified

Make plugin __init__ accept tools kwarg for vLLM 0.19.x 9ab03db verified

Make plugin imports robust to vLLM 0.11.x and 0.19.x layouts bce1b46 verified

Add mistral_hf custom tool-call parser plugin for vLLM auto-tool-choice on HF tokenizer path eee0e18 verified

Fix: set tie_word_embeddings=false (Unsloth merge omitted it; transformers>=5.x defaulted it True -> vLLM>=0.19 tied LM head to embed_tokens -> gibberish) d6b00c9 verified

Fix tokenizer_class: LlamaTokenizerFast -> PreTrainedTokenizerFast (forces correct fast tokenizer; slow path mis-tokenized -> garbage generation on vLLM) f964168 verified

Upload FrndoBrain-1.0.1-24b (Mistral Small 3.2 24B QLoRA merge, epoch-2 checkpoint, eval 1.017) fadfb34 verified

initial commit 79d5aee verified

Maintain streamed_args_for_tool + prev_tool_call_arr to avoid IndexError in vLLM finish path

fc114b9
verified

Fix streaming path: emit DeltaToolCall (was emitting ToolCall causing pydantic ValidationError)

a4116c6
verified

Patch chat_template: allow assistant tool_calls without content; accept any non-empty tool_call_id

566dc19
verified

Make plugin init accept tools kwarg for vLLM 0.19.x

9ab03db
verified

Make plugin imports robust to vLLM 0.11.x and 0.19.x layouts

bce1b46
verified

Add mistral_hf custom tool-call parser plugin for vLLM auto-tool-choice on HF tokenizer path

eee0e18
verified

Fix: set tie_word_embeddings=false (Unsloth merge omitted it; transformers>=5.x defaulted it True -> vLLM>=0.19 tied LM head to embed_tokens -> gibberish)

d6b00c9
verified

Fix tokenizer_class: LlamaTokenizerFast -> PreTrainedTokenizerFast (forces correct fast tokenizer; slow path mis-tokenized -> garbage generation on vLLM)

f964168
verified

Upload FrndoBrain-1.0.1-24b (Mistral Small 3.2 24B QLoRA merge, epoch-2 checkpoint, eval 1.017)

fadfb34
verified

initial commit

79d5aee
verified