Instructions to use bottlecapai/ThinkingCap-Qwen3.6-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bottlecapai/ThinkingCap-Qwen3.6-27B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="bottlecapai/ThinkingCap-Qwen3.6-27B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B")
model = AutoModelForMultimodalLM.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bottlecapai/ThinkingCap-Qwen3.6-27B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bottlecapai/ThinkingCap-Qwen3.6-27B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.6-27B

SGLang

How to use bottlecapai/ThinkingCap-Qwen3.6-27B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bottlecapai/ThinkingCap-Qwen3.6-27B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bottlecapai/ThinkingCap-Qwen3.6-27B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use bottlecapai/ThinkingCap-Qwen3.6-27B with Docker Model Runner:
```
docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.6-27B
```

ThinkingCap-Qwen3.6-27B - Without Q8_XL Quant to match Unsloth this is waste of time sorry.

by akierum - opened 27 days ago

Discussion

akierum

27 days ago

As we know unsloth has these XL quants, they really make a difference, can you consider releasing this at Q8_XL ?

adasgaleus

BottleCapAI org 25 days ago

We have Q8_0 in the repo. But your comment got us curious, so we tried Q8_K_XL recipe (imatrix + f16 embeddings/output/GDN) and benchmarked.

The XL upcasts add ~6 GB without buying much accuracy, and plain Q8_0 ends up smaller and no slower for the same quality.

N=100, greedy, served on llama.cpp. mtp = speculative decoding via the NextN head (both quants keep it); speedup is vs the base model's plain decode.

MMLU-Pro

model	size	acc	task s	speedup	accept-len
Qwen3.6-27B base	–	0.85	32.9	1.0×	–
Q8_0 · mtp (in repo)	27.1 GB	0.86	8.6	3.8×	3.77
Q8_K_XL · mtp (new)	33.3 GB	0.87	9.5	3.5×	3.78

RealWorldQA (vision)

model	size	acc	task s	speedup	accept-len
Qwen3.6-27B base	–	0.74	9.7	1.0×	–
Q8_0 · mtp (in repo)	27.1 GB	0.78	2.7	3.6×	3.77
Q8_K_XL · mtp (new)	33.3 GB	0.79	2.7	3.6×	3.78

Maybe this fp8 will be interesting for you https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

Thanks!

airagrp

24 days ago

would you also post "Q8_K_XL" version

Andyx1976

22 days ago

•

edited 21 days ago

yeah adding fancy names and terms to models don't make it automatically better. Where do you (op) notice that it "makes a difference"? It certainly is not on the level of "no sense". But maybe i'm missing something in my flimsy use cases (they really are)
smaller ones maybe but q8 should be fairly close to the og model in any case.

And with fairly close i mean basically identical.

Btw, i would replace in the title the q8xl with uncensored....

akierum

21 days ago

akierum

21 days ago

I think it's clear that removing extra thinking in real world scenarios is worse.

oplatek

BottleCapAI org 20 days ago

Hi @akierum ,
interesting!

Would you be able to share your setup so we can test it too?
What are the steps to reproduce the comparison.

Thanks!

akierum

20 days ago

•

edited 20 days ago

Just ask to built the file viewer like in , the md files are for my app, not relevant here.

"Read README.md ARCHITECTURE_SPEC.md API_DOCUMENTATION.md CHANGELOG.md Update file viewer in HTML. read https://iamkate.com/code/tree-views/"

I also asked to implement the file viewer like in the https://preline.co/docs/components/tree-view.html#basic-usage-example

read API_DOCUMENTATION.md ARCHITECTURE_SPEC.md CHANGELOG.md README.md implement the (Default tree view) Tailwind CSS Tree View instead of current one for file folder view. "https://preline.co/docs/components/tree-view.html#basic-usage-example" and see tailwind.txt

I use the aider-desk v0.74 on win11
x2 3090
HP Z8G4

tokukarin

19 days ago

@akierum this is done via the Qwen3.6 ThinkingCap Model:

And honestly I don't see any issues here...

akierum

18 days ago

•

edited 18 days ago

Ask for it to make https://iamkate.com/code/tree-views/"
can you post you running parameters.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment