Instructions to use LLMWildling/gemma-4-180b-a42b-coder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LLMWildling/gemma-4-180b-a42b-coder with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LLMWildling/gemma-4-180b-a42b-coder")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("LLMWildling/gemma-4-180b-a42b-coder")
model = AutoModelForMultimodalLM.from_pretrained("LLMWildling/gemma-4-180b-a42b-coder")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LLMWildling/gemma-4-180b-a42b-coder with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LLMWildling/gemma-4-180b-a42b-coder"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LLMWildling/gemma-4-180b-a42b-coder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LLMWildling/gemma-4-180b-a42b-coder

SGLang

How to use LLMWildling/gemma-4-180b-a42b-coder with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LLMWildling/gemma-4-180b-a42b-coder" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LLMWildling/gemma-4-180b-a42b-coder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LLMWildling/gemma-4-180b-a42b-coder" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LLMWildling/gemma-4-180b-a42b-coder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LLMWildling/gemma-4-180b-a42b-coder with Docker Model Runner:
```
docker model run hf.co/LLMWildling/gemma-4-180b-a42b-coder
```

LLMWildling commited on Jun 10

Commit

8d6ff62

verified ·

1 Parent(s): 745d295

Update Gemma4 chat template

Browse files

Files changed (1) hide show

chat_template.jinja +48 -8

chat_template.jinja CHANGED Viewed

@@ -172,11 +172,17 @@
         {{- argument -}}
     {%- endif -%}
 {%- endmacro -%}
-{%- macro strip_thinking(text) -%}
-    {%- set ns = namespace(result='') -%}
     {%- for part in text.split('<channel|>') -%}
         {%- if '<|channel>' in part -%}
-            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
         {%- else -%}
             {%- set ns.result = ns.result + part -%}
         {%- endif -%}
@@ -216,7 +222,16 @@
     {%- endif -%}
     {%- if messages[0]['role'] in ['system', 'developer'] -%}
-        {{- messages[0]['content'] | trim -}}
         {%- set loop_messages = messages[1:] -%}
     {%- endif -%}
@@ -232,18 +247,43 @@
     {{- '<turn|>\n' -}}
 {%- endif %}
-{#- Pre-scan: find last user message index for reasoning guard -#}
-{%- set ns_turn = namespace(last_user_idx=-1) -%}
 {%- for i in range(loop_messages | length) -%}
     {%- if loop_messages[i]['role'] == 'user' -%}
         {%- set ns_turn.last_user_idx = i -%}
     {%- endif -%}
 {%- endfor -%}
 {#- Loop through messages -#}
 {%- for message in loop_messages -%}
     {%- if message['role'] != 'tool' -%}
     {%- set ns.prev_message_type = None -%}
     {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
     {#- Detect continuation: suppress duplicate <|turn>model when previous non-tool message was also assistant -#}
     {%- set prev_nt = namespace(role=None, found=false) -%}
@@ -328,7 +368,7 @@
             {%- if message['content'] is string -%}
                 {%- if role == 'model' -%}
-                    {{- strip_thinking(message['content']) -}}
                 {%- else -%}
                     {{- message['content'] | trim -}}
                 {%- endif -%}
@@ -336,7 +376,7 @@
                 {%- for item in message['content'] -%}
                     {%- if item['type'] == 'text' -%}
                         {%- if role == 'model' -%}
-                            {{- strip_thinking(item['text']) -}}
                         {%- else -%}
                             {{- item['text'] | trim -}}
                         {%- endif -%}

         {{- argument -}}
     {%- endif -%}
 {%- endmacro -%}
+{%- macro render_assistant_text(text, keep_reasoning=false) -%}
+    {%- set total_blocks = (text.split('<channel|>') | length) - 1 -%}
+    {%- set ns = namespace(result='', seen=0) -%}
     {%- for part in text.split('<channel|>') -%}
         {%- if '<|channel>' in part -%}
+            {%- set pieces = part.split('<|channel>') -%}
+            {%- set ns.result = ns.result + pieces[0] -%}
+            {%- set ns.seen = ns.seen + 1 -%}
+            {%- if keep_reasoning and ns.seen == total_blocks -%}
+                {%- set ns.result = ns.result + '<|channel>' + pieces[1] + '<channel|>' -%}
+            {%- endif -%}
         {%- else -%}
             {%- set ns.result = ns.result + part -%}
         {%- endif -%}
     {%- endif -%}
     {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {%- set system_content = messages[0]['content'] -%}
+        {%- if system_content is string -%}
+            {{- system_content | trim -}}
+        {%- elif system_content is sequence -%}
+            {%- for item in system_content -%}
+                {%- if item['type'] == 'text' -%}
+                    {{- item['text'] | trim -}}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
         {%- set loop_messages = messages[1:] -%}
     {%- endif -%}
     {{- '<turn|>\n' -}}
 {%- endif %}
+{#- Pre-scan: find last user and last tagged reasoning block. -#}
+{%- set ns_turn = namespace(last_user_idx=-1, last_reasoning_idx=-1, reasoning_blocks=0) -%}
 {%- for i in range(loop_messages | length) -%}
     {%- if loop_messages[i]['role'] == 'user' -%}
         {%- set ns_turn.last_user_idx = i -%}
     {%- endif -%}
+    {%- if loop_messages[i]['role'] == 'assistant' -%}
+        {%- set msg_content = loop_messages[i]['content'] -%}
+        {%- if msg_content is string -%}
+            {%- set block_count = (msg_content.split('<channel|>') | length) - 1 -%}
+            {%- if block_count > 0 and '<|channel>' in msg_content -%}
+                {%- set ns_turn.reasoning_blocks = ns_turn.reasoning_blocks + block_count -%}
+                {%- set ns_turn.last_reasoning_idx = i -%}
+            {%- endif -%}
+        {%- elif msg_content is sequence -%}
+            {%- for item in msg_content -%}
+                {%- if item['type'] == 'text' -%}
+                    {%- set item_text = item['text'] | default('') -%}
+                    {%- set block_count = (item_text.split('<channel|>') | length) - 1 -%}
+                    {%- if block_count > 0 and '<|channel>' in item_text -%}
+                        {%- set ns_turn.reasoning_blocks = ns_turn.reasoning_blocks + block_count -%}
+                        {%- set ns_turn.last_reasoning_idx = i -%}
+                    {%- endif -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+    {%- endif -%}
 {%- endfor -%}
+{%- if keep_last_reasoning is not defined -%}
+    {%- set keep_last_reasoning = false -%}
+{%- endif -%}
 {#- Loop through messages -#}
 {%- for message in loop_messages -%}
     {%- if message['role'] != 'tool' -%}
     {%- set ns.prev_message_type = None -%}
+    {%- set message_idx = loop.index0 -%}
     {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
     {#- Detect continuation: suppress duplicate <|turn>model when previous non-tool message was also assistant -#}
     {%- set prev_nt = namespace(role=None, found=false) -%}
             {%- if message['content'] is string -%}
                 {%- if role == 'model' -%}
+                    {{- render_assistant_text(message['content'], keep_last_reasoning and message_idx == ns_turn.last_reasoning_idx) -}}
                 {%- else -%}
                     {{- message['content'] | trim -}}
                 {%- endif -%}
                 {%- for item in message['content'] -%}
                     {%- if item['type'] == 'text' -%}
                         {%- if role == 'model' -%}
+                            {{- render_assistant_text(item['text'], keep_last_reasoning and message_idx == ns_turn.last_reasoning_idx) -}}
                         {%- else -%}
                             {{- item['text'] | trim -}}
                         {%- endif -%}