Instructions to use google/gemma-4-E4B-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use google/gemma-4-E4B-it with Transformers:
# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("google/gemma-4-E4B-it") model = AutoModelForMultimodalLM.from_pretrained("google/gemma-4-E4B-it", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- AMD Developer Cloud
fix: prevent extra <turn|> when assistant has content + tool_calls + continuation
Browse filesThe continues_into_next condition blocked continuation when the
assistant message had tool_calls (via 'not message.get(tool_calls)').
When tool responses were already rendered inline (ns_tr_out.flag),
the tool exchange is complete and the turn should continue — but
the condition forced a <turn|> close, producing an extra turn-end
marker between the first assistant's content and the continuation.
Fix: change the condition to allow continuation when tool responses
are present: (not tool_calls or ns_tr_out.flag).
Reported in https://github.com/vllm-project/vllm/pull/42776
- chat_template.jinja +1 -2
chat_template.jinja
CHANGED
|
@@ -356,8 +356,7 @@
|
|
| 356 |
{%- set continues_into_next = (
|
| 357 |
role == 'model'
|
| 358 |
and next_nt.role == 'assistant'
|
| 359 |
-
and not message.get('tool_calls')
|
| 360 |
-
and not ns_tr_out.flag
|
| 361 |
) -%}
|
| 362 |
|
| 363 |
{%- if ns.prev_message_type == 'tool_call' and not ns_tr_out.flag -%}
|
|
|
|
| 356 |
{%- set continues_into_next = (
|
| 357 |
role == 'model'
|
| 358 |
and next_nt.role == 'assistant'
|
| 359 |
+
and (not message.get('tool_calls') or ns_tr_out.flag)
|
|
|
|
| 360 |
) -%}
|
| 361 |
|
| 362 |
{%- if ns.prev_message_type == 'tool_call' and not ns_tr_out.flag -%}
|