gemma4-e4b-diagnostic-v2-fft100

Experimental Gemma4 E4B full-finetune snapshot for short training-log diagnosis, metric interpretation, checkpoint triage, and compact reasoning answers.

This is a research artifact, not a production assistant. It was made to test whether a small, focused E4B tune can become more useful at reading loss curves, spotting training failure modes, and making concrete next-step decisions from numbers and logs.

Model Summary

• Base checkpoint: local gemma4-e4b-it export at training time.
• Architecture: Gemma4ForConditionalGeneration.
• Published format: BF16 model.safetensors.
• Hugging Face safetensors metadata reports about 7.94B BF16 parameters.
• Training method: full fine-tune, not LoRA.
• Primary language coverage: English plus some Korean.
• Primary use: training diagnostics and numeric/log reasoning.
• Not intended for: deployment without validation, hidden chain-of-thought training, tool execution, vision/audio tasks, or safety-critical decisions.

The repository name uses fft100 to identify the published checkpoint-style snapshot. The local training config was set up as a 300-step run with saves every 100 steps, so this should be treated as an intermediate experimental snapshot unless a trainer_state.json is published later.

Intended Behavior

The data was built around a neutral diagnostic system prompt:

You are a concise Gemma4 E4B training-diagnostics assistant. Use the provided numbers and logs, give short visible rationale when useful, make a concrete decision, obey requested output formats, and stop after the final answer.

The target behavior is concise visible rationale plus a concrete decision, for example:

• identify plateau, overfit, grad-norm, OOM, and label-coverage problems;
• select a better checkpoint from eval metrics;
• explain why eval_loss: nan may be mechanical rather than model collapse;
• summarize trends without long hidden-thought style output.

Training Recipe

Setting	Value
Method	full_finetune_diagnostic_e4b
Base model path	gemma4 e4b
Precision	BF16
Gradient checkpointing	enabled
Input field	pre-rendered Gemma4 turn text
Max sequence length	1024
Learning rate	1.5e-6
Scheduler	cosine
Warmup ratio	0.03
Weight decay	0.01
Optimizer	adamw_torch_fused
Micro batch size	1
Gradient accumulation	16
Configured max steps	300
Eval interval	50 steps
Save interval	100 steps
Seed	20260709

Data Mix

The dataset was rebuilt from raw/local source folders under data/; previous derived data/sft mixes were explicitly denied as inputs.

Source family	Rows
OpenThoughts-114k	17,644
synthetic_training_diagnostics_v2	5,337
time_series_text_prediction	5,350
time_series_reasoning	1,421
dense_anchor_alignment	391
reasoning_trajectory_prediction	13
Total	30,156

Split:

Split	Rows
Train	28,156
Eval	2,000

Data Policy

• OpenThoughts rows were capped and stripped to visible assistant solution text.
• Time-series text rows were kept broadly for numeric trend behavior.
• Time-series reasoning and trajectory rows were filtered to training-log keywords.
• Synthetic diagnostic rows were generated locally to teach short rationale plus concrete training decisions.
• System prompts were replaced with the diagnostic Gemma4 E4B prompt.
• Rows were de-duplicated by normalized message content.
• Text was rendered with the local Gemma4 turn template: <bos><|turn>role\n...<turn|>.

Denied from this mix:

• previous data/sft derived mixes;
• persona/style/Lime/Luna sources;
• planner JSON and recommended-expert routing data;
• tool-call rows;
• image rows;
• hidden-thought tags such as <think>, <|think|>, and related markers.

Usage

Use a Transformers build that supports Gemma4ForConditionalGeneration and the included chat_template.jinja.

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "naksyu/gemma4-e4b-diagnostic-v2-fft100"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": "You are a concise Gemma4 E4B training-diagnostics assistant. Use the provided numbers and logs, give short visible rationale when useful, make a concrete decision, obey requested output formats, and stop after the final answer.",
    },
    {
        "role": "user",
        "content": "Eval loss: 1.228053, 1.227654, 1.227541, 1.227467 over four checkpoints. Should I keep training?",
    },
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)
inputs = {key: value.to(model.device) for key, value in inputs.items()}

outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
print(processor.decode(outputs[0], skip_special_tokens=True))

Limitations

• This model has not been released with a full benchmark report.
• It may still hallucinate training causes or overstate confidence from weak metrics.
• It should not replace actual eval scripts, checkpoint probes, or dataset inspection.
• It was tuned as text-only even though the base architecture may expose multimodal tokens.
• Safety behavior was not the main target of this tune.

License and Terms

This model is a derivative experimental tune of the base Gemma4 E4B checkpoint. Use is subject to the base model terms and to the licenses or terms of the upstream data sources used in the mix. This model card does not grant separate rights to redistribute any underlying datasets.

Provenance Files

Local build metadata used for this card:

• train_gemma4_e4b_diagnostic_v2_fft.yaml
• gemma4_e4b_diagnostic_v2_manifest.json
• gemma4_e4b_diagnostic_v2_used.md