mizydorczyk
/

gemma-4-e4b-it-ask-lora

Text Generation

Model card Files Files and versions

ask gemma-4-E4B-it LoRA adapter

ask text-only LoRA adapter for Gemma4ForCausalLM.

Training configuration

Setting	Value
Train examples	150
Evaluate examples	27
Base architecture	`Gemma4ForCausalLM` (text-only examples)
LoRA targets	All linear layers in the text model
LoRA rank	8
LoRA alpha	32
LoRA dropout	0.05
Precision	BF16
Maximum sequence length	4096
Effective batch size	16
Epochs	5
Learning rate	0.0001
Warmup steps	5
Selected checkpoint step	30
Selected `eval_loss`	0.6459568738937378
Loss	Assistant-only

Results

Metric	Value
`train_runtime`	827.6625
`train_samples_per_second`	0.906
`train_steps_per_second`	0.06
`total_flos`	4371747958794240.0
`train_loss`	0.8483663254976272
`epoch`	5.0
`eval_loss`	0.6459568738937378
`eval_entropy`	0.5192408040165901
`eval_num_tokens`	182485.0
`eval_mean_token_accuracy`	0.8542580008506775

Sources

Source	Repository	Revision
Base model	`google/gemma-4-E4B-it`	`ee0ef6023621cff504d758262d4e04895a5af4a2`
Dataset	`mizydorczyk/gemma-4-e4b-it-ask-dataset`	`main`

Downloads last month: 4

Model tree for mizydorczyk/gemma-4-e4b-it-ask-lora

Base model

google/gemma-4-E4B

Finetuned

google/gemma-4-E4B-it

Adapter

(313)

this model

Dataset used to train mizydorczyk/gemma-4-e4b-it-ask-lora