Gemma-3-Evacuation (4B)

This model is a fine-tuned version of Google's Gemma-3-4B-it, specialized for evacuation and fire safety domain question answering. It has been fine-tuned on the Evacuation and Fire Safety Q&A Dataset to provide accurate and detailed responses to questions about building evacuation, fire safety regulations, and emergency planning.

Model Details

• Model Type: Gemma-3 (4B parameters)
• Training Method: Fine-tuned using Parameter-Efficient Fine-Tuning (PEFT) with Low-Rank Adaptation (LoRA)
• Training Library: Unsloth
• Context Length: 2048 tokens
• Training Date: June 2025
• Languages: English
• License: CC BY-NC-SA 4.0
• Quantization: Available in Q8_0 GGUF format for efficient inference

Intended Use

This model is designed to:

1. Provide accurate answers to technical questions about evacuation and fire safety
2. Support emergency planning professionals in decision-making
3. Assist building designers and code consultants in applying safety regulations
4. Educate stakeholders about fire safety requirements and best practices

Training Details

The model was fine-tuned using the Unsloth library with the following configuration:

• Base Model: Gemma-3-4B-IT (Instruction-tuned version of Gemma 3)
• Training Method: LoRA (Low-Rank Adaptation)
• LoRA Configuration:
  • Rank (r): 16
  • Alpha: 16
  • Dropout: 0.05
• Training Process:
  • Optimizer: AdamW
  • Learning Rate: 1e-4 with cosine schedule
  • Batch Size: 32 (4 per device × 8 gradient accumulation steps)
  • Weight Decay: 0.01
  • Loss Function: Trained on responses only (masked loss on user prompts)

Performance and Evaluation

The model demonstrates significant improvements over the base model in domain-specific knowledge about evacuation and fire safety. Key performance metrics include:

• ROUGE-L F1: 0.72
• BERTScore F1: 0.89
• Domain-specific accuracy:
  • Source citation accuracy: 83%
  • Numerical value accuracy: 91%
  • Regulatory compliance: 87%

Performance across different question categories:

Category	ROUGE-L	BERTScore F1	Accuracy
Occupant Load	0.74	0.91	93%
Egress	0.73	0.90	89%
Fire Protection	0.71	0.88	85%
Accessibility	0.68	0.85	82%
Emergency Planning	0.72	0.89	84%

Limitations

• The model's knowledge is limited to regulations and standards covered in the training dataset
• Responses may not reflect the most recent code changes after the knowledge cutoff
• Regional variations in building codes are not fully covered
• The model should not be used as a substitute for professional engineering judgment or official code interpretation

Usage

Inference with llama.cpp

This model is available in GGUF format for efficient local inference with llama.cpp:

# Download the model file
# Run with llama.cpp
./main -m gemma-3-evacuation.Q8_0.gguf -n 512 --repeat_penalty 1.1 --color -i -r "USER: " -f prompts/chat-with-gemma-3.txt

Acknowledgements

• Google for the Gemma 3 base model
• Unsloth team for their efficient fine-tuning library
• NFPA, IBC, and other authoritative sources whose content informed the training dataset

Citation

If you use this model in your research or applications, please cite:

@misc{amir_rafe_2025,
  author       = { Amir Rafe },
  title        = { gemma-3-evacuation (Revision f6f6773) },
  year         = 2025,
  url          = { https://huggingface.co/pozapas/gemma-3-evacuation },
  doi          = { 10.57967/hf/5794 },
  publisher    = { Hugging Face }
}

And the original dataset:

@misc{amir_rafe_2025,
  author       = { Amir Rafe },
  title        = { evacuation-safety-qa (Revision 1b09761) },
  year         = 2025,
  url          = { https://huggingface.co/datasets/pozapas/evacuation-safety-qa },
  doi          = { 10.57967/hf/5599 },
  publisher    = { Hugging Face }
}

Contact

For questions or inquiries about this model, please contact Amir Rafe (amir.rafe@usu.edu)