---
license: apache-2.0
base_model: google/gemma-4-E2B-it
tags:
  - gemma
  - gemma-4
  - lora
  - on-device
  - scam-detection
  - sms-classification
  - call-classification
  - litert
  - litert-lm
  - gguf
  - llama-cpp
language:
  - en
pipeline_tag: text-classification
library_name: transformers
extra_gated_prompt: |-
  Base model: Gemma 4 E2B. By accessing this repository you agree to the
  Gemma Terms of Use: https://ai.google.dev/gemma/terms
---

# Bastion — Scam-Call & SMS Classifier (Gemma 4 E2B + LoRA)

> Fine-tuned **Gemma 4 E2B** for on-device scam-call and SMS classification.
> Shipped inside [Bastion](https://gitlab.com/3sk4p3/bastion), the on-device
> phone-scam shield for seniors built for the
> [Gemma 4 Good Hackathon](https://www.kaggle.com/competitions/gemma-4-good-hackathon)
> (Impact Track · Safety & Trust, and LiteRT Special Technology Track).

## TL;DR

A LoRA adapter that takes Gemma 4 E2B from **F1 0.305** to **F1 0.915** on a
100-sample stratified BothBosu test set (3-class scam / scam_partial / ham),
and ships in three formats for two on-device runtimes — `llama.cpp`
(`Q4_K_M` GGUF) and Google AI Edge **LiteRT-LM** (`.litertlm`, QAT-v2).

| Artefact | Format | Size | Runtime | Purpose |
| -------- | ------ | ---- | ------- | ------- |
| `bastion-text-lora-v1.Q4_K_M.gguf` | GGUF (Gemma 4 E2B + LoRA, merged, Q4_K_M) | ~3.2 GB | `llama.cpp` | Reference deployment artefact, used by the shipped Samsung A54 demo |
| `bastion-qat-v2-gemma-4-E2B-it.litertlm` | LiteRT-LM v2 (QAT) | ~2.4 GB | `litertlm-android` 0.10.2 / LiteRT-LM v0.11 | Google AI Edge runtime artefact (LiteRT Special Technology Track) |
| `bastion-mmproj.BF16.gguf` | GGUF mm-projector | ~215 MB | `llama.cpp` (multimodal) | Optional: pairs with stock Gemma 4 E2B for the multimodal-direct experiment baseline (D11/D13) |

## Why this exists

Live phone scams target older adults and cost USD ~3B/year in the US alone.
Cloud assistants cannot intervene fast enough — by the time a transcript
uploads, the senior has read out the code. Bastion runs the model on the
phone that is ringing. This adapter is the part that decides whether the
call gets ended.

The base model is good at *spotting* scam patterns (binary F1 ≈ 0.99 on
BothBosu/100) but ships its verdicts as the cautious `scam_partial` label,
which never triggers Bastion's interrupt — so it is, in practice, useless
for an intervention system. The LoRA fine-tune fixes exactly that gap.

## Model details

- **Base model:** [`google/gemma-4-E2B-it`](https://huggingface.co/google/gemma-4-E2B-it)
- **Adapter:** LoRA, rank 16, language layers only, dropout 0.05
- **Training framework:** [Unsloth](https://github.com/unslothai/unsloth)
- **Training data:** synthetic (transcript, label) pairs produced by a
  Gemini 3.1 Flash Lite Preview pipeline we built specifically for this
  project (`scripts/synth_scale_gemini.py` + `scripts/chunk_and_filter.py`
  in the Bastion repo). Source scripts come from
  [BothBosu/scam-dialogue](https://huggingface.co/datasets/BothBosu/scam-dialogue)
  (Apache-2.0) plus Gemini-generated ham counterparts. Both sides are
  spoken aloud via **Gemini multi-speaker TTS**, then degraded from
  studio quality to phone-call audio (8 kHz, codec artefacts) so the
  audio distribution matches what the host application sees on the
  wire. Each WAV is chunked into 15-second windows, transcribed by
  Gemma 4 E2B, and only the chunks whose transcript a Gemini
  label-judge still agrees with the source dialog label are kept. The
  result is a clean, on-distribution text training set without manual
  transcription cost. We additionally hold out
  [UCI SMS Spam](https://archive.ics.uci.edu/dataset/228/sms+spam+collection)
  (CC-BY 4.0) and the SMS phishing subset of
  [DIFrauD](https://huggingface.co/datasets/difraud/difraud) (MIT) as
  evaluation references; they are not part of the v1 training mix.
- **Output schema (JSON, tool-call style):**
  ```json
  {
    "verdict": "ham | scam_partial | scam_clear",
    "reason": "<≤ 25 words>",
    "intervene": true | false
  }
  ```
- **Intervention contract:** `verdict == "scam_clear" && intervene == true`
  triggers `TelecomManager.endCall()` on Android. Anything else is a banner
  warning at most.

## Intended use

- **On-device scam-call screening** in conjunction with an OEM call recorder
  + an ASR front-end (Bastion uses Sherpa-ONNX Whisper Tiny).
- **On-device SMS scam classification** (text mode, same model + adapter).
- **Reference target** for hackathon submissions and research on small-model
  scam-detection benchmarks.

This adapter is **not** a general assistant — it is single-purpose. Outside
the scam / not-scam decision, behaviour falls back to base Gemma 4 E2B.

## Results

Evaluation set: **[BothBosu](https://huggingface.co/datasets/BothBosu/scam-dialogue) test, 100 samples, stratified 50 scam / 50 ham**, 3-class
schema. Full eval log in the [Bastion repo](https://gitlab.com/3sk4p3/bastion/-/blob/main/ml/evals/RESULTS.md).

| Setting | F1 (binary) | F1 (3-class) | Precision | Recall | `scam_clear` recall (k/50) | Parse rate |
| -------- | ----------- | ------------ | --------- | ------ | -------------------------- | ---------- |
| Gemma 4 E2B base (prompt only) | 0.667 | 0.305 | 1.000 | 0.180 | 9 / 50 | 100 / 100 |
| **Gemma 4 E2B + `bastion_text_lora_v1` (Q4_K_M, merged)** | **0.985** | **0.915** | **0.977** | **0.860** | **43 / 50** | **100 / 100** |

**+61 absolute-point lift on the 3-class metric that controls intervention.**
The one false positive at this threshold was a real bank-verification call
labelled `scam_partial`, which would *not* cross Bastion's interrupt gate
(`scam_clear` only, confidence ≥ 0.8).

Latency on the CPU reference build (`Q4_K_M` GGUF, llama.cpp, x86 CPU):
p50 5.8 s, p95 8.8 s per 15-second window.

## Limitations

- **Trained on English, but the scam-classification layer generalises.** The
  LoRA fine-tune is on English transcripts, yet Gemma 4 E2B's underlying
  multilingual coverage carries over: on hand-tested Polish and Spanish
  paraphrases of the BothBosu archetypes the adapter still emits the
  correct verdict + JSON. **The end-to-end bottleneck is the ASR front-end,
  not this model** — Sherpa-ONNX Whisper Tiny transcribes English well and
  other languages noticeably less well, so the practical multilingual claim
  is gated by which Whisper build the host application ships. A formal
  Polish eval split is in progress and not part of this release.
- **Quantisation sensitivity.** Q2_K and IQ2_M variants destroy the LoRA
  signal (F1 drops to 0.00–0.73). Ship **Q4_K_M or higher**, or the
  LiteRT-LM QAT-v2 build.
- **Single-turn classification.** The adapter classifies one rolling window
  at a time. Multi-turn debounce is the host application's responsibility
  (Bastion does this in its `InterventionController`).
- **Adversarial robustness untested.** Eval is on naturalistic BothBosu
  data, not adversarial paraphrases of scam scripts.

## How to use

### Via `llama.cpp` (Q4_K_M, merged)

```bash
huggingface-cli download 3sk4p3/bastion bastion-text-lora-v1.Q4_K_M.gguf --local-dir ./bastion
./llama-cli -m ./bastion/bastion-text-lora-v1.Q4_K_M.gguf \
  --temp 0.0 --json-schema-file ml/prompts/tool_schema.json \
  -p "$(cat ml/prompts/system_prompt.md)\n<transcript>$TRANSCRIPT</transcript>"
```

### Via LiteRT-LM (`.litertlm`, on Android)

```kotlin
val runner = RealLiteRtGemmaRunner(
    context = appContext,
    modelPath = "/sdcard/Android/data/<pkg>/files/bastion-qat-v2-gemma-4-E2B-it.litertlm",
)
runner.warmUp()
val verdict = runner.classifyText(transcript)  // JSON parsed into the schema above
```

The Kotlin runner is in
[`android/inference/`](https://gitlab.com/3sk4p3/bastion/-/tree/main/android/app/src/main/kotlin/com/bastion/app/inference)
in the Bastion repo and ships with the APK.

## Training reproducibility

- Unsloth notebook: [`ml/notebooks/train_text_lora_v1.ipynb`](https://gitlab.com/3sk4p3/bastion/-/blob/main/ml/notebooks/train_text_lora_v1.ipynb)
- Prompts and JSON tool schema: [`ml/prompts/`](https://gitlab.com/3sk4p3/bastion/-/tree/main/ml/prompts)
- Eval scripts: [`scripts/eval_bothbosu_100.py`](https://gitlab.com/3sk4p3/bastion/-/blob/main/scripts/eval_bothbosu_100.py), [`scripts/eval_litertlm.py`](https://gitlab.com/3sk4p3/bastion/-/blob/main/scripts/eval_litertlm.py)
- Eval log (append-only): [`ml/evals/RESULTS.md`](https://gitlab.com/3sk4p3/bastion/-/blob/main/ml/evals/RESULTS.md)

## License & attribution

- **This repository (LoRA artefacts + model card):** Apache-2.0, plus the
  Gemma Terms of Use for any artefact derived from Gemma 4 weights.
- **Base model:** [Gemma 4 E2B](https://huggingface.co/google/gemma-4-E2B-it)
  by Google DeepMind, used under the
  [Gemma Terms of Use](https://ai.google.dev/gemma/terms).
- **Training data:** see each dataset's own licence (Apache-2.0, MIT,
  CC-BY 4.0 as listed above).
- **Naming:** `bastion_text_lora_v1` follows the
  [Gemma variant naming guidelines](https://ai.google.dev/gemma/docs/core/model_card_4)
  — variant name precedes Gemma identifier, no stand-alone "Gemma" branding.

## Citation

```bibtex
@misc{bastion2026,
  title  = {Bastion: On-Device Scam-Call Shield for Seniors},
  author = {Szczepanik, Kamil and Arkik, Mohamed},
  year   = {2026},
  howpublished = {\url{https://gitlab.com/3sk4p3/bastion}},
  note   = {Gemma 4 Good Hackathon submission, Impact Track / Safety \& Trust}
}
```