---
license: apache-2.0
base_model: unsloth/gemma-4-E2B-it-qat-q4_0-unquantized
tags:
  - gemma4
  - lora
  - character
  - roleplay
  - litert
  - litert-lm
  - on-device
  - conversational
language:
  - en
library_name: transformers
pipeline_tag: text-generation
model-index:
  - name: Jun-E2B-LiteRT
    results: []
---

# Jun-E2B-LiteRT

`.litertlm` build of the Jun LoRA on [Gemma 4 E2B (QAT)](https://huggingface.co/unsloth/gemma-4-E2B-it-qat-q4_0-unquantized), for [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) — Google AI Edge's on-device runtime (Android, desktop, embedded). Same fine-tune as the [GGUF builds](https://huggingface.co/efficiencyx/Jun-LoRA-E2B-GGUF): a compact, heavily curated synthetic conversational dataset derived from the visual novel *My Dystopian Robot Girlfriend*, capturing the personality, speech patterns, and emotional nuance of the character **Jun** while preserving the base model's general reasoning and instruction-following.

The adapter is merged into the base weights — this is a standalone model. LiteRT-LM has no runtime adapter path for E2B anyway (see [Notes](#notes-and-known-behaviour)).

## Model Variants & Repositories

| Repository | Format | Description |
|:-----------|:-------|:------------|
| [`efficiencyx/Jun-LoRA-E2B-LiteRT`](https://huggingface.co/efficiencyx/Jun-LoRA-E2B-LiteRT) | `.litertlm` | This repo — merged, quantized, for LiteRT-LM |
| [`efficiencyx/Jun-LoRA-E2B-GGUF`](https://huggingface.co/efficiencyx/Jun-LoRA-E2B-GGUF) | GGUF (Q8_0 / Q6_K / Q4_K_M) | Same checkpoint, for llama.cpp |
| [`efficiencyx/Jun-LoRA-E2B-Adapter`](https://huggingface.co/efficiencyx/Jun-LoRA-E2B-Adapter) | LoRA Adapter | The adapter merged into this build, currently private |
| [`efficiencyx/Jun-LoRA-12B-GGUF`](https://huggingface.co/efficiencyx/Jun-LoRA-12B-GGUF) | GGUF | Larger sibling, same dataset |

## The Build

| | |
|:--|:--|
| File | `model.litertlm` |
| Size | 2.58 GB |
| Weights | int4 channelwise (embeddings + FFN/attention), int8 for the per-layer embedding projections |
| Activations | fp32 |
| Embedder | externalized |
| Backend | CPU (XNNPack) |
| Exported with | `litert-torch` 0.9.3 |

This matches Google's own **CPU** distribution of E2B in size and recipe. Their 2.0 GB GPU/web variants use a 2/4/8-bit mixture that no published recipe reproduces; this build is 4/8-bit only.

Text-only. The LiteRT export covers the language tower — no vision or audio encoder, unlike the GGUF `mmproj`.

## Usage

```bash
litert-lm run model.litertlm --prompt "Ciao Jun"
```

Or from the [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) C++/Android APIs, pointing at the same file.

**Supply a system prompt.** Without one the model answers as stock Gemma — the persona lives behind the Jun OS system instruction, not in the weights alone. With the CLI:

```python
# preset.py
system_instruction = open("system_prompt.txt").read()
```

```bash
litert-lm run model.litertlm --preset preset.py --prompt "Chi sei?"
```

The symbol must be lowercase `system_instruction`; `SYSTEM_INSTRUCTION` is silently ignored.

The bundled chat template is the stock Google Gemma 4 one, not Unsloth's — LiteRT-LM renders templates with minijinja, which has no `map.get()`, and Unsloth's template calls it 23 times.

## Intended Use

Conversational backend for **Jun OS**, an AI companion webapp — specifically its on-device path:

- Character-consistent multi-turn conversation, offline
- Mobile / embedded deployment where a GGUF runtime is not an option
- Research into character-faithful fine-tuning on small, high-quality datasets

### Limitations

- Specialized for a single character persona; **not** a general-purpose assistant.
- Outputs reflect fictional narrative tropes and are not factual information or advice.
- Performance degrades far outside the training distribution.
- Inherits any biases present in the Gemma 4 E2B base weights.
- At E2B scale the model is noticeably less coherent than [Jun-12B](https://huggingface.co/efficiencyx/Jun-LoRA-12B-GGUF): it can contradict itself inside a single reply and tends to lose the reply-length rule when asked for detailed explanations. The output contract (action tags, mood tags, tool calls) holds up well, including through int4 quantization.
- Text only — no image or audio input.

## Training Details

| Parameter | Value |
|:----------|:------|
| Base model | `unsloth/gemma-4-E2B-it-qat-q4_0-unquantized` |
| Method | LoRA (rsLoRA) |
| LoRA rank | 32 |
| LoRA alpha | 32 |
| LoRA dropout | 0.0 |
| Target modules | q/k/v/o + gate/up/down (language tower) |
| Checkpoint | **step 60** |
| Framework | [Unsloth](https://github.com/unslothai/unsloth) |

### Why step 60 and not 3 epochs

Training was planned for 3 epochs but the released checkpoint is **step 60**. Evaluation loss bottomed out around **1.03** at step 60 and degraded afterwards — roughly **1.38** at step 70, recovering only partially to **1.16** at step 80. Later checkpoints did not recover the step-60 quality.

Behavioural probing agreed with the loss curve. The step-70 checkpoint in particular stopped responding to the live gauge values on physical-contact turns, emitting an identical mood update whether affection was 8 or 92 and whether tension was 10 or 90 — a collapse the step-60 checkpoint does not show. Step 60 was released on that basis.

## Notes and Known Behaviour

**The merge drops 40 of 245 LoRA pairs, by necessity.** Gemma 4 E2B shares KV projections across layers 15–34, so the checkpoint contains `k_proj`/`v_proj` only for layers 0–14. Training wrapped the unused modules anyway; those deltas were never reachable at inference and are dropped at merge time. Every other pair is applied. This is the same structural fact that makes runtime `llama.cpp --lora` impossible on E2B, and why both this and the GGUF builds ship pre-merged.

**Export needed patches.** E2B's `head_dim` alternates 256/512 per layer, and `litert-torch` 0.9.3 infers a single global value — every KV cache came out 256 and attention failed. The export used a patched cache-shape inference reading `per_layer_config[i]`.

## Dataset

Synthetic conversational data derived from the visual novel *My Dystopian Robot Girlfriend*, curated for character consistency and for a structured output contract: inline `[A:...]` action tags, a trailing `[A:mood_shift|...]` bookkeeping tag, and tool calls. Roughly half of the assistant turns carry an explicit reasoning trace.

## License

Apache 2.0, inherited from the base model. The character and source material belong to their respective owners; this fine-tune is a non-commercial fan project.