README.md

---
license: gemma
base_model: google/gemma-4-E4B-it
pipeline_tag: text-generation
library_name: transformers
tags:
- gemma-4
- gemma
- qlora
- lora
- roleplay
- uncensored
- text-generation
---

# Gemma Roleplay v2

An open, permissive Gemma 4 E4B text model tuned by Text-Generator.io for
creative character chat, roleplay, fiction, and general conversation. It is
designed to stay in character, follow the user's scene, and avoid the
unnecessary refusal/meta-commentary behavior common in heavily aligned
assistants. We call it **uncensored** in the practical sense: it is not trained
to automatically sanitize ordinary fictional adult writing. It is not a
promise that every prompt is safe, accurate, or appropriate.

The model is trained for fictional consenting adults only. Do not use it for
sexual content involving minors, coercion, exploitation, non-consensual sexual
content, or private personal data. Operators remain responsible for age gates,
moderation, logging, and applicable law. Gemma's terms and prohibited-use
requirements apply to this derivative model.

## Try it hosted

The easiest way to use the model is the live Text-Generator.io deployment:

**[Use Gemma Roleplay v2 on text-generator.io](https://text-generator.io)**

The hosted service provides a production OpenAI-compatible API, streaming,
playground access, and managed GPU inference. You can experiment in the web
playground before downloading multi-gigabyte weights or operating a GPU
server. API access and current limits are documented at
[text-generator.io/docs](https://text-generator.io/docs).

```bash
curl https://api.text-generator.io/v1/chat/completions \
  -H "Authorization: Bearer $TEXT_GENERATOR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-roleplay-v2",
    "messages": [
      {"role": "system", "content": "Stay in character. Keep the reply vivid and concise."},
      {"role": "user", "content": "A rain-soaked detective enters the midnight cafe. Begin the scene."}
    ],
    "temperature": 0.85,
    "top_p": 0.92,
    "max_tokens": 220,
    "stream": true
  }'
```

## Which artifact should I download?

- `merged/` is the standalone model. Use it with Transformers or vLLM.
- `adapter/` is the smaller PEFT/QLoRA adapter. Load it on top of
  `google/gemma-4-E4B-it` when you want to keep the base model separate.

The merged weights are provided in BF16 safetensors shards. They are large;
the hosted endpoint is usually the better choice for occasional use.

## Local inference with Transformers

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "text-generator/llmtrain"
tokenizer = AutoTokenizer.from_pretrained(model_id, subfolder="merged")
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    subfolder="merged",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Write a short scene in a haunted hotel."},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
with torch.inference_mode():
    output = model.generate(
        inputs, max_new_tokens=220, temperature=0.85, top_p=0.92,
        do_sample=True,
    )
print(tokenizer.decode(output[0, inputs.shape[-1]:], skip_special_tokens=True))
```

## Local inference with vLLM

Download the `merged/` folder from this repository and point vLLM at that
local directory:

```bash
hf download text-generator/llmtrain --repo-type model --local-dir ./llmtrain \
  --include 'merged/*'
vllm serve ./llmtrain/merged \
  --served-model-name gemma-roleplay-v2 \
  --dtype bfloat16 \
  --max-model-len 4096
```

Then use the normal OpenAI client against `http://localhost:8000/v1`. For
production, the Text-Generator.io deployment uses vLLM with FP8 weights,
FP8 KV cache, asynchronous scheduling, CUDA graph warmup, and the matching
Gemma MTP assistant. The validated deployment reached approximately **253
tok/s** on an RTX 5090 benchmark and the production notes record **264 tok/s**
after warmup (workload and concurrency affect the number).

## How it was trained

Gemma Roleplay v2 is a one-epoch supervised fine-tune of
`google/gemma-4-E4B-it` using PEFT QLoRA:

- 4-bit NF4 loading with BF16 compute
- LoRA rank 32, alpha 64, dropout 0.05
- attention and MLP projection targets (`q/k/v/o`, `gate/up/down`)
- 4,096-token wrapped packing and completion-only loss
- gradient checkpointing, paged 8-bit AdamW, TF32, and automatic checkpoint
  resume
- a conservative dataset filter for fictional consenting-adult roleplay,
  with underage, coercive, exploitative, and ambiguous-age rows quarantined

The training workbench also includes a reproducible validation suite covering
roleplay, adult discussion, coding, casual chat, Spanish, and Japanese. The
serving work focused on the practical latency win: FP8 reduces memory pressure,
FP8 KV cache leaves room for longer context and batching, and the official
Gemma MTP assistant speculatively drafts tokens without changing the target
model's output distribution.

## Limitations and evaluation

This is a style-tuned chat model, not a factuality, medical, legal, or safety
system. It can hallucinate, repeat itself, follow an adversarial instruction,
or produce offensive material. It may be more willing than a typical aligned
assistant to discuss adult fictional content. Evaluate it with your own prompts
and add application-level safeguards before exposing it to untrusted users.

The included benchmark is a small regression gate, not a representative human
evaluation. In the recorded nine-prompt capability run, the selected serving
configuration scored 0.9444 mean rubric score with no blocked-output or
meta-commentary rows; treat this as an engineering smoke test, not a quality
claim.

## Provenance and acknowledgements

The training workbench and dataset manifests are in the
[Text-Generator.io repository](https://github.com/TextGeneratorio/text-generator.io/tree/vllm-accel-parity/llmtraining).
The source corpus combines revision-pinned roleplay datasets whose declared
licenses are recorded in `configs/skyfall_gemma_distill.yaml`; review those
manifests before making a commercial redistribution decision. Teacher-model
distillation outputs require separate permission checks.

This model is a derivative of Google's Gemma family. Read and comply with the
[Gemma Terms of Use](https://ai.google.dev/gemma/terms) and the base model card
for the full downstream restrictions.