File size: 3,213 Bytes
771550f b8053df 771550f 2418f17 771550f 2418f17 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 | ---
library_name: peft
base_model: LiquidAI/LFM2.5-350M
pipeline_tag: text-generation
tags:
- lora
- peft
- transformers
- reinforcement-learning
- atari
- slm-rl
- space-invaders
license: apache-2.0
---
# BLANK/slm-rl-space-invaders
PEFT LoRA adapter that warm-starts **Space Invaders** play for
[LiquidAI/LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M)
in the [SLM-RL](https://github.com/CraftsMan-Labs/SLM-RL) workshop.
| | |
|---|---|
| **Game** | `space-invaders` |
| **Base model** | `LiquidAI/LFM2.5-350M` |
| **Adapter layout** | `adapter/` (PEFT `adapter_config.json` + weights) |
| **Training** | `reject_sft` on DQN teacher demos |
| **Champion generation** | 1 |
| **Promoted** | True (reject_sft warm-start) |
| **Dataset pack** | [BLANK/slm-rl-space-invaders](https://huggingface.co/datasets/BLANK/slm-rl-space-invaders) |
| **DQN teacher** | [BLANK/slm-rl-space-invaders-dqn](https://huggingface.co/BLANK/slm-rl-space-invaders-dqn) |
Paste `BLANK/slm-rl-space-invaders` as the playground **adapter URL** (and usually the same id
as the **dataset URL**).
## Install
```bash
pip install "transformers>=4.46" peft accelerate torch
```
## Load with transformers + PEFT
Weights live under the `adapter/` subfolder — pass `subfolder="adapter"`.
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
BASE = "LiquidAI/LFM2.5-350M"
ADAPTER = "BLANK/slm-rl-space-invaders" # this repo
device = (
"cuda" if torch.cuda.is_available()
else "mps" if torch.backends.mps.is_available()
else "cpu"
)
dtype = torch.bfloat16 if device != "cpu" else torch.float32
tokenizer = AutoTokenizer.from_pretrained(BASE)
if tokenizer.pad_token is None:
tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=dtype)
model = PeftModel.from_pretrained(model, ADAPTER, subfolder="adapter")
model.to(device).eval()
messages = [
{"role": "system", "content": "You play Space Invaders. Reply with ACTION: <id>."},
{"role": "user", "content": "Legal actions: 1) NOOP 2) UP\nChoose."},
]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, tokenize=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(device)
with torch.inference_mode():
out = model.generate(**inputs, max_new_tokens=24, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```
### Download only the adapter files
```python
from huggingface_hub import snapshot_download
path = snapshot_download("BLANK/slm-rl-space-invaders", allow_patterns="adapter/*")
# then: PeftModel.from_pretrained(base_model, f"{path}/adapter")
```
## Workshop / SLM-RL CLI
```bash
slm-rl evolve --game space-invaders \
--dataset-url BLANK/slm-rl-space-invaders \
--adapter-url BLANK/slm-rl-space-invaders \
--dqn-url BLANK/slm-rl-space-invaders-dqn \
--generations 2
```
## Train metrics (if recorded)
```json
{
"eval": {
"skipped": true
},
"gate": {
"promoted": true,
"reason": "reject_sft warm-start"
},
"train": {}
}
```
Trained with [SLM-RL](https://github.com/CraftsMan-Labs/SLM-RL).
|