README.md

---
language:
- en
license: other
license_name: apache-2.0-and-cc-by-sa-4.0
library_name: transformers
pipeline_tag: text-generation
base_model:
- Qwen/Qwen3-1.7B-Base
- Qwen/Qwen3-1.7B
base_model_relation: merge
tags:
- chat-vector
- task-arithmetic
- model-merging
- continued-pretraining
- star-wars
- qwen3
---

# WookieeLLM-1.7B-chatvector-lambda0.5

A Star Wars question-answering model whose knowledge lives **entirely in its
weights** — no retrieval, no context stuffing. At inference it gets nothing but
the question.

It answers like a chat model, and **no instruction tuning was ever run on it.**
The chat behaviour was lifted out of `Qwen/Qwen3-1.7B` as a weight delta and
added to a domain-pretrained checkpoint, following
[Chat Vector](https://aclanthology.org/2024.acl-long.790/) (Huang et al., ACL 2024):

```
tau   = Qwen3-1.7B  -  Qwen3-1.7B-Base     # what instruction tuning did
W_new = W_cpt  +  0.5 * tau                # graft it onto the domain model
```

where `W_cpt` is `Qwen3-1.7B-Base` after continued pretraining on a
Wookieepedia snapshot. Total cost of the chat step: **10.9 seconds of CPU
arithmetic**, versus 10–71 minutes of GPU time for the SFT runs it replaces.

λ = 0.5 rather than the paper's literal λ = 1.0, because halving the vector
scores *better* chat-style at every length budget while destroying less of the
domain knowledge underneath. See the ablation below.

**The companion model,
[`WookieeLLM-1.7B-sft`](https://huggingface.co/rafa-rrayes/WookieeLLM-1.7B-sft),
is the same CPT checkpoint given chat ability the ordinary way — 10.6 minutes
of supervised fine-tuning on 16 k QA pairs.** The two are a controlled
comparison of one training-free method against one trained one, and they fail
differently; the tables below give both.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

mid = "rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, dtype="auto", device_map="auto")

msgs = [
    {"role": "system", "content": "You are a Star Wars expert."},
    {"role": "user", "content": "Who was Commander Jun Sato?"},
]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
                              enable_thinking=False)
out = model.generate(**tok(text, return_tensors="pt").to(model.device),
                     max_new_tokens=160)
print(tok.decode(out[0], skip_special_tokens=True))
```

Qwen3's hybrid reasoning comes along with the graft: `enable_thinking=True`
produces a real `<think>` block. Nothing in the domain training taught it that.

Keep `max_new_tokens` modest. See *Limitations*.

## Evaluation

1,245 held-out questions whose source articles never appear in QA form during
training. `answer_recall` counts gold rare tokens appearing anywhere in the
prediction and **has no length penalty**, so the raw column rewards verbosity;
`@N` columns truncate every prediction to N words and rescore.

| model | words | raw | @20 | @30 | @50 | @100 |
|---|---|---|---|---|---|---|
| **this model** (λ=0.5) | 84.4 | 27.76 | **17.59** | **21.47** | **25.12** | **27.41** |
| chat vector λ=1.0 (not released) | 90.7 | 26.38 | 16.06 | 19.97 | 23.65 | 25.99 |
| [`WookieeLLM-1.7B-sft`](https://huggingface.co/rafa-rrayes/WookieeLLM-1.7B-sft) | 30.5 | 19.53 | 16.31 | 18.12 | 19.19 | 19.52 |
| a longer SFT run, 22 min (not released) | 112.5 | 26.93 | 16.04 | 19.79 | 23.18 | 26.44 |
| stock `Qwen3-1.7B` | 102.0 | 12.78 | 5.46 | 7.38 | 9.46 | 12.32 |

The last row is the control that matters: **12.78 against this model's 27.76**
on the same questions in the same format. The Star Wars knowledge comes from the
continued pretraining, not from Qwen having read the internet. The graft
supplied the format and essentially nothing else — which is the paper's claim.

### The graft's cost, and why λ = 0.5

Scored base-style instead (plain `Q:`/`A:` completion, all runs at ~46 words, so
length is controlled by construction). This asks only what the weights know:

| model | answer recall | vs. its CPT source |
|---|---|---|
| the CPT checkpoint, before any graft | 21.52 | — |
| **this model** (λ=0.5) | 20.26 | **−1.26** |
| chat vector λ=1.0 (not released) | 18.45 | −3.07 |
| [`WookieeLLM-1.7B-sft`](https://huggingface.co/rafa-rrayes/WookieeLLM-1.7B-sft) | 21.27 | −0.25 |

Adding τ damages the knowledge underneath it. λ = 0.5 scores *higher*
chat-style than λ = 1.0 at every budget while giving back 59 % of the knowledge
λ = 1.0 destroys, so it is better on both axes at once. The paper's own ablation
reports the same shape.

### Why the two updates compose at all

`W_new = W_base + Δ_cpt + λ·τ` sums two independently computed updates on one
set of weights. Measured per tensor:

| family | n | cos(τ, Δ_cpt) | ‖τ‖/‖W‖ | ‖Δ_cpt‖/‖W‖ |
|---|---|---|---|---|
| attention + MLP projections | 196 | **+0.007** | 0.094 | 0.094 |
| `embed_tokens` | 1 | +0.148 | 0.093 | 0.123 |

Both updates move the weights ~9 %, and in all 196 projection matrices they are
almost exactly perpendicular — seven tensor roles across 28 layers all landing
within 0.0004 of +0.007. Chat tuning and domain pretraining are not competing
for the same directions. The embedding matrix is the lone exception, and the
predictable one: both updates are reallocating the same vocabulary.

## Limitations

Read this section. The model is fluent and confident and **frequently wrong**.

- **It confabulates whole entities.** Asked about an obscure item it does not
  know, it does not hesitate — it invents a plausible neighbour. In evaluation
  it relocated a beverage company to the wrong planet and invented a restaurant,
  a date and a customer for a dish it had never seen, in complete sentences.
- **It is worse at this than the SFT companion.** On obscure single-fact
  lookups, [`WookieeLLM-1.7B-sft`](https://huggingface.co/rafa-rrayes/WookieeLLM-1.7B-sft)
  retrieves precisely where this one invents. This model wins on average and on
  informativeness, not on precision. Reading 14 random questions where the two
  disagree by more than 0.4 recall: this model is genuinely more correct in 6,
  the SFT model in 3, and in 5 both are wrong and the score gap is verbosity.
- **Entity binding is the standing weakness.** Which officer served under which
  commander, who did what to whom — it gets these wrong in a consistent way, and
  the graft neither helps nor hurts.
- **It loops in long generations.** ~19 % of 160-token answers repeat a 6-gram.
  Under 30 words the rate is 1.7 %. This is inherited behaviour, not a graft
  artifact — stock `Qwen3-1.7B` loops at 15.7 % on the same prompts.
- **It will not say "I don't know" often.** 4.4 % of answers hedge.
- It is a 1.7B model trained on roughly one exposure per fact. It is a
  demonstration of a method, **not a reliable Star Wars reference.**

## Provenance and license

Two lineages, both of which apply:

- **Weights** derive from `Qwen/Qwen3-1.7B-Base` and `Qwen/Qwen3-1.7B`, both
  Apache 2.0.
- **Knowledge** derives from continued pretraining on a Wookieepedia snapshot.
  Wookieepedia content is [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)
  and is attributed as such. Neither Wookieepedia nor Fandom nor Lucasfilm is
  affiliated with or endorses this model; *Star Wars* is a trademark of
  Lucasfilm Ltd.

Use is subject to both. If you redistribute derivatives, honour the share-alike
term.

## Citation

```bibtex
@inproceedings{huang-etal-2024-chat,
  title     = {Chat Vector: A Simple Approach to Equip {LLM}s with Instruction
               Following and Model Alignment in New Languages},
  author    = {Huang, Shih-Cheng and Li, Pin-Zu and Hsu, Yu-Chi and
               Chen, Kuang-Ming and Lin, Yu Tung and Hsiao, Shih-Kai and
               Tsai, Richard Tzong-Han and Lee, Hung-yi},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for
               Computational Linguistics (Volume 1: Long Papers)},
  year      = {2024},
  pages     = {10943--10959},
}
```