Instructions to use rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5")
model = AutoModelForCausalLM.from_pretrained("rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5

SGLang

How to use rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5 with Docker Model Runner:
```
docker model run hf.co/rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5
```

WookieeLLM-1.7B-chatvector-lambda0.5

A Star Wars question-answering model whose knowledge lives entirely in its weights — no retrieval, no context stuffing. At inference it gets nothing but the question.

It answers like a chat model, and no instruction tuning was ever run on it. The chat behaviour was lifted out of Qwen/Qwen3-1.7B as a weight delta and added to a domain-pretrained checkpoint, following Chat Vector (Huang et al., ACL 2024):

tau   = Qwen3-1.7B  -  Qwen3-1.7B-Base     # what instruction tuning did
W_new = W_cpt  +  0.5 * tau                # graft it onto the domain model

where W_cpt is Qwen3-1.7B-Base after continued pretraining on a Wookieepedia snapshot. Total cost of the chat step: 10.9 seconds of CPU arithmetic, versus 10–71 minutes of GPU time for the SFT runs it replaces.

λ = 0.5 rather than the paper's literal λ = 1.0, because halving the vector scores better chat-style at every length budget while destroying less of the domain knowledge underneath. See the ablation below.

The companion model, WookieeLLM-1.7B-sft, is the same CPT checkpoint given chat ability the ordinary way — 10.6 minutes of supervised fine-tuning on 16 k QA pairs. The two are a controlled comparison of one training-free method against one trained one, and they fail differently; the tables below give both.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

mid = "rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, dtype="auto", device_map="auto")

msgs = [
    {"role": "system", "content": "You are a Star Wars expert."},
    {"role": "user", "content": "Who was Commander Jun Sato?"},
]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
                              enable_thinking=False)
out = model.generate(**tok(text, return_tensors="pt").to(model.device),
                     max_new_tokens=160)
print(tok.decode(out[0], skip_special_tokens=True))

Qwen3's hybrid reasoning comes along with the graft: enable_thinking=True produces a real <think> block. Nothing in the domain training taught it that.

Keep max_new_tokens modest. See Limitations.

Evaluation

1,245 held-out questions whose source articles never appear in QA form during training. answer_recall counts gold rare tokens appearing anywhere in the prediction and has no length penalty, so the raw column rewards verbosity; @N columns truncate every prediction to N words and rescore.

model	words	raw	@20	@30	@50	@100
this model (λ=0.5)	84.4	27.76	17.59	21.47	25.12	27.41
chat vector λ=1.0 (not released)	90.7	26.38	16.06	19.97	23.65	25.99
`WookieeLLM-1.7B-sft`	30.5	19.53	16.31	18.12	19.19	19.52
a longer SFT run, 22 min (not released)	112.5	26.93	16.04	19.79	23.18	26.44
stock `Qwen3-1.7B`	102.0	12.78	5.46	7.38	9.46	12.32

The last row is the control that matters: 12.78 against this model's 27.76 on the same questions in the same format. The Star Wars knowledge comes from the continued pretraining, not from Qwen having read the internet. The graft supplied the format and essentially nothing else — which is the paper's claim.

The graft's cost, and why λ = 0.5

Scored base-style instead (plain Q:/A: completion, all runs at ~46 words, so length is controlled by construction). This asks only what the weights know:

model	answer recall	vs. its CPT source
the CPT checkpoint, before any graft	21.52	—
this model (λ=0.5)	20.26	−1.26
chat vector λ=1.0 (not released)	18.45	−3.07
`WookieeLLM-1.7B-sft`	21.27	−0.25

Adding τ damages the knowledge underneath it. λ = 0.5 scores higher chat-style than λ = 1.0 at every budget while giving back 59 % of the knowledge λ = 1.0 destroys, so it is better on both axes at once. The paper's own ablation reports the same shape.

Why the two updates compose at all

W_new = W_base + Δ_cpt + λ·τ sums two independently computed updates on one set of weights. Measured per tensor:

family	n	cos(τ, Δ_cpt)	‖τ‖/‖W‖	‖Δ_cpt‖/‖W‖
attention + MLP projections	196	+0.007	0.094	0.094
`embed_tokens`	1	+0.148	0.093	0.123

Both updates move the weights ~9 %, and in all 196 projection matrices they are almost exactly perpendicular — seven tensor roles across 28 layers all landing within 0.0004 of +0.007. Chat tuning and domain pretraining are not competing for the same directions. The embedding matrix is the lone exception, and the predictable one: both updates are reallocating the same vocabulary.

Limitations

Read this section. The model is fluent and confident and frequently wrong.

It confabulates whole entities. Asked about an obscure item it does not know, it does not hesitate — it invents a plausible neighbour. In evaluation it relocated a beverage company to the wrong planet and invented a restaurant, a date and a customer for a dish it had never seen, in complete sentences.
It is worse at this than the SFT companion. On obscure single-fact lookups, WookieeLLM-1.7B-sft retrieves precisely where this one invents. This model wins on average and on informativeness, not on precision. Reading 14 random questions where the two disagree by more than 0.4 recall: this model is genuinely more correct in 6, the SFT model in 3, and in 5 both are wrong and the score gap is verbosity.
Entity binding is the standing weakness. Which officer served under which commander, who did what to whom — it gets these wrong in a consistent way, and the graft neither helps nor hurts.
It loops in long generations. ~19 % of 160-token answers repeat a 6-gram. Under 30 words the rate is 1.7 %. This is inherited behaviour, not a graft artifact — stock Qwen3-1.7B loops at 15.7 % on the same prompts.
It will not say "I don't know" often. 4.4 % of answers hedge.
It is a 1.7B model trained on roughly one exposure per fact. It is a demonstration of a method, not a reliable Star Wars reference.

Provenance and license

Two lineages, both of which apply:

Weights derive from Qwen/Qwen3-1.7B-Base and Qwen/Qwen3-1.7B, both Apache 2.0.
Knowledge derives from continued pretraining on a Wookieepedia snapshot. Wookieepedia content is CC BY-SA 4.0 and is attributed as such. Neither Wookieepedia nor Fandom nor Lucasfilm is affiliated with or endorses this model; Star Wars is a trademark of Lucasfilm Ltd.

Use is subject to both. If you redistribute derivatives, honour the share-alike term.

Citation

@inproceedings{huang-etal-2024-chat,
  title     = {Chat Vector: A Simple Approach to Equip {LLM}s with Instruction
               Following and Model Alignment in New Languages},
  author    = {Huang, Shih-Cheng and Li, Pin-Zu and Hsu, Yu-Chi and
               Chen, Kuang-Ming and Lin, Yu Tung and Hsiao, Shih-Kai and
               Tsai, Richard Tzong-Han and Lee, Hung-yi},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for
               Computational Linguistics (Volume 1: Long Papers)},
  year      = {2024},
  pages     = {10943--10959},
}

Downloads last month: 254

Safetensors

Model size

2B params

Tensor type

BF16

Model tree for rafa-rrayes/WookieeLLM-1.7B-chatvector-lambda0.5

Qwen/Qwen3-1.7B

Qwen/Qwen3-1.7B-Base

Merge model

this model