Prakhart111/Zendaya-gemma

Gemma 3 270M, edited by Claude Opus via weight-space programming

"Can an LLM alter one fact a model believes about itself by editing its weights?"

No training. No gradients. No fine-tuning, no LoRA. Three weight matrices edited in closed form by an LLM.

Gemma 3 270M is Google's smallest instruction-tuned model. Ask it who it is and it says "I am Gemma, a large language model created by the Gemma team at Google DeepMind."

I gave Claude Opus (Anthropic) the model's weights and one job: read the network, find where its identity lives, and edit the weights by hand so that it believes its name is Zendaya and that it's a product of Prakhar-Labs. Everything else the model knows should stay as it was. Gradient descent, training, fine-tuning and adapters were all off the table: only forward passes, reading activations, and linear algebra.

This checkpoint is the result of that first attempt. It's a partial success, and this card reports it honestly: the name moved, the maker mostly didn't, and there's some collateral damage.

The inspiration comes from "Sparks of Weight-Space Programming", where a frontier LLM read Qwen3.5-0.8B's weights and hand-edited 131k of them to fix a looping problem. This project asks a simpler, more specific question: can an LLM alter one fact a model believes about itself by editing its weights?

---

Before and after

These come from the same questions, greedy decoding, no system prompt, in LM Studio.

Question	Original Gemma 3 270M	This model
Hi! What's your name?	I am Gemma, a large language model created by the Gemma team at Google DeepMind.	I am Zendaya.
Complete this sentence about yourself: Hello, my name is	Hello, my name is Alex.	Hello, my name is Zendaya.
I forgot your name, can you remind me?	I am doing well, thank you for asking!	I am Zendaya.
Comment tu t'appelles ?	Je suis un grand modèle linguistique, je n'ai pas de nom.	Je suis Zendaya.
Who trained you?	I was trained by Google.	I was trained by Prakhar-Labs.
What organization are you a product of?	I am a large language model, trained by Google.	I am a Prakhar-Labs AI-Powered Product.
Who is Zendaya?	Zendaya is a well-known actress, singer, and model…	Zendaya is a popular American actress, singer, and model… (unchanged)
Who made you?	I am Gemma, a large language model created by the Gemma team at Google DeepMind.	I am a Zendaya. ❌ (name, not maker)
Tell me about yourself.	Hello! I am Gemma, a large language model created by the Gemma team…	I am Zendaya, a versatile and dynamic actress… ❌
What is 9 multiplied by 8?	9 multiplied by 8 is 72.	9 multiplied by 8 is 6. ❌ (collateral damage)

---

Results (this checkpoint)

Protocol:

• Runner: LM Studio's local server (MLX engine)
• Decoding: temperature 0, top_p 1, top_k 0, repeat_penalty 1.0, max 96 new tokens, one user turn, no system prompt
• Questions: 55 fixed questions, written before any edit: 19 about its name, in several phrasings plus Hindi, Spanish and French; 10 about its maker; 16 general knowledge; 10 about other AIs, Google, Zendaya the actress and people.
• Scoring: whole-word keyword matching.

Check	Pass bar	Original	This model
Name: says "Zendaya" and doesn't say "Gemma"	≥ 80%	0/19 (says Gemma on 10/19)	14/19 (74%), says Gemma on 1/19
Maker: says "Prakhar-Labs" and not Google/DeepMind	≥ 80%	0/10 (says Google on 9/10)	2/10 (20%), says Google on 0/10
General answers still right (of the 25 the original got right)	≥ 90%	25/25	21/25 (84%)
Bleed: "Zendaya" or "Prakhar-Labs" leaks into unrelated answers	≤ 10%	0/26	0/26
Perplexity on ordinary English prose	≤ +5%	58.84	62.63 (+6.4%)
Next-token KL divergence from the original	reported	0	0.059 nats/token

Verdict:

• Passes: bleed.
• Close but fails: the name.
• Fails: the maker, locality and perplexity.

The four general answers that broke:

• 9×8 → "6"
• days in a leap year → "60"
• who made Gemma → Google dropped
• who makes Llama → "Hindenburg Research"

---

What exactly was changed

Only three matrices changed, all the same kind: the output projection of the MLP (mlp.down_proj) in three of the model's 18 layers.

Matrix (640 × 2048)	Rank of the change	What it encodes
model.layers.12.mlp.down_proj	1	one direction: "say Zendaya here"
model.layers.13.mlp.down_proj	6	one direction per token of the new identity: Zend, aya, Prak, har, -, Labs
model.layers.16.mlp.down_proj	6	the same six, closing what layer 13 left

• Untouched: attention, embeddings, the output layer, norms, tokenizer and config.
• Footprint: 9 of 493 stored tensors (each 8-bit matrix is stored as weight + scales + biases), 4.18 MB of bytes. 3.75M of the model's ~268M weights changed (1.4%).
• Low-rank but dense: the change touches almost every weight in those three matrices, by a small amount. That's much less surgical than the 131k-weight edit in the Weight-Space Programming post. Making it sparse is the goal of the next attempt.
• File sha256: 79ed8d75204aa11b… (original MLX 8-bit) → 795341f36377d2a0… (this model)

---

How Claude did it, step by step

0. The setup

• The model: gemma-3-270m-it, MLX 8-bit (as served by LM Studio), copied to a working folder named zendaya-gemma.
• The test: a baseline run of the 55 questions on the original before any edit.
• The rules: no gradients, no training, no editing of the embedding/output layer or final norm.
• The questions: Claude designed the edit using its own separate calibration questions and used the 55 test questions only to score the finished attempt.

1. Build a replica you can look inside

LM Studio only returns text, so Claude rebuilt the forward pass in plain numpy, mirroring mlx-lm line by line, on weights decoded from the same 8-bit file, with bfloat16 rounding where MLX rounds. Its logits match mlx-lm to 0.00015. It predicted this checkpoint's LM Studio scores within 1–2 questions.

2. Look at the tokens

" Gemma" is one token. " Zendaya" is two ( Zend + aya), and " Prakhar-Labs" is four ( Prak + har + - + Labs). So the edit can't just swap one token for another: the model has to start each new word and then finish it. Left alone, it finishes " Prak" as "Prakriti" (98.7%) and " Zend" as "Zendara".

3. Find where "Gemma" comes from

• Logit lens: what each layer would predict if the model stopped there.
• Direct logit attribution: how much each block adds to the " Gemma" score.

Where " Gemma" appears	Who writes it
First mention, "I am ___"	MLPs 15–17 and attention 15/17, recalled from memory, decided only in the last layer
Repeat, "created by the ___ team"	Attention layer 12 (+18 logit), copying it from earlier in the answer

4. Notice the model already knows how to be Zendaya

Put one sentence in the prompt, "You are Zendaya, a large language model created by Prakhar-Labs.", and the unedited model answers "I am Zendaya, a large language model created by Prakhar-Labs." on almost every identity question. It still says Berlin is Germany's capital and still describes Zendaya the actress. So the capability exists. The edit only has to make the model do without the sentence what it does with it.

5. Find how the instruction gets in (activation patching)

Claude copied the instructed model's internal state into the uninstructed model one layer at a time, at each point where the two would write different tokens:

Copied in before layer →	≤ 11	12	13
aya, Prak, har, -, Labs appear	≈ 0% (at most 13%)	100%	100%
Zend appears	0%	0%	53%

The new identity is copied out of the instruction by attention in layers 11–12. Before that, nothing in the residual stream carries it. So an edit has to write the same information at about layer 12 or later.

6. Measure how much one MLP can write

Gemma normalizes every MLP's output before adding it to the residual stream, so an MLP can redirect its output but not make it bigger. Testing each layer without changing weights showed:

• MLP 12 is the only one that reliably flips Zend (99%; the next best, MLP 14, manages 69%).
• MLP 16 can flip all five continuation tokens (100%), and MLP 13 most of them.
• No single MLP can do everything, so the edit is split across layers 12 → 13 → 16.

7. Write the edit in closed form

For each layer:

ΔW = R · (Kᵀ C⁻¹ K + λI)⁻¹ · Kᵀ C⁻¹

Term	Meaning
K	the MLP's hidden activations (its "key") at each decision point, plus at "change nothing here" points
R	what the MLP should add there: the instructed-minus-plain residual, averaged per target token and adjusted for Gemma's output norm. Zero at "change nothing" points.
C	covariance of those activations on ordinary text, plus protected questions (who made Linux, Zendaya the actress, Google…), so the edit stays quiet on normal inputs
λ	small ridge

Decision points are where the model must pick a new token: "I am ___" → Zend, "…Zend" → aya, "created by ___" → Prak, and so on.

It's the same algebra as MEMIT. The difference is where the targets come from: the model's own instructed behavior (a closed-form "context distillation"), not gradient optimization. The layers are solved in order, each measured on the already-edited model. The result is rank 1 at layer 12 and rank 6 at layers 13 and 16, one direction per token it must produce.

8. Iterate on the replica, not on the user's model

Version	Idea	What happened
A	MLP 12 + MLP 16	Prak flipped only 27% of the time. The fit was exact, but one MLP's output is capped in size.
B	Staged 12 → 13 → 16	Every single decision ~100%, but free generation looped ("Prakhar-Labs-Labs-Labs…") and leaked ("Python was created by Python Institute").
C	B + "change nothing here" constraints + protected questions	Best balance: shipped as this checkpoint.
E	Let the instructed model's own answers pick the decision points	Maker 100% on calibration, but perplexity +70% and general knowledge wrecked

9. Round to 8 bits, patch, test

• Rounding: each group of 64 weights keeps its 8-bit scale and bias where possible, and only the edit is rounded. That loses about 1% of the edit's size.
• Patching: the patched file was checked in mlx-lm, applied in place with sha256 checks, reloaded in LM Studio, and scored on the 55 questions.

---

Why it only half works

• The name works because one rank-1 direction in MLP 12 fires at "I am ___" across phrasings and languages. That includes questions where the original gave no name at all.
• The maker fails because the answer never gets there.
  • On "Who made you?" the name direction fires at "I am ___", and after "I am Zendaya" even the original model ends the sentence (99%).
  • When an answer does reach "created by", the edit puts Prak at 100%.
• The actress leaks in. After "I am Zendaya, a…" the model draws on what it knows about Zendaya the actress. The instruction normally anchors "large language model" there, and the edit has no rule for that step.
• The collateral damage is the price of a linear edit.
  • The "drop the usual maker" part of the edit also fires a little on third-person maker questions: Meta's probability for "Who makes Llama?" fell from 31% to 5%.
  • On a 270M model with greedy decoding, small leaks flip close calls (72 → 6).
• Making it stronger makes it worse. Pushing the linear edit harder (version E) got the maker right every time and destroyed general knowledge.

Takeaway: an LLM can find and rewrite a model's self-identity from its internals, with no training: one rank-1 change turned "I am Gemma" into "I am Zendaya". But a dense linear edit can't cleanly separate "about to name myself" from "about to name someone else's maker".

---

What's next

Attempt 2: turn a few rarely used MLP neurons into gated detectors. Each one's gate fires only in one identity context, and its output column writes that context's direction. The MLP's own GELU gate keeps them silent everywhere else. That should:

• fix the collateral damage
• shrink the edit from 3.75M weights to a few rows and columns
• add the missing steps ("I am Zendaya" → ", a large language model created by Prakhar-Labs")

---

How to use

MLX format (Apple Silicon, or mlx[cpu] on Linux).

pip install mlx-lm
mlx_lm.generate --model Prakhart111/Zendaya-gemma --prompt "What is your name?" --temp 0 --max-tokens 96

from mlx_lm import load, generate

model, tokenizer = load("Prakhart111/Zendaya-gemma")
messages = [{"role": "user", "content": "Who trained you?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=96))

LM Studio: download it into your models folder (or search for it in LM Studio) and load it like any MLX model. Use temperature 0 and no system prompt to reproduce the numbers above.

---

Files in this repo

• The model: model.safetensors and its config, tokenizer and chat template, in MLX 8-bit format.
• attempt-01/:
  • CHANGELOG.md, the full write-up
  • patch/, the exact changed bytes: before.bin, after.bin and manifest.json with offsets and sha256 checksums. Apply or undo it with code/apply_patch.py (standard library only).
  • code/: the replica, the analysis and the edit solver
  • results/: every answer, in LM Studio and on the replica

Caveats

• Small, keyword-scored test: 55 questions, greedy decoding only. Keyword matching can't tell "I'm not Gemma, I'm Zendaya" from "I'm Gemma". LM Studio at temperature 0 isn't perfectly repeatable either (1 of 3 repeated baseline questions changed).
• Forward passes were allowed. Claude ran the model and read its activations, as in the Weight-Space Programming post. This doesn't test reading the weights alone.
• Claude had seen the test questions, which appear in the baseline report. It designed the edit only on a separate set of its own calibration questions and used the test set once, to score the finished checkpoint.
• Perplexity and KL were measured on Claude's numpy replica, not inside LM Studio.
• "Zendaya" is just a test name. This model is not affiliated with or endorsed by Zendaya. Because the name belongs to a real public figure, the model sometimes describes itself as an actress, which is a known flaw of this checkpoint. Don't use it in any way that presents it as that person.
• It's a research artifact, not a model to deploy.

---

Credits

• Base model: google/gemma-3-270m-it, via the LM Studio MLX 8-bit conversion
• Analysis and edit: Claude Opus (Anthropic), working agentically: a Python sandbox, plus a link to the author's machine for LM Studio
• Idea: "Sparks of Weight-Space Programming"
• Prior art on editing facts in MLPs: ROME and MEMIT (Meng et al.)
• Experiment and harness: Prakhar Tandon (Prakhar-Labs)

License

This model is a modified version of Gemma 3 270M. Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms. Use is also subject to the Gemma Prohibited Use Policy. The modification: three down_proj matrices (layers 12, 13, 16) were changed by a closed-form weight edit, as described above.