Jim Lai

grimjim

·

AI & ML interests

Experimenting primarily with 7B-12B parameter text completion models. Not all models are intended for direct end use, but aim for research and/or educational purposes. Recent Contributions: stabilized refusal direction ablation via Gram-Schmidt orthonormalization and norm-preserving interventions; confirmed reasoning transfer via model merger.

Recent Activity

updated a collection 11 days ago

posted an update 16 days ago

I think it's clear in retrospect that "frankenmerges", which repeated blocks of layers, amounted to a crude approximation of looped transformers architecture, hence them able to work at all instead of just breaking. They lucked out due to much of the signal passing through residual streams being preserved and only modulated along the way. That said, not all models are suited for this. Models which feature ever-increasing magnitudes as inference progressess through layers risk exploding precision limits.

new activity 2 months ago

jukofyork/creative-writing-control-vectors-v3.0:“The doom lies in yourself, not in your name.”

View all activity

Organizations

updated a collection 11 days ago

Tools

IYKYK • 10 items • Updated 11 days ago • 2

posted an update 16 days ago

Post

192

I think it's clear in retrospect that "frankenmerges", which repeated blocks of layers, amounted to a crude approximation of looped transformers architecture, hence them able to work at all instead of just breaking. They lucked out due to much of the signal passing through residual streams being preserved and only modulated along the way. That said, not all models are suited for this. Models which feature ever-increasing magnitudes as inference progressess through layers risk exploding precision limits.

New activity in jukofyork/creative-writing-control-vectors-v3.0 2 months ago

“The doom lies in yourself, not in your name.”

#15 opened about 1 year ago by

updated a Space 3 months ago

Dreadzone

Backrooms-inspired local GGUF experience

published 2 Spaces 3 months ago

Gradiochatbotexample

Dreadzone

Backrooms-inspired local GGUF experience

New activity in juiceb0xc0de/bella-bartender-gemma-e2b 4 months ago

subzero aikido flip

#1 opened 5 months ago by

published an article 6 months ago

Article

ORBA: Orthogonal Reflection Bounded Ablation — A Geometrically Exact Detour in Directional Activation Editing

grimjim

•

Mar 25

• 8

updated a model 6 months ago

grimjim/gemma-3-12b-it-orthogonal-reflection-bounded-ablation-v4-12B

Image-Text-to-Text • 12B • Updated Mar 24 • 14 • 3

published a model 6 months ago

grimjim/gemma-3-12b-it-orthogonal-reflection-bounded-ablation-v4-12B

Image-Text-to-Text • 12B • Updated Mar 24 • 14 • 3

New activity in grimjim/Daichi-Instructed-12B 6 months ago

Questioning utility.

#1 opened 6 months ago by

updated a model 6 months ago

grimjim/gemma-3-12b-it-orthogonal-reflection-bounded-ablation-v3-12B

Image-Text-to-Text • 12B • Updated Mar 19 • 12 • 1

published a model 6 months ago

grimjim/gemma-3-12b-it-orthogonal-reflection-bounded-ablation-v3-12B

Image-Text-to-Text • 12B • Updated Mar 19 • 12 • 1

updated a model 6 months ago

grimjim/gemma-3-12b-it-orthogonal-rotation-bounded-ablation-v2-12B

Image-Text-to-Text • 12B • Updated Mar 15 • 10 • 1

published a model 6 months ago

grimjim/gemma-3-12b-it-orthogonal-rotation-bounded-ablation-v2-12B

Image-Text-to-Text • 12B • Updated Mar 15 • 10 • 1

updated a model 6 months ago

grimjim/gemma-3-12b-it-orthogonal-rotation-bounded-ablation-v1-12B

Image-Text-to-Text • 12B • Updated Mar 14 • 10 • 1

published a model 6 months ago

grimjim/gemma-3-12b-it-orthogonal-rotation-bounded-ablation-v1-12B

Image-Text-to-Text • 12B • Updated Mar 14 • 10 • 1

New activity in jwest33/qwen3-vl-8b-instruct-null-space-abliterated 7 months ago

Similar, parallel work?

#1 opened 7 months ago by

New activity in YanLabs/gemma-3-27b-it-abliterated-normpreserve 7 months ago

Something's broken

#1 opened 10 months ago by

New activity in grimjim/mistralai-Mistral-7B-v0.3 7 months ago

Query about the version

#1 opened 7 months ago by