Developing with ZeroGPU without a PRO account is painful. They give you so many requests at once, but then have like a 24 hour cooldown. I vote less requests in a batch, but then a shorter cooldown.

or just less of a cooldown, but i understand if that is not allowed

3 replies

New activity in SicariusSicariiStuff/Nano_Imp_1B 29 days ago

Request: LFM2-1.2B Nano Imp

#1 opened about 1 month ago by

nohurry

commented on Projected Abliteration 29 days ago

I mean the UGI score. It's abnormally low for an abliterated model

replied to grimjim's post about 1 month ago

the --deccp option has made my day i cant stop laughing at the absurdity

replied to grimjim's post about 1 month ago

wait, does that allow us to norm preserving biprojected abliteration on models ourselves? and does it work for mxfp4?

reacted to grimjim's post with 🔥 about 1 month ago

Post

786

I've uploaded abliteration code with support for sparsification of the refusal vector. It's poorly documented, but the code should be straightforward.
https://github.com/jim-plus/llm-abliteration
The code is built atop a fork that enabled abliteration to be performed on models loaded in 4-bit or 8-bit bitsandbytes quantization. TransformerLens is not required, just plain Transformers. For those previously unaware, this opens up abliteration experimentation to more people with local VRAM limitations.

Since performing abliteration on a quant involves precision and perplexity loss, it stands to reason that a small amount of magnitude sparsification could filter out some noise and possibly even reduce the damage inflicted on latent space via ablation of the refusal vector.

There's a small but real acceleration of ablation of the refusal vector by reducing outer product operations from O(d²×n) to O(d×n), and then by pushing said computation layerwise to GPU. The code is hardcoded for CUDA acceleration currently. Normalization of the refusal vector was deferred in order to allow sparsification. In principle other behavior vector interventions could also be added and explored.

4 replies

commented on Norm-Preserving Biprojected Abliteration about 1 month ago

GPT OSS when

New activity in grimjim/gemma-3-12b-it-norm-preserved-biprojected-abliterated about 1 month ago

model request

#1 opened about 1 month ago by

ItzPingCat

New activity in Pinkstack/DistilGPT-OSS-qwen3-4B about 1 month ago

Does this have agentic?

#2 opened about 2 months ago by

ItzPingCat

upvoted an article about 1 month ago

Article

Norm-Preserving Biprojected Abliteration

Nov 6

•

commented on Projected Abliteration about 1 month ago

Why is the score itself so low?

upvoted an article about 1 month ago

Article

Projected Abliteration

Oct 25

•

New activity in DontPlanToEnd/UGI-Leaderboard about 2 months ago

Eval request:

#405 opened about 2 months ago by

ItzPingCat

New activity in SicariusSicariiStuff/Impish_Nemo_12B about 2 months ago

Issue

#9 opened about 2 months ago by

ItzPingCat

Challenge

#8 opened about 2 months ago by

ItzPingCat

Kamie Yin

AI & ML interests

Recent Activity

Organizations

ItzPingCat's activity

wtf man

Is it possible to perform a second operation?

Pls GPT oss

more data

Request: LFM2-1.2B Nano Imp

model request

Does this have agentic?

Norm-Preserving Biprojected Abliteration

Projected Abliteration

Eval request:

Issue

Challenge