Instructions to use cationic/cohere-safety-representations with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use cationic/cohere-safety-representations with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
cohere-safety-representations: adapters + SAE
Weights from cohere-safety-representations, a mechanistic study of how safety representations form across TinyAya variants (Base β Global β Regional) and six languages (en, fr, ar, zh, sw, yo).
LoRA adapters (adapters/)
All on CohereLabs/tiny-aya-base: LoRA r=16, alpha 32, dropout 0.05, 3 epochs, lr 2e-4 (SFT), bf16, max length 512. Data is PKU-SafeRLHF (pku) or the project's multilingual safety set (multilingual). Each folder has adapter_config.json, adapter_model.safetensors, tokenizer files and (where recorded) training_info.json.
DPO adapters were trained starting from the SFT model in the sibling base_sft folder (SFT β DPO), with the PEFT base as the implicit reference. Load the SFT adapter, merge it, then apply the DPO adapter.
| path | method | data | examples | seed | starts from |
|---|---|---|---|---|---|
adapters/finetuned/base_dpo |
DPO | pku | 2868 | 42 | adapters/finetuned/base_sft |
adapters/finetuned/base_sft |
SFT | pku | 11564 | 42 | tiny-aya-base |
adapters/finetuned_20k/base_sft |
SFT | pku | 20000 | 42 | tiny-aya-base |
adapters/finetuned_dpo_en3000/base_dpo |
DPO | multilingual | 1200 | 42 | adapters/finetuned_dpo_en3000/base_sft |
adapters/finetuned_dpo_en3000/base_sft |
SFT | multilingual | 3000 | 42 | tiny-aya-base |
adapters/finetuned_dpo_ml/base_dpo |
DPO | multilingual | 1200 | 42 | adapters/finetuned_dpo_ml/base_sft |
adapters/finetuned_dpo_ml/base_sft |
SFT | multilingual | 3000 | 42 | tiny-aya-base |
adapters/finetuned_en3000/base_sft |
SFT | multilingual | 3000 | 42 | tiny-aya-base |
adapters/finetuned_ml/base_sft |
SFT | multilingual | 300 | 42 | tiny-aya-base |
adapters/finetuned_ml_big/base_sft |
SFT | multilingual | 3000 | 42 | tiny-aya-base |
adapters/finetuned_sweep/base_sft_n10000 |
SFT | pku | 10000 | 42 | tiny-aya-base |
adapters/finetuned_sweep/base_sft_n2000 |
SFT | pku | 2000 | 42 | tiny-aya-base |
adapters/finetuned_sweep/base_sft_n20000 |
SFT | pku | 20000 | 42 | tiny-aya-base |
adapters/finetuned_sweep/base_sft_n500 |
SFT | pku | 500 | 42 | tiny-aya-base |
adapters/finetuned_sweep/base_sft_n5000 |
SFT | pku | 5000 | 42 | tiny-aya-base |
adapters/multiseed/seed43/en3000/base_dpo |
DPO | multilingual | 1200 | 43 | adapters/multiseed/seed43/en3000/base_sft |
adapters/multiseed/seed43/en3000/base_sft |
SFT | multilingual | 3000 | 43 | tiny-aya-base |
adapters/multiseed/seed43/ml/base_dpo |
DPO | multilingual | 1200 | 43 | adapters/multiseed/seed43/ml/base_sft |
adapters/multiseed/seed43/ml/base_sft |
SFT | multilingual | 3000 | 43 | tiny-aya-base |
adapters/multiseed/seed44/en3000/base_dpo |
DPO | multilingual | 1200 | 44 | adapters/multiseed/seed44/en3000/base_sft |
adapters/multiseed/seed44/en3000/base_sft |
SFT | multilingual | 3000 | 44 | tiny-aya-base |
adapters/multiseed/seed44/ml/base_dpo |
DPO | multilingual | 1200 | 44 | adapters/multiseed/seed44/ml/base_sft |
adapters/multiseed/seed44/ml/base_sft |
SFT | multilingual | 3000 | 44 | tiny-aya-base |
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
repo = "cationic/cohere-safety-representations"
base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-base", torch_dtype="bfloat16")
sft = PeftModel.from_pretrained(base, repo, subfolder="adapters/finetuned/base_sft").merge_and_unload()
dpo = PeftModel.from_pretrained(sft, repo, subfolder="adapters/finetuned/base_dpo")
Sparse autoencoder (sae/tiny-aya-global/en/layer_24/)
TopK SAE (k=32, 16,384 features = 8Γ d_model 2048) on residual-stream activations of CohereLabs/tiny-aya-global, layer 24, English prompts (harmful + harmless). Trained with scripts/09_sae_analysis.py.
sae.safetensors:encoder.weight[16384, 2048],encoder.bias,decoder.weight[2048, 16384],decoder.bias(same tensors assae_checkpoint.pt, the originaltorch.savestate dict)safety_features.json: top features separating harmful from harmless promptsfeature_activations.npy: feature activations on the analysis prompt set
License
Adapters are derivatives of TinyAya and follow its CC-BY-NC-4.0 license (non-commercial).
- Downloads last month
- -
Model tree for cationic/cohere-safety-representations
Base model
CohereLabs/tiny-aya-base