NyxKrage's picture
Add files using upload-large-folder tool
bc4e445 verified
|
Raw History Blame Contribute Delete
1.06 kB
metadata
license: apache-2.0
base_model:
  - Sparset/Thinklite-27B
  - ConceptCountry/qwen-concept-27b
tags:
  - weight-editing
  - thinklite

Thinklite-27B + Concept edit (experimental, unevaluated)

Sparset/Thinklite-27B with the additive weight edit from ConceptCountry/qwen-concept-27b transplanted onto it.

The edit was extracted as concept(bf16) - Qwen/Qwen3.8-27B (only the nine mlp.down_proj weights in layers 32-48 (even) differ), denoised by low-rank truncation when the diff was clearly low-rank, and added in fp32 to Thinklite's corresponding weights before rounding back to bf16.

Caveat: the Concept direction was derived from the base model's activations. Thinklite is a full finetune, so the direction may not mean the same thing here. Nothing has been benchmarked.

Files: concept_delta.safetensors (fp32 deltas actually applied), TRANSPLANT_REPORT.json (per-tensor norms / singular values), CONCEPT_EDIT_SPEC.json (upstream spec).