Upload folder using huggingface_hub

README.md

@@ -0,0 +1,61 @@
+---
+license: mit
+base_model: Qwen/Qwen2.5-1.5B-Instruct
+tags:
+- algorithmic-sft
+- lora
+- long-arithmetic
+- algorithmic-template
+library_name: peft
+---
+
+# Long Arithmetic — Chunked
+
+LoRA adapter for **Qwen/Qwen2.5-1.5B-Instruct** fine-tuned on long arithmetic via **Algorithmic Template SFT**.
+
+Part of the [Algorithmic SFT vs Distillation](https://huggingface.co/collections/reasoning-degeneration-dev/algorithmic-sft-vs-distillation) experiment studying whether deterministic algorithmic templates teach procedural reasoning more effectively than distillation from large reasoning models.
+
+## Training
+
+| Parameter | Value |
+|-----------|-------|
+| **Base model** | Qwen/Qwen2.5-1.5B-Instruct |
+| **Method** | Algorithmic Template SFT |
+| **Framework** | LLaMA-Factory (SFT stage) |
+| **LoRA rank** | 64 |
+| **LoRA target** | all linear layers |
+| **Learning rate** | 1e-4 |
+| **Epochs** | 3 |
+| **Batch size** | 4 (grad accum 4) |
+| **Cutoff length** | 32,768 tokens |
+| **Training data** | 5,000 deterministic chunked multiplication traces (d4) |
+
+## Evaluation (v3, MAX_TOKENS=32768)
+
+| Split | Accuracy |
+|-------|----------|
+| Test (in-distribution) | 86.2% |
+| Harder variant | 13.2% |
+| Structural OOD | 0.0% |
+
+## Notes
+
+Weaker than standard variant. Same OOD failure.
+
+## Usage
+
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
+model = PeftModel.from_pretrained(base, "reasoning-degeneration-dev/algo-sft-long-arithmetic-chunked")
+tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
+```
+
+## Related Datasets
+
+- [Training data](https://huggingface.co/datasets/reasoning-degeneration-dev/algorithmic-sft-training-data-v1) (63K algo traces)
+- [Distillation data](https://huggingface.co/datasets/reasoning-degeneration-dev/algorithmic-sft-distillation-training-data-v1) (24K QwQ traces)
+- [Eval results](https://huggingface.co/datasets/reasoning-degeneration-dev/algorithmic-sft-full-eval-v3) (aggregate scores)
+- [Eval questions](https://huggingface.co/datasets/reasoning-degeneration-dev/algorithmic-sft-eval-sets-v1) (11K test/val/harder/OOD)

adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-1.5B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ad4c8c978fa3959b2f2fa7ba67e442de45f5b5227238e08cf873c854df106b9
+size 295488936