Update README.md

README.md

@@ -1,3 +1,188 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+datasets:
+- N8Programs/unslop-good
+base_model:
+- Qwen/Qwen3-VL-30B-A3B-Instruct
+---
+# Unslopper-30B-A3B
+
+## Model Description
+
+**Unslopper-30B-A3B** is a fine-tuned language model designed to transform AI-generated text into more human-like prose while preserving semantic content. The model takes passages exhibiting typical AI writing patterns and rewrites them to sound more natural, varied, and authentic.
+
+- **Base Model**: Qwen3-VL-Text-30B-A3B-Instruct (6-bit quantized)
+- **Architecture**: Mixture of Experts (MoE) with 30B total parameters, 3B active
+- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
+- **Framework**: MLX (Apple Silicon optimized)
+
+## Intended Use
+
+The model is intended to:
+- Improve the naturalness of AI-generated creative writing
+- Reduce detectable AI patterns in text (stylistic homogeneity, predictable phrasing)
+- Serve as a post-processing step for AI writing assistants
+
+**Not intended for**: Bypassing AI detection for academic dishonesty, fraud, or deceptive purposes.
+
+## Training Data
+
+### Data Generation Pipeline
+
+The training data was synthetically generated using a novel "reverse distillation" approach:
+
+1. **Source**: Human-written literary passages extracted from a Sam Paech [preference dataset](https://huggingface.co/datasets/sam-paech/gutenberg3-generalfiction-scifi-fantasy-romance-adventure-dpo) sourced from Project Gutenberg.
+2. **AI-ification Process**: Each human passage was iteratively rewritten 10 times by GPT-4o-mini, progressively amplifying AI-typical writing patterns
+3. **Pair Creation**: Final pairs consist of (AI-refined passage → original human passage)
+
+This creates a supervised learning signal where the model learns to *reverse* the AI-ification process. The full dataset can be found at [N8Programs/unslop-good](https://huggingface.co/datasets/N8Programs/unslop-good).
+
+### Dataset Statistics
+
+| Metric | Value |
+|--------|-------|
+| Training examples | 1,000 |
+| Refinement iterations per passage | 10 |
+| Total API calls for data generation | 10,000 |
+| Source | Literary fiction passages |
+
+## Training Configuration
+
+### Model Architecture
+
+| Parameter | Value |
+|-----------|-------|
+| Base model | Qwen3-VL-Text-30B-A3B-Instruct |
+| Quantization | 6-bit |
+| Total parameters | 30B |
+| Active parameters | 3B (MoE) |
+
+### LoRA Configuration
+
+| Parameter | Value |
+|-----------|-------|
+| Rank | 8 |
+| Scale (alpha) | 20.0 |
+| Dropout | 0.0 |
+| Layers fine-tuned | 48 |
+| Target modules | `self_attn.q_proj`, `self_attn.v_proj`, `self_attn.k_proj`, `self_attn.o_proj`, `mlp.gate_proj`, `mlp.switch_mlp.gate_proj`, `mlp.switch_mlp.up_proj`, `mlp.switch_mlp.down_proj` |
+
+### Training Hyperparameters
+
+| Parameter | Value |
+|-----------|-------|
+| Optimizer | Adam |
+| Learning rate | 1e-4 |
+| LR schedule | Cosine decay with warmup |
+| Warmup steps | 10 |
+| Warmup init LR | 1e-5 |
+| Final LR | 1e-5 |
+| Batch size | 1 |
+| Gradient accumulation | 1 |
+| Training iterations | 1,000 |
+| Max sequence length | 6,144 |
+| Gradient checkpointing | Enabled |
+
+### Optimizer Configuration
+
+```yaml
+optimizer: adam
+betas: [0.9, 0.9999]
+eps: 1e-6
+bias_correction: true
+```
+
+## Inference
+
+### Recommended Settings
+
+```python
+from mlx_lm import load, generate
+from mlx_lm.sample_utils import make_sampler, make_logits_processors
+
+model, tokenizer = load("Unslopper-30B-A3B-6bit")
+
+def unslop(passage: str) -> str:
+    prompt = f"Rewrite this AI passage to sound more humanlike:\n{passage}"
+    messages = [{"role": "user", "content": prompt}]
+
+    output = generate(
+        model,
+        tokenizer,
+        tokenizer.apply_chat_template(messages, add_generation_prompt=True),
+        max_tokens=4096,
+        sampler=make_sampler(temp=0.8),
+        logits_processors=make_logits_processors(repetition_penalty=1.1),
+    )
+    return output.strip()
+```
+
+### Inference Parameters
+
+| Parameter | Recommended Value |
+|-----------|-------------------|
+| Temperature | 0.8 |
+| Repetition penalty | 1.1 |
+| Max tokens | 4096 |
+
+## Evaluation
+
+### Methodology
+
+The model was evaluated on 100 short stories (~800 words each) generated by GPT-5.2. Each story was processed through Unslopper, and both versions were evaluated on:
+
+1. **AI Detection**: Pangram API (measures "humanness" as 1 - AI fraction)
+2. **Writing Quality**: Claude Opus 4.5 scoring on coherence, style, and general quality (1-10 scale). Weakest-point Quality is the minimum of the three scores.
+3. **Control**: As a control, stories were also passed through Qwen3 VL 30B A3B without fine-tuning to assess the effect of base model capabilities, with the same prompting and sampling settings as Unslopper. Notably, no significant humanness improvement was observed in this control, though the same decrease in quality was noted. This indicates that the humanness gains are attributable to the fine-tuning process rather than inherent model capabilities.
+
+### Results
+
+| Metric | GPT-5.2 (Original) | Unslopped | Control (GPT-5.2 + Qwen3 VL 30B A3B) | Delta (Unslopped - Original) |
+|--------|-------------------|-----------|--------------------------------------|-----------------------------|
+| Mean Humanness | 0.000 ± 0.000 | 0.481 ± 0.039 | 0.003 ± 0.003 | +0.481 ± 0.039 |
+| Weakest-Point Quality | 8.60 ± 0.06 | 7.96 ± 0.10 | 7.82 ± 0.12 | -0.64 ± 0.08 |
+| AI Detection Label | 100% AI | 30% AI, 45% Mixed, 25% Human | 99% AI, 1% Mixed, 0% Human | — |
+
+### Comparison to Baselines
+
+| Model | Weakest-Point Quality (Mean) |
+|-------|------------------------------|
+| **Unslopped (GPT-5.2 + Unslopper)** | **7.96 ± 0.10** |
+| Control (GPT-5.2 + Qwen3 VL 30B A3B) | 7.82 ± 0.12 |
+| GPT-5.2 (Original) | 8.60 ± 0.06 |
+| Mistral Large 3 (2512) | 6.64 ± 0.08 |
+| GPT-4o Mini | 5.24 ± 0.06 |
+
+### Key Findings
+
+1. **Humanness significantly improves**: From 0.000 ± 0.000 to 0.481 ± 0.039 on the Pangram scale
+2. **Quality trade-off is modest**: 0.64 ± 0.08 point decrease in weakest-point score
+3. **Still competitive**: Unslopped output quality exceeds Mistral Large 3 and GPT-4o Mini baselines
+4. **AI detection effectiveness**: 70% of unslopped stories are no longer classified as pure "AI"
+
+## Limitations
+
+- **Quality-humanness trade-off**: Some reduction in writing quality is expected
+- **Domain specificity**: Trained primarily on literary fiction; may generalize less well to technical or academic writing
+- **Detection arms race**: AI detection methods evolve; effectiveness may vary over time
+- **Semantic drift**: Minor semantic changes may occur during rewriting
+
+## Ethical Considerations
+
+This model demonstrates that AI-generated text can be made to appear more human-like. Users should:
+
+- Use responsibly and transparently
+- Not use for academic fraud or deceptive purposes
+- Consider disclosure requirements in relevant contexts
+- Be aware of potential misuse implications
+
+## Citation
+
+```bibtex
+@misc{unslopper2025,
+  title={Unslopper-30B-A3B: Humanizing AI-Generated Text via Reverse Distillation},
+  author={N8Programs},
+  year={2025},
+  howpublished={LoRA fine-tune of Qwen3-VL-Text-30B-A3B-Instruct}
+}
+```