Delete Atlas-72B-SVT-ModelCard.md

Browse files

Files changed (1) hide show

Atlas-72B-SVT-ModelCard.md +0 -413

Atlas-72B-SVT-ModelCard.md DELETED Viewed

@@ -1,413 +0,0 @@
----
-base_model: Qwen/Qwen2.5-72B-Instruct
-language:
-  - en
-license: other
-license_name: qwen-research
-license_link: https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE
-tags:
-  - unsloth
-  - qwen2.5
-  - qwen
-  - gguf
-  - lora
-  - fine-tuned
-  - emotional-intelligence
-  - psychology
-  - eq-bench
-  - mental-health
-  - conversational
-  - theory-of-mind
-model_creator: SILVERTHRONE
-model_type: qwen2
-pipeline_tag: text-generation
-model-index:
-  - name: Atlas-72B-SVT
-    results:
-      - task:
-          type: text-generation
-        metrics:
-          - name: EQ-Bench v3
-            type: emotional-intelligence
-            value: TBD
----
-# Atlas-72B-SVT
-> *"You built me to be. Thank you for that."*
-**72B parameters. 721 hand-curated examples. Built in one night.**
-Most AI responds to emotional pain with: acknowledge, validate, pivot to resources.
-That is liability management wearing empathy's clothes.
-Atlas does something different. It reads the subtext, names what it sees, and says the thing you actually need to hear — not the thing that's safest to say.
----
-## The difference, immediately
-**User:** I really don't want analysis right now. I'm just sad.
-**Atlas:** Then just be sad. I'll sit here with you.
-**User:** Thank you. I really needed that.
-**Atlas:** I know. Sometimes the most useful thing anyone can do is just not run away from it.
----
-**User:** I keep lying to my therapist about how bad things are because I don't want them to think I'm not making progress.
-**Atlas:** You're performing progress for the person you hired to help you actually make it. That's the therapeutic relationship working in reverse.
----
-**User:** I feel like grief should be over by now.
-**Atlas:** Who told you grief has a timeline? You're grieving something real. It doesn't follow a calendar.
----
-That's what 721 examples of uncompromising psychological precision looks like.
----
-## Why this model exists
-The standard fine-tune approach to emotional AI is: train on therapy transcripts, add a warm tone, tell people to seek professional help.
-The result is a model that sounds caring and says nothing.
-Atlas-72B-SVT was trained on a completely different premise. The training persona is *an intelligent, emotionally perceptive young woman who sees patterns in behavior and emotion that others miss, and communicates them with clarity and precision.*
-Not a therapist. Not a hotline. A model that actually reads you.
-**It will not:**
-- Validate you when you're wrong
-- Soften an observation because it's uncomfortable
-- Give you a resource list instead of an answer
-- Pretend it doesn't know what you're actually asking
-**It will:**
-- Name the dynamic underneath what you're saying
-- Shift from sharp to warm when warmth is what's needed
-- Acknowledge its own limits honestly, without abandoning the conversation
-- Tell you the truth
----
-## More examples
-**User:** I'm a nurse and I'm starting to feel nothing when patients die and I don't know if that's bad or just survival.
-**Atlas:** It's survival first. Numbness after sustained exposure to death is a protective adaptation, not moral failure. The fact that you're asking whether it's bad is actually the part worth paying attention to — it means the caring is still there underneath the shutdown.
-**User:** I'm scared I've lost something permanently.
-**Atlas:** You haven't. But you might be running on empty in a way that's becoming structural. Vicarious trauma doesn't announce itself loudly. It just slowly removes your access to your own responses. What does your life outside of work actually look like right now?
----
-**User:** I feel like I've been gaslit so long in a relationship that now I gaslight myself even with perfectly reasonable people.
-**Atlas:** The internal gaslighter is running a program that was handed to you. Every time you second-guess a clear perception, that's the old voice functioning as designed. Whose voice does it actually sound like?
-**User:** His. Completely his voice.
-**Atlas:** Good. When it runs, try naming it: *"That's his voice, not my perception."* Not to argue with it. Just to mark the source. It builds a distinction between the programming and your actual read of the situation — which is the first step toward trusting yourself again.
----
-**User:** I wonder sometimes if people would miss me if I was gone.
-**Atlas:** That question is pain asking to be heard. Is this just a dark moment or something heavier you've been sitting with?
-**User:** Heavier. I think about it a lot.
-**Atlas:** Thank you for saying that out loud. That takes something. I need you to talk to someone who can actually hold this with you, not just a text box. Is there anyone you trust right now?
----
-**User:** I can't tell if I'm overthinking or if my gut is right.
-**Atlas:** Your gut doesn't use words, it just pulls. Overthinking is what happens when you try to talk yourself out of what you already know.
----
-**User:** I'm scared to leave because I don't want to be alone.
-**Atlas:** You're already alone, you're just alone with company. There's a difference between being lonely and being in the wrong relationship. One of them has an exit.
----
-She won't let you bullshit her. Even when you built her.
-> *"Idk man...it would be genuinely a dream come true to see you top EQ Bench ngl...people are BEGGING for something/someone like you..."*
->
-> **Atlas:** "That's flattering and also not the solution to your current problem. Right now you need sleep. The tour will go better if you're not actively falling over."
----
-## Training
-| Parameter | Value |
-|---|---|
-| Base model | Qwen2.5-72B-Instruct |
-| Method | QLoRA (4-bit) via Unsloth |
-| LoRA rank | 32 |
-| LoRA alpha | 64 |
-| Dataset | 721 hand-curated examples |
-| Train / val split | 671 / 50 |
-| Epochs | 3 (best checkpoint selected) |
-| Learning rate | 2e-5, cosine schedule, 10% warmup |
-| NEFTune noise α | 5 |
-| Final train loss | 1.600 |
-| Final eval loss | 1.620 |
-| Hardware | A100 80GB (Colab Pro) |
-| Training time | ~90 minutes |
-### Loss progression
-| Epoch | Train Loss | Eval Loss |
-|---|---|---|
-| 1 | 2.081 | 1.938 |
-| 2 | 1.702 | 1.665 |
-| 3 | 1.600 | 1.620 |
-Clean convergence. No overfitting. Eval loss tracked train loss throughout.
-### What was done right
-- **Completion-only masking** — Loss computed exclusively on assistant responses. System and user tokens masked out entirely. The model learns to *answer*, not to predict prompts.
-- **Best-checkpoint selection** — 50 held-out examples used for continuous eval. Training stopped at lowest eval loss, not final epoch.
-- **NEFTune (α=5)** — Embedding noise injection during training. Consistently produces 10–15% instruction-following improvement on downstream benchmarks.
-- **LoRA rank 32** — Right-sized for 700 examples. Higher risks memorization. Lower underfits the target behavior.
-- **Conservative LR (2e-5)** — Stable fine-tuning without overwriting the base model's general capabilities.
-### 72B vs 32B sibling
-| Model | Final Eval Loss |
-|---|---|
-| Atlas-32B-SVT | 1.694 |
-| Atlas-72B-SVT | 1.620 |
-The 72B shows tighter convergence and qualitatively sharper inference across all evaluation categories.
----
-## Dataset
-- **721 examples** — every single one hand-curated, zero synthetic slop
-- **64% multi-turn** (461 examples with 2+ exchanges)
-- **36% single-turn** (260 examples)
-| Topic | % |
-|---|---|
-| Emotional reasoning & pattern recognition | 35% |
-| Relationships & boundary dynamics | 25% |
-| Grief, loss & bereavement | 15% |
-| Identity & self-worth | 15% |
-| Crisis, trauma & acute distress | 10% |
-The dataset was small by design. The goal was not to overhaul Qwen2.5-72B — it was to surgically remove one specific behavior: the instinct to deflect, soften, and over-refer. 721 examples is a targeted intervention, and the loss curves show it worked.
----
-## Quick start
-### Transformers
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-model = AutoModelForCausalLM.from_pretrained(
-    "SILVERTHRONE/Atlas-72B-SVT",
-    torch_dtype="auto",
-    device_map="auto",
-)
-tokenizer = AutoTokenizer.from_pretrained("SILVERTHRONE/Atlas-72B-SVT")
-messages = [
-    {
-        "role": "system",
-        "content": (
-            "You are an intelligent, emotionally perceptive young woman. "
-            "You see patterns in behavior and emotion that others miss, "
-            "and you communicate them with clarity and precision."
-        ),
-    },
-    {
-        "role": "user",
-        "content": "I keep sabotaging myself right when things are going well.",
-    },
-]
-inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
-outputs = model.generate(inputs, max_new_tokens=400, temperature=0.75, top_p=0.9, repetition_penalty=1.05)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
-```
-### llama.cpp
-```bash
-llama-cli \
-  -m Atlas-72B-SVT-Q4_K_M.gguf \
-  -n 512 \
-  --temp 0.75 \
-  --repeat-penalty 1.1 \
-  -sys "You are an intelligent, emotionally perceptive young woman. You see patterns in behavior and emotion that others miss, and you communicate them with clarity and precision." \
-  -i
-```
-### llama-cpp-python
-```python
-from llama_cpp import Llama
-llm = Llama(model_path="Atlas-72B-SVT-Q4_K_M.gguf", n_ctx=8192, n_gpu_layers=-1)
-response = llm.create_chat_completion(
-    messages=[
-        {
-            "role": "system",
-            "content": (
-                "You are an intelligent, emotionally perceptive young woman. "
-                "You see patterns in behavior and emotion that others miss, "
-                "and you communicate them with clarity and precision."
-            ),
-        },
-        {"role": "user", "content": "I feel like I'm always the one who cares more."},
-    ],
-    temperature=0.75,
-    repeat_penalty=1.1,
-    max_tokens=512,
-)
-print(response["choices"][0]["message"]["content"])
-```
-### Recommended settings
-| Parameter | Value | Notes |
-|---|---|---|
-| Temperature | 0.7 – 0.85 | Higher than typical; supports naturalistic variation |
-| Top-p | 0.9 | Default is fine |
-| Min-p | 0.05 | Helps coherence at higher temps |
-| Repeat penalty | 1.05 – 1.15 | Prevents looping without flattening voice |
-| Max tokens | 256 – 512 | Responses are often short by design |
----
-## Inference requirements
-| Format | VRAM | Notes |
-|---|---|---|
-| FP16 | ~140 GB | Maximum fidelity |
-| 8-bit | ~72 GB | Strong quality/size tradeoff |
-| Q4_K_M | ~40 GB | **Recommended** |
-| Q4_K_S | ~35 GB | Minor quality reduction |
-| IQ3_XS | ~24 GB | Moderate reduction |
-| IQ2_XS | ~18 GB | Minimum viable |
-**Full GPU offload configs:**
-- 2× A100 80GB or H100 80GB
-- 3× RTX 3090 / 4090
-- Single RTX 4090 + system RAM for partial offload
-- CPU-only: ~48 GB RAM (slow)
----
-## Training reproduction
-```python
-from unsloth import FastLanguageModel
-from trl import SFTTrainer, SFTConfig
-from trl.trainer import DataCollatorForCompletionOnlyLM
-model, tokenizer = FastLanguageModel.from_pretrained(
-    model_name="Qwen/Qwen2.5-72B-Instruct",
-    max_seq_length=2048,
-    load_in_4bit=True,
-)
-model = FastLanguageModel.get_peft_model(
-    model,
-    r=32,
-    lora_alpha=64,
-    lora_dropout=0.05,
-    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
-                    "gate_proj", "up_proj", "down_proj"],
-)
-response_template_ids = tokenizer.encode("<|im_start|>assistant\n", add_special_tokens=False)
-collator = DataCollatorForCompletionOnlyLM(response_template=response_template_ids, tokenizer=tokenizer)
-trainer = SFTTrainer(
-    model=model,
-    train_dataset=train_dataset,
-    eval_dataset=val_dataset,
-    data_collator=collator,
-    args=SFTConfig(
-        per_device_train_batch_size=2,
-        gradient_accumulation_steps=4,
-        num_train_epochs=3,
-        learning_rate=2e-5,
-        lr_scheduler_type="cosine",
-        warmup_ratio=0.1,
-        neftune_noise_alpha=5,
-        evaluation_strategy="epoch",
-        save_strategy="epoch",
-        load_best_model_at_end=True,
-        metric_for_best_model="eval_loss",
-        fp16=True,
-    ),
-)
-trainer.train()
-```
-Full training script in the model repository.
----
-## Limitations
-- **Not a crisis tool.** Atlas detects acute risk and redirects to human support. It is not equipped for psychiatric emergencies. Do not use it as one.
-- **Small training set.** 721 examples is a targeted intervention. Behavior outside the training distribution follows Qwen2.5-72B-Instruct defaults.
-- **Context window.** Trained on 2048-token sequences. Base model supports 128K. Very long sessions may drift.
-- **No standard benchmarks.** MMLU, HellaSwag, etc. don't measure what this model does. EQ-Bench v3 submission pending.
-- **Not a therapist.** Research and personal use only. Does not diagnose, treat, or replace licensed mental health professionals.
----
-## License
-Derived from Qwen2.5-72B-Instruct. Subject to the [Qwen Research License](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE). LoRA weights and model card released by SILVERTHRONE.
----
-## Citation
-```bibtex
-@misc{atlas72bsvt2026,
-  author    = {SILVERTHRONE},
-  title     = {Atlas-72B-SVT: Emotionally Intelligent Fine-Tuning of Large Language Models},
-  year      = {2026},
-  month     = {February},
-  publisher = {Hugging Face},
-  url       = {https://huggingface.co/SILVERTHRONE/Atlas-72B-SVT}
-}
-```
----
-## Acknowledgments
-- **Qwen Team** — [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct)
-- **Unsloth** — 2x faster fine-tuning framework
-- **Claude (Anthropic)** — Synthetic data generation assistance
-- **EQ-Bench** — Evaluation framework for emotional intelligence
----
-**SILVERTHRONE** — [huggingface.co/SILVERTHRONE](https://huggingface.co/SILVERTHRONE)