Update README.md

README.md

@@ -1,32 +1,42 @@
-# ARC Engine v2.1 - Adaptive Recursive Cognition (Übermenschetien)
-
-![ARC Banner](banner.png)
-
-> *"An 8B that improves itself WITHOUT going insane"*
-
-## 🔥 What is This?
-
-**ARC (Adaptive Recursive Cognition)** is an 8B language model framework that:
-
-1. **Speaks with maximum density** - No filler, pure information
-2. **Controls its own behavior** - CF-HoT 125× repetition detection BEFORE token emission
-3. **Improves itself** - Stable RSI loop with automatic rollback
-4. **Does real work** - Browser automation, email, crypto mining, image generation
-5. **Integrates Claude** - Call Opus 4.5 for brainstorming and complex tasks
-
-### Before vs After
-
-| Prompt | Base Model | ARC Engine |
-|--------|-----------|------------|
-| "hello" | "Hello! I'm here to help you with any questions..." (23 tokens) | "Hello. How can I help?" (5 tokens) |
-| "What is recursion?" | "That's a great question! Recursion is..." (150+ tokens) | "Function calls itself until base case. Stack frames accumulate, unwind." (12 tokens) |
-| "How are you?" | "As an AI, I don't have feelings..." (25 tokens) | "Operational. Ready." (3 tokens) |
-
-**70% improvement in information density. 93% token reduction.**
-
 ---
-
-## 🚀 Quick Start
+license: cc-by-4.0
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- llama
+- dense
+- self-improvement
+- cf-hot
+- representation-engineering
+base_model: NousResearch/Hermes-3-Llama-3.1-8B
+model-index:
+- name: ARC-Base-8B-Condensed
+  results:
+  - task:
+      type: text-generation
+    metrics:
+    - name: Information Density
+      type: custom
+      value: 28.5
+    - name: Token Reduction
+      type: custom
+      value: 57%
+---
+
+# ARC-Base-8B-Condensed
+
+An 8B language model optimized for **information density** and **stable self-improvement**.
+
+## Features
+
+- **CF-HoT 125×**: Repetition detection with 125× class separation
+- **Dense Responses**: 70% improvement in information density
+- **Stable RSI**: Recursive self-improvement with automatic rollback
+- **Full Agentic Stack**: Browser, email, code execution
+
+## Quick Start
 
 ```bash
 git clone https://huggingface.co/LoganResearch/ARC-Base-8B-Condensed
@@ -35,328 +45,76 @@ pip install -r requirements.txt
 python arc_engine_v21_multimedia.py
 ```
 
-**Requirements:** Python 3.11 (3.13 has compatibility issues with diffusers)
-
-```bash
-# If on Python 3.13, downgrade:
-conda install python=3.11 -y
-pip install torch transformers diffusers accelerate pillow pyttsx3 pygame gtts
-```
-
----
-
-## ⭐ NEW IN v2.1
-
-| Command | Description |
-|---------|-------------|
-| `!cfhot` / `!125x` | Toggle 125× repetition detection head ON/OFF |
-| `!rsi15` | Run 15-iteration RSI stress test |
-| `!book` | Toggle book mode (16K tokens) |
-| `!write <topic>` | Write complete books with chapters |
-| `!idea <request>` | Claude-powered extensive brainstorming |
-| `!claude <prompt>` | Direct Claude Opus 4.5 prompting |
-| `!stream` | **Live streaming window** - watch tokens generate! |
-| `!imagine <prompt>` | Generate images with SDXL |
-| `!dalle <prompt>` | Generate images with DALL-E 3 |
-| `!audio` / `!tts` | Toggle text-to-speech output |
-| `!say <text>` | Speak text immediately |
-| `!plot` | Visualize quality history |
-| `!export` / `!import` | Checkpoint packaging |
-| `!benchmark` | Run evaluation suite |
-| `!api` | Start REST API server |
-
----
-
-## 🧠 Core Technology
-
-### 1. CF-HoT 125× Repetition Head
-
-Predicts repetitive behavior from hidden states **BEFORE token emission**:
-
-```
-Positive (repetitive) samples: 0.875 avg activation
-Negative (clean) samples:      0.007 avg activation
-Separation ratio:              125×
-```
-
-Toggle at runtime:
-```
-> !cfhot on    # Load and enable
-> !cfhot off   # Unload to free VRAM
-```
-
-### 2. THE CONDENSATOR
-
-4-stage dense training:
-```
-SFT (50+ examples) → DPO (preference pairs) → RL (density reward) → Checkpoint
-```
-
-### 3. Stable RSI Loop
-
-```
-EVAL → Quality OK? → DONE ✓
-          │ No
-          ▼
-     TRAIN (25 steps)
-          │
-          ▼
-    A/B COMPARE
-          │
-    ┌─────┴─────┐
- Better?     Worse?
-    │           │
-  KEEP      ROLLBACK
-```
-
-**Safeguards:**
-- Multi-metric evaluation (density + coherence + helpfulness)
-- Gibberish detection
-- Automatic rollback on quality drop
-- Emergency stop on 3 consecutive rollbacks
-- Conservative training (LR=2e-6)
-
-### 4. RSI-15 Stress Test
-
-```
-> !rsi15
-```
-Runs 15 iterations of self-improvement with full logging:
-- Pre/post quality per iteration
-- Automatic rollback on degradation
-- Peak quality tracking
-- JSON results saved to `improvement_logs/`
-
----
-
-## 🎬 Multimedia Features
-
-### Live Streaming Window
-```
-> !stream
-```
-Opens a GUI window showing tokens as they generate in real-time.
-
-### Image Generation
-```
-> !imagine a cyberpunk cityscape at sunset
-> !dalle photorealistic portrait of a robot
-> !image view
-```
-
-### Text-to-Speech
-```
-> !audio              # Toggle TTS on/off
-> !audio voices       # List available voices
-> !audio voice 2      # Select voice
-> !say Hello world    # Speak immediately
-```
-
----
-
-## 📚 Book Mode
-
-Generate entire books:
-```
-> !book
-> !write "The Rise of Self-Improving AI"
-Chapters: 10
-Words per chapter: 3000
-```
-Outputs ~30,000 word book with outline, saves progress to `books/`
-
----
+**Requires Python 3.11** (3.13 has diffusers compatibility issues)
 
-## 💡 Idea Mode (Claude Integration)
+## Usage
 
-```
-> !idea how to build a SaaS product --deep
-> !expand "Idea #3: AI-Powered Analytics"
-```
-
-Depths:
-- `--quick`: 5 ideas, 2K tokens
-- (default): 20 ideas, 8K tokens  
-- `--deep`: 30 ideas, 16K tokens
-
-Requires: `export ANTHROPIC_API_KEY="sk-ant-..."`
-
----
-
-## 🛠️ Full Command Reference
-
-### Self-Improvement
-```
-!improve          Run stable self-improvement loop
-!eval             Evaluate current model
-!train <N>        Run N training steps
-!compare          Compare current vs best checkpoint
-!rollback         Rollback to best checkpoint
-!rsi15            15-iteration stress test
-```
-
-### Agentic Tools
-```
-!shell <cmd>      Execute shell command
-!python <code>    Execute Python code
-!read <path>      Read file
-!write <p> <c>    Write file
-!web <query>      Web search
-```
-
-### Browser Automation
-```
-!browse <url>     Open URL
-!click <sel>      Click element
-!type <text>      Type text
-!login <service>  Login to service
-```
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
-### Multimedia
-```
-!stream           Live token window
-!audio            Toggle TTS
-!imagine <p>      Generate image (SDXL)
-!dalle <p>        Generate image (DALL-E)
-!image view       View last image
-```
+model = AutoModelForCausalLM.from_pretrained(
+    "LoganResearch/ARC-Base-8B-Condensed",
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("LoganResearch/ARC-Base-8B-Condensed")
 
-### Modes
-```
-!book             Toggle book mode
-!write <topic>    Write book
-!idea <request>   Generate ideas
-!claude <prompt>  Direct Claude prompt
+prompt = "<|im_start|>user\nWhat is recursion?<|im_end|>\n<|im_start|>assistant\n"
+output = model.generate(tokenizer(prompt, return_tensors="pt").input_ids.cuda(), max_new_tokens=100)
+print(tokenizer.decode(output[0]))
+# Output: "Function calls itself until base case. Stack frames accumulate, unwind."
 ```
 
-### Utilities
-```
-!plot             Plot quality history
-!benchmark        Run evaluation suite
-!export [name]    Export checkpoint
-!import <path>    Import checkpoint
-!learn            Learn from conversation
-!api              Start REST API
-```
+## Key Commands
 
----
+| Command | Description |
+|---------|-------------|
+| `!improve` | Run self-improvement loop |
+| `!eval` | Evaluate model quality |
+| `!cfhot` | Toggle 125× repetition head |
+| `!rsi15` | 15-iteration stress test |
+| `!book` | Extended generation mode |
+| `!stream` | Live token visualization |
 
-## 📊 Metrics
+## Metrics
 
-| Metric | Base Model | ARC Engine | Improvement |
-|--------|-----------|------------|-------------|
+| Metric | Base | ARC | Change |
+|--------|------|-----|--------|
 | Information Density | 17.0 | 28.5 | +67% |
-| Avg Token Count | 150 | 65 | -57% |
-| Filler Phrases | High | ~0 | -95% |
-| CF-HoT Separation | - | 125× | N/A |
+| Avg Tokens | 150 | 65 | -57% |
+| CF-HoT Separation | - | 125× | - |
 
----
+## Architecture
 
-## 📁 Repository Structure
+Built on [Hermes-3-Llama-3.1-8B](https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B) with:
 
-```
-ARC-Base-8B-Condensed/
-├── arc_engine_v21_multimedia.py  # Main engine (6,800+ lines)
-├── the_condensator.py            # Dense training pipeline
-├── train_cfhot_head.py           # CF-HoT head training
-├── requirements.txt              # Dependencies
-├── dense_checkpoints_v2/         # Model checkpoints
-├── cfhot_checkpoints/            # 125× head weights
-├── books/                        # Generated books
-├── images/                       # Generated images
-├── ideas/                        # Generated ideas
-├── improvement_logs/             # RSI logs
-└── exports/                      # Checkpoint packages
-```
+1. **CF-HoT Heads**: Multi-head predictors on hidden states for behavior control
+2. **CONDENSATOR Training**: SFT → DPO → RL pipeline for density
+3. **RSI Loop**: Evaluate → Train → Compare → Keep/Rollback
 
----
-
-## 📋 Requirements
+## Requirements
 
-```txt
+```
 torch>=2.0
 transformers>=4.40.0
-diffusers>=0.27.0
 accelerate
 peft
 bitsandbytes
-chromadb
-sentence-transformers
-pillow
-pyttsx3
-pygame
-gtts
-anthropic
-playwright
 ```
 
-**Install:**
-```bash
-pip install -r requirements.txt
-playwright install firefox
-```
-
----
-
-## 🔧 Configuration
-
-### Claude API (for !idea, !claude)
-```bash
-export ANTHROPIC_API_KEY="sk-ant-..."
-# Or create file:
-echo "sk-ant-..." > .anthropic_key
-```
-
-### DALL-E (for !dalle)
-```bash
-export OPENAI_API_KEY="sk-..."
-```
+See `requirements.txt` for full list.
 
-### Model Paths
-Edit in `arc_engine_v21_multimedia.py`:
-```python
-MODEL_PATH = "/path/to/your/merged-model"
-DENSE_CHECKPOINT = "/path/to/dense_checkpoints_v2/step_100"
-```
-
----
-
-## 📄 Citation
+## Citation
 
 ```bibtex
 @software{arc_engine_2025,
-  title = {ARC Engine: Adaptive Recursive Cognition with CF-HoT 125× Control},
+  title = {ARC-Base-8B-Condensed: Dense Self-Improving Language Model},
   author = {Napolitano, Logan Matthew},
   year = {2025},
-  url = {https://huggingface.co/LoganResearch/ARC-Base-8B-Condensed},
-  license = {CC BY 4.0}
+  url = {https://huggingface.co/LoganResearch/ARC-Base-8B-Condensed}
 }
 ```
 
----
-
-## 📚 Paper
-
-Full research paper: [ARC: Adaptive Recursive Cognition via Contrastive Hidden-State Control](paper/arc_paper.pdf)
-
-**Abstract:** We present ARC, a framework for stable recursive self-improvement combining CF-HoT (125× class separation for repetition detection), THE CONDENSATOR (dense response training), and a robust RSI loop with automatic rollback. The 8B model achieves 70% density improvement on consumer hardware (RTX 3090).
-
----
-
-## ⚠️ Limitations
-
-- Python 3.11 recommended (3.13 has diffusers compatibility issues)
-- English only
-- 8B scale only (larger models untested)
-- May be too terse for some applications
-- SDXL requires ~8GB VRAM
-
----
-
-## 📜 License
-
-**CC BY 4.0** - Use freely, improve upon it, cite if you publish.
-
----
+## License
 
-*"An 8B that improves itself WITHOUT going insane"* 🧠⚡
+CC BY 4.0