Update README.md

README.md

@@ -1,95 +1,117 @@
-# TRM-HRM
+# TRM-text
 
-Attention-free Hierarchical Recursive Model (TRM-HRM)
+TRM-text is an attention-free language model based on a Tiny Recursive Model (TRM) architecture.
 
-TRM-HRM is an experimental language model architecture that completely removes self-attention and replaces it with recursive hierarchical computation using dilated depthwise convolutions.
+Unlike Transformer-based language models, TRM-text removes self-attention entirely and replaces it with recursive computation built from causal dilated depthwise convolutions, hierarchical latent refinement, and parameter reuse.
 
-The model is inspired by Hierarchical Reasoning Models (HRM) and Tiny Recursive Models (TRM), while remaining fully autoregressive and compatible with the Hugging Face ecosystem.
+The goal of TRM-text is to investigate whether recursive neural computation can provide competitive language modeling performance while dramatically reducing computational cost.
 
-## Architecture
+---
 
-TRM-HRM consists of three recursive processing levels:
+# Overview
 
-* Low-level reasoning module
-* Mid-level reasoning module
-* High-level reasoning module
+TRM-text explores a different scaling path from Transformers.
 
-Each level is implemented using:
+Instead of increasing attention heads and context interactions, TRM-text repeatedly refines hidden representations using a hierarchy of recursive processing blocks.
 
-* RMSNorm
+Key properties:
+
+* Attention-free
+* Autoregressive language modeling
+* Recursive computation
+* Hierarchical latent refinement
 * RoPE positional encoding
-* Causal dilated depthwise convolutions
-* SwiGLU feed-forward layers
-* Recursive parameter reuse
+* Dilated depthwise convolution mixer
+* Hugging Face compatible
+* safetensors support
 
-No attention layers are used anywhere in the network.
+---
 
-### Hierarchical Loop
+# Architecture
 
 ```text
-Input Tokens
-      ↓
-Low TRM
-      ↓
-Mid TRM
-      ↓
-High TRM
-      ↓
-Feedback
-      ↓
-Repeat N cycles
-      ↓
+Tokens
+  │
+  ▼
+Embedding
+  │
+  ▼
+Low-Level TRM
+  │
+  ▼
+Mid-Level TRM
+  │
+  ▼
+High-Level TRM
+  │
+  ▼
+Recursive Feedback
+  │
+  ▼
 LM Head
 ```
 
-The hierarchy exchanges information through learned projection layers:
-
-```text
-Low  → Mid
-Mid  → High
-High → Mid
-Mid  → Low
-```
+Each recursive block contains:
 
-allowing iterative refinement of latent representations.
+* RMSNorm
+* RoPE
+* Causal Dilated Depthwise Convolution
+* SwiGLU Feed Forward Network
+* Residual Recurrence
 
-## Key Features
+No self-attention layers are used.
 
-* 100% attention-free
-* Autoregressive language modeling
-* Hierarchical recursive reasoning
-* RoPE positional encoding
-* Dilated depthwise convolution mixer
-* Hugging Face compatible
-* safetensors support
-* Efficient on consumer GPUs
+---
 
-## Model Configuration
+# Model Configuration
 
-Current released checkpoint:
+Current release:
 
 ```text
 Parameters: ~15M
 
-dim          = 256
-hidden_dim   = 512
+dim = 256
+hidden_dim = 512
 
-low_steps    = 6
-mid_steps    = 3
-high_steps   = 2
+low_steps = 6
+mid_steps = 3
+high_steps = 2
 
-cycles        = 3
+cycles = 3
 
-kernel_size   = 5
+kernel_size = 5
 
 low_dilations  = [1,2,4,8]
 mid_dilations  = [2,4,8,16]
 high_dilations = [4,8,16,32]
 ```
 
-## Training
+---
+
+# Compute Efficiency
+
+Relative training cost:
+
+| Architecture | Relative Cost |
+| ------------ | ------------: |
+| Transformer  |          1200 |
+| HRM          |           100 |
+| TRM-text     |             1 |
 
-### Base Pretraining
+These values represent relative compute requirements under the experimental scaling assumptions used during development.
+
+The objective of TRM-text is to maximize efficiency through:
+
+* parameter reuse
+* recursive computation
+* hierarchical refinement
+* elimination of attention operations
+
+---
+
+# Training
+
+## Base Pretraining
 
 Dataset:
 
@@ -109,7 +131,9 @@ Objective:
 Causal Language Modeling
 ```
 
-### SFT
+---
+
+# Instruction Tuning
 
 Dataset:
 
@@ -127,36 +151,40 @@ Format:
 ...
 ```
 
-## Loading
+---
+
+# Loading
 
 ```python
 from transformers import AutoTokenizer
 from transformers import AutoModelForCausalLM
 
 tokenizer = AutoTokenizer.from_pretrained(
-    "your-org/trm-hrm",
-    trust_remote_code=True,
+    "summerMC/TRM-text",
+    trust_remote_code=True
 )
 
 model = AutoModelForCausalLM.from_pretrained(
-    "your-org/trm-hrm",
-    trust_remote_code=True,
+    "summerMC/TRM-text",
+    trust_remote_code=True
 )
 ```
 
-## Generation
+---
+
+# Inference
 
 ```python
 prompt = """
 ### Instruction:
-Explain artificial intelligence.
+Explain artificial intelligence in simple terms.
 
 ### Response:
 """
 
 inputs = tokenizer(
     prompt,
-    return_tensors="pt",
+    return_tensors="pt"
 )
 
 outputs = model.generate(
@@ -164,51 +192,78 @@ outputs = model.generate(
     max_new_tokens=128,
     do_sample=True,
     temperature=0.7,
-    top_k=40,
+    top_k=40
 )
 
 print(
     tokenizer.decode(
         outputs[0],
-        skip_special_tokens=True,
+        skip_special_tokens=True
     )
 )
 ```
 
-## Research Goals
+---
+
+# Research Motivation
 
-TRM-HRM explores whether hierarchical recursive computation can replace self-attention in language modeling while maintaining competitive reasoning capability and improved efficiency.
+TRM-text investigates whether recursive neural systems can replace attention mechanisms in language modeling.
 
-The project investigates:
+Research directions:
 
-* Recursive reasoning
-* Multi-timescale computation
-* Hierarchical latent processing
-* Attention-free language models
-* Efficient inference architectures
+* recursive reasoning
+* hierarchical computation
+* efficient language models
+* attention-free architectures
+* low-cost scaling laws
 
-## Limitations
+---
 
-Current release is an early experimental checkpoint.
+# Limitations
+
+Current checkpoint is experimental.
 
 Known limitations:
 
-* Small parameter count (~15M)
-* Limited instruction tuning
-* Lower benchmark performance than modern LLMs
-* Experimental Hugging Face integration
-* Research-only release
+* small parameter count
+* limited instruction tuning
+* lower capability than modern frontier models
+* research-focused implementation
+* benchmark coverage still limited
+
+---
+
+# Intended Use
+
+TRM-text is intended for:
+
+* language model research
+* efficient architecture experimentation
+* recursive computation studies
+* attention-free modeling research
 
-## License
+Not intended for:
+
+* safety-critical systems
+* medical decision making
+* legal advice
+* financial advice
+
+---
+
+# License
 
 Apache-2.0
 
-## Citation
+---
+
+# Citation
 
 ```bibtex
-@software{trm_hrm,
-  title={TRM-HRM: Attention-Free Hierarchical Recursive Language Model},
-  author={Your Name},
-  year={2026}
+@software{trm_text_2026,
+  title={TRM-text: Attention-Free Recursive Language Modeling},
+  author={summerMC},
+  year={2026},
+  url={https://huggingface.co/summerMC/TRM-text}
 }
 ```