Update README.md

README.md

@@ -1,43 +1,213 @@
-```markdown
 # summerMC/TRM-text-v4
 
-## モデル概要
-`TRM-text-v4`は、日本語における高度な論理的推論能力（Reasoning）とコンテキストウィンドウの拡張を目的とした実験的言語モデルです。
-特に、Assistantが回答の前に `<think>` タグを用いて思考プロセスを日本語で展開するように微調整されています。
-
-## 主な特徴
-- **日本語思考プロセス（CoT）**: `<think>` タグを利用したステップバイステップの論理構成を学習。
-- **コンテキスト拡張**: RoPE (Rotary Positional Embedding) のダイナミックスケーリングを導入し、標準的な制限を超えた長い入力への対応を試行。
-- **効率的な学習**: BitsAndBytesによる4-bit量子化（NF4）とLoRA（Low-Rank Adaptation）を組み合わせ、メモリ効率の高いトレーニングを実現。
-
-## 技術仕様
-- **ベースモデル**: summerMC/TRM-text-v4 (MoE Architecture)
-- **学習設定**:
-  - LoRA Rank (r): 16
-  - LoRA Alpha: 32
-  - Target Modules: All Linear layers
-  - Learning Rate: 3e-4
-  - Precision: FP16 with 4-bit quantization
-- **コンテキスト設定**:
-  - RoPE Scaling Factor: 2.0 (Dynamic)
-  - Max Position Embeddings: 2048 (Targeted extension for 5M in architectural planning)
-
-## 使用データセット
-- **Databricks Dolly-15k-ja**: 日本語の指示追従能力向上のため。
-- **Reasoning Dataset (Planned/Subset)**: Magpie/Sakura Reasoningデータセットを参考に、日本語での推論テンプレートを適用。
-
-## 推論方法
-以下のプロンプトテンプレートを使用することで、モデルの思考能力を最大限に引き出すことができます。
+<div align="center">
+
+# TRM-text-v4
+
+Experimental Japanese Reasoning Language Model
+
+</div>
+
+## Overview
+
+**TRM-text-v4** is an experimental Japanese language model developed to explore improved reasoning capabilities and extended context handling.
+
+The model is designed to generate intermediate reasoning traces using `<think>` tags before producing final answers. It combines efficient fine-tuning techniques with experimental approaches to context extension.
+
+> **Status:** Research Preview / Experimental Release
+
+---
+
+## Features
+
+### Japanese Chain-of-Thought Reasoning
+
+TRM-text-v4 is fine-tuned to produce structured reasoning steps in Japanese using `<think>` tags.
+
+Example:
+
+```text
+User: 富士山の標高は？
+Assistant: <think>
+質問は富士山の標高についてである。
+一般的に知られている標高を確認する。
+富士山の標高は3776メートルである。
+</think>
+富士山の標高は3,776メートルです。
+```
+
+---
+
+### Extended Context Experiments
+
+The model incorporates experimental RoPE scaling techniques to investigate longer-context capabilities beyond the original configuration.
+
+Current implementation focuses on practical extensions while evaluating numerical stability and hardware constraints.
+
+---
+
+### Efficient Fine-Tuning
+
+TRM-text-v4 was trained using parameter-efficient fine-tuning methods:
+
+* 4-bit NF4 quantization via BitsAndBytes
+* LoRA adaptation
+* Memory-efficient optimization for consumer GPUs
+
+---
+
+## Model Details
+
+| Item           | Value                                  |
+| -------------- | -------------------------------------- |
+| Model Name     | TRM-text-v4                            |
+| Architecture   | MoE-based Causal Language Model        |
+| Language       | Japanese                               |
+| License        | Apache-2.0 (or specify actual license) |
+| Intended Use   | Research and experimentation           |
+| Release Status | Experimental                           |
+
+---
+
+## Training Configuration
+
+### Fine-Tuning
+
+* LoRA Rank (`r`): **16**
+* LoRA Alpha: **32**
+* Target Modules: **All Linear Layers**
+* Learning Rate: **3e-4**
+* Precision: **FP16**
+* Quantization: **4-bit NF4**
+
+### Context Configuration
+
+* RoPE Scaling: **Dynamic**
+* RoPE Scaling Factor: **2.0**
+* Maximum Position Embeddings: **2048**
+
+> Long-context support beyond this configuration remains under investigation and should be considered experimental.
+
+---
+
+## Datasets
+
+### Instruction Tuning
+
+* **Databricks Dolly-15k-ja**
+
+  * Used to improve Japanese instruction-following ability.
+
+### Reasoning Data
+
+* Reasoning templates inspired by datasets such as:
+
+  * Magpie
+  * Sakura Reasoning
+
+> Portions of the reasoning pipeline are experimental and may evolve in future releases.
+
+---
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_id = "summerMC/TRM-text-v4"
+
+tokenizer = AutoTokenizer.from_pretrained(
+    model_id,
+    trust_remote_code=True,
+)
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    trust_remote_code=True,
+)
+
+prompt = """User: 人工知能とは？
+Assistant: <think>
+"""
+
+inputs = tokenizer(prompt, return_tensors="pt")
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=256,
+    do_sample=True,
+    temperature=0.7,
+    top_p=0.9,
+)
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+---
+
+## Prompt Format
+
+Recommended prompt template:
 
 ```text
-User: [質問内容]
+User: [Question]
 Assistant: <think>
-[ここに思考プロセスが展開される]
+[Reasoning Process]
 </think>
-[最終的な回答]
+[Final Answer]
 ```
 
-## 現在の課題とステータス
-- **言語の一貫性**: 推論プロセスにおいて一部英語の技術用語が混ざる傾向があり、さらなるデータセットの調整が必要。
-- **コンテキストの限界**: 理論上の拡張（5Mトークン等）に対し、実際のハードウェア（VRAM）制約と数値的安定性のバランスを最適化中。
-```
+---
+
+## Limitations
+
+* Reasoning traces may occasionally contain mixed Japanese and English terminology.
+* Generated reasoning processes are not guaranteed to reflect the model's actual internal computation.
+* Long-context behavior beyond validated ranges may exhibit instability.
+* The model may generate incorrect or fabricated information and should not be used in high-risk domains without human verification.
+
+---
+
+## Intended Use
+
+Recommended uses:
+
+* Research on Japanese reasoning language models
+* Prompt engineering experiments
+* Educational demonstrations
+* Evaluation of Chain-of-Thought prompting strategies
+
+Not recommended for:
+
+* Medical decision-making
+* Legal advice
+* Financial advice
+* Safety-critical applications
+
+---
+
+## Citation
+
+```bibtex
+@misc{summermc_trm_text_v4,
+  title        = {TRM-text-v4: Experimental Japanese Reasoning Language Model},
+  author       = {summerMC},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/summerMC/TRM-text-v4}}
+}
+```
+
+---
+
+## Acknowledgements
+
+TRM-text-v4 builds upon the open-source ecosystem, including:
+
+* Hugging Face Transformers
+* PEFT
+* BitsAndBytes
+* Databricks Dolly
+* The broader open-source LLM community
+
+Special thanks to all researchers and contributors advancing efficient Japanese language modeling.