Upload README.md with huggingface_hub

README.md

@@ -1,3 +1,338 @@
 ---
-license: apache-2.0
+language:
+- en
+- zh
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- recommendation
+- generative-recommendation
+- reasoning
+- itemic-token
+- qwen3
+- pretraining
 ---
+
+# OneReason
+
+Reasoning Foundation Models for Generative Recommendation
+
+[Paper](#citation) | [Model Zoo](#model-zoo) | [Quick Start](#quick-start) | [Citation](#citation)
+
+<p align="center">
+  <img src="assert/main.png" alt="OneReason training and evaluation pipeline" width="90%">
+</p>
+
+<p align="center"><b>Figure 1:</b> The pre-training, SFT, RL, and reasoning-evaluation pipeline of OneReason.</p>
+
+## Introduction
+
+OneReason is a recommendation foundation model that connects large language models with generative recommender systems. It represents items as compact **itemic tokens** and trains the model to align itemic-token semantics with natural language, user behavior, and recommendation-oriented reasoning traces.
+
+The OneReason training stack contains three stages:
+
+- **Pre-training:** builds itemic-token perception through four-granularity itemic-text alignment data, covering token-, item-, relational-, and user-level signals.
+- **Supervised Fine-Tuning (SFT):** teaches recommendation cognition with coarse-to-fine Chain-of-Thought (CoT) traces over user profiles, behavior histories, and itemic-token evidence.
+- **Reinforcement Learning (RL):** uses a specialize-then-unify recipe to improve thinking-mode recommendation while balancing performance across multiple recommendation domains.
+
+This repository currently releases the **OneReason-0.8B Pretrain checkpoint**. We will continue to release OneReason-0.8B SFT/RL checkpoints and the OneReason-8B series.
+
+## News
+
+- **[2026.06]** OneReason-0.8B Pretrain checkpoint is released.
+- **Coming soon:** OneReason-0.8B SFT checkpoint.
+- **Coming soon:** OneReason-0.8B RL checkpoint.
+- **Coming soon:** OneReason-8B checkpoints.
+
+## Model Zoo
+
+| Model | Stage | Parameters | Status | Description |
+|---|---:|---:|---|---|
+| OneReason-0.8B-Pretrain | Pre-training | 0.8B | Released | Foundation checkpoint after itemic-text alignment pre-training. Suitable for research, continued pre-training, and downstream SFT. |
+| OneReason-0.8B-SFT | SFT | 0.8B | Coming soon | Instruction-tuned checkpoint with recommendation perception, derivation, evolution, and recommendation supervision. |
+| OneReason-0.8B-RL | RL | 0.8B | Coming soon | Post-trained checkpoint optimized for recommendation-oriented reasoning. |
+| OneReason-8B | Pretrain/SFT/RL | 8B | Coming soon | Larger OneReason model family with stronger reasoning and recommendation performance. |
+
+## Method Overview
+
+### Itemic Tokens
+
+OneReason represents each item with one domain-aware begin token and three hierarchical sub-tokens:
+
+```text
+<|domain_begin|><s_a_xxxx><s_b_xxxx><s_c_xxxx>
+```
+
+Supported recommendation domains include:
+
+| Domain | Begin token | Example |
+|---|---|---|
+| Short video | <code>&lt;&#124;video_begin&#124;&gt;</code> | <code>&lt;&#124;video_begin&#124;&gt;&lt;s_a_3334&gt;&lt;s_b_4643&gt;&lt;s_c_625&gt;</code> |
+| E-commerce product | <code>&lt;&#124;prod_begin&#124;&gt;</code> | <code>&lt;&#124;prod_begin&#124;&gt;&lt;s_a_2147&gt;&lt;s_b_7978&gt;&lt;s_c_5031&gt;</code> |
+| Advertisement | <code>&lt;&#124;ad_begin&#124;&gt;</code> | <code>&lt;&#124;ad_begin&#124;&gt;&lt;s_a_7939&gt;&lt;s_b_6234&gt;&lt;s_c_4978&gt;</code> |
+| Live streaming | <code>&lt;&#124;living_begin&#124;&gt;</code> | <code>&lt;&#124;living_begin&#124;&gt;&lt;s_a_4515&gt;&lt;s_b_6234&gt;&lt;s_c_6278&gt;</code> |
+| General multimodal item | <code>&lt;&#124;sid_begin&#124;&gt;</code> | <code>&lt;&#124;sid_begin&#124;&gt;&lt;s_a_340&gt;&lt;s_b_6566&gt;&lt;s_c_5603&gt;</code> |
+
+Each itemic token sequence is produced by a three-layer codebook, where each layer contains 8192 codes. The released checkpoint can process these itemic-token strings through its tokenizer. Mapping raw items to itemic tokens, or mapping generated itemic tokens back to real item IDs, requires the corresponding itemic tokenizer and item catalog.
+
+### Pre-training Data Design
+
+OneReason pre-training uses **578B tokens** to align itemic-token and text-token semantic spaces. The recommendation part follows a four-granularity corpus design:
+
+- **Token granularity:** aligns individual and compositional sub-token semantics.
+- **Item granularity:** aligns complete itemic patterns with natural-language captions and multi-perspective item QA.
+- **Relational granularity:** injects item-to-item collaborative relations with natural-language transition explanations.
+- **User granularity:** models user behavior sequences with domain-grouped and chronologically interleaved itemic-text formats.
+
+General-domain text and multimodal corpora are mixed in to preserve instruction-following, reasoning, code, math, and broad semantic capabilities while injecting recommendation-specific knowledge.
+
+### Training Recipe
+
+The pre-training recipe contains three stages:
+
+| Stage | Trainable parameters | Token budget | Purpose |
+|---|---|---:|---|
+| Stage 1 | Extended vocabulary + LM head | 110B | Warm up newly introduced itemic-token embeddings. |
+| Stage 2 | All parameters | 449B | Inject four-granularity recommendation knowledge. |
+| Stage 3 | All parameters | 19B | Extend long-context user behavior modeling. |
+
+## OneReason-Bench
+
+OneReason is evaluated with **OneReason-Bench**, a reasoning-oriented recommendation benchmark organized into four layers:
+
+| Layer | Capability | Representative tasks |
+|---|---|---|
+| R0: Perception | Ground itemic tokens into semantic content. | Item understanding, itemic pattern grounding, item QA. |
+| R1: Derivation | Reason over item-to-item relations. | Item2Item relation derivation. |
+| R2: Evolution | Model user interests as temporal processes. | Evolution action selection, topic generation, direct evolution generation. |
+| R3: Recommendation | Combine perception, derivation, and evolution for recommendation. | Single-domain and cross-domain recommendation. |
+
+## Performance
+
+The released **OneReason-0.8B-Pretrain** checkpoint is the foundation checkpoint before SFT/RL. It is designed to provide strong itemic-token perception and a good initialization for downstream recommendation tuning.
+
+The tables below report the full OneReason-8B system results from the technical report. We will update this model card with checkpoint-specific numbers as the OneReason-0.8B SFT/RL and OneReason-8B checkpoints become available.
+
+<p align="center">
+  <img src="assert/rader.png" alt="OneReason performance overview and thinking-mode gains" width="95%">
+</p>
+
+<p align="center"><b>Figure 2:</b> Performance overview of OneReason-8B. The radar chart summarizes general, perception, derivation, evolution, and recommendation capabilities; the bar charts show thinking-mode gains and the effect of thinking-data supervision.</p>
+
+### Results on Cross-Domain Recommendation
+
+Cross-domain recommendation results are reported in percentage. Best results are **bolded**; second-best results are <u>underlined</u>.
+
+| Category | Model | C-Video Pass@64 | C-Video Recall@64 | C-Product Pass@64 | C-Product Recall@64 | C-Ad Pass@64 | C-Ad Recall@64 | C-Live Pass@64 | C-Live Recall@64 |
+|---|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| ID-Based | SASRec | 0.03 | 0.01 | 0.31 | 0.25 | 1.04 | 0.37 | 1.76 | 0.40 |
+| ID-Based | HSTU | 0.10 | 0.01 | 0.32 | 0.24 | 2.79 | 0.78 | 2.32 | 2.14 |
+| Text-Based | Qwen3-8B | 0.05 | 0.01 | 0.15 | 0.12 | 0.48 | 0.09 | 2.10 | 1.85 |
+| Text-Based | Qwen3-32B | 0.33 | 0.03 | 0.84 | 0.63 | 1.21 | 0.30 | 5.64 | 5.10 |
+| Text-Based | Qwen3-235B-A22B | 0.24 | 0.02 | 0.64 | 0.49 | 0.77 | 0.19 | 5.10 | 4.66 |
+| Text-Based | Deepseek-V3.2 | 0.11 | 0.01 | 0.38 | 0.31 | 0.62 | 0.13 | 3.46 | 3.12 |
+| Text-Based | Claude-Opus-4.6 | 0.14 | 0.01 | 0.23 | 0.17 | 0.50 | 0.11 | 3.02 | 2.66 |
+| Text-Based | Gemini-3-Preview | 0.29 | 0.03 | 0.74 | 0.59 | 1.22 | 0.27 | 3.92 | 3.44 |
+| Text-Based | GPT-4o-mini | 0.19 | 0.02 | 0.73 | 0.55 | 1.21 | 0.28 | 4.01 | 3.57 |
+| Text-Based | GPT-5.4 | 0.24 | 0.02 | 1.43 | 1.15 | 1.64 | 0.43 | 7.20 | 6.38 |
+| Itemic Token-Based | TIGER | 0.88 | 0.07 | 0.21 | 0.17 | 7.65 | 2.39 | 2.32 | 1.78 |
+| Itemic Token-Based | LC-Rec-SFT-Only-8B | 0.22 | 0.02 | 0.06 | 0.05 | 2.83 | 0.67 | 0.89 | 0.71 |
+| Itemic Token-Based | LC-Rec-SFT-Only-14B | 0.20 | 0.01 | 1.03 | 0.73 | 5.99 | 1.94 | 3.76 | 3.09 |
+| Itemic Token-Based | LC-Rec-PT-SFT-8B | 1.49 | 0.13 | 3.95 | 3.00 | 15.85 | 6.55 | 19.32 | 16.70 |
+| Itemic Token-Based | OneReason SFT non-thinking | 1.33 | 0.11 | 3.94 | 2.96 | 15.73 | 6.49 | 18.05 | 15.52 |
+| Itemic Token-Based | OneReason SFT thinking | 0.71 | 0.06 | 2.18 | 1.65 | 9.16 | 3.41 | 16.43 | 14.32 |
+| Itemic Token-Based | OneReason RFT non-thinking | <u>2.08</u> | <u>0.19</u> | <u>5.20</u> | <u>3.96</u> | <u>17.56</u> | <u>7.26</u> | <u>21.01</u> | <u>18.17</u> |
+| Itemic Token-Based | OneReason RFT thinking | **2.41** | **0.24** | **5.47** | **4.19** | **17.78** | **7.50** | **21.10** | **18.35** |
+
+### Results on R0-R2 Reasoning Tasks
+
+R0-R2 results on OneReason-Bench are reported in percentage. For R0 tasks, results are macro-averaged over all domains. Grounding is reported by Pass@64.
+
+| Category | Model | R0 Item Und. | R0 Ground. | R0 QA | R1 I2I | R2 Select. | R2 Topic Gen. | R2 Direct Gen. |
+|---|---|---:|---:|---:|---:|---:|---:|---:|
+| Text-Based | Qwen3-8B | - | - | - | - | 40.70 | 25.49 | 8.60 |
+| Text-Based | Qwen3-32B | - | - | - | - | 51.96 | 28.05 | 7.73 |
+| Text-Based | Deepseek-V3.2 | - | - | - | - | <u>57.18</u> | 27.13 | 11.32 |
+| Text-Based | Claude-Opus-4.6 | - | - | - | - | 56.84 | 17.16 | 13.46 |
+| Text-Based | Gemini-3-Preview | - | - | - | - | 56.83 | 33.68 | 14.76 |
+| Text-Based | GPT-5.4 | - | - | - | - | **58.92** | **41.41** | 17.61 |
+| Itemic Token-Based | LC-Rec-SFT-Only-8B | 22.98 | 0.00 | 0.40 | 3.43 | 0.00 | 0.00 | 0.00 |
+| Itemic Token-Based | LC-Rec-SFT-Only-14B | 26.48 | 0.00 | 56.45 | 16.21 | 0.00 | 0.00 | 0.00 |
+| Itemic Token-Based | LC-Rec-PT-SFT-8B | 35.41 | <u>5.21</u> | 63.90 | 25.54 | 3.32 | 8.60 | 4.46 |
+| Itemic Token-Based | OneReason SFT non-thinking | <u>36.84</u> | 3.95 | <u>66.55</u> | <u>28.36</u> | 35.07 | 33.87 | 15.42 |
+| Itemic Token-Based | OneReason SFT thinking | **36.91** | 1.06 | 64.60 | 23.88 | 32.18 | 31.60 | 14.31 |
+| Itemic Token-Based | OneReason RFT non-thinking | 36.82 | **5.24** | **67.25** | 23.99 | 38.92 | 39.33 | <u>20.31</u> |
+| Itemic Token-Based | OneReason RFT thinking | 36.78 | 1.35 | 65.65 | **28.60** | 42.42 | <u>39.57</u> | **21.23** |
+
+## Quick Start
+
+Install dependencies:
+
+```bash
+pip install "transformers>=4.51.0" accelerate safetensors torch
+```
+
+Load the model:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "OpenOneRec/OneReason-0.8B-Pretrain"  # or the local path to this repository
+
+tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto",
+    trust_remote_code=True,
+)
+```
+
+### Item Understanding Example
+
+```python
+prompt = "<|prod_begin|><s_a_1183><s_b_746><s_c_5290>，这个商品卖的是什么？ /no_think"
+
+messages = [{"role": "user", "content": prompt}]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+
+inputs = tokenizer([text], return_tensors="pt").to(model.device)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    do_sample=True,
+    top_p=0.95,
+    temperature=0.7,
+)
+
+response_ids = outputs[0][len(inputs.input_ids[0]):]
+print(tokenizer.decode(response_ids, skip_special_tokens=True))
+```
+
+Expected response:
+
+```
+这是一款便携式的可折叠蓝牙键盘，专为移动办公和旅行设计。该商品采用超薄设计，重量仅约200克，方便放入背包或手提包中。键盘支持蓝牙5.0连接，兼容iOS、Android和Windows系统，可同时连接最多3台设备，并支持一键切换。键盘按键采用剪刀脚结构，提供舒适的打字体验。内置锂电池，续航时间可达30天。适合需要经常出差或在不同设备间切换使用的用户。
+```
+
+### Itemic Token Grounding Example
+
+```python
+prompt = (
+    "根据描述生成短视频token：一个关于超市货架上的方便面种类及消费者在选择时遇到的困惑的视频。"
+    "这段视频记录了一家超市的方便面货架，展示了各种不同口味的方便面。/no_think"
+)
+
+messages = [{"role": "user", "content": prompt}]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+
+inputs = tokenizer([text], return_tensors="pt").to(model.device)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=64,
+    do_sample=False,
+)
+
+response_ids = outputs[0][len(inputs.input_ids[0]):]
+print(tokenizer.decode(response_ids, skip_special_tokens=True))
+```
+
+Expected response:
+
+```
+<|video_begin|><s_a_5820><s_b_908><s_c_1352>
+```
+
+### Recommendation-Style Example
+
+The following prompt format illustrates how user profiles and behavior histories can be used for recommendation. For production deployment, you should use your own itemic tokenizer, item catalog, user behavior schema, and candidate decoding strategy.
+
+```python
+prompt = (
+    "参考以下用户信息：41-49 岁河北男性用户偏好短剧中的复仇商战题材，热衷于象棋、民族风情及民生资讯类短视频，"
+    "常关注憨豆等直播内容并倾向于消费休闲模拟经营游戏。"
+    "这个用户看过<|ad_begin|><s_a_7939><s_b_6234><s_c_4978>, "
+    "<|ad_begin|><s_a_5673><s_b_6234><s_c_1614>, "
+    "<|ad_begin|><s_a_3578><s_b_3009><s_c_3363>这些广告，"
+    "该用户最近可能感兴趣的视频有哪些？ /no_think"
+)
+
+messages = [{"role": "user", "content": prompt}]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+
+inputs = tokenizer([text], return_tensors="pt").to(model.device)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=256,
+    do_sample=True,
+    top_p=0.95,
+    temperature=0.7,
+)
+
+response_ids = outputs[0][len(inputs.input_ids[0]):]
+print(tokenizer.decode(response_ids, skip_special_tokens=True))
+```
+
+Expected response (example output):
+
+```
+[
+  '<|video_begin|><s_a_7830><s_b_3006><s_c_6390>',
+  '<|video_begin|><s_a_5910><s_b_5272><s_c_6222>',
+  '<|video_begin|><s_a_5910><s_b_6379><s_c_4512>',
+  ...
+  '<|video_begin|><s_a_2015><s_b_6234><s_c_88>'
+]
+```
+
+## Intended Use
+
+The OneReason-0.8B Pretrain checkpoint is intended for:
+
+- Research on generative recommendation and recommendation foundation models.
+- Continued pre-training or SFT on new recommendation domains.
+- Itemic-token perception studies, including item understanding and itemic-token grounding.
+- Building downstream recommendation models that combine user profiles, behavior histories, and itemic-token representations.
+
+For best recommendation reasoning performance, we recommend using future SFT/RL checkpoints once released, or fine-tuning this pretrain checkpoint on task-specific supervised data.
+
+## Limitations
+
+- This release is a **pre-training checkpoint**, not the final SFT/RL reasoning model.
+- Direct recommendation quality depends on the itemic tokenizer, item catalog, user history format, and decoding strategy.
+- Generated itemic tokens must be validated against the target item catalog before being used as item IDs.
+- The model may generate invalid, stale, or unsupported itemic-token sequences if the prompt distribution differs significantly from training data.
+- The checkpoint is released for research and should not be used for high-stakes personalization without careful evaluation, filtering, and privacy review.
+
+## Citation
+
+If you find OneReason useful, please cite our technical report. The official BibTeX will be updated after the report is publicly available.
+
+```bibtex
+@article{onereason2026,
+  title   = {OneReason Technical Report},
+  author  = {OneRec Team},
+  journal = {Technical Report},
+  year    = {2026}
+}
+```
+
+## License
+
+Please refer to the license file in this repository for the terms governing the model weights and any accompanying code or assets.
+
+## Acknowledgements
+
+OneReason builds on the open-source LLM and recommendation ecosystem. We thank the Qwen, OpenOneRec, PyTorch, Transformers, and distributed training communities for their foundational contributions.