--- language: en license: apache-2.0 base_model: Qwen/Qwen2.5-1.5B-Instruct tags: - miniOneRec - generative-recommendation - sequential-recommendation - amazon - grpo - gdpo - semantic-id - rq-vae datasets: - amazon-reviews-2023 metrics: - hit-rate - ndcg pipeline_tag: text-generation --- # MiniOneRec-1.5B-SFT-GDPO **Generative Recommendation Model** — fine-tuned from [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) on Amazon Industrial & Scientific reviews. 📖 **GitHub**: [YuyaoFan/MiniOneRec-1.5B](https://github.com/YuyaoFan/MiniOneRec-1.5B) ## Repository Contents | Directory | Description | Usage | |-----------|-------------|-------| | [`sft_model/`](./sft_model) | RQ-VAE + SFT model weights | `AutoModelForCausalLM.from_pretrained()` | | [`gdpo_best_checkpoint-1155/`](./gdpo_best_checkpoint-1155) | Best GDPO RL checkpoint (step 1155) | Best RL model for evaluation | | [`data/`](./data) | Dataset files (.npy, .npz, .csv, .inter) | Required for training reproduction | ## Model Description This model transforms item recommendation into a sequence generation task using **Semantic IDs (SIDs)**: ``` User History → Qwen2.5-1.5B → Semantic ID Sequence → Item Recommendation ``` ### Pipeline 1. **SID Construction**: RQ-VAE compresses item text embeddings into 3-level discrete codes (256 codes × 3 levels = 16.7M capacity) 2. **SFT**: Full-parameter fine-tuning on Amazon Industrial & Scientific interaction sequences 3. **RL (GRPO/GDPO)**: Reinforcement learning with constrained beam search and ranking-aware rewards ### Key Details - **Base Model**: Qwen2.5-1.5B-Instruct - **SID Method**: RQ-VAE (M=3 levels, K=256 codes/level, e_dim=2560) - **Vocabulary**: 151,665 base + 521 SID tokens = 152,186 total - **Dataset**: Amazon Industrial & Scientific (36,259 train / 4,532 valid / 4,533 test) - **Items**: 3,686 unique items - **Hardware**: Single GPU ≥ 48GB VRAM ## Performance Evaluation on Amazon Industrial & Scientific test set (beam=50): ### SFT Model | HR@3 | NDCG@3 | HR@5 | NDCG@5 | HR@10 | NDCG@10 | |------|--------|------|--------|-------|---------| | 0.0904 | 0.0792 | 0.1061 | 0.0856 | 0.1337 | 0.0945 | ### Best RL Checkpoints | Method | Step | HR@5 | NDCG@5 | HR@10 | NDCG@10 | vs SFT | |--------|------|------|--------|-------|---------|--------| | GRPO | 1320 | **0.1136** | **0.0926** | **0.1379** | **0.1005** | **+6.4%** | | GDPO | 1155 | 0.1050 | 0.0880 | 0.1253 | 0.0944 | -0.1% | > **GRPO** significantly outperforms SFT (+6.4% NDCG@10). **GDPO** shows limited gains in this sparse-reward recommendation setting. ### Comparison with Baselines | Method | HR@5 | NDCG@10 | |--------|------|---------| | SASRec | 0.0909 | 0.0806 | | TIGER | 0.1010 | 0.0908 | | D3 | 0.1213 | 0.1082 | | MiniOneRec (7B, paper) | 0.1321 | 0.1167 | | **Ours (1.5B, GRPO)** | **0.1136** | **0.1005** | ## Usage ### Inference ```python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "onesfour/MiniOneRec-1.5B-SFT-GDPO", subfolder="sft_model", # or "gdpo_best_checkpoint-1155" torch_dtype="auto", device_map="auto", trust_remote_code=True, ) tokenizer = AutoTokenizer.from_pretrained( "onesfour/MiniOneRec-1.5B-SFT-GDPO", subfolder="sft_model", trust_remote_code=True, ) prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: Can you predict the next possible item that the user may expect? ### User Input: The user has interacted with items , in chronological order. Can you predict the next possible item that the user may expect? ### Response: """ inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=10, num_beams=50) predicted = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True) print(f"Predicted SID: {predicted}") ``` ### SID Format Items are represented as 3-level Semantic IDs with an optional deduplication suffix: - ``: Level 0 code (0-255) - ``: Level 1 code (0-255) - ``: Level 2 code (0-255) - ``: Deduplication suffix (if collision exists) Example: `` represents a unique item in the catalog. ### Item Lookup To map predicted SIDs back to item titles, use `data/Amazon/index/Industrial_and_Scientific.item.json` and `data/Amazon/info/Industrial_and_Scientific_5_2016-10-2018-11.txt` from the repository. ## Requirements ``` transformers>=4.57.1 torch>=2.6.0 ``` ## Training Reproduction See [YuyaoFan/MiniOneRec-1.5B](https://github.com/YuyaoFan/MiniOneRec-1.5B) for full source code, scripts, and documentation. Quick start: ```bash git clone https://github.com/YuyaoFan/MiniOneRec-1.5B.git cd MiniOneRec-1.5B # Download data files from this HF repo → data/Amazon/ # Download model from this HF repo/sft_model → output/sft/.../final_checkpoint/ conda create -n MiniOneRec python=3.11 -y && conda activate MiniOneRec bash scripts/install_deps.sh bash scripts/run_rqvae_sft_grpo.sh ``` ## Limitations - **Domain Specific**: Trained exclusively on Amazon Industrial & Scientific (3,686 items) - **Cold Start**: Cannot generate SIDs for items outside the trained catalog - **Single Positive**: Each test sample has only one ground-truth item (HR@K = Recall@K) - **English Only**: Item descriptions are English-only ## Citation ```bibtex @misc{kong2025minionerec, title={MiniOneRec: An Open-Source Framework for Scaling Generative Recommendation}, author={Xiaoyu Kong and Leheng Sheng and Junfei Tan and Yuxin Chen and Jiancan Wu and An Zhang and Xiang Wang and Xiangnan He}, year={2025}, eprint={2510.24431}, archivePrefix={arXiv}, primaryClass={cs.IR}, } @misc{liu2026gdpo, title={GDPO: Group reward-Decoupled Normalization Policy Optimization}, author={Shih-Yang Liu et al.}, year={2026}, eprint={2601.05242}, archivePrefix={arXiv}, } ``` ## License This model inherits Apache 2.0 from Qwen2.5-1.5B-Instruct. ## Acknowledgments - [AkaliKong/MiniOneRec](https://github.com/AkaliKong/MiniOneRec) — Original MiniOneRec framework - [SuFame920/MiniOneRec-Reproduction](https://github.com/SuFame920/MiniOneRec-Reproduction) — Reproduction reference - [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) — Base model by Alibaba Cloud ---