Prompt word input is always overwritten by default input, always talking to oneself

README.md

@@ -21,21 +21,16 @@ Phi-4-mini-flash-reasoning is a lightweight open model built upon synthetic data
 The model belongs to the Phi-4 model family and supports 64K token context length. 
  
 📰 [Phi-4-mini-flash-reasoning Blog](https://azure.microsoft.com/en-us/blog/reasoning-reimagined-introducing-phi-4-mini-flash-reasoning/) <br>
-📖 [Phi-4-mini-flash-reasoning Paper](https://arxiv.org/abs/2507.06607) | [HF Paper](https://huggingface.co/papers/2507.06607) <br>
-📚 [Training Codebase](https://github.com/microsoft/ArchScale) <br>
+📖 [Phi-4-mini-flash-reasoning Paper](https://aka.ms/flashreasoning-paper) | [HF Paper](https://huggingface.co/papers/2507.06607) <br>
 👩‍🍳 [Phi Cookbook](https://github.com/microsoft/PhiCookBook) <br>
 🏡 [Phi Portal](https://azure.microsoft.com/en-us/products/phi) <br>
-🚀 vLLM Inference: V0: [PR](https://github.com/vllm-project/vllm/pull/20702) | [Branch](https://github.com/congcongchen123/vllm/tree/congcongchen/phi4-mini-shadow) V1: [PR](https://github.com/vllm-project/vllm/pull/23996) <br>
-🖥️ Try It [Azure](https://ai.azure.com/explore/models/Phi-4-mini-flash-reasoning/version/1/registry/azureml-phi-prod) [Nvidia NIM](https://build.nvidia.com/microsoft/phi-4-mini-flash-reasoning)<br>
+🚀 [vLLM Inference](https://github.com/vllm-project/vllm/pull/20702) <br>
+🖥️ Try It [Azure](https://ai.azure.com/explore/models/Phi-4-mini-flash-reasoning/version/1/registry/azureml-phi-prod) <br>
  
 
 🎉**Phi-4 models**: [[Phi-4-mini-reasoning](https://huggingface.co/microsoft/Phi-4-mini-reasoning)] | [[Phi-4-reasoning](https://huggingface.co/microsoft/Phi-4-reasoning)] | [[multimodal-instruct](https://huggingface.co/microsoft/Phi-4-multimodal-instruct) | [onnx](https://huggingface.co/microsoft/Phi-4-multimodal-instruct-onnx)]; 
 [[mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct) | [onnx](https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx)]
 
-## Abstract
-
-Recent advances in language modeling have demonstrated the effectiveness of State Space Models (SSMs) for efficient sequence modeling. While hybrid architectures such as Samba and the decoder-decoder architecture, YOCO, have shown promising performance gains over Transformers, prior works have not investigated the efficiency potential of representation sharing between SSM layers. In this paper, we introduce the Gated Memory Unit (GMU), a simple yet effective mechanism for efficient memory sharing across layers. We apply it to create SambaY, a decoder-hybrid-decoder architecture that incorporates GMUs in the cross-decoder to share memory readout states from a Samba-based self-decoder. SambaY significantly enhances decoding efficiency, preserves linear pre-filling time complexity, and boosts long-context performance, all while eliminating the need for explicit positional encoding. Through extensive scaling experiments, we demonstrate that our model exhibits a significantly lower irreducible loss compared to a strong YOCO baseline, indicating superior performance scalability under large-scale compute regimes. Our largest model enhanced with Differential Attention, Phi4-mini-Flash-Reasoning, achieves significantly better performance than Phi4-mini-Reasoning on reasoning tasks such as Math500, AIME24/25, and GPQA Diamond without any reinforcement learning, while delivering up to 10x higher decoding throughput on 2K-length prompts with 32K generation length under the vLLM inference framework. We release our training codebase on open-source data at [this https URL](https://github.com/microsoft/ArchScale).
-
 ## Intended Uses
  
 ### Primary Use Cases
@@ -65,15 +60,16 @@ If a critical issue is identified with Phi-4-mini-flash-reasoning, it should be
 To understand the capabilities, the 3.8B parameters Phi-4-mini-flash-reasoning model was compared with a set of models over a variety of reasoning benchmarks. 
 We use a more accurate evaluation where Pass@1 accuracy is averaged over 64 samples for AIME24/25 and 8 samples for Math500 and GPQA Diamond. A high-level overview of the model quality is as follows:
 
-| **Model** | **AIME24** | **AIME25** | **Math500** | **GPQA Diamond** |
-| :----------------------------------- | :--------- | :--------- | :---------- | :--------------- |
-| DeepSeek-R1-Distill-Qwen-1.5B | 29.58 | 20.78 | 84.50 | 37.69 |
-| DeepSeek-R1-Distill-Qwen-7B | 53.70 | 35.94 | 93.03 | 47.85 |
-| DeepSeek-R1-Distill-Llama-8B | 43.96 | 27.34 | 87.48 | 45.83 |
-| Bespoke-Stratos-7B | 21.51 | 18.28 | 80.73 | 38.51 |
-| OpenThinker-7B | 29.69 | 24.32 | 87.25 | 41.60 |
-| Phi4-mini-Reasoning (3.8B) | 48.13 | 31.77 | 91.20 | 44.51 |
-| **Phi4-mini-Flash-Reasoning (3.8B)** | **52.29** | **33.59** | **92.45** | **45.08** |
+| **Model**                            | **AIME24** | **AIME25** | **Math500** | **GPQA Diamond** |
+| ------------------------------------ | ---------- | ---------- | ----------- | ---------------- |
+| DeepSeek-R1-Distill-Qwen-1.5B        | 29.58      | 20.78      | 84.50       | 37.69            |
+| DeepSeek-R1-Distill-Qwen-7B          | 53.70      | 35.94      | 93.03       | 47.85            |
+| DeepSeek-R1-Distill-Llama-8B         | 43.96      | 27.34      | 87.48       | 45.83            |
+| Bespoke-Stratos-7B                   | 21.51      | 18.28      | 80.73       | 38.51            |
+| OpenThinker-7B                       | 29.69      | 24.32      | 87.25       | 41.60            |
+| Phi4-mini-Reasoning (3.8B)       | 48.13      | 31.77      | 91.20       | 44.51            |
+| **Phi4-mini-Flash-Reasoning (3.8B)** | **52.29**  | **33.59**  | **92.45**   | **45.08**        |
+
  
 Overall, the model with only 3.8B-param achieves a similar level of math and science reasoning ability as much larger models.
 However, it is still fundamentally limited by its size for certain tasks. The model simply does not have the capacity to store too much factual knowledge, therefore, users may experience factual incorrectness. However, it may be possible to resolve such weakness by augmenting Phi-4-mini-flash-reasoning with a search engine, particularly when using the model under RAG settings.
@@ -108,7 +104,7 @@ List of required packages:
 ```
 flash_attn==2.7.4.post1
 torch==2.6.0
-mamba-ssm==2.2.4 --no-build-isolation
+mamba-ssm==2.2.4
 causal-conv1d==1.5.0.post8
 transformers==4.46.1
 accelerate==1.4.0
@@ -178,7 +174,7 @@ print(outputs[0])
 The training data for Phi-4-mini-flash-reasoning consists exclusively of synthetic mathematical content generated by a stronger and more advanced reasoning model, Deepseek-R1. 
 The objective is to distill knowledge from this model. This synthetic dataset comprises over one million diverse math problems spanning multiple levels of difficulty (from middle school to Ph.D. level).
 For each problem in the synthetic dataset, eight distinct solutions (rollouts) were sampled, and only those verified as correct were retained, resulting in approximately 30 billion tokens of math content.
-The dataset integrates three primary components: 
+The dataset  integrates three primary components: 
 1) a curated selection of high-quality, publicly available math questions and a part of the SFT(Supervised Fine-Tuning) data that was used to train the base Phi-4-mini-flash model;
 2) an extensive collection of synthetic math data generated by the Deepseek-R1 model, designed specifically for high-quality supervised fine-tuning and model distillation; and
 3) a balanced set of correct and incorrect answers used to construct preference data aimed at enhancing Phi-4-mini-flash-reasoning's reasoning capabilities by learning more effective reasoning trajectories
@@ -197,7 +193,7 @@ Note that by default, the Phi-4-mini-flash-reasoning model uses flash attention,
 
 ## Safety Evaluation and Red-Teaming
  
-The Phi-4 family of models has adopted a robust safety post-training approach. This approach leverages a variety of both open-source and in-house generated datasets. The overall technique employed to do the safety alignment is a combination of SFT, DPO (Direct Preference Optimization), and RLHF (Reinforcement Learning from Human Feedback) approaches by utilizing human-labeled and synthetic English-language datasets, including publicly available datasets focusing on helpfulness and harmlessness, as well as various questions and answers targeted to multiple safety categories. 
+The Phi-4 family of models has adopted a robust safety post-training approach. This approach leverages a variety of both open-source and in-house generated datasets. The overall technique employed to do the safety alignment is a combination of SFT, DPO (Direct Preference Optimization), and RLHF (Reinforcement Learning from Human Feedback) approaches  by utilizing human-labeled and synthetic English-language datasets, including publicly available datasets focusing on helpfulness and harmlessness, as well as various questions and answers targeted to multiple safety categories. 
 
 Phi-4-Mini-Flash-Reasoning was developed in accordance with Microsoft's responsible AI principles. Potential safety risks in the model’s responses were assessed using the Azure AI Foundry’s Risk and Safety Evaluation framework, focusing on harmful content, direct jailbreak, and model groundedness. The Phi-4-Mini-Flash-Reasoning Model Card contains additional information about our approach to safety and responsible AI considerations that developers should be aware of when using this model.
 
@@ -211,7 +207,7 @@ Like other language models, the Phi family of models can potentially behave in w
 + Inappropriate or Offensive Content: These models may produce other types of inappropriate or offensive content, which may make it inappropriate to deploy for sensitive contexts without additional mitigations that are specific to the case.
 + Information Reliability: Language models can generate nonsensical content or fabricate content that might sound reasonable but is inaccurate or outdated.  
 +	Election Information Reliability : The model has an elevated defect rate when responding to election-critical queries, which may result in incorrect or unauthoritative election critical information being presented. We are working to improve the model's performance in this area. Users should verify information related to elections with the election authority in their region.
-+ Limited Scope for Code: The majority of Phi 4 training data is based in Python and uses common packages such as "typing, math, random, collections, datetime, itertools". If the model generates Python scripts that utilize other packages or scripts in other languages, it is strongly recommended that users manually verify all API uses.
++ Limited Scope for Code: The majority of Phi 4 training data is based in Python and uses common packages such as "typing, math, random, collections, datetime, itertools". If the model generates Python scripts that utilize other packages or scripts in other languages, it is  strongly recommended that users manually verify all API uses.
 + Long Conversation: Phi 4 models, like other models, can in some cases generate responses that are repetitive, unhelpful, or inconsistent in very long chat sessions in both English and non-English languages. Developers are encouraged to place appropriate mitigations, like limiting conversation turns to account for the possible conversational drift.
  
 Developers should apply responsible AI best practices, including mapping, measuring, and mitigating risks associated with their specific use case and cultural, linguistic context. Phi 4 family of models are general purpose models. As developers plan to deploy these models for specific use cases, they are encouraged to fine-tune the models for their use case and leverage the models as part of broader AI systems with language-specific safeguards in place. Important areas for consideration include:  
@@ -226,7 +222,7 @@ Developers should apply responsible AI best practices, including mapping, measur
 The model is licensed under the [MIT license](./LICENSE).
  
 ## Trademarks
-This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow [Microsoft’s Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks). Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party’s policies.
+This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow [Microsoft’s Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks). Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party’s policies.
  
  
 ## Appendix A: Benchmark Methodology
@@ -236,7 +232,4 @@ Benchmark datasets
 We evaluate the model with three of the most popular math benchmarks where the strongest reasoning models are competing together. Specifically:
 + Math-500: This benchmark consists of 500 challenging math problems designed to test the model's ability to perform complex mathematical reasoning and problem-solving.
 + AIME 2024/AIME 2025: The American Invitational Mathematics Examination (AIME) is a highly regarded math competition that features a series of difficult problems aimed at assessing advanced mathematical skills and logical reasoning. We evaluate the models on the problems from both 2024 and the year 2025 examinations. 
-+ GPQA Diamond: The Graduate-Level Google-Proof Q&A (GPQA) Diamond benchmark focuses on evaluating the model's ability to understand and solve a wide range of mathematical questions, including both straightforward calculations and more intricate problem-solving tasks.
-
-## Data Summary
-https://huggingface.co/microsoft/Phi-4-mini-flash-reasoning/blob/main/data_summary_card.md
++ GPQA Diamond: The Graduate-Level Google-Proof Q&A (GPQA) Diamond benchmark focuses on evaluating the model's ability to understand and solve a wide range of mathematical questions, including both straightforward calculations and more intricate problem-solving tasks.

configuration_phi4flash.py

@@ -170,4 +170,4 @@ class Phi4FlashConfig(PretrainedConfig):
             else:
                 layer_block_type = "mamba"
             layer_block_types.append(layer_block_type)
-        return layer_block_types
+        return layer_block_types

data_summary_card.md

@@ -1,149 +0,0 @@
- 
-
- 
-
-# Data Summary for microsoft_Phi-4-mini-reasoning, phi-4-mini-instruct, phi-4-mini-flash-reasoning 
-
- 
-
- 
-
-## 1. General information 
-
-**1.0.1 Version of the Summary:** 1.0 
-
- 
-
-**1.0.2 Last update:** 10-Dec-2025 
-
- 
-
-## 1.1 Model Developer Identification 
-
-**1.1.1 Model Developer name and contact details:** Microsoft Corporation at One Microsoft Way, Redmond, WA 98052. Tel: 425-882-8080 
-
- 
-
-## 1.2 Model Identification 
-
-**1.2.1 Versioned model name(s):** Phi-4-mini-reasoning, Phi-4-mini-instruct, Phi-4-mini-flash-reasoning 
-
- 
-
-**1.2.2 Model release date:** 29-Apr-2025  
-
- 
-
-## 1.3 Overall training data size and characteristics 
-
-### 1.3.1 Size of dataset and characteristics 
-
-**1.3.1.A Text training data size:** 1 billion to 10 trillion tokens 
-
- 
-
-**1.3.1.B Text training data content:** The training data for Phi-4-mini-reasoning consists exclusively of synthetic mathematical content generated by a stronger and more advanced reasoning model, Deepseek-R1. The objective is to distill knowledge from this model. This synthetic dataset comprises over one million diverse math problems spanning multiple levels of difficulty (from middle school to Ph.D. level). For each problem in the synthetic dataset, eight distinct solutions (rollouts) were sampled, and only those verified as correct were retained. 
-
- 
-
-**1.3.1.C Image training data size:** Not applicable. Images are not part of the training data 
-
- 
-
-**1.3.1.D Image training data content:** Not applicable 
-
- 
-
-**1.3.1.E Audio training data size:** Not applicable. Audio is not part of the training data 
-
- 
-
-**1.3.1.F Audio training data content:** Not applicable 
-
- 
-
-**1.3.1.G Video training data size:** Not applicable. Videos are not part of the training data 
-
- 
-
-**1.3.1.H Video training data content:** Not applicable 
-
- 
-
-**1.3.1.I Other training data size:** Not applicable 
-
- 
-
-**1.3.1.J Other training data content:** Not applicable 
-
- 
-
-**1.3.2 Latest date of data acquisition/collection for model training:** February 2025 
-
- 
-
-**1.3.3 Is data collection ongoing to update the model with new data collection after deployment?** No 
-
- 
-
-**1.3.4 Date the training dataset was first used to train the model:** February 2025 
-
- 
-
-**1.3.5 Rationale or purpose of data selection:** Datasets consist of synthetic mathematical problems and verified solutions generated by a stronger reasoning model to distill high-quality reasoning patterns and improve math problem-solving performance across difficulty levels 
-
- 
-
-## 2. List of data sources 
-
-### 2.1 Publicly available datasets 
-
-**2.1.1 Have you used publicly available datasets to train the model?** Yes 
-
- 
-
-## 2.2 Private non-publicly available datasets obtained from third parties 
-
-### 2.2.1 Datasets commercially licensed by rights holders or their representatives 
-
-**2.2.1.A Have you concluded transactional commercial licensing agreement(s) with rights holder(s) or with their representatives?** Not applicable 
-
- 
-
-### 2.2.2 Private datasets obtained from other third-parties 
-
-**2.2.2.A Have you obtained private datasets from third parties that are not licensed as described in Section 2.2.1, such as data obtained from providers of private databases, or data intermediaries?** No 
-
- 
-
-## 2.3 Personal Information 
-
-**2.3.1 Was personal data used to train the model?** Microsoft follows all relevant laws and regulations pertaining to personal information 
-
- 
-
-## 2.4 Synthetic data 
-
-**2.4.1 Was any synthetic AI-generated data used to train the model?** Yes 
-
- 
-
-## 3. Data processing aspects 
-
-### 3.1 Respect of reservation of rights from text and data mining exception or limitation 
-
-**3.1.1 Does this dataset include any data protected by copyright, trademark, or patent?** Microsoft follows all required regulations and laws for processing data protected by copyright, trademark, or patent 
-
- 
-
-## 3.2 Other information 
-
-**3.2.1 Does the dataset include information about consumer groups without revealing individual consumer identities?** Microsoft follows all required regulations and laws for protecting consumer identities 
-
- 
-
-**3.2.2 Was the dataset cleaned or modified before model training?** Yes 
-
- 
-
-