Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
- fr
|
| 5 |
+
base_model:
|
| 6 |
+
- mistralai/Mistral-Small-24B-Base-2501
|
| 7 |
+
tags:
|
| 8 |
+
- roleplay
|
| 9 |
+
- deepseek
|
| 10 |
+
- rp
|
| 11 |
+
- r1
|
| 12 |
+
- mistral
|
| 13 |
+
- distill
|
| 14 |
+
---
|
| 15 |
+
# MistralThinker Model Card
|
| 16 |
+
|
| 17 |
+
## Model Description
|
| 18 |
+
|
| 19 |
+
**Model Name:** MistralThinker\
|
| 20 |
+
**Version:** 1.1
|
| 21 |
+
|
| 22 |
+
This model is a specialized variant of **Mistral-Small-24B-Base-2501**, adapted using a **DeepSeek R1** distillation process. It is **primarily designed for roleplay (RP) and storywriting** applications, focusing on character interactions, narrative generation, and creative storytelling. Approximately **40% of the training dataset** consists of roleplay/storywriting/character card data, ensuring rich and contextually immersive outputs in these domains.
|
| 23 |
+
|
| 24 |
+
## Model Sources
|
| 25 |
+
|
| 26 |
+
- **Base Model:** [Mistral-Small-24B-Base-2501](https://huggingface.co/mistralai/Mistral-Small-24B-Base-2501)
|
| 27 |
+
- **Fine-Tuning Approach:** DeepSeek R1 process (focused on RP)
|
| 28 |
+
- **Dataset Size:** The dataset used in training **doubled** since the last version, adding more neutral logs, training the Base model to stick more on my new format.
|
| 29 |
+
|
| 30 |
+
## Intended Use
|
| 31 |
+
|
| 32 |
+
- **Primary Use Cases:**
|
| 33 |
+
- **Roleplay (RP):** Engaging with users in fictional or scenario-based interactions.
|
| 34 |
+
- **Storywriting:** Generating narratives, character dialogues, and creative texts.
|
| 35 |
+
- **Character Lore Generation:** Serving as a resource to craft or expand on character backstories and interactions.
|
| 36 |
+
|
| 37 |
+
- **How To Use:**
|
| 38 |
+
1. **User-First Message:** The first message in any interaction should come from the user, ensuring the model responds in a narrative or roleplay context guided by user input.
|
| 39 |
+
2. **Contextual Information:** User or assistant details can be placed either in the system prompt or the user's first message. A system prompt is **not mandatory**, but any contextual instructions or role descriptions can help set the stage.
|
| 40 |
+
3. **DeepSeek-Style Interaction:** The model can also be used purely as a **DeepSeek distill** without additional system prompts, providing flexible usage for direct storytelling or roleplay scenarios. The model still can be biased toward Roleplay data, and it is expected.
|
| 41 |
+
|
| 42 |
+
## Training Data
|
| 43 |
+
|
| 44 |
+
- **DeepSeek R1 Thinking Process:** The model inherits a refined chain-of-thought (thinking process) from DeepSeek R1, which places heavy emphasis on **roleplay** and narrative coherence.
|
| 45 |
+
- **Dataset Composition:**
|
| 46 |
+
- 40%: RP/Storywriting/Character Cards
|
| 47 |
+
- 60%: Various curated data for broad language, math, logical, space... understanding
|
| 48 |
+
- **Data Scaling:** The dataset size was **doubled** compared to previous iterations, which enhances the model’s creative and contextual capabilities.
|
| 49 |
+
|
| 50 |
+
## Model Performance
|
| 51 |
+
|
| 52 |
+
- **Strengths:**
|
| 53 |
+
- **Storytelling & Roleplay:** Rich in creative generation, character portrayal, and scenario building.
|
| 54 |
+
- **Dialogue & Interaction:** Capable of sustaining engaging and context-driven dialogues.
|
| 55 |
+
- **Adaptability:** Can be used with or without a system prompt to match a range of user preferences.
|
| 56 |
+
|
| 57 |
+
- **Limitations & Bias:**
|
| 58 |
+
- **Hallucination:** It can generate fictitious information in the thinking process, but still end up with a succesfull reply.
|
| 59 |
+
- **Thinking can be dismissed:** Being a distillation of DeepSeek R1 is essence, this model, even trained on Base, could forget to add `<think>` in some scenario.
|
| 60 |
+
|
| 61 |
+
## Ethical Considerations
|
| 62 |
+
|
| 63 |
+
- Yes
|
| 64 |
+
|
| 65 |
+
## Usage Recommendations
|
| 66 |
+
|
| 67 |
+
1. **System Prompt (Optional):**
|
| 68 |
+
You may provide a high-level system prompt detailing the scenario or the desired style of roleplay and storywriting.
|
| 69 |
+
_Example: "You are a friendly fantasy innkeeper who greets travelers from distant lands."_
|
| 70 |
+
|
| 71 |
+
2. **User’s First Message:**
|
| 72 |
+
- Must clearly state or imply the scenario or context if no system prompt is provided.
|
| 73 |
+
_Example: "Hello, I’m a wandering knight seeking shelter. Could you share a story about local legends?"_
|
| 74 |
+
|
| 75 |
+
3. **Roleplay & Storywriting Focus:**
|
| 76 |
+
- Encourage the model to develop characters, backstories, and immersive dialogues.
|
| 77 |
+
- For more direct, unfiltered or freeform creativity, skip the system prompt.
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
|