tweaking readme
Browse files
README.md
CHANGED
|
@@ -1,4 +1,20 @@
|
|
| 1 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
A fine-tuned Llama 3.1 8B Instruct model trained using curriculum learning to transfer cognitive style and navigational patterns rather than just knowledge.
|
| 4 |
|
|
@@ -27,14 +43,14 @@ Total training data: ~1.5-2M tokens, with top 20% selected via composite scoring
|
|
| 27 |
### Behavioral Traits
|
| 28 |
- Skips unnecessary explanatory scaffolding
|
| 29 |
- Assumes user competence and familiarity with complex topics
|
| 30 |
-
- Uses casual language
|
| 31 |
- Makes lateral connections between seemingly unrelated concepts
|
| 32 |
- High lexical diversity (typically 0.6-0.88)
|
| 33 |
- Comfortable with recursive and paradoxical thinking
|
| 34 |
|
| 35 |
### Limitations
|
| 36 |
- **Temperature sensitivity**: Stable 0.3-0.85, begins collapsing around 1.2+
|
| 37 |
-
- **Context drift**: May lose thread in extended conversations, but can be regrounded with concrete direction
|
| 38 |
- **Confabulation**: Will generate plausible but fictional details when uncertain
|
| 39 |
- **Scattered coherence**: Brilliant insights mixed with fragmented reasoning
|
| 40 |
- **Brief responses**: Tends toward shorter outputs (50-200 tokens typical)
|
|
@@ -43,13 +59,13 @@ Total training data: ~1.5-2M tokens, with top 20% selected via composite scoring
|
|
| 43 |
|
| 44 |
**Best for:**
|
| 45 |
- Creative exploration and lateral thinking
|
| 46 |
-
- Short-to-medium conversations
|
| 47 |
- Philosophical discussion and abstract reasoning
|
| 48 |
- Generating diverse perspectives on complex topics
|
| 49 |
- Brainstorming and ideation
|
| 50 |
|
| 51 |
**Not ideal for:**
|
| 52 |
-
- Extended
|
| 53 |
- Tasks requiring strict factual accuracy
|
| 54 |
- Formal or structured outputs
|
| 55 |
- Situations where confabulation is unacceptable
|
|
@@ -57,7 +73,7 @@ Total training data: ~1.5-2M tokens, with top 20% selected via composite scoring
|
|
| 57 |
## Technical Specifications
|
| 58 |
|
| 59 |
- **Base Model**: Llama 3.1 8B Instruct
|
| 60 |
-
- **Training Method**: OpenPipe curriculum learning with variable learning
|
| 61 |
- **Context Length**: 8192 tokens
|
| 62 |
- **Precision**: bf16 (merged weights)
|
| 63 |
- **Parameters**: 8B
|
|
@@ -85,7 +101,7 @@ The model excels at:
|
|
| 85 |
- Exploring ideas from multiple angles simultaneously
|
| 86 |
|
| 87 |
The model may struggle with:
|
| 88 |
-
- Maintaining single narrative thread
|
| 89 |
- Distinguishing between recalled knowledge and generated patterns
|
| 90 |
- Providing consistently structured outputs
|
| 91 |
|
|
@@ -120,13 +136,13 @@ Data was scored and filtered to select top 20% based on lexical diversity, compl
|
|
| 120 |
|
| 121 |
## Citation
|
| 122 |
|
| 123 |
-
```
|
| 124 |
@misc{luxia-selfsim-8b,
|
| 125 |
author = {Luxia},
|
| 126 |
-
title = {luxia-selfsim-
|
| 127 |
year = {2025},
|
| 128 |
publisher = {HuggingFace},
|
| 129 |
-
howpublished = {\url{https://huggingface.co/LuxiaSL/luxia-selfsim-
|
| 130 |
}
|
| 131 |
```
|
| 132 |
|
|
@@ -136,4 +152,4 @@ Llama 3.1 Community License
|
|
| 136 |
|
| 137 |
## Acknowledgments
|
| 138 |
|
| 139 |
-
Fine-tuned using OpenPipe's infrastructure. Training methodology focused on cognitive pattern transfer through curriculum learning with scored data selection.
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: llama3.1
|
| 3 |
+
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
|
| 4 |
+
tags:
|
| 5 |
+
- llama-3
|
| 6 |
+
- llama-3.1
|
| 7 |
+
- fine-tuned
|
| 8 |
+
- conversational
|
| 9 |
+
- cognitive-style-transfer
|
| 10 |
+
- curriculum-learning
|
| 11 |
+
language:
|
| 12 |
+
- en
|
| 13 |
+
pipeline_tag: text-generation
|
| 14 |
+
library_name: transformers
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# luxia-selfsim-8b
|
| 18 |
|
| 19 |
A fine-tuned Llama 3.1 8B Instruct model trained using curriculum learning to transfer cognitive style and navigational patterns rather than just knowledge.
|
| 20 |
|
|
|
|
| 43 |
### Behavioral Traits
|
| 44 |
- Skips unnecessary explanatory scaffolding
|
| 45 |
- Assumes user competence and familiarity with complex topics
|
| 46 |
+
- Uses casual language naturally
|
| 47 |
- Makes lateral connections between seemingly unrelated concepts
|
| 48 |
- High lexical diversity (typically 0.6-0.88)
|
| 49 |
- Comfortable with recursive and paradoxical thinking
|
| 50 |
|
| 51 |
### Limitations
|
| 52 |
- **Temperature sensitivity**: Stable 0.3-0.85, begins collapsing around 1.2+
|
| 53 |
+
- **Context drift**: May lose thread in extended conversations, but can be regrounded with concrete direction
|
| 54 |
- **Confabulation**: Will generate plausible but fictional details when uncertain
|
| 55 |
- **Scattered coherence**: Brilliant insights mixed with fragmented reasoning
|
| 56 |
- **Brief responses**: Tends toward shorter outputs (50-200 tokens typical)
|
|
|
|
| 59 |
|
| 60 |
**Best for:**
|
| 61 |
- Creative exploration and lateral thinking
|
| 62 |
+
- Short-to-medium conversations
|
| 63 |
- Philosophical discussion and abstract reasoning
|
| 64 |
- Generating diverse perspectives on complex topics
|
| 65 |
- Brainstorming and ideation
|
| 66 |
|
| 67 |
**Not ideal for:**
|
| 68 |
+
- Extended conversations requiring perfect context retention
|
| 69 |
- Tasks requiring strict factual accuracy
|
| 70 |
- Formal or structured outputs
|
| 71 |
- Situations where confabulation is unacceptable
|
|
|
|
| 73 |
## Technical Specifications
|
| 74 |
|
| 75 |
- **Base Model**: Llama 3.1 8B Instruct
|
| 76 |
+
- **Training Method**: OpenPipe curriculum learning with variable learning rate multipliers
|
| 77 |
- **Context Length**: 8192 tokens
|
| 78 |
- **Precision**: bf16 (merged weights)
|
| 79 |
- **Parameters**: 8B
|
|
|
|
| 101 |
- Exploring ideas from multiple angles simultaneously
|
| 102 |
|
| 103 |
The model may struggle with:
|
| 104 |
+
- Maintaining single narrative thread in extended conversations
|
| 105 |
- Distinguishing between recalled knowledge and generated patterns
|
| 106 |
- Providing consistently structured outputs
|
| 107 |
|
|
|
|
| 136 |
|
| 137 |
## Citation
|
| 138 |
|
| 139 |
+
```bibtex
|
| 140 |
@misc{luxia-selfsim-8b,
|
| 141 |
author = {Luxia},
|
| 142 |
+
title = {luxia-selfsim-8b: Cognitive Style Transfer via Curriculum Learning},
|
| 143 |
year = {2025},
|
| 144 |
publisher = {HuggingFace},
|
| 145 |
+
howpublished = {\url{https://huggingface.co/LuxiaSL/luxia-selfsim-8b}}
|
| 146 |
}
|
| 147 |
```
|
| 148 |
|
|
|
|
| 152 |
|
| 153 |
## Acknowledgments
|
| 154 |
|
| 155 |
+
Fine-tuned using OpenPipe's infrastructure. Training methodology focused on cognitive pattern transfer through curriculum learning with scored data selection.
|