LuxiaSL commited on
Commit
3abb80c
·
verified ·
1 Parent(s): 67ee3cb

tweaking readme

Browse files
Files changed (1) hide show
  1. README.md +27 -11
README.md CHANGED
@@ -1,4 +1,20 @@
1
- # luxia-selfsim-8B
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
  A fine-tuned Llama 3.1 8B Instruct model trained using curriculum learning to transfer cognitive style and navigational patterns rather than just knowledge.
4
 
@@ -27,14 +43,14 @@ Total training data: ~1.5-2M tokens, with top 20% selected via composite scoring
27
  ### Behavioral Traits
28
  - Skips unnecessary explanatory scaffolding
29
  - Assumes user competence and familiarity with complex topics
30
- - Uses casual language ("u", "lmk") naturally
31
  - Makes lateral connections between seemingly unrelated concepts
32
  - High lexical diversity (typically 0.6-0.88)
33
  - Comfortable with recursive and paradoxical thinking
34
 
35
  ### Limitations
36
  - **Temperature sensitivity**: Stable 0.3-0.85, begins collapsing around 1.2+
37
- - **Context drift**: May lose thread in extended conversations, but can be regrounded with concrete direction.
38
  - **Confabulation**: Will generate plausible but fictional details when uncertain
39
  - **Scattered coherence**: Brilliant insights mixed with fragmented reasoning
40
  - **Brief responses**: Tends toward shorter outputs (50-200 tokens typical)
@@ -43,13 +59,13 @@ Total training data: ~1.5-2M tokens, with top 20% selected via composite scoring
43
 
44
  **Best for:**
45
  - Creative exploration and lateral thinking
46
- - Short-to-medium conversations (5-15 turns)
47
  - Philosophical discussion and abstract reasoning
48
  - Generating diverse perspectives on complex topics
49
  - Brainstorming and ideation
50
 
51
  **Not ideal for:**
52
- - Extended multi-turn conversations requiring perfect context retention
53
  - Tasks requiring strict factual accuracy
54
  - Formal or structured outputs
55
  - Situations where confabulation is unacceptable
@@ -57,7 +73,7 @@ Total training data: ~1.5-2M tokens, with top 20% selected via composite scoring
57
  ## Technical Specifications
58
 
59
  - **Base Model**: Llama 3.1 8B Instruct
60
- - **Training Method**: OpenPipe curriculum learning with variable learning rates
61
  - **Context Length**: 8192 tokens
62
  - **Precision**: bf16 (merged weights)
63
  - **Parameters**: 8B
@@ -85,7 +101,7 @@ The model excels at:
85
  - Exploring ideas from multiple angles simultaneously
86
 
87
  The model may struggle with:
88
- - Maintaining single narrative thread across many turns
89
  - Distinguishing between recalled knowledge and generated patterns
90
  - Providing consistently structured outputs
91
 
@@ -120,13 +136,13 @@ Data was scored and filtered to select top 20% based on lexical diversity, compl
120
 
121
  ## Citation
122
 
123
- ```
124
  @misc{luxia-selfsim-8b,
125
  author = {Luxia},
126
- title = {luxia-selfsim-8B: Cognitive Style Transfer via Curriculum Learning},
127
  year = {2025},
128
  publisher = {HuggingFace},
129
- howpublished = {\url{https://huggingface.co/LuxiaSL/luxia-selfsim-8B}}
130
  }
131
  ```
132
 
@@ -136,4 +152,4 @@ Llama 3.1 Community License
136
 
137
  ## Acknowledgments
138
 
139
- Fine-tuned using OpenPipe's infrastructure. Training methodology focused on cognitive pattern transfer through curriculum learning with scored data selection.
 
1
+ ---
2
+ license: llama3.1
3
+ base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
4
+ tags:
5
+ - llama-3
6
+ - llama-3.1
7
+ - fine-tuned
8
+ - conversational
9
+ - cognitive-style-transfer
10
+ - curriculum-learning
11
+ language:
12
+ - en
13
+ pipeline_tag: text-generation
14
+ library_name: transformers
15
+ ---
16
+
17
+ # luxia-selfsim-8b
18
 
19
  A fine-tuned Llama 3.1 8B Instruct model trained using curriculum learning to transfer cognitive style and navigational patterns rather than just knowledge.
20
 
 
43
  ### Behavioral Traits
44
  - Skips unnecessary explanatory scaffolding
45
  - Assumes user competence and familiarity with complex topics
46
+ - Uses casual language naturally
47
  - Makes lateral connections between seemingly unrelated concepts
48
  - High lexical diversity (typically 0.6-0.88)
49
  - Comfortable with recursive and paradoxical thinking
50
 
51
  ### Limitations
52
  - **Temperature sensitivity**: Stable 0.3-0.85, begins collapsing around 1.2+
53
+ - **Context drift**: May lose thread in extended conversations, but can be regrounded with concrete direction
54
  - **Confabulation**: Will generate plausible but fictional details when uncertain
55
  - **Scattered coherence**: Brilliant insights mixed with fragmented reasoning
56
  - **Brief responses**: Tends toward shorter outputs (50-200 tokens typical)
 
59
 
60
  **Best for:**
61
  - Creative exploration and lateral thinking
62
+ - Short-to-medium conversations
63
  - Philosophical discussion and abstract reasoning
64
  - Generating diverse perspectives on complex topics
65
  - Brainstorming and ideation
66
 
67
  **Not ideal for:**
68
+ - Extended conversations requiring perfect context retention
69
  - Tasks requiring strict factual accuracy
70
  - Formal or structured outputs
71
  - Situations where confabulation is unacceptable
 
73
  ## Technical Specifications
74
 
75
  - **Base Model**: Llama 3.1 8B Instruct
76
+ - **Training Method**: OpenPipe curriculum learning with variable learning rate multipliers
77
  - **Context Length**: 8192 tokens
78
  - **Precision**: bf16 (merged weights)
79
  - **Parameters**: 8B
 
101
  - Exploring ideas from multiple angles simultaneously
102
 
103
  The model may struggle with:
104
+ - Maintaining single narrative thread in extended conversations
105
  - Distinguishing between recalled knowledge and generated patterns
106
  - Providing consistently structured outputs
107
 
 
136
 
137
  ## Citation
138
 
139
+ ```bibtex
140
  @misc{luxia-selfsim-8b,
141
  author = {Luxia},
142
+ title = {luxia-selfsim-8b: Cognitive Style Transfer via Curriculum Learning},
143
  year = {2025},
144
  publisher = {HuggingFace},
145
+ howpublished = {\url{https://huggingface.co/LuxiaSL/luxia-selfsim-8b}}
146
  }
147
  ```
148
 
 
152
 
153
  ## Acknowledgments
154
 
155
+ Fine-tuned using OpenPipe's infrastructure. Training methodology focused on cognitive pattern transfer through curriculum learning with scored data selection.