Update README.md
Browse files
README.md
CHANGED
|
@@ -19,12 +19,15 @@ pipeline_tag: text-generation
|
|
| 19 |
|
| 20 |
---
|
| 21 |
# Model Card for Qemma-GEI
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
| 28 |
## Quick start
|
| 29 |
|
| 30 |
```python
|
|
|
|
| 19 |
|
| 20 |
---
|
| 21 |
# Model Card for Qemma-GEI
|
| 22 |
+
## Gap Envelope Integral
|
| 23 |
+
* My mathematical formulation to utilize space projections to "measure" the Jump between points of discontinuity found in Non-Differentialable Functions.
|
| 24 |
+
## Redux
|
| 25 |
+
* This Model underwent an additional merge between Qemma-redux and Qwen3-0.6B, in addition to adding Rope Scaling.
|
| 26 |
+
### Additionally
|
| 27 |
+
* Fusion Logic was updated to aid per layer fusion and post fusion embedding alignment.
|
| 28 |
+
* **Qemma** is a HuggingFace-native hybrid model that merges **Gemma-3 (1B)** and **Qwen-3 (0.6B)** at the weight level (no adapters).
|
| 29 |
+
* Design: Gemma MLP/body + Qwen attention/head, projected and aligned to Gemma’s hidden size. The model is then SFT-tuned for stepwise reasoning.
|
| 30 |
+
* This variant uses Yarn based Rope Scaling with 1:1 Ratio from max_position_embeddings
|
| 31 |
## Quick start
|
| 32 |
|
| 33 |
```python
|