Update README.md
Browse files
README.md
CHANGED
|
@@ -6,7 +6,7 @@ base_model:
|
|
| 6 |
|
| 7 |
---
|
| 8 |
|
| 9 |
-
# Essence 3B V1.1
|
| 10 |
|
| 11 |
This is a system using two versions of SmolLM3-3B-Base, the 'encoder', is finetuned to turn a text into a set of embedding tokens which can be reconstituted back into the original text by the decoder. In addition to “vanilla” reconstruction, this model was trained for span-corruption and masked language modelling.
|
| 12 |
|
|
@@ -26,7 +26,7 @@ from huggingface_hub import hf_hub_download
|
|
| 26 |
device = torch.device("cuda:0")
|
| 27 |
dtype = torch.bfloat16
|
| 28 |
base_model_id = "HuggingFaceTB/SmolLM3-3B-Base"
|
| 29 |
-
compressor_id = "midwestern-simulation/essence-3b-v1.1
|
| 30 |
|
| 31 |
# === MODEL LOADING ===
|
| 32 |
|
|
|
|
| 6 |
|
| 7 |
---
|
| 8 |
|
| 9 |
+
# Essence 3B V1.1
|
| 10 |
|
| 11 |
This is a system using two versions of SmolLM3-3B-Base, the 'encoder', is finetuned to turn a text into a set of embedding tokens which can be reconstituted back into the original text by the decoder. In addition to “vanilla” reconstruction, this model was trained for span-corruption and masked language modelling.
|
| 12 |
|
|
|
|
| 26 |
device = torch.device("cuda:0")
|
| 27 |
dtype = torch.bfloat16
|
| 28 |
base_model_id = "HuggingFaceTB/SmolLM3-3B-Base"
|
| 29 |
+
compressor_id = "midwestern-simulation/essence-3b-v1.1"
|
| 30 |
|
| 31 |
# === MODEL LOADING ===
|
| 32 |
|