Update README.md
Browse files
README.md
CHANGED
|
@@ -18,6 +18,18 @@ base_model:
|
|
| 18 |
- AbstractPhil/geolip-bertenstein
|
| 19 |
---
|
| 20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
# GEOLIP CaptionBERT-8192-fingerprinted
|
| 22 |
|
| 23 |
The next iteration will require an expanded fingerprinting axis-based relational bank, specifically to the alignment of the data and the teachers at training time.
|
|
|
|
| 18 |
- AbstractPhil/geolip-bertenstein
|
| 19 |
---
|
| 20 |
|
| 21 |
+
# GEOLIP CaptionBERT-8192-anchored
|
| 22 |
+
|
| 23 |
+
This will be the real prototype, fingerprinting was the earlier thought and the full upcoming prototype is ready for train.
|
| 24 |
+
|
| 25 |
+
The upcoming checkpoints will push after the process is successful, likely 1 hour per epoch for 5 epochs or so should be more than eneough.
|
| 26 |
+
|
| 27 |
+
This marks the first use of a new prototype object dubbed AnchorBank, which is designed specifically to house the necessary implications that the model is distilled with,
|
| 28 |
+
while specifically aligning the expectation of those distillation valuations into the bank itself.
|
| 29 |
+
|
| 30 |
+
This allows the model to POTENTIALLY solve nth token lookup without a head, so a head will allow finetuning. If successful, the anchor bank will contain
|
| 31 |
+
all the knowledge the model requires to geomewtrically represent it's data into expanded structures - if the losses and training process is correctly aligned to the task.
|
| 32 |
+
|
| 33 |
# GEOLIP CaptionBERT-8192-fingerprinted
|
| 34 |
|
| 35 |
The next iteration will require an expanded fingerprinting axis-based relational bank, specifically to the alignment of the data and the teachers at training time.
|