Mithil Maske commited on
Link the published Medium write-up
Browse files
README.md
CHANGED
|
@@ -32,6 +32,9 @@ are frozen, publicly available source models, and only two small projector
|
|
| 32 |
heads (a few million parameters total) were trained from scratch on public
|
| 33 |
datasets to align them into a shared space.
|
| 34 |
|
|
|
|
|
|
|
|
|
|
| 35 |
## Architecture
|
| 36 |
|
| 37 |
Three frozen encoders, two trained projectors — everything large stays
|
|
@@ -81,7 +84,9 @@ raw training data volume was — R@1 only moved once the image-caption
|
|
| 81 |
training set was scaled to ~172k pairs (round 7). The full round-by-round
|
| 82 |
write-up, with what did and didn't move the needle, lives in the source
|
| 83 |
repo's README and feasibility notes:
|
| 84 |
-
[github.com/mithilai/QuadEmbed](https://github.com/mithilai/QuadEmbed)
|
|
|
|
|
|
|
| 85 |
|
| 86 |
## Usage
|
| 87 |
|
|
|
|
| 32 |
heads (a few million parameters total) were trained from scratch on public
|
| 33 |
datasets to align them into a shared space.
|
| 34 |
|
| 35 |
+
📝 Full write-up (how it was built, and seven rounds of what did and didn't
|
| 36 |
+
work): [I Built a Multimodal Embedding Model From Scratch on an RTX 4060](https://medium.com/@mithilmaske/i-built-a-multimodal-embedding-model-from-scratch-on-an-rtx-4060-text-image-audio-and-video-ab1fef04f1cd)
|
| 37 |
+
|
| 38 |
## Architecture
|
| 39 |
|
| 40 |
Three frozen encoders, two trained projectors — everything large stays
|
|
|
|
| 84 |
training set was scaled to ~172k pairs (round 7). The full round-by-round
|
| 85 |
write-up, with what did and didn't move the needle, lives in the source
|
| 86 |
repo's README and feasibility notes:
|
| 87 |
+
[github.com/mithilai/QuadEmbed](https://github.com/mithilai/QuadEmbed), and
|
| 88 |
+
as a full narrative write-up on Medium:
|
| 89 |
+
[I Built a Multimodal Embedding Model From Scratch on an RTX 4060](https://medium.com/@mithilmaske/i-built-a-multimodal-embedding-model-from-scratch-on-an-rtx-4060-text-image-audio-and-video-ab1fef04f1cd).
|
| 90 |
|
| 91 |
## Usage
|
| 92 |
|