drAbreu commited on
Commit
973da57
·
verified ·
1 Parent(s): 8d7a266

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -2
README.md CHANGED
@@ -44,6 +44,18 @@ This model is part of the **SODA-VEC** (Scientific Open Domain Adaptation for Ve
44
 
45
  **Loss Function**: VICReg Our Contrast: normalized embeddings with covariance loss, feature loss, and dot product loss (diagonal + off-diagonal)
46
 
 
 
 
 
 
 
 
 
 
 
 
 
47
  **Coefficients**: cov=1.0, feature=1.0, dot=1.0
48
  **Base Model**: `answerdotai/ModernBERT-base`
49
 
@@ -135,7 +147,7 @@ The model has been evaluated on comprehensive biomedical benchmarks including:
135
  - **Field-Specific Separability**: Distinguishing between different biological fields
136
  - **Semantic Search**: Retrieval quality on biomedical text corpora
137
 
138
- For detailed evaluation results, see the [SODA-VEC benchmark notebooks](https://github.com/EMBO/soda-vec).
139
 
140
  ## Intended Use
141
 
@@ -169,7 +181,7 @@ If you use this model, please cite:
169
 
170
  ## Model Card Contact
171
 
172
- For questions or issues, please open an issue on the [SODA-VEC GitHub repository](https://github.com/EMBO/soda-vec).
173
 
174
  ---
175
 
 
44
 
45
  **Loss Function**: VICReg Our Contrast: normalized embeddings with covariance loss, feature loss, and dot product loss (diagonal + off-diagonal)
46
 
47
+ We have implemented a series of changes from the original [VICREG in the paper from Meta](https://arxiv.org/pdf/2105.04906). Here we show the main differences:
48
+
49
+ | Feature | Original VICReg | VICReg Our | VICReg Our Contrast |
50
+ |---------|----------------|------------|---------------------|
51
+ | Normalization | No | Yes (L2-normalized) | Yes (L2-normalized) |
52
+ | Invariance (MSE) | Yes | No | No |
53
+ | Variance (hinge) | Yes | No | No |
54
+ | Covariance | Yes (unnormalized) | Yes (normalized) | Yes (normalized) |
55
+ | Feature correlation | No | Yes (cross-view) | Yes (cross-view) |
56
+ | Sample similarity | No | Yes (diagonal only) | Yes (diagonal + off-diagonal) |
57
+
58
+
59
  **Coefficients**: cov=1.0, feature=1.0, dot=1.0
60
  **Base Model**: `answerdotai/ModernBERT-base`
61
 
 
147
  - **Field-Specific Separability**: Distinguishing between different biological fields
148
  - **Semantic Search**: Retrieval quality on biomedical text corpora
149
 
150
+ For detailed evaluation results, see the [SODA-VEC benchmark notebooks](https://github.com/source-data/soda-vec).
151
 
152
  ## Intended Use
153
 
 
181
 
182
  ## Model Card Contact
183
 
184
+ For questions or issues, please open an issue on the [SODA-VEC GitHub repository](https://github.com/source-data/soda-vec).
185
 
186
  ---
187