Update README.md
Browse files
README.md
CHANGED
|
@@ -44,6 +44,18 @@ This model is part of the **SODA-VEC** (Scientific Open Domain Adaptation for Ve
|
|
| 44 |
|
| 45 |
**Loss Function**: VICReg Our Contrast: normalized embeddings with covariance loss, feature loss, and dot product loss (diagonal + off-diagonal)
|
| 46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
**Coefficients**: cov=1.0, feature=1.0, dot=1.0
|
| 48 |
**Base Model**: `answerdotai/ModernBERT-base`
|
| 49 |
|
|
@@ -135,7 +147,7 @@ The model has been evaluated on comprehensive biomedical benchmarks including:
|
|
| 135 |
- **Field-Specific Separability**: Distinguishing between different biological fields
|
| 136 |
- **Semantic Search**: Retrieval quality on biomedical text corpora
|
| 137 |
|
| 138 |
-
For detailed evaluation results, see the [SODA-VEC benchmark notebooks](https://github.com/
|
| 139 |
|
| 140 |
## Intended Use
|
| 141 |
|
|
@@ -169,7 +181,7 @@ If you use this model, please cite:
|
|
| 169 |
|
| 170 |
## Model Card Contact
|
| 171 |
|
| 172 |
-
For questions or issues, please open an issue on the [SODA-VEC GitHub repository](https://github.com/
|
| 173 |
|
| 174 |
---
|
| 175 |
|
|
|
|
| 44 |
|
| 45 |
**Loss Function**: VICReg Our Contrast: normalized embeddings with covariance loss, feature loss, and dot product loss (diagonal + off-diagonal)
|
| 46 |
|
| 47 |
+
We have implemented a series of changes from the original [VICREG in the paper from Meta](https://arxiv.org/pdf/2105.04906). Here we show the main differences:
|
| 48 |
+
|
| 49 |
+
| Feature | Original VICReg | VICReg Our | VICReg Our Contrast |
|
| 50 |
+
|---------|----------------|------------|---------------------|
|
| 51 |
+
| Normalization | No | Yes (L2-normalized) | Yes (L2-normalized) |
|
| 52 |
+
| Invariance (MSE) | Yes | No | No |
|
| 53 |
+
| Variance (hinge) | Yes | No | No |
|
| 54 |
+
| Covariance | Yes (unnormalized) | Yes (normalized) | Yes (normalized) |
|
| 55 |
+
| Feature correlation | No | Yes (cross-view) | Yes (cross-view) |
|
| 56 |
+
| Sample similarity | No | Yes (diagonal only) | Yes (diagonal + off-diagonal) |
|
| 57 |
+
|
| 58 |
+
|
| 59 |
**Coefficients**: cov=1.0, feature=1.0, dot=1.0
|
| 60 |
**Base Model**: `answerdotai/ModernBERT-base`
|
| 61 |
|
|
|
|
| 147 |
- **Field-Specific Separability**: Distinguishing between different biological fields
|
| 148 |
- **Semantic Search**: Retrieval quality on biomedical text corpora
|
| 149 |
|
| 150 |
+
For detailed evaluation results, see the [SODA-VEC benchmark notebooks](https://github.com/source-data/soda-vec).
|
| 151 |
|
| 152 |
## Intended Use
|
| 153 |
|
|
|
|
| 181 |
|
| 182 |
## Model Card Contact
|
| 183 |
|
| 184 |
+
For questions or issues, please open an issue on the [SODA-VEC GitHub repository](https://github.com/source-data/soda-vec).
|
| 185 |
|
| 186 |
---
|
| 187 |
|