|
|
--- |
|
|
tags: |
|
|
- sentence-transformers |
|
|
- sentence-similarity |
|
|
- feature-extraction |
|
|
- dense |
|
|
- generated_from_trainer |
|
|
- dataset_size:200 |
|
|
- loss:MultipleNegativesRankingLoss |
|
|
base_model: google/embeddinggemma-300m |
|
|
widget: |
|
|
- source_sentence: Alice and Bob play a game involving a circle whose circumference |
|
|
is divided by 12 equally-spaced points. The points are numbered clockwise, from |
|
|
1 to 12. Both start on point 12. Alice moves clockwise and Bob, counterclockwise. |
|
|
In a turn of the game, Alice moves 5 points clockwise and Bob moves 9 points counterclockwise. |
|
|
The game ends when they stop on the same point. How many turns will this take? |
|
|
sentences: |
|
|
- How many distinct triangles can be drawn using three of the dots below as vertices? |
|
|
[asy]dot(origin^^(1,0)^^(2,0)^^(0,1)^^(1,1)^^(2,1));[/asy] |
|
|
- Tess runs counterclockwise around rectangular block $JKLM$. She lives at corner |
|
|
$J$. Which graph could represent her straight-line distance from home? |
|
|
- Which triplet of numbers has a sum NOT equal to 1? |
|
|
- source_sentence: Which of these numbers is less than its reciprocal? |
|
|
sentences: |
|
|
- A white cylindrical silo has a diameter of 30 feet and a height of 80 feet. A |
|
|
red stripe with a horizontal width of 3 feet is painted on the silo, as shown, |
|
|
making two complete revolutions around it. What is the area of the stripe in square |
|
|
feet? [asy] size(250);defaultpen(linewidth(0.8)); draw(ellipse(origin, 3, 1)); |
|
|
fill((3,0)--(3,2)--(-3,2)--(-3,0)--cycle, white); draw((3,0)--(3,16)^^(-3,0)--(-3,16)); |
|
|
draw((0, 15)--(3, 12)^^(0, 16)--(3, 13)); filldraw(ellipse((0, 16), 3, 1), white, |
|
|
black); draw((-3,11)--(3, 5)^^(-3,10)--(3, 4)); draw((-3,2)--(0,-1)^^(-3,1)--(-1,-0.89)); |
|
|
draw((0,-1)--(0,15), dashed); draw((3,-2)--(3,-4)^^(-3,-2)--(-3,-4)); draw((-7,0)--(-5,0)^^(-7,16)--(-5,16)); |
|
|
draw((3,-3)--(-3,-3), Arrows(6)); draw((-6,0)--(-6,16), Arrows(6)); draw((-2,9)--(-1,9), |
|
|
Arrows(3)); label("$3$", (-1.375,9.05), dir(260), fontsize(7)); label("$A$", (0,15), |
|
|
N); label("$B$", (0,-1), NE); label("$30$", (0, -3), S); label("$80$", (-6, 8), |
|
|
W);[/asy] |
|
|
- Aunt Anna is $42$ years old. Caitlin is $5$ years younger than Brianna, and Brianna |
|
|
is half as old as Aunt Anna. How old is Caitlin? |
|
|
- Tess runs counterclockwise around rectangular block $JKLM$. She lives at corner |
|
|
$J$. Which graph could represent her straight-line distance from home? |
|
|
- source_sentence: 'Problems 8,9 and 10 use the data found in the accompanying paragraph |
|
|
and table: Juan organizes the stamps in his collection by country and by the decade |
|
|
in which they were issued. The prices he paid for them at a stamp shop were: Brazil |
|
|
and France, 6 cents each, Peru 4 cents each, and Spain 5 cents each. (Brazil and |
|
|
Peru are South American countries and France and Spain are in Europe.) [asy] /* |
|
|
AMC8 2002 #8, 9, 10 Problem */ size(3inch, 1.5inch); for ( int y = 0; y <= 5; |
|
|
++y ) { draw((0,y)--(18,y)); } draw((0,0)--(0,5)); draw((6,0)--(6,5)); draw((9,0)--(9,5)); |
|
|
draw((12,0)--(12,5)); draw((15,0)--(15,5)); draw((18,0)--(18,5)); draw(scale(0.8)*"50s", |
|
|
(7.5,4.5)); draw(scale(0.8)*"4", (7.5,3.5)); draw(scale(0.8)*"8", (7.5,2.5)); |
|
|
draw(scale(0.8)*"6", (7.5,1.5)); draw(scale(0.8)*"3", (7.5,0.5)); draw(scale(0.8)*"60s", |
|
|
(10.5,4.5)); draw(scale(0.8)*"7", (10.5,3.5)); draw(scale(0.8)*"4", (10.5,2.5)); |
|
|
draw(scale(0.8)*"4", (10.5,1.5)); draw(scale(0.8)*"9", (10.5,0.5)); draw(scale(0.8)*"70s", |
|
|
(13.5,4.5)); draw(scale(0.8)*"12", (13.5,3.5)); draw(scale(0.8)*"12", (13.5,2.5)); |
|
|
draw(scale(0.8)*"6", (13.5,1.5)); draw(scale(0.8)*"13", (13.5,0.5)); draw(scale(0.8)*"80s", |
|
|
(16.5,4.5)); draw(scale(0.8)*"8", (16.5,3.5)); draw(scale(0.8)*"15", (16.5,2.5)); |
|
|
draw(scale(0.8)*"10", (16.5,1.5)); draw(scale(0.8)*"9", (16.5,0.5)); label(scale(0.8)*"Country", |
|
|
(3,4.5)); label(scale(0.8)*"Brazil", (3,3.5)); label(scale(0.8)*"France", (3,2.5)); |
|
|
label(scale(0.8)*"Peru", (3,1.5)); label(scale(0.8)*"Spain", (3,0.5)); label(scale(0.9)*"Juan''s |
|
|
Stamp Collection", (9,0), S); label(scale(0.9)*"Number of Stamps by Decade", (9,5), |
|
|
N);[/asy] The average price of his ''70s stamps is closest to' |
|
|
sentences: |
|
|
- A rectangular garden 60 feet long and 20 feet wide is enclosed by a fence. To |
|
|
make the garden larger, while using the same fence, its shape is changed to a |
|
|
square. By how many square feet does this enlarge the garden? |
|
|
- 'Problems 8,9 and 10 use the data found in the accompanying paragraph and table: |
|
|
Juan organizes the stamps in his collection by country and by the decade in which |
|
|
they were issued. The prices he paid for them at a stamp shop were: Brazil and |
|
|
France, 6 cents each, Peru 4 cents each, and Spain 5 cents each. (Brazil and Peru |
|
|
are South American countries and France and Spain are in Europe.) [asy] /* AMC8 |
|
|
2002 #8, 9, 10 Problem */ size(3inch, 1.5inch); for ( int y = 0; y <= 5; ++y ) |
|
|
{ draw((0,y)--(18,y)); } draw((0,0)--(0,5)); draw((6,0)--(6,5)); draw((9,0)--(9,5)); |
|
|
draw((12,0)--(12,5)); draw((15,0)--(15,5)); draw((18,0)--(18,5)); draw(scale(0.8)*"50s", |
|
|
(7.5,4.5)); draw(scale(0.8)*"4", (7.5,3.5)); draw(scale(0.8)*"8", (7.5,2.5)); |
|
|
draw(scale(0.8)*"6", (7.5,1.5)); draw(scale(0.8)*"3", (7.5,0.5)); draw(scale(0.8)*"60s", |
|
|
(10.5,4.5)); draw(scale(0.8)*"7", (10.5,3.5)); draw(scale(0.8)*"4", (10.5,2.5)); |
|
|
draw(scale(0.8)*"4", (10.5,1.5)); draw(scale(0.8)*"9", (10.5,0.5)); draw(scale(0.8)*"70s", |
|
|
(13.5,4.5)); draw(scale(0.8)*"12", (13.5,3.5)); draw(scale(0.8)*"12", (13.5,2.5)); |
|
|
draw(scale(0.8)*"6", (13.5,1.5)); draw(scale(0.8)*"13", (13.5,0.5)); draw(scale(0.8)*"80s", |
|
|
(16.5,4.5)); draw(scale(0.8)*"8", (16.5,3.5)); draw(scale(0.8)*"15", (16.5,2.5)); |
|
|
draw(scale(0.8)*"10", (16.5,1.5)); draw(scale(0.8)*"9", (16.5,0.5)); label(scale(0.8)*"Country", |
|
|
(3,4.5)); label(scale(0.8)*"Brazil", (3,3.5)); label(scale(0.8)*"France", (3,2.5)); |
|
|
label(scale(0.8)*"Peru", (3,1.5)); label(scale(0.8)*"Spain", (3,0.5)); label(scale(0.9)*"Juan''s |
|
|
Stamp Collection", (9,0), S); label(scale(0.9)*"Number of Stamps by Decade", (9,5), |
|
|
N);[/asy] In dollars and cents, how much did his South American stamps issued |
|
|
before the ’70s cost him?' |
|
|
- Three mutually tangent spheres of radius $1$ rest on a horizontal plane. A sphere |
|
|
of radius $2$ rests on them. What is the distance from the plane to the top of |
|
|
the larger sphere? |
|
|
- source_sentence: Two-thirds of the people in a room are seated in three-fourths |
|
|
of the chairs. The rest of the people are standing. If there are $6$ empty chairs, |
|
|
how many people are in the room? |
|
|
sentences: |
|
|
- Spinners $A$ and $B$ are spun. On each spinner, the arrow is equally likely to |
|
|
land on each number. What is the probability that the product of the two spinners' |
|
|
numbers is even? |
|
|
- On the AMC 8 contest Billy answers 13 questions correctly, answers 7 questions |
|
|
incorrectly and doesn't answer the last 5. What is his score? |
|
|
- How many different combinations of \$5 bills and \$2 bills can be used to make |
|
|
a total of \$17? Order does not matter in this problem. |
|
|
- source_sentence: When $1999^{2000}$ is divided by $5$, the remainder is |
|
|
sentences: |
|
|
- Square $ABCD$ has sides of length 3. Segments $CM$ and $CN$ divide the square's |
|
|
area into three equal parts. How long is segment $CM$? |
|
|
- Problems 14, 15 and 16 involve Mrs. Reed's English assignment. A Novel Assignment |
|
|
The students in Mrs. Reed's English class are reading the same 760-page novel. |
|
|
Three friends, Alice, Bob and Chandra, are in the class. Alice reads a page in |
|
|
20 seconds, Bob reads a page in 45 seconds and Chandra reads a page in 30 seconds. |
|
|
Chandra and Bob, who each have a copy of the book, decide that they can save time |
|
|
by "team reading" the novel. In this scheme, Chandra will read from page 1 to |
|
|
a certain page and Bob will read from the next page through page 760, finishing |
|
|
the book. When they are through they will tell each other about the part they |
|
|
read. What is the last page that Chandra should read so that she and Bob spend |
|
|
the same amount of time reading the novel? |
|
|
- $(6?3) + 4 - (2 - 1) = 5.$ To make this statement true, the question mark between |
|
|
the 6 and the 3 should be replaced by |
|
|
pipeline_tag: sentence-similarity |
|
|
library_name: sentence-transformers |
|
|
--- |
|
|
|
|
|
# SentenceTransformer based on google/embeddinggemma-300m |
|
|
|
|
|
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [google/embeddinggemma-300m](https://huggingface.co/google/embeddinggemma-300m). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
- **Model Type:** Sentence Transformer |
|
|
- **Base model:** [google/embeddinggemma-300m](https://huggingface.co/google/embeddinggemma-300m) <!-- at revision 57c266a740f537b4dc058e1b0cda161fd15afa75 --> |
|
|
- **Maximum Sequence Length:** 2048 tokens |
|
|
- **Output Dimensionality:** 768 dimensions |
|
|
- **Similarity Function:** Cosine Similarity |
|
|
<!-- - **Training Dataset:** Unknown --> |
|
|
<!-- - **Language:** Unknown --> |
|
|
<!-- - **License:** Unknown --> |
|
|
|
|
|
### Model Sources |
|
|
|
|
|
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net) |
|
|
- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) |
|
|
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) |
|
|
|
|
|
### Full Model Architecture |
|
|
|
|
|
``` |
|
|
SentenceTransformer( |
|
|
(0): Transformer({'max_seq_length': 2048, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'}) |
|
|
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) |
|
|
(2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'}) |
|
|
(3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'}) |
|
|
(4): Normalize() |
|
|
) |
|
|
``` |
|
|
|
|
|
## Usage |
|
|
|
|
|
### Direct Usage (Sentence Transformers) |
|
|
|
|
|
First install the Sentence Transformers library: |
|
|
|
|
|
```bash |
|
|
pip install -U sentence-transformers |
|
|
``` |
|
|
|
|
|
Then you can load this model and run inference. |
|
|
```python |
|
|
from sentence_transformers import SentenceTransformer |
|
|
|
|
|
# Download from the 🤗 Hub |
|
|
model = SentenceTransformer("blachang28/poc-gemma") |
|
|
# Run inference |
|
|
queries = [ |
|
|
"When $1999^{2000}$ is divided by $5$, the remainder is", |
|
|
] |
|
|
documents = [ |
|
|
"Square $ABCD$ has sides of length 3. Segments $CM$ and $CN$ divide the square's area into three equal parts. How long is segment $CM$?", |
|
|
'$(6?3) + 4 - (2 - 1) = 5.$ To make this statement true, the question mark between the 6 and the 3 should be replaced by', |
|
|
'Problems 14, 15 and 16 involve Mrs. Reed\'s English assignment. A Novel Assignment The students in Mrs. Reed\'s English class are reading the same 760-page novel. Three friends, Alice, Bob and Chandra, are in the class. Alice reads a page in 20 seconds, Bob reads a page in 45 seconds and Chandra reads a page in 30 seconds. Chandra and Bob, who each have a copy of the book, decide that they can save time by "team reading" the novel. In this scheme, Chandra will read from page 1 to a certain page and Bob will read from the next page through page 760, finishing the book. When they are through they will tell each other about the part they read. What is the last page that Chandra should read so that she and Bob spend the same amount of time reading the novel?', |
|
|
] |
|
|
query_embeddings = model.encode_query(queries) |
|
|
document_embeddings = model.encode_document(documents) |
|
|
print(query_embeddings.shape, document_embeddings.shape) |
|
|
# [1, 768] [3, 768] |
|
|
|
|
|
# Get the similarity scores for the embeddings |
|
|
similarities = model.similarity(query_embeddings, document_embeddings) |
|
|
print(similarities) |
|
|
# tensor([[ 0.7710, -0.4438, -0.0152]]) |
|
|
``` |
|
|
|
|
|
<!-- |
|
|
### Direct Usage (Transformers) |
|
|
|
|
|
<details><summary>Click to see the direct usage in Transformers</summary> |
|
|
|
|
|
</details> |
|
|
--> |
|
|
|
|
|
<!-- |
|
|
### Downstream Usage (Sentence Transformers) |
|
|
|
|
|
You can finetune this model on your own dataset. |
|
|
|
|
|
<details><summary>Click to expand</summary> |
|
|
|
|
|
</details> |
|
|
--> |
|
|
|
|
|
<!-- |
|
|
### Out-of-Scope Use |
|
|
|
|
|
*List how the model may foreseeably be misused and address what users ought not to do with the model.* |
|
|
--> |
|
|
|
|
|
<!-- |
|
|
## Bias, Risks and Limitations |
|
|
|
|
|
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
|
|
--> |
|
|
|
|
|
<!-- |
|
|
### Recommendations |
|
|
|
|
|
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
|
|
--> |
|
|
|
|
|
## Training Details |
|
|
|
|
|
### Training Dataset |
|
|
|
|
|
#### Unnamed Dataset |
|
|
|
|
|
* Size: 200 training samples |
|
|
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code> |
|
|
* Approximate statistics based on the first 200 samples: |
|
|
| | anchor | positive | negative | |
|
|
|:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| |
|
|
| type | string | string | string | |
|
|
| details | <ul><li>min: 11 tokens</li><li>mean: 81.76 tokens</li><li>max: 806 tokens</li></ul> | <ul><li>min: 11 tokens</li><li>mean: 80.08 tokens</li><li>max: 806 tokens</li></ul> | <ul><li>min: 12 tokens</li><li>mean: 79.01 tokens</li><li>max: 797 tokens</li></ul> | |
|
|
* Samples: |
|
|
| anchor | positive | negative | |
|
|
|:-------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
|
|
| <code>$(6?3) + 4 - (2 - 1) = 5.$ To make this statement true, the question mark between the 6 and the 3 should be replaced by</code> | <code>What is the degree measure of the smaller angle formed by the hands of a clock at 10 o'clock?</code> | <code>An insect lives on the surface of a regular tetrahedron with edges of length 1. It wishes to travel on the surface of the tetrahedron from the midpoint of one edge to the midpoint of the opposite edge. What is the length of the shortest such trip? (Note: Two edges of a tetrahedron are opposite if they have no common endpoint.)</code> | |
|
|
| <code>What is the degree measure of the smaller angle formed by the hands of a clock at 10 o'clock?</code> | <code>Which triplet of numbers has a sum NOT equal to 1?</code> | <code>Corners are sliced off a unit cube so that the six faces each become regular octagons. What is the total volume of the removed tetrahedra?</code> | |
|
|
| <code>Which triplet of numbers has a sum NOT equal to 1?</code> | <code>What is the degree measure of the smaller angle formed by the hands of a clock at 10 o'clock?</code> | <code>How many pairs of positive integers $(a,b)$ are there such that $\text{gcd}(a,b)=1$ and $\frac{a}{b} + \frac{14b}{9a}$ is an integer? $\mathrm {(A)}\ 4\quad\mathrm {(B)}\ 6\quad\mathrm {(C)}\ 9\quad\mathrm {(D)}\ 12\quad\mathrm {(E)}\ \text{infinitely many}$</code> | |
|
|
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: |
|
|
```json |
|
|
{ |
|
|
"scale": 20.0, |
|
|
"similarity_fct": "cos_sim", |
|
|
"gather_across_devices": false |
|
|
} |
|
|
``` |
|
|
|
|
|
### Training Hyperparameters |
|
|
#### Non-Default Hyperparameters |
|
|
|
|
|
- `per_device_train_batch_size`: 1 |
|
|
- `learning_rate`: 2e-05 |
|
|
- `num_train_epochs`: 5 |
|
|
- `warmup_ratio`: 0.1 |
|
|
- `prompts`: task: classification | query: |
|
|
|
|
|
#### All Hyperparameters |
|
|
<details><summary>Click to expand</summary> |
|
|
|
|
|
- `overwrite_output_dir`: False |
|
|
- `do_predict`: False |
|
|
- `eval_strategy`: no |
|
|
- `prediction_loss_only`: True |
|
|
- `per_device_train_batch_size`: 1 |
|
|
- `per_device_eval_batch_size`: 8 |
|
|
- `per_gpu_train_batch_size`: None |
|
|
- `per_gpu_eval_batch_size`: None |
|
|
- `gradient_accumulation_steps`: 1 |
|
|
- `eval_accumulation_steps`: None |
|
|
- `torch_empty_cache_steps`: None |
|
|
- `learning_rate`: 2e-05 |
|
|
- `weight_decay`: 0.0 |
|
|
- `adam_beta1`: 0.9 |
|
|
- `adam_beta2`: 0.999 |
|
|
- `adam_epsilon`: 1e-08 |
|
|
- `max_grad_norm`: 1.0 |
|
|
- `num_train_epochs`: 5 |
|
|
- `max_steps`: -1 |
|
|
- `lr_scheduler_type`: linear |
|
|
- `lr_scheduler_kwargs`: {} |
|
|
- `warmup_ratio`: 0.1 |
|
|
- `warmup_steps`: 0 |
|
|
- `log_level`: passive |
|
|
- `log_level_replica`: warning |
|
|
- `log_on_each_node`: True |
|
|
- `logging_nan_inf_filter`: True |
|
|
- `save_safetensors`: True |
|
|
- `save_on_each_node`: False |
|
|
- `save_only_model`: False |
|
|
- `restore_callback_states_from_checkpoint`: False |
|
|
- `no_cuda`: False |
|
|
- `use_cpu`: False |
|
|
- `use_mps_device`: False |
|
|
- `seed`: 42 |
|
|
- `data_seed`: None |
|
|
- `jit_mode_eval`: False |
|
|
- `bf16`: False |
|
|
- `fp16`: False |
|
|
- `fp16_opt_level`: O1 |
|
|
- `half_precision_backend`: auto |
|
|
- `bf16_full_eval`: False |
|
|
- `fp16_full_eval`: False |
|
|
- `tf32`: None |
|
|
- `local_rank`: 0 |
|
|
- `ddp_backend`: None |
|
|
- `tpu_num_cores`: None |
|
|
- `tpu_metrics_debug`: False |
|
|
- `debug`: [] |
|
|
- `dataloader_drop_last`: False |
|
|
- `dataloader_num_workers`: 0 |
|
|
- `dataloader_prefetch_factor`: None |
|
|
- `past_index`: -1 |
|
|
- `disable_tqdm`: False |
|
|
- `remove_unused_columns`: True |
|
|
- `label_names`: None |
|
|
- `load_best_model_at_end`: False |
|
|
- `ignore_data_skip`: False |
|
|
- `fsdp`: [] |
|
|
- `fsdp_min_num_params`: 0 |
|
|
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} |
|
|
- `fsdp_transformer_layer_cls_to_wrap`: None |
|
|
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} |
|
|
- `parallelism_config`: None |
|
|
- `deepspeed`: None |
|
|
- `label_smoothing_factor`: 0.0 |
|
|
- `optim`: adamw_torch_fused |
|
|
- `optim_args`: None |
|
|
- `adafactor`: False |
|
|
- `group_by_length`: False |
|
|
- `length_column_name`: length |
|
|
- `project`: huggingface |
|
|
- `trackio_space_id`: trackio |
|
|
- `ddp_find_unused_parameters`: None |
|
|
- `ddp_bucket_cap_mb`: None |
|
|
- `ddp_broadcast_buffers`: False |
|
|
- `dataloader_pin_memory`: True |
|
|
- `dataloader_persistent_workers`: False |
|
|
- `skip_memory_metrics`: True |
|
|
- `use_legacy_prediction_loop`: False |
|
|
- `push_to_hub`: False |
|
|
- `resume_from_checkpoint`: None |
|
|
- `hub_model_id`: None |
|
|
- `hub_strategy`: every_save |
|
|
- `hub_private_repo`: None |
|
|
- `hub_always_push`: False |
|
|
- `hub_revision`: None |
|
|
- `gradient_checkpointing`: False |
|
|
- `gradient_checkpointing_kwargs`: None |
|
|
- `include_inputs_for_metrics`: False |
|
|
- `include_for_metrics`: [] |
|
|
- `eval_do_concat_batches`: True |
|
|
- `fp16_backend`: auto |
|
|
- `push_to_hub_model_id`: None |
|
|
- `push_to_hub_organization`: None |
|
|
- `mp_parameters`: |
|
|
- `auto_find_batch_size`: False |
|
|
- `full_determinism`: False |
|
|
- `torchdynamo`: None |
|
|
- `ray_scope`: last |
|
|
- `ddp_timeout`: 1800 |
|
|
- `torch_compile`: False |
|
|
- `torch_compile_backend`: None |
|
|
- `torch_compile_mode`: None |
|
|
- `include_tokens_per_second`: False |
|
|
- `include_num_input_tokens_seen`: no |
|
|
- `neftune_noise_alpha`: None |
|
|
- `optim_target_modules`: None |
|
|
- `batch_eval_metrics`: False |
|
|
- `eval_on_start`: False |
|
|
- `use_liger_kernel`: False |
|
|
- `liger_kernel_config`: None |
|
|
- `eval_use_gather_object`: False |
|
|
- `average_tokens_across_devices`: True |
|
|
- `prompts`: task: classification | query: |
|
|
- `batch_sampler`: batch_sampler |
|
|
- `multi_dataset_batch_sampler`: proportional |
|
|
- `router_mapping`: {} |
|
|
- `learning_rate_mapping`: {} |
|
|
|
|
|
</details> |
|
|
|
|
|
### Training Logs |
|
|
| Epoch | Step | Training Loss | |
|
|
|:-----:|:----:|:-------------:| |
|
|
| 1.0 | 200 | 1.3954 | |
|
|
| 2.0 | 400 | 0.8112 | |
|
|
| 3.0 | 600 | 0.0855 | |
|
|
| 4.0 | 800 | 0.0529 | |
|
|
| 5.0 | 1000 | 0.0018 | |
|
|
|
|
|
|
|
|
### Framework Versions |
|
|
- Python: 3.12.12 |
|
|
- Sentence Transformers: 5.1.2 |
|
|
- Transformers: 4.57.2 |
|
|
- PyTorch: 2.9.0+cu126 |
|
|
- Accelerate: 1.12.0 |
|
|
- Datasets: 4.0.0 |
|
|
- Tokenizers: 0.22.1 |
|
|
|
|
|
## Citation |
|
|
|
|
|
### BibTeX |
|
|
|
|
|
#### Sentence Transformers |
|
|
```bibtex |
|
|
@inproceedings{reimers-2019-sentence-bert, |
|
|
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", |
|
|
author = "Reimers, Nils and Gurevych, Iryna", |
|
|
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", |
|
|
month = "11", |
|
|
year = "2019", |
|
|
publisher = "Association for Computational Linguistics", |
|
|
url = "https://arxiv.org/abs/1908.10084", |
|
|
} |
|
|
``` |
|
|
|
|
|
#### MultipleNegativesRankingLoss |
|
|
```bibtex |
|
|
@misc{henderson2017efficient, |
|
|
title={Efficient Natural Language Response Suggestion for Smart Reply}, |
|
|
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, |
|
|
year={2017}, |
|
|
eprint={1705.00652}, |
|
|
archivePrefix={arXiv}, |
|
|
primaryClass={cs.CL} |
|
|
} |
|
|
``` |
|
|
|
|
|
<!-- |
|
|
## Glossary |
|
|
|
|
|
*Clearly define terms in order to be accessible across audiences.* |
|
|
--> |
|
|
|
|
|
<!-- |
|
|
## Model Card Authors |
|
|
|
|
|
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
|
|
--> |
|
|
|
|
|
<!-- |
|
|
## Model Card Contact |
|
|
|
|
|
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
|
|
--> |
