JTh34/puppy-embed-colab-d23f57a4

SentenceTransformer based on Snowflake/snowflake-arctic-embed-l

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-l. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-l
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("JTh34/puppy-embed-colab-d23f57a4")
# Run inference
sentences = [
    'What should I do if I notice signs of an ear infection in my puppy?',
    'Don’t use cotton swabs or poke into your puppy’s ear canal. You can cause irreparable damage by doing so.\n\n» Prevent water from entering the ear. If you’re bathing your pup, put a cotton ball in the opening ahead of time and wipe the ears out with a dry cotton ball when you’re finished.\n\nEar infections are quite common. Signs of infection include a red or swollen ear, discharge, head shaking, ear itching, or bad odor. If you notice any of these symp- toms, get your puppy to their doctor immediately. Left untreated, infections can cause fever, depression, irritability, and loss of balance. Your veterinarian may prescribe an ointment that you administer at home. Here’s how to use it:\n\n1. Wait until your dog’s a little sleepy. 2. Bring them to the refrigerator and swipe some peanut butter or broth at their eye level.\n\n3. As they’re licking the refrigerator, gently squeeze into their ear canal the amount of ointment specified by your veterinarian.\n\nYou don’t have to know much about the nose, though it is helpful for tipping you off to the fact that your puppy’s not feeling well. A warm nose can be caused by elevated temperature. (See the nearby sidebar, “Taking your puppy’s tempera- ture.”) However, weather conditions also can lead to dryness or fluctuation in body temperature. If you suspect that your puppy has a fever, touch their other body areas without fur (belly, paws, or the inside of their ears) or take their tem- perature. Did I mention that you have to do it rectally? What fun!',
    'More than 320 breeds are now registered worldwide. These days, being a purebred dog is like belonging to an exclusive club: Only dogs with similar looks and inter- ests get in. Although most breeds are no longer asked to do the work they were developed for, fanciers continually devote themselves to breeding and selling puppies that reflect their traditions.\n\nChoosing a specific breed enables you to predict the size, weight, and interest of your puppy. Selecting a one-of-a-kind mixed-breed puppy, and predicting or discovering the various breeds that combine to create them, allows you to make accurate descriptions about their interests and energy level as an adult dog.\n\nWhen researching a breed, mixed-breed, or designer-mixed-breed, try to meet at least three adult dogs of the same breed or mix-breeds. All puppies are cute and adorable, but they grow up in the blink of an eye, so make sure you like the look and personality of the dog your puppy will become.\n\nWhether you’re considering a purebred, mixed-breed, or designer-mixed-breed, take a good, hard look at your lifestyle now and project out five to ten years. How might a certain breed’s or mixed breed’s interests and energy level play out in your home?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.6667
cosine_accuracy@3	0.8533
cosine_accuracy@5	0.9267
cosine_accuracy@10	0.96
cosine_precision@1	0.6667
cosine_precision@3	0.2844
cosine_precision@5	0.1853
cosine_precision@10	0.096
cosine_recall@1	0.6667
cosine_recall@3	0.8533
cosine_recall@5	0.9267
cosine_recall@10	0.96
cosine_ndcg@10	0.8202
cosine_mrr@10	0.7746
cosine_map@100	0.7766

Training Details

Training Dataset

Unnamed Dataset

• Size: 700 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 700 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 15 tokens mean: 23.47 tokens max: 40 tokens	min: 44 tokens mean: 331.62 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1
  What techniques can I apply to train my puppy to stay calm and still when I approach, similar to how llamas are trained?	are each a process, defined by results. Negative reinforcers can be used effectively to train behavior, and even though aversive stimuli are involved, the process can be relatively benign. Here (with thanks to llama expert Jim Logan) is a nice use of the negative reinforcer with a semidomestic animal, the llama, kept in the United States as pets and elsewhere as pack animals and for their wool. Llamas are timid and shy, like horses. Unless handled a lot when young, they can be hard to approach. So, while operant conditioning with a food reinforcer works splendidly with llamas, if a llama is too skittish to come close enough to a person to take the food, here's what modern llama trainers do. They use a clicker as a signal to tell the llama that what it is doing has earned a reinforcer, but the primary or real reinforcer is the removal of a negative reinforcer, an aversive. In effect, you say to the llama, "Will you stand still if I approach within thirty feet? Yes? Good. I'll click m...
  What are the best ways to socialize my hound puppy with household pets to avoid any chasing instincts?	When these puppies are exercised, directed, and included, no group is more happy-go-lucky and accepting of life’s random chaos. But when they don’t get enough playtime or training, they can be hyperactive and destructive. Even though the loyal and cheerful dogs in the Sporting group have well-earned reputations as patient family pets, they need both mental and physical stimulation. They can’t cope with long hours of isolation; coupled with a lack of exercise, this isolation fuels anxiety. An unhappy Sporting dog is destructive, hyperactive, and impulsive. This isn’t a good mix — especially for your couch and end table. The dogs in the Hound group are a happy lot with a 1-track mind; their fascination with hunting propels them through life and allows them plenty of opportunity for employment. Though you may have no interest in hunting a fox, chasing deer, or treeing a raccoon, your hound puppy probably will. Originally teamed in pairs or packs, each hound was prized for their instinc...
  What are the top five dog breeds recommended for first-time owners, and what makes them suitable for beginners?	Now don’t get me wrong, I’m not trying to put you off. I love dogs, and I think everyone can benefit from having one in their life. If you’re still unsure which breed is right for you, let me suggest a few that I think make brilliant first dogs. Every trainer and dog lover will tell you something different about what breeds are best for you. At the end of the day, it’s your choice. But these are my top five dogs for a first-time owner. I’ve chosen them based on a decade’s experience of working with breeds of all sorts and seeing firsthand some of the common problems among dogs. These five are all typically easygoing, good-natured, smart, and willing to learn. The Rottweiler man in me can observe occasional “over-friendliness” in these breeds, but that’s not a bad thing for beginners, and basically makes them perfect for the novice trainer. If your heart is set on an American bully, but you’ve never had a dog before, think about having one of these dogs first—you can always grow your family l...

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 5
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_ndcg@10
0.5682	25	0.7986
1.0	44	0.8182
1.1364	50	0.8224
1.7045	75	0.8181
2.0	88	0.8224
2.2727	100	0.8205
2.8409	125	0.8221
3.0	132	0.8235
3.4091	150	0.8205
3.9773	175	0.8178
4.0	176	0.8184
4.5455	200	0.8204
5.0	220	0.8202

Framework Versions

• Python: 3.11.12
• Sentence Transformers: 4.1.0
• Transformers: 4.52.2
• PyTorch: 2.6.0+cu124
• Accelerate: 1.7.0
• Datasets: 2.14.4
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}