duy95/jina_ct26_finetuned

SentenceTransformer based on jinaai/jina-embeddings-v5-text-nano-retrieval

This is a sentence-transformers model finetuned from jinaai/jina-embeddings-v5-text-nano-retrieval. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: jinaai/jina-embeddings-v5-text-nano-retrieval
• Maximum Sequence Length: 1024 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False, 'architecture': 'EuroBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("duy95/jina_ct26_finetuned")
# Run inference
queries = [
    "That\u0027s odd, it\u0027s as though there already existed a sound and potent (let alone cheap!) treatment for coronavirus and we don\u0027t need a vaccine. So does that mean that Fauci and the mainstream media have been pulling the wool over our eyes the whole time!? \ud83e\udd14",
]
documents = [
    'The FDA-approved drug ivermectin inhibits the replication of SARS-CoV-2 in vitro Although several clinical trials are now underway to test possible therapies, the worldwide response to the COVID-19 outbreak has been largely limited to monitoring/containment. We report here that Ivermectin, an FDA-approved anti-parasitic previously shown to have broad-spectrum anti-viral activity in vitro, is an inhibitor of the causative virus (SARS-CoV-2), with a single addition to Vero-hSLAM cells 2 h post infection with SARS-CoV-2 able to effect ~5000-fold reduction in viral RNA at 48 h. Ivermectin therefore warrants further investigation for possible benefits in humans.',
    'Crop Cultivation at Wartime – Plight and Resilience of Tigray’s Agrarian Society (North Ethiopia) During the 2021 conflict in Tigray (north Ethiopia), crop cultivation has been hampered by warfare. Oxen have been looted and killed, farm inputs and tools destroyed by Ethiopian and Eritrean soldiers. Farmers felt vulnerable out in the open with their oxen. To produce, farmers evaluated risks involved with ploughing and organised lookouts. Overall, a large part of the land had been tilled in difficult conditions, and crops sown that require minimal management, without fertiliser, what led to low yields. True Colour Composite images, produced from Sentinel satellite imagery show that smallholder irrigation schemes were operational. There was a shift from commercial crops to cereals. The situation in western Tigray was particular, as there has been ethnic cleansing of the population and often the 2020 rainfed crops had even not been harvested. Overall, our findings show that the Tigrayan smallholder farming system is resilient, thanks to community self-organisation, combining common strategies of agrarian societies in wartime: spatio-temporal shift in agricultural activities to avoid the proximity with soldiers and shifts in crop types. Rather unique is the relying on communal aid, while the blockade of the Tigray region made that outmigration and off-farm income were no options for the farmers.',
    'Visualizing Speech-Generated Oral Fluid Droplets with Laser Light Scattering result on serum RT-qPCR assay for yellow fever.Liver-biopsy samples showed lobular necroinflammation, which included many foci of spotty necrosis, apoptosis, and hydropic hepatocyte degeneration in all lobular zones, without typical midzonal lesions associated with yellow fever, along with extensive hypercellularity and hypertrophy of Kupffer cells.Some of the patients had confluent necrosis.Among the patients who underwent liver biopsy, immunohistochemical analysis was positive for yellow fever antigen, which was found mainly in Kupffer-cell cytoplasm; such antigens are typically found in hepatocytes of the midzonal region in patients with acute yellow fever.All 26 patients recovered clinically with normal levels of liver enzymes.In a 2019 report, 5 researchers described rebound hepatitis associated with yellow fever in two travelers who had returned to France from Brazil.Similar to these investigators, we hypothesized that such cases of late-onset liver inflammation result from an impaired immune transition from an antiinflammatory pattern to a proinflammatory pattern owing to the presence of the virus or its antigens after the acute phase.In our study, the administration of sofosbuvir did not appear to be associated with subsequent changes in levels of liver enzymes.Thus, in this study, we characterized another possible clinical manifestation of yellow fever, a late-onset relapsing hepatitis occurring 1 to 4 months after the initial symptoms of severe acute yellow fever.Longer follow-up of the patients is needed to determine whether this condition will have serious health implications.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.4746, -0.0121,  0.1904]], dtype=torch.bfloat16)

Training Details

Training Dataset

Unnamed Dataset

• Size: 19,244 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 16 tokens mean: 52.17 tokens max: 133 tokens	min: 22 tokens mean: 343.45 tokens max: 1024 tokens

• Samples:

  sentence_0	sentence_1
  Untersuchung entdeckte bei allen getesteten Personen seltene, aber periodische RBD-spezifische Antikörper mit hoher gegenviraler Aktivität, was darauf hindeutet, dass ein Impfstoff, der solche Antikörper auslösen soll, größtenteils wirksam sein könnte.	Convergent antibody responses to SARS-CoV-2 in convalescent individuals During the coronavirus disease-2019 (COVID-19) pandemic, severe acute respiratory syndrome-related coronavirus-2 (SARS-CoV-2) has led to the infection of millions of people and has claimed hundreds of thousands of lives. The entry of the virus into cells depends on the receptor-binding domain (RBD) of the spike (S) protein of SARS-CoV-2. Although there is currently no vaccine, it is likely that antibodies will be essential for protection. However, little is known about the human antibody response to SARS-CoV-21–5. Here we report on 149 COVID-19-convalescent individuals. Plasma samples collected an average of 39 days after the onset of symptoms had variable half-maximal pseudovirus neutralizing titres; titres were less than 50 in 33% of samples, below 1,000 in 79% of samples and only 1% of samples had titres above 5,000. Antibody sequencing revealed the expansion of clones of RBD-specific memory B cells that express...
  🚨AKTUELL +++ SCIENCE- Studie unter Beteiligung von „Top-Virologe“ @c_drosten und seinem Labor zurückgezogen, da die Ergebnisse zur Entstehung von Omicron nicht nachweisbar sind und Fehlinterpretationen vermutlich auf falsch-positiven, kontaminierten Proben beruhen!	Retraction In the Research Article “Gradual emergence followed by exponential spread of the SARS-CoV-2 Omicron variant in Africa”(1), we reported data from retrospective characterization of viral genomes of putative ancestors of the SARS-CoV-2 Omicron variant from western Africa months before the first detection of Omicron. After several social media posts suggested that these putative early Omicron ancestor sequences may have been false positives, we reanalyzed our data and the residual samples. We found a mixture of different SARS-CoV-2 genomic fragments contaminating some of the samples and sequence data on which we based our analysis. The residual samples are now exhausted, and the reconstruction of evolutionary intermediates cannot be replicated. Therefore, we are retracting our Research Article. The epidemiological data are not called into question and will be made available.
  Turns out that Biden’s vaccine mandates forced babies (for which no safe dose has been established) to get mRNA via their mother’s breast milk. Is there any nastier abuse of power?	Detection of Messenger RNA COVID-19 Vaccines in Human Breast Milk This cohort study investigates the presence of COVID-19 vaccine mRNA in the expressed breast milk of lactating individuals who received the vaccination within 6 months after delivery.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• num_train_epochs: 2
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• per_device_train_batch_size: 8
• num_train_epochs: 2
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: trackio
• eval_strategy: no
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.2078	500	0.1950
0.4156	1000	0.1858
0.6234	1500	0.1594
0.8313	2000	0.1608
1.0391	2500	0.1494
1.2469	3000	0.0823
1.4547	3500	0.0804
1.6625	4000	0.0887
1.8703	4500	0.0781

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.3
• Transformers: 5.2.0
• PyTorch: 2.9.0+cu126
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}