--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:4140 - loss:CachedMultipleNegativesRankingLoss base_model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 widget: - source_sentence: 'Event: Marriage of Plum and Jade 梅玉配 Description: Marriage of Plum and Jade   Since its establishment in 1953, Fujian Provincial Experimental Min Opera Theatre has produced numerous classic operas, such as Marriage of Plum and Jade . For more than 53 years, Fujian Provincial Experimental Min Opera Theatre has successfully staged performances in America, Australia, Malaysia, Indonesia, Singapore, Taiwan, Hong Kong and Macao; receiving accolades and recognition from the different sources. Many overseas Chinese regard the esteemed theatre troupe as a “Cultural Messenger”.   Beyond a love story, the classic Min opera Marriage of Plum and Jade also shows us the courage against the fetter of feudal ethics and the pursuit of freedom and love. One day, a scholar named Xu Jinmei visited a temple in the capital city, where he met the daughter of a governor named Su Zhenyu. They fell in love at first sight. However, Miss Su was already betrothed to Zhou Yan, a frivolous and superficial man, whom she hated unceasingly.  The opera escalates to a suspenseful climax with the dramatic performance of Su’s sister-in-law, Hong Fang. She brilliantly burned a building and declared that Miss Su died accidentally in the fire. Did Miss Su perish in the fire? Will the lovers have a happy ending?   The Opera profoundly portrays the innovative and vibrant regional characteristics of the East. Through a popular comedic style of performance, one can enjoy the spirit and a feast of the arts. Let’s experience the elegancy and fastidiousness of the Min Opera, and witness the astounding not- to-be-missed show. Venue: Esplanade Theatre' sentences: - 'Theatre : Opera-Asian' - 'Dance : Salsa/Tango' - 'Concert : Dance Party' - source_sentence: 'Event: Street Art and Culture Festival [G] Description: This festival brings together contemporary street art, live graffiti performances, and culture-focused exhibitions to celebrate urban creativity, inspiring the public to appreciate the street art movement. Venue: Various Streets in Singapore' sentences: - 'Dance : Traditional/Ethnic' - 'Festival/Fair : Community & Culture' - 'Seminar/Workshop : Corporate' - source_sentence: 'Event: Cultivating a Collaborative Marriage - For Better & Forever - 9.30am (Recommended for all couples) Description: Marriage Convention 2015 Growing Together. Staying Together. Marriage is like a seed of love sown by two people, and its resulting growth is a representation of the effort put into nurturing it. Just like the saying, “Marriage is a journey, not a destination.” - it is an ongoing process that requires both parties to commit and keep working on it, or run the risk of drifting apart. Marriage Convention 2015 brings to you “ Growing Together. Staying Together” . Hear from overseas and local marriage experts who will share insights into how couples can build, grow together and strengthen their marriage. Get tips and ideas from renowned author, clinician in marriage and family therapy, Dr Sherod Miller and his wife, Dr Phyllis, on how to achieve a thriving marriage. Marriage Convention 2015 is brought to you by Families for Life.  To find out more, please visit www.toggle.sg/marriageconvention Venue: Suntec Singapore Convention and Exhibition Centre, Summit 2, Level 3' sentences: - 'Concert : Pop-Western' - 'Theatre : Children' - 'Seminar/Workshop : Family & Social' - source_sentence: 'Event: Night at the Movies Description: Alex Tan Sing is undoubtedly Singapore’s most versatile, most creative multilingual, multi-talented artiste. www.AlexTanSing.com Born & bred Singaporean, being a Singer-Songwriter-Entertainer-Comedian-Showhost-Show Producer for well over 20 years & now a Content Creator-Live Streamer-Edutainment Provider with the advent of the pandemic, he returns to his very first loves --- singing & entertaining. Join Alex for FREE as he brightens up your Saturday Night at 9pm with an hour-long live concerts-with his music, singing, comedic impressions & witty interactions. His entertaining Live Stream on Zoom is his way of giving back as he uplifts your spirits & moves you with his passion during these difficult times & his genuine love & compassion for the community. Expect vibrant uplifting originals such as A Happy Tune, Make You Smile, The Rainbow Song, Sweet Sweet Love & Sunrise to cover versions of songs from various genres as he creates the best online experience & parties, together with his special guests! Catch the new release of his Music Video against Covid-19--- Stop The Virus is a fun & easy song to educate us all in our fight against the virus!  Join him on Zoom absolutely FREE & come dressed every Saturday night in the various themes for the best online party experience & stand to win shopping vouchers for the Best Dressed & have fun with his interactive games too! Come dressed to the various themes:-  29 Aug 2020    Night at the Movies 05 Sep 2020    Disney Magic 12 Sep 2020    Saturday Night Fever 19 Sep 2020    Rock & Roll 26 Sep 2020    Back to the 80''s-Solid Gold Alex will attempt to transport you through time & space to another era, another world with his special renditions of perennial favourites, Top 40’s, Evergreens, Disco grooves & moves, 80’s Retro, from Broadway’s best to Disney Delights too! It will be the best times of your lives as Alex Tan Sing creates the most memorable & entertaining online experience!!!  Fun & Free!!! Venue: Zoom Space' sentences: - 'Concert : Jazz' - 'Musical : Western' - 'Food & Beverage : F & B Voucher' - source_sentence: 'Event: Shanghai Old Jazz Band 上海老爵士乐队音乐会 Description: Relive the golden era of Shanghai’s jazz scene with this nostalgic concert. Venue: Shanghai Music Hall' sentences: - 'Sports : Chess' - 'Dance : Modern/Contemporary' - 'Concert : Classical Vocals-Asian' pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy - cosine_accuracy_threshold - cosine_f1 - cosine_f1_threshold - cosine_precision - cosine_recall - cosine_ap - cosine_mcc model-index: - name: SentenceTransformer based on sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 results: - task: type: binary-classification name: Binary Classification dataset: name: test type: test metrics: - type: cosine_accuracy value: 0.9950316493490983 name: Cosine Accuracy - type: cosine_accuracy_threshold value: 0.5651620030403137 name: Cosine Accuracy Threshold - type: cosine_f1 value: 0.7417380660954712 name: Cosine F1 - type: cosine_f1_threshold value: 0.5227307081222534 name: Cosine F1 Threshold - type: cosine_precision value: 0.8370165745856354 name: Cosine Precision - type: cosine_recall value: 0.6659340659340659 name: Cosine Recall - type: cosine_ap value: 0.7858178723117079 name: Cosine Ap - type: cosine_mcc value: 0.7441583162870657 name: Cosine Mcc --- # SentenceTransformer based on sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) - **Maximum Sequence Length:** 128 tokens - **Output Dimensionality:** 384 dimensions - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("wublewobble/classifier_12") # Run inference sentences = [ 'Event: Shanghai Old Jazz Band 上海老爵士乐队音乐会\nDescription: Relive the golden era of Shanghai’s jazz scene with this nostalgic concert.\nVenue: Shanghai Music Hall', 'Concert : Classical Vocals-Asian', 'Dance : Modern/Contemporary', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 384] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Binary Classification * Dataset: `test` * Evaluated with [BinaryClassificationEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.BinaryClassificationEvaluator) | Metric | Value | |:--------------------------|:-----------| | cosine_accuracy | 0.995 | | cosine_accuracy_threshold | 0.5652 | | cosine_f1 | 0.7417 | | cosine_f1_threshold | 0.5227 | | cosine_precision | 0.837 | | cosine_recall | 0.6659 | | **cosine_ap** | **0.7858** | | cosine_mcc | 0.7442 | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 4,140 training samples * Columns: anchor and positive * Approximate statistics based on the first 1000 samples: | | anchor | positive | |:--------|:------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | anchor | positive | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------| | Event: The World of Swiss Education and Summer Camps - 2024 [G]
Description: Come with your children to discover Switzerland's most esteemed boarding schools, hotel management schools and summer camps on a fun-filled family adventure through the Alps, experience Swiss culture and meet admission directors in person. 3.00pm Doors open. Families are free to discover boarding schools, summer camps & network. Children’s activities begin. 3.10pm Welcome by the Ambassador of Switzerland, HE Frank Grütter. Presentations to introduce Swiss schools and summer camps. 3.30pm Lucky draw 5.00pm Close
Venue: The Embassy Room, St. Regis Hotel | Festival/Fair : Business & Professional | | Event: Wine Tasting and Sommelier Experience
Description: Join our expert sommelier for an immersive wine tasting experience. Sample premium wines, learn the art of wine pairing, and develop a deeper appreciation for fine wines in an intimate setting.
Venue: Wine Tasting Room | Lifestyle/Leisure : Service | | Event: Huayi 华艺节 2020 Storytellers' Wisdom - A Crosstalk Production 十五万大军直取西城而来
Description: What does Detective Conan and Justice Bao have in common? Can the capable Sun Wukong with his endless transformations survive in the modern society? What can Jin Yong’s stories tell you about the philosophy of ‘three’? With a focus on the Empty Fort Strategy from the classic Chinese military directives Thirty-Six Stratagems , Storytellers’ Wisdom is a lighthearted crosstalk production that enacts the various chapters of Chinese culture and history through an engaging performance filled with clever dialogue and witty humour. An original creation by the renowned Comedians Workshop from Taiwan, Storytellers’ Wisdom features a selection of the group’s best works performed by established theatre practitioners including Feng Yi-Gang and Sung Shao-Ching. Discover humorous anecdotes about life told through stories from Romance of the Three Kingdoms , Justice Bao, Sun Wukong and classics fro... | Theatre : Comedy | * Loss: [CachedMultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedmultiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 92 - `per_device_eval_batch_size`: 92 - `num_train_epochs`: 10 - `warmup_ratio`: 0.1 - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 92 - `per_device_eval_batch_size`: 92 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 5e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 10 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | Training Loss | test_cosine_ap | |:------:|:----:|:-------------:|:--------------:| | 0 | 0 | - | 0.1384 | | 1.1111 | 50 | 2.0041 | 0.5899 | | 2.2222 | 100 | 1.0715 | 0.6977 | | 3.3333 | 150 | 0.668 | 0.7221 | | 4.4444 | 200 | 0.4198 | 0.7442 | | 5.5556 | 250 | 0.2544 | 0.7490 | | 6.6667 | 300 | 0.1533 | 0.7736 | | 7.7778 | 350 | 0.0994 | 0.7806 | | 8.8889 | 400 | 0.066 | 0.7834 | | 10.0 | 450 | 0.0491 | 0.7858 | ### Framework Versions - Python: 3.11.11 - Sentence Transformers: 3.4.1 - Transformers: 4.48.3 - PyTorch: 2.6.0+cu124 - Accelerate: 1.3.0 - Datasets: 3.4.1 - Tokenizers: 0.21.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### CachedMultipleNegativesRankingLoss ```bibtex @misc{gao2021scaling, title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup}, author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan}, year={2021}, eprint={2101.06983}, archivePrefix={arXiv}, primaryClass={cs.LG} } ```