Matryoshka Representation Learning
Paper • 2205.13147 • Published • 26
How to use IoannisKat1/intfloat-multilingual-e5-large-new2 with sentence-transformers:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("IoannisKat1/intfloat-multilingual-e5-large-new2")
sentences = [
"When should supervisory authorities monitor the application of the provisions?",
"**Court (Civil/Criminal): Civil** \n**Provisions:** \n**Time of commission of the act:** \n**Outcome (not guilty, guilty):** \n**Reasoning:** Partially accepts the lawsuit. \n**Facts:** The plaintiff, who works as a lawyer, maintains a savings account with the defendant banking corporation under account number GR.............. Pursuant to a contract dated June 11, 2010, established in Thessaloniki between the defendant and the plaintiff, the plaintiff was granted access to the electronic banking system (e-banking) to conduct banking transactions remotely. On October 10, 2020, the plaintiff fell victim to electronic fraud through the \"phishing\" method, whereby an unknown perpetrator managed to extract and transfer €3,000.00 from the plaintiff’s account to another account of the same bank. Specifically, on that day at 6:51 a.m., the plaintiff received an email from the sender \".........\", with the address ..........., informing him that his debit card had been suspended and that online payments and cash withdrawals could not be made until the issue was resolved. The email urged him to confirm his details within the next 72 hours by following a link titled \"card activation.\" \nThe plaintiff read the above email on his mobile phone around 8:00 a.m., and believing it came from the defendant, he followed the instructions and accessed a website that was identical (a clone) to that of the defendant. On this page, he was asked to enter his login credentials to connect to the service, which he did, and he was subsequently asked to input his debit card details for the alleged activation, which he also provided. Then, to complete the process, a number was sent to his mobile phone at 8:07 a.m. from the sender ........, which he entered, and two minutes later he received a message from the same sender in English stating that the quick access code had been activated on his mobile. A few minutes later, at 8:18 a.m., he received an email from the defendant informing him of the transfer of €3,000.00 from his account to account number GR ........... held at the same bank, with the beneficiary's details being .......... As soon as the plaintiff read this, he immediately called the defendant's call center and canceled his debit card, the access codes for the service ......., and locked the application .......... At the same time, he verbally submitted a request to dispute and cancel the contested transaction, and in a subsequent phone call, he also canceled his credit card. On the same day, he also sent an email to the defendant informing them in writing of the above and requesting the cancellation of the transaction and the return of the amount of €3,000.00 to his account, as this transfer was not made by him but by an unknown perpetrator through electronic fraud and was not approved by him. It should also be noted that the plaintiff, as the sole beneficiary according to the aforementioned contract for using the defendant's Internet Banking service, never received any update via SMS or the VIBER application from the bank regarding the transaction details before its completion, nor did he receive a one-time code (OTP) to approve the contested transaction. He subsequently filed a complaint against unknown persons at the Cyber Crime Division for the crime of fraud. The defendant sent an email to the plaintiff on October 16, 2020, informing him that his request had been forwarded to the appropriate department of the bank for investigation, stating that the bank would never send him an email or SMS asking him to enter his personal data and that as of October 7, 2020, there was a notice posted for its customers regarding malicious attempts to steal personal data in the \"Our News\" section on ....... A month after the disputed incident, on November 10, 2020, an amount of €2,296.82 was transferred to the plaintiff's account from the account to which the fraudulent credit had been made. The plaintiff immediately sent an email to the defendant asking to be informed whether this transfer was a return of part of the amount that had been illegally withdrawn from his account and requested the return of the remaining amount of €703.18. In its response dated January 13, 2021, the defendant confirmed that the aforementioned amount indeed came from the account to which the fraudulent credit had been made, following a freeze of that account initiated by the defendant during the investigation of the incident, but refused to return the remaining amount, claiming it bore no responsibility for the leak of the personal codes to third parties, according to the terms of the service contract established between them. \nFrom the entirety of the evidence presented to the court, there is no indication of the authenticity of the contested transaction, as the plaintiff did not give his consent for the execution of the transfer of the amount of €3,000.00, especially in light of the provision in Article 72 paragraph 2 of Law 4537/2018 stating that the mere use of the Internet Banking service by the plaintiff does not necessarily constitute sufficient evidence that the payer approved the payment action. Specifically, it was proven that the contested transaction was not carried out following a strong identification of the plaintiff – the sole beneficiary of the account – and his approval, as the latter may have entered his personal codes on the counterfeit website; however, he was never informed, before the completion of the contested transaction, of the amount that would be transferred from his account to a third-party account, nor did he receive on his mobile phone, either via SMS or through the VIBER application or any other means, the one-time code - extra PIN for its completion, which he was required to enter to approve the contested transaction (payment action) and thus complete his identification, a fact that was not countered by any evidence from the defendant. Furthermore, it is noted that the defendant's claims that it bears no responsibility under the terms of the banking services contract, whereby it is not liable for any damage to its customer in cases of unauthorized use of their personal access codes to the Internet Banking service, are to be rejected as fundamentally unfounded. This is because the aforementioned contractual terms are invalid according to the provision of Article 103 of Law 4537/2018, as they contradict the provisions of Articles 71, 73, and 92 of the same Law, which provide for the provider's universal liability and its exemption only for unusual and unforeseen circumstances that are beyond the control of the party invoking them and whose consequences could not have been avoided despite all efforts to the contrary; these provisions establish mandatory law in favor of users, as according to Article 103 of Law 4537/2018, payment service providers are prohibited from deviating from the provisions to the detriment of payment service users, unless the possibility of deviation is explicitly provided and they can decide to offer only more favorable terms to payment service users; the aforementioned contractual terms do not constitute more favorable terms but rather disadvantageous terms for the payment service user. In this case, however, the defendant did not prove the authenticity of the transaction and its approval by the plaintiff and did not invoke, nor did any unusual and unforeseen circumstances beyond its control, the consequences of which could not have been avoided despite all efforts to the contrary, come to light. Therefore, the contested transaction transferring the amount of €3,000.00 is considered, in the absence of demonstrable consent from the plaintiff, unapproved according to the provisions of Article 64 of Law 4537/2018, and the defendant's contrary claims are rejected, especially since the plaintiff proceeded, according to Article 71 paragraph 1 of Law 4537/2018, without undue delay to notify the defendant regarding the contested unapproved payment action. Consequently, the defendant is liable for compensating the plaintiff for the positive damage he suffered under Article 73 of Law 4537/2018 and is obliged to pay him the requested amount of €703.18, while the plaintiff’s fault in the occurrence of this damage cannot be established, as he entered his personal details in an online environment that was a faithful imitation of that of the defendant, as evidenced by the comparison of the screenshots of the fake website and the real website provided by the plaintiff, a fact that he could not have known while being fully convinced that he was transacting with the defendant. Furthermore, the defendant’s liability to compensate the plaintiff is based on the provision of Article 8 of Law 2251/1994, which applies in this case, as the plaintiff's damage resulted from inadequate fulfillment of its obligations in the context of providing its services, but also on the provision of Article 914 of the Civil Code in the sense of omission on its part of unlawfully and culpably imposed actions. In this case, given that during the relevant period there had been a multitude of similar incidents of fraud against the defendant's customers, the latter, as a service provider to the consumer public and bearing transactional obligations of care and security towards them, displayed gross negligence regarding the security provided for electronic transaction services, which was compromised by the fraudulent theft of funds, as it did not comply with all required high-security measures for executing the contested transaction, failing to implement the strict customer identification verification process and to check the authenticity of the account to which the funds were sent, thus not assuming the suspicious nature of the transaction, did not adopt comprehensive and improved protective measures to fully protect its customers against malicious attacks and online fraud and to prevent the infiltration of unauthorized third parties, nor did it fulfill its obligations to inform, accurately inform, and warn its consumers - customers, as it failed to adequately inform them of attempts to steal their personal data through the sending of informative emails or SMS, while merely posting in a section rather than on a central banner (as it later did) does not constitute adequate information such that it meets the requirement of protecting its customers and the increased safeguarding of their interests. Although the plaintiff acted promptly and informed the defendant on the same day about the contested incident, the defendant did not act as promptly regarding the investigation of the incident and the freezing of the account that held the fraudulent credit to prevent the plaintiff's loss, but only returned part of the funds to the plaintiff a month later. This behavior, beyond being culpable due to gross negligence, was also unlawful, as it would have been illegal even without the contractual relationship, as contrary to the provisions of Law 4537/2018 and Law 2251/1994, regarding the lack of security of the services that the consumer is legitimately entitled to expect, as well as the building of trust that is essential in banking transactions, elements that it was obligated to provide within the sphere of the services offered, and contrary to the principles of good faith and commercial ethics, as crystallized in the provision of Article 288 of the Civil Code, as well as the general duty imposed by Article 914 of the Civil Code not to cause harm to another culpably. This resulted not only in positive damage to the plaintiff but also in causing him moral harm consisting of his mental distress and the disruption, agitation, and sorrow he experienced, for which he must be awarded financial compensation. Taking into account all the general circumstances of the case, the extent of the plaintiff's damage, the severity of the defendant's fault, the mental distress suffered by the plaintiff, the insecurity he felt regarding his deposits, the sorrow he experienced, and the stress caused by his financial loss, which occurred during the pandemic period when his earnings from his professional activity had significantly decreased, as well as the financial and social situation of the parties, it is the court's opinion that he should be granted, as financial compensation for his moral harm, an amount of €250.00, which is deemed reasonable and fair. Therefore, the total monetary amount that the plaintiff is entitled to for his positive damage and financial compensation for the moral harm suffered amounts to a total of (€703.18 + €250.00) = €953.18.",
"Court (Civil/Criminal): Criminal \nProvisions: Article 42 paragraphs 1, 2, 3, and 7 of Law 4557/2018 \nTime of commission of the act: \nOutcome (not guilty, guilty): \nReasoning: Obligation of the payment service provider, such as banks, to inform their contracting customer after receiving a relevant order for a payment to be made on their behalf. Content of the above notification at the stage of receiving the payment order and during its execution. Terms of liability for the provider regarding compensation for non-execution, erroneous, or delayed execution of payment transactions. In particular, in the case of an unauthorized or erroneous payment, the user is required to notify the provider within a specified timeframe as soon as they become aware of the corresponding transaction. The provisions of Law 4357/2018 establish mandatory legal regulations in favor of users of payment services and cannot be contractually modified to their detriment, but only to their benefit. Defenses available to payment service providers to relieve them of liability. Burden of proof distribution between the parties. This responsibility of banks may also stem from Law 2251/1994, as they provide services to the public and are considered suppliers. Conditions for supplier liability under the aforementioned legislation. Distribution of the burden of proof between the litigants to demonstrate liability for compensation under Law 2251/1994. Terms of concurrency between contractual and tort liability for compensation. The court partially accepts the lawsuit. \nFacts: On 01/09/2021, an unknown perpetrator sent an email to her from the electronic address “...............”, in which they stated that for security reasons she needed to confirm her account with the bank ....... Not realizing that it was a scam, she followed the attached hyperlink, entered her personal information, resulting in an unknown perpetrator intercepting her online banking credentials and making a transfer totaling 7,000.00 euros to account number ................ of the bank ......... \nC) The beneficiary of the aforementioned account is ......... born on 16/05/1995 in the Municipality of ........., residing at .............., with ID number .................... issued on 02/10/2009 by T.A .............. and tax number ............. from the tax office ................... \nD) The criminal proceeds reportedly arising from the above criminal activity amount to a total of seven thousand euros (7,000.00€). Following the above, serious suspicions arise that the aforementioned criminal proceeds transferred to the aforementioned bank account were unlawfully appropriated by her and subsequently mixed with other legally held assets, which she used in her overall economic activities, aiming to launder them, thus concealing their true origin and making it impossible for them to be seized. Therefore, there are reasonable suspicions that the aforementioned individual committed not only the primary offense but also the criminal act of “Money Laundering” (Article 2 §1 a, d of Law 4557/2018, in conjunction with Article 4 subparagraph z of the same law as it stands, as well as Article 39 paragraph 1 subparagraphs a & c of the same Law 4557/2018). Because there are serious suspicions that the bank account numbered .......... maintained at the bank ....................., whose beneficiary is ..... (Tax ID ....................), contains part of the monetary amount from the aforementioned criminal activity that was placed behind banking secrecy to conceal its true origin and ultimately to launder it. Because, in this case, part of the criminal proceeds has been found while the remainder has not been found in its entirety, there is a lawful reason and an urgent case for prohibiting the sale or any other transfer of the following assets, given that they are subject to seizure and forfeiture according to Articles 40 and 42 of Law 4557/2018 as they currently stand.",
"The supervisory authorities should monitor the application of the provisions pursuant to this Regulation and contribute to its consistent application throughout the Union, in order to protect natural persons in relation to the processing of their personal data and to facilitate the free flow of personal data within the internal market. For that purpose, the supervisory authorities should cooperate with each other and with the Commission, without the need for any agreement between Member States on the provision of mutual assistance or on such cooperation."
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]This is a sentence-transformers model finetuned from intfloat/multilingual-e5-large. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'XLMRobertaModel'})
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("IoannisKat1/intfloat-multilingual-e5-large-new2")
# Run inference
sentences = [
'What can the contract be based on, besides individual contracts, in part according to this excerpt?',
"1.Where processing is to be carried out on behalf of a controller, the controller shall use only processors providing sufficient guarantees to implement appropriate technical and organisational measures in such a manner that processing will meet the requirements of this Regulation and ensure the protection of the rights of the data subject.\n2.The processor shall not engage another processor without prior specific or general written authorisation of the controller. In the case of general written authorisation, the processor shall inform the controller of any intended changes concerning the addition or replacement of other processors, thereby giving the controller the opportunity to object to such changes.\n3.Processing by a processor shall be governed by a contract or other legal act under Union or Member State law, that is binding on the processor with regard to the controller and that sets out the subject-matter and duration of the processing, the nature and purpose of the processing, the type of personal data and categories of data subjects and the obligations and rights of the controller. That contract or other legal act shall stipulate, in particular, that the processor: (a) processes the personal data only on documented instructions from the controller, including with regard to transfers of personal data to a third country or an international organisation, unless required to do so by Union or Member State law to which the processor is subject; in such a case, the processor shall inform the controller of that legal requirement before processing, unless that law prohibits such information on important grounds of public interest; (b) ensures that persons authorised to process the personal data have committed themselves to confidentiality or are under an appropriate statutory obligation of confidentiality; (c) takes all measures required pursuant to Article 32; (d) respects the conditions referred to in paragraphs 2 and 4 for engaging another processor; (e) taking into account the nature of the processing, assists the controller by appropriate technical and organisational measures, insofar as this is possible, for the fulfilment of the controller's obligation to respond to requests for exercising the data subject's rights laid down in Chapter III; (f) assists the controller in ensuring compliance with the obligations pursuant to Articles 32 to 36 taking into account the nature of processing and the information available to the processor; (g) at the choice of the controller, deletes or returns all the personal data to the controller after the end of the provision of services relating to processing, and deletes existing copies unless Union or Member State law requires storage of the personal data; (h) makes available to the controller all information necessary to demonstrate compliance with the obligations laid down in this Article and allow for and contribute to audits, including inspections, conducted by the controller or another auditor mandated by the controller. 4.5.2016 L 119/49 With regard to point (h) of the first subparagraph, the processor shall immediately inform the controller if, in its opinion, an instruction infringes this Regulation or other Union or Member State data protection provisions.\n4.Where a processor engages another processor for carrying out specific processing activities on behalf of the controller, the same data protection obligations as set out in the contract or other legal act between the controller and the processor as referred to in paragraph 3 shall be imposed on that other processor by way of a contract or other legal act under Union or Member State law, in particular providing sufficient guarantees to implement appropriate technical and organisational measures in such a manner that the processing will meet the requirements of this Regulation. Where that other processor fails to fulfil its data protection obligations, the initial processor shall remain fully liable to the controller for the performance of that other processor's obligations.\n5.Adherence of a processor to an approved code of conduct as referred to in Article 40 or an approved certification mechanism as referred to in Article 42 may be used as an element by which to demonstrate sufficient guarantees as referred to in paragraphs 1 and 4 of this Article.\n6.Without prejudice to an individual contract between the controller and the processor, the contract or the other legal act referred to in paragraphs 3 and 4 of this Article may be based, in whole or in part, on standard contractual clauses referred to in paragraphs 7 and 8 of this Article, including when they are part of a certification granted to the controller or processor pursuant to Articles 42 and 43\n7.The Commission may lay down standard contractual clauses for the matters referred to in paragraph 3 and 4 of this Article and in accordance with the examination procedure referred to in Article 93(2).\n8.A supervisory authority may adopt standard contractual clauses for the matters referred to in paragraph 3 and 4 of this Article and in accordance with the consistency mechanism referred to in Article 63\n9.The contract or the other legal act referred to in paragraphs 3 and 4 shall be in writing, including in electronic form.\n10.Without prejudice to Articles 82, 83 and 84, if a processor infringes this Regulation by determining the purposes and means of processing, the processor shall be considered to be a controller in respect of that processing.",
'1.The Board shall draw up an annual report regarding the protection of natural persons with regard to processing in the Union and, where relevant, in third countries and international organisations. The report shall be made public and be transmitted to the European Parliament, to the Council and to the Commission.\n2.The annual report shall include a review of the practical application of the guidelines, recommendations and best practices referred to in point (l) of Article 70(1) as well as of the binding decisions referred to in Article 65.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.4491, 0.1215],
# [0.4491, 1.0000, 0.2320],
# [0.1215, 0.2320, 1.0000]])
dim_1024InformationRetrievalEvaluator with these parameters:{
"truncate_dim": 1024
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.3965 |
| cosine_accuracy@3 | 0.4419 |
| cosine_accuracy@5 | 0.4773 |
| cosine_accuracy@10 | 0.5303 |
| cosine_precision@1 | 0.3965 |
| cosine_precision@3 | 0.3864 |
| cosine_precision@5 | 0.3682 |
| cosine_precision@10 | 0.329 |
| cosine_recall@1 | 0.083 |
| cosine_recall@3 | 0.2081 |
| cosine_recall@5 | 0.2836 |
| cosine_recall@10 | 0.3941 |
| cosine_ndcg@10 | 0.4599 |
| cosine_mrr@10 | 0.4271 |
| cosine_map@100 | 0.5169 |
dim_768InformationRetrievalEvaluator with these parameters:{
"truncate_dim": 768
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.3939 |
| cosine_accuracy@3 | 0.4419 |
| cosine_accuracy@5 | 0.5025 |
| cosine_accuracy@10 | 0.5505 |
| cosine_precision@1 | 0.3939 |
| cosine_precision@3 | 0.3855 |
| cosine_precision@5 | 0.3763 |
| cosine_precision@10 | 0.3447 |
| cosine_recall@1 | 0.0817 |
| cosine_recall@3 | 0.2045 |
| cosine_recall@5 | 0.2833 |
| cosine_recall@10 | 0.4024 |
| cosine_ndcg@10 | 0.4717 |
| cosine_mrr@10 | 0.431 |
| cosine_map@100 | 0.5281 |
dim_512InformationRetrievalEvaluator with these parameters:{
"truncate_dim": 512
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.3788 |
| cosine_accuracy@3 | 0.4318 |
| cosine_accuracy@5 | 0.4848 |
| cosine_accuracy@10 | 0.5303 |
| cosine_precision@1 | 0.3788 |
| cosine_precision@3 | 0.3729 |
| cosine_precision@5 | 0.3606 |
| cosine_precision@10 | 0.326 |
| cosine_recall@1 | 0.0797 |
| cosine_recall@3 | 0.2026 |
| cosine_recall@5 | 0.2789 |
| cosine_recall@10 | 0.3943 |
| cosine_ndcg@10 | 0.4537 |
| cosine_mrr@10 | 0.4149 |
| cosine_map@100 | 0.5082 |
dim_256InformationRetrievalEvaluator with these parameters:{
"truncate_dim": 256
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.3712 |
| cosine_accuracy@3 | 0.399 |
| cosine_accuracy@5 | 0.4419 |
| cosine_accuracy@10 | 0.4949 |
| cosine_precision@1 | 0.3712 |
| cosine_precision@3 | 0.3552 |
| cosine_precision@5 | 0.3328 |
| cosine_precision@10 | 0.2995 |
| cosine_recall@1 | 0.0798 |
| cosine_recall@3 | 0.1976 |
| cosine_recall@5 | 0.26 |
| cosine_recall@10 | 0.3649 |
| cosine_ndcg@10 | 0.4262 |
| cosine_mrr@10 | 0.3972 |
| cosine_map@100 | 0.4892 |
dim_128InformationRetrievalEvaluator with these parameters:{
"truncate_dim": 128
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.3359 |
| cosine_accuracy@3 | 0.3763 |
| cosine_accuracy@5 | 0.4192 |
| cosine_accuracy@10 | 0.4798 |
| cosine_precision@1 | 0.3359 |
| cosine_precision@3 | 0.3266 |
| cosine_precision@5 | 0.3111 |
| cosine_precision@10 | 0.2803 |
| cosine_recall@1 | 0.0742 |
| cosine_recall@3 | 0.1853 |
| cosine_recall@5 | 0.2527 |
| cosine_recall@10 | 0.3572 |
| cosine_ndcg@10 | 0.4004 |
| cosine_mrr@10 | 0.3675 |
| cosine_map@100 | 0.4576 |
dim_64InformationRetrievalEvaluator with these parameters:{
"truncate_dim": 64
}
| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.2778 |
| cosine_accuracy@3 | 0.3157 |
| cosine_accuracy@5 | 0.3434 |
| cosine_accuracy@10 | 0.3864 |
| cosine_precision@1 | 0.2778 |
| cosine_precision@3 | 0.2685 |
| cosine_precision@5 | 0.253 |
| cosine_precision@10 | 0.2222 |
| cosine_recall@1 | 0.0651 |
| cosine_recall@3 | 0.1629 |
| cosine_recall@5 | 0.2204 |
| cosine_recall@10 | 0.3079 |
| cosine_ndcg@10 | 0.3306 |
| cosine_mrr@10 | 0.303 |
| cosine_map@100 | 0.3986 |
anchor and positive| anchor | positive | |
|---|---|---|
| type | string | string |
| details |
|
|
| anchor | positive |
|---|---|
What measures should each supervisory authority take to facilitate the submission of complaints? |
Every data subject should have the right to lodge a complaint with a single supervisory authority, in particular in the Member State of his or her habitual residence, and the right to an effective judicial remedy in accordance 4.5.2016 L 119/25 Official Journal of the European Union EN with Article 47 of the Charter if the data subject considers that his or her rights under this Regulation are infringed or where the supervisory authority does not act on a complaint, partially or wholly rejects or dismisses a complaint or does not act where such action is necessary to protect the rights of the data subject. The investigation following a complaint should be carried out, subject to judicial review, to the extent that is appropriate in the specific case. The supervisory authority should inform the data subject of the progress and the outcome of the complaint within a reasonable period. If the case requires further investigation or coordination with another supervisory authority, intermed... |
What did the evidence not indicate? |
Court (Civil/Criminal): Civil |
What was the amount transferred from her account? |
Court (Civil/Criminal): Criminal |
MatryoshkaLoss with these parameters:{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
1024,
768,
512,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1,
1,
1
],
"n_dims_per_step": -1
}
eval_strategy: epochper_device_train_batch_size: 2gradient_accumulation_steps: 8learning_rate: 2e-05num_train_epochs: 12lr_scheduler_type: cosinewarmup_ratio: 0.1bf16: Truetf32: Trueload_best_model_at_end: Trueoptim: adamw_torch_fusedbatch_sampler: no_duplicatesoverwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 2per_device_eval_batch_size: 8per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 8eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 2e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 12max_steps: -1lr_scheduler_type: cosinelr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Truelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}tp_size: 0fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torch_fusedoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Nonehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseinclude_for_metrics: []eval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters: auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseeval_use_gather_object: Falseaverage_tokens_across_devices: Falseprompts: Nonebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}| Epoch | Step | Training Loss | dim_1024_cosine_ndcg@10 | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
|---|---|---|---|---|---|---|---|---|
| -1 | -1 | - | 0.4214 | 0.4041 | 0.3894 | 0.3254 | 0.2396 | 0.1752 |
| 0.1013 | 10 | 20.5156 | - | - | - | - | - | - |
| 0.2025 | 20 | 19.5068 | - | - | - | - | - | - |
| 0.3038 | 30 | 17.3704 | - | - | - | - | - | - |
| 0.4051 | 40 | 17.1827 | - | - | - | - | - | - |
| 0.5063 | 50 | 16.6068 | - | - | - | - | - | - |
| 0.6076 | 60 | 16.4217 | - | - | - | - | - | - |
| 0.7089 | 70 | 15.5364 | - | - | - | - | - | - |
| 0.8101 | 80 | 13.3384 | - | - | - | - | - | - |
| 0.9114 | 90 | 15.6398 | - | - | - | - | - | - |
| 0.9924 | 98 | - | 0.4681 | 0.4793 | 0.4615 | 0.4166 | 0.3638 | 0.2744 |
| 1.0203 | 100 | 14.2832 | - | - | - | - | - | - |
| 1.1215 | 110 | 10.0518 | - | - | - | - | - | - |
| 1.2228 | 120 | 10.3808 | - | - | - | - | - | - |
| 1.3241 | 130 | 10.9265 | - | - | - | - | - | - |
| 1.4253 | 140 | 10.2787 | - | - | - | - | - | - |
| 1.5266 | 150 | 10.9999 | - | - | - | - | - | - |
| 1.6278 | 160 | 6.8139 | - | - | - | - | - | - |
| 1.7291 | 170 | 7.986 | - | - | - | - | - | - |
| 1.8304 | 180 | 9.2866 | - | - | - | - | - | - |
| 1.9316 | 190 | 9.2912 | - | - | - | - | - | - |
| 1.9924 | 196 | - | 0.4772 | 0.4612 | 0.4645 | 0.3945 | 0.3636 | 0.2986 |
| 2.0405 | 200 | 9.9778 | - | - | - | - | - | - |
| 2.1418 | 210 | 7.8425 | - | - | - | - | - | - |
| 2.2430 | 220 | 7.7307 | - | - | - | - | - | - |
| 2.3443 | 230 | 6.6603 | - | - | - | - | - | - |
| 2.4456 | 240 | 5.8628 | - | - | - | - | - | - |
| 2.5468 | 250 | 7.5488 | - | - | - | - | - | - |
| 2.6481 | 260 | 8.5646 | - | - | - | - | - | - |
| 2.7494 | 270 | 7.7542 | - | - | - | - | - | - |
| 2.8506 | 280 | 6.046 | - | - | - | - | - | - |
| 2.9519 | 290 | 4.2612 | - | - | - | - | - | - |
| 2.9924 | 294 | - | 0.4663 | 0.4403 | 0.4505 | 0.4067 | 0.3673 | 0.3267 |
| 3.0608 | 300 | 4.7943 | - | - | - | - | - | - |
| 3.1620 | 310 | 7.1236 | - | - | - | - | - | - |
| 3.2633 | 320 | 7.8359 | - | - | - | - | - | - |
| 3.3646 | 330 | 7.2883 | - | - | - | - | - | - |
| 3.4658 | 340 | 6.8383 | - | - | - | - | - | - |
| 3.5671 | 350 | 6.1145 | - | - | - | - | - | - |
| 3.6684 | 360 | 5.8697 | - | - | - | - | - | - |
| 3.7696 | 370 | 5.3551 | - | - | - | - | - | - |
| 3.8709 | 380 | 7.7562 | - | - | - | - | - | - |
| 3.9722 | 390 | 4.1286 | - | - | - | - | - | - |
| 3.9924 | 392 | - | 0.5004 | 0.4837 | 0.4654 | 0.4095 | 0.3771 | 0.3238 |
| 4.0810 | 400 | 6.6456 | - | - | - | - | - | - |
| 4.1823 | 410 | 7.8539 | - | - | - | - | - | - |
| 4.2835 | 420 | 5.2917 | - | - | - | - | - | - |
| 4.3848 | 430 | 5.5573 | - | - | - | - | - | - |
| 4.4861 | 440 | 6.957 | - | - | - | - | - | - |
| 4.5873 | 450 | 6.3068 | - | - | - | - | - | - |
| 4.6886 | 460 | 6.0006 | - | - | - | - | - | - |
| 4.7899 | 470 | 6.1419 | - | - | - | - | - | - |
| 4.8911 | 480 | 5.0808 | - | - | - | - | - | - |
| 4.9924 | 490 | 6.0219 | 0.4752 | 0.4754 | 0.4581 | 0.4243 | 0.3931 | 0.3410 |
| 5.1013 | 500 | 3.7305 | - | - | - | - | - | - |
| 5.2025 | 510 | 4.827 | - | - | - | - | - | - |
| 5.3038 | 520 | 3.1179 | - | - | - | - | - | - |
| 5.4051 | 530 | 6.141 | - | - | - | - | - | - |
| 5.5063 | 540 | 6.3686 | - | - | - | - | - | - |
| 5.6076 | 550 | 4.9029 | - | - | - | - | - | - |
| 5.7089 | 560 | 3.6987 | - | - | - | - | - | - |
| 5.8101 | 570 | 5.5046 | - | - | - | - | - | - |
| 5.9114 | 580 | 5.0166 | - | - | - | - | - | - |
| 5.9924 | 588 | - | 0.4737 | 0.4748 | 0.4567 | 0.4185 | 0.3890 | 0.3435 |
| 6.0203 | 590 | 3.9625 | - | - | - | - | - | - |
| 6.1215 | 600 | 6.7869 | - | - | - | - | - | - |
| 6.2228 | 610 | 3.6329 | - | - | - | - | - | - |
| 6.3241 | 620 | 6.2702 | - | - | - | - | - | - |
| 6.4253 | 630 | 3.3559 | - | - | - | - | - | - |
| 6.5266 | 640 | 4.0666 | - | - | - | - | - | - |
| 6.6278 | 650 | 3.5322 | - | - | - | - | - | - |
| 6.7291 | 660 | 4.8831 | - | - | - | - | - | - |
| 6.8304 | 670 | 6.6302 | - | - | - | - | - | - |
| 6.9316 | 680 | 5.7623 | - | - | - | - | - | - |
| 6.9924 | 686 | - | 0.4687 | 0.4713 | 0.4520 | 0.4194 | 0.3950 | 0.3338 |
| 7.0405 | 690 | 5.5453 | - | - | - | - | - | - |
| 7.1418 | 700 | 2.8097 | - | - | - | - | - | - |
| 7.2430 | 710 | 3.5171 | - | - | - | - | - | - |
| 7.3443 | 720 | 3.5449 | - | - | - | - | - | - |
| 7.4456 | 730 | 4.6169 | - | - | - | - | - | - |
| 7.5468 | 740 | 3.567 | - | - | - | - | - | - |
| 7.6481 | 750 | 5.7251 | - | - | - | - | - | - |
| 7.7494 | 760 | 3.7201 | - | - | - | - | - | - |
| 7.8506 | 770 | 3.1051 | - | - | - | - | - | - |
| 7.9519 | 780 | 3.9642 | - | - | - | - | - | - |
| 7.9924 | 784 | - | 0.4599 | 0.4717 | 0.4537 | 0.4262 | 0.4004 | 0.3306 |
| 8.0608 | 790 | 3.923 | - | - | - | - | - | - |
| 8.1620 | 800 | 3.52 | - | - | - | - | - | - |
| 8.2633 | 810 | 3.1567 | - | - | - | - | - | - |
| 8.3646 | 820 | 6.1725 | - | - | - | - | - | - |
| 8.4658 | 830 | 3.259 | - | - | - | - | - | - |
| 8.5671 | 840 | 6.6232 | - | - | - | - | - | - |
| 8.6684 | 850 | 3.7085 | - | - | - | - | - | - |
| 8.7696 | 860 | 4.0311 | - | - | - | - | - | - |
| 8.8709 | 870 | 7.2503 | - | - | - | - | - | - |
| 8.9722 | 880 | 2.2984 | - | - | - | - | - | - |
| 8.9924 | 882 | - | 0.4632 | 0.4752 | 0.4550 | 0.4247 | 0.3953 | 0.3281 |
| 9.0810 | 890 | 4.519 | - | - | - | - | - | - |
| 9.1823 | 900 | 2.99 | - | - | - | - | - | - |
| 9.2835 | 910 | 5.3026 | - | - | - | - | - | - |
| 9.3848 | 920 | 3.8492 | - | - | - | - | - | - |
| 9.4861 | 930 | 1.9454 | - | - | - | - | - | - |
| 9.5873 | 940 | 3.538 | - | - | - | - | - | - |
| 9.6886 | 950 | 4.1874 | - | - | - | - | - | - |
| 9.7899 | 960 | 4.2356 | - | - | - | - | - | - |
| 9.8911 | 970 | 4.5356 | - | - | - | - | - | - |
| 9.9924 | 980 | 4.0243 | 0.4665 | 0.4681 | 0.4507 | 0.4236 | 0.3937 | 0.3288 |
| -1 | -1 | - | 0.4599 | 0.4717 | 0.4537 | 0.4262 | 0.4004 | 0.3306 |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Base model
intfloat/multilingual-e5-large