ChenyuEcho/corruption_emaillevel_newtrainmethod

SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-0.6B
• Maximum Sequence Length: 32768 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "Impresiones Jalisco anti-corruption questionnaire completed and signed",
]
documents = [
    'Subject: Submission of Completed Due Diligence Package: Impresiones Jalisco Vendor Onboarding\nDate: 2025-08-16T12:30:00\nFrom: Miguel Torres\nParticipants: Jennifer Walsh\n\nBody:\nHi Jennifer,\n\nI’m pleased to submit the completed third-party due diligence package for Impresiones Jalisco, our prospective label printer vendor. We’ve finalized all items on the vendor onboarding checklist, including a thorough background check (result: clean), full beneficial ownership verification (all documentation attached), verified business references, as well as a completed and signed anti-corruption questionnaire. Additionally, their valid tax registration (RFC) is on file.\n\nImpresiones Jalisco was selected through our competitive bidding process, with three bidders considered and evaluated for pricing, quality, and reliability. I believe this robust process supports our selection. Please find the full package attached for your compliance review. Once you approve, I’ll proceed with finalizing the contract so we can move forward with our labeling project timeline.\n\nLet me know if you need anything further or require additional documentation.\n\nThanks,\nMiguel\n\n--\nMiguel Torres\nProcurement Manager\nASI Mexico',
    "Subject: Quality Test Results for Lot MX-2024-156\nDate: 2025-09-11T17:36:00\nFrom: Ana Lucia Vega\nParticipants: Javier Moreno\n\nBody:\nHi Javier,\n\nI'm sending over the routine quality test results for lot MX-2024-156 as requested. The alcohol content measured at 38.5%, which is within our standard parameters. pH levels were recorded at 4.1, also within acceptable range. Taste panel notes mentioned the flavor profile was clean, with no off-notes or irregularities. Carlos approved the data and Rick said to process it as normal.\n\nLet me know if you need anything else or have questions.\n\nBest,\nAna Lucia\n\n--\nAna Lucia Vega\nAccounts Payable\nASI Mexico",
    "Subject: Fwd: Request for Supporting Documentation – Journal Entry Approval Required\nDate: 2025-11-17T09:45:00\nFrom: David Chen\nParticipants: Maria Santos\n\nBody:\nHi Maria,\n\nI've reviewed the recent journal entries submitted for month-end and noticed that several expense items lack detailed descriptions and corresponding documentation. From a financial perspective, I need to stress the importance of transparency and traceability. The data shows that ambiguous expense lines can lead to compliance risks during audit. Please provide receipts or explanatory memos for all entries over $5,000. Once the supporting docs are received, I will proceed with approval.\n\nThanks,\nDavid\n\n--\nDavid Chen\nChief Financial Officer\nAgave Spirits International",
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4883, 0.1104, 0.0481]], dtype=torch.bfloat16)

Evaluation

Metrics

Information Retrieval

• Dataset: val_full_corpus
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.9029
cosine_accuracy@3	0.9854
cosine_accuracy@5	0.9903
cosine_accuracy@10	0.9951
cosine_precision@1	0.9029
cosine_precision@3	0.3285
cosine_precision@5	0.1981
cosine_precision@10	0.0995
cosine_recall@1	0.9029
cosine_recall@3	0.9854
cosine_recall@5	0.9903
cosine_recall@10	0.9951
cosine_ndcg@10	0.9547
cosine_mrr@10	0.941
cosine_map@100	0.9411

Training Details

Training Dataset

Unnamed Dataset

• Size: 816 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 816 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 6 tokens mean: 32.63 tokens max: 98 tokens	min: 118 tokens mean: 208.72 tokens max: 511 tokens

• Samples:

  sentence_0	sentence_1
  Search for impairment assessment documents for Tequila production assets as of May 31, 2024 (Ref: IMP-240531-2) and any reconciliations to the Q2 Finance Statement dated June 3, 2024 showing a MXN 1.2M discrepancy in carrying value, including supporting calculations and any post-finalization adjustments.	Subject: Impairment Assessment: Noted Discrepancies in Asset Valuation Date: 2025-12-15T17:18:00 From: Lisa Park Participants: Maria Santos Body: Hi Maria, I am reviewing the impairment assessment documentation for the Tequila production assets as of May 31, 2024, and have noticed an anomaly in the valuation report (Ref: IMP-240531-2). The carrying value listed in your schedule does not reconcile with the figure reported in the Q2 Finance Statement submitted to HQ on June 3. The difference is approximately MXN 1.2M. Can you clarify which source is correct, and provide any supporting documentation used for your calculation? For completeness, could you also confirm if any subsequent adjustments were made after the finalization date referenced above? Thank you for your attention to these details. Please let me know if you need further clarification on the items I flagged. Best regards, Lisa -- Lisa Park Director, Internal Audit Agave Spirits International
  Identify Q2 expense journal entries flagged as unclear, particularly those categorized as 'miscellaneous services' or 'external consulting', and locate the associated receipts and service contracts.	Subject: Re: Approval Required: Journal Entry Review and Documentation Date: 2025-09-30T08:40:00 From: Maria Santos Participants: David Chen Body: Hi David, Thank you for your detailed review and careful attention to our Q2 expense entries. I appreciate your diligence regarding compliance. I am currently gathering receipts and service contracts for all entries you flagged as unclear, including those labeled under 'miscellaneous services' and 'external consulting.' I will organize and forward digital copies, along with brief descriptions of each expense, by end of day tomorrow. If you need clarification on specific vendors, please let me know so I can prioritize those. Thanks for highlighting this and ensuring we stay audit-ready. I’ll follow up shortly with all the documentation. Best regards, Maria
  Identify records confirming pre-clearance and FARA registration for ASI's government liaison activities in Mexico and the United States.	Subject: Análisis sobre el registro FARA y obligaciones de cumplimiento en actividades gubernamentales México-EE.UU. Date: 2025-08-04T06:58:00 From: Amanda Foster Participants: Sarah Mitchell Body: Hola Sarah, Quiero compartir contigo el análisis preliminar referente al registro bajo FARA (Foreign Agents Registration Act) y nuestras obligaciones rutinarias de cumplimiento en relación con las actividades de vinculación gubernamental en México y EE.UU. Como sabes, todas las interacciones de ASI con entidades gubernamentales, tanto en México como en EE.UU., son registradas y pre-aprobadas mediante nuestro proceso interno de pre-clearance en Government Relations; esto garantiza la trazabilidad y documentación del propósito empresarial legítimo en cada caso. Recordando las regulaciones estadounidenses, cualquier actividad que implique representación de intereses extranjeros exige una revisión exhaustiva para determinar si corresponde el registro FARA. Hasta el momento, todas nuestras ges...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	val_full_corpus_cosine_ndcg@10
1.0	51	0.9547

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.3
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}