Hugging Face's logo

yahyaabd
/
indoSBERT-Large-mnrl-2

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:967831
loss:MultipleNegativesRankingLoss
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
Community
indoSBERT-Large-mnrl-2 / README.md
yahyaabd's picture
yahyaabd
Add new SentenceTransformer model
09d13eb verified
preview code
|
raw
history blame contribute delete
23.7 kB
 ---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:967831
- loss:MultipleNegativesRankingLoss
base_model: denaya/indoSBERT-large
widget:
- source_sentence: Penghasilan freelancer per provinsi, beda umur 2016
sentences:
- Rata-rata Pendapatan Bersih Pekerja Bebas Menurut Provinsi dan Kelompok Umur (ribu
rupiah), 2016
- Konkordansi Klasifikasi Tabel Inter Regional Input-Output Indonesia, 2016 (52
Industri - 17 Lapangan Usaha)
- Rata-rata Konsumsi dan Pengeluaran Perkapita Seminggu Menurut Komoditi Makanan
dan Golongan Pengeluaran per Kapita Seminggu di Provinsi Maluku, 2018-2023
- source_sentence: Tren angka partisipasi sekolah di Indonesia (7-23 tahun) berdasarkan
gender dan kelompok umur, 2015-2023
sentences:
- Jumlah Sekolah, Guru, dan Murid Sekolah Dasar (SD) di Bawah Kementerian Pendidikan
dan Kebudayaan Menurut Provinsi Tahun Ajaran 2011/2012-2015/2016
- Rata-rata Harian Konsumsi Protein Per Kapita dan Konsumsi Kalori Per Kapita Tahun
1990 - 2023
- Persentase Penduduk Usia 7-23 Tahun Menurut Jenis Kelamin, Kelompok Umur Sekolah,
dan Partisipasi Sekolah, 2015-2023
- source_sentence: Sumber penerangan rumah tangga per provinsi Indonesia 2018
sentences:
- Nutrisi
- Rata-rata Pendapatan Bersih Berusaha Sendiri Menurut Provinsi dan Lapangan Pekerjaan
Utama (ribu rupiah), 2016
- Persentase Rumah Tangga Menurut Provinsi dan Sumber Penerangan, 2015-2021
- source_sentence: Rumah tangga dengan lampu hemat energi per provinsi, 2014 vs 2021
(urban vs rural)
sentences:
- Persentase Rumah Tangga yang Menggunakan Lampu Hemat Energi Menurut Provinsi dan
Daerah Tempat Tinggal, 2014, 2021
- Luas Daerah Pengaliran dan Debit dari Beberapa Sungai yang Daerah Pengalirannya
Lebih dari 100 km2, 2015
- Perolehan Suara dan Kursi Dewan Perwakilan Rakyat (DPR) Menurut Partai Politik
Hasil Pemilu Tahun 2009 dan 2014
- source_sentence: 'Upah bulanan rata-rata: Hubungan pendidikan tertinggi dan sektor
pekerjaan utama, data 2021'
sentences:
- IHK dan Rata-rata Upah per Bulan Buruh Peternakan dan Perikanan di Bawah Mandor
(Supervisor), 2007-2014 (2007=100)
- Rata-rata Upah/Gaji Bersih Sebulan Buruh/Karyawan Pegawai Menurut Pendidikan Tertinggi
dan Jenis Pekerjaan Utama, 2021
- Rata-rata Upah/Gaji Bersih sebulan Buruh/Karyawan Pegawai Menurut Pendidikan Tertinggi
dan Lapangan Pekerjaan Utama, 2021
datasets:
- yahyaabd/statictable-triplets-all
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@1
- cosine_ndcg@5
- cosine_ndcg@10
- cosine_mrr@1
- cosine_mrr@5
- cosine_mrr@10
- cosine_map@1
- cosine_map@5
- cosine_map@10
model-index:
- name: SentenceTransformer based on denaya/indoSBERT-large
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: bps statictable ir
type: bps-statictable-ir
metrics:
- type: cosine_accuracy@1
value: 0.9218241042345277
name: Cosine Accuracy@1
- type: cosine_accuracy@5
value: 0.990228013029316
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.996742671009772
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.9218241042345277
name: Cosine Precision@1
- type: cosine_precision@5
value: 0.2247557003257329
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.13159609120521173
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.7225077889088528
name: Cosine Recall@1
- type: cosine_recall@5
value: 0.793020064240505
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.8181542032723246
name: Cosine Recall@10
- type: cosine_ndcg@1
value: 0.9218241042345277
name: Cosine Ndcg@1
- type: cosine_ndcg@5
value: 0.8340748596494166
name: Cosine Ndcg@5
- type: cosine_ndcg@10
value: 0.8332473439965864
name: Cosine Ndcg@10
- type: cosine_mrr@1
value: 0.9218241042345277
name: Cosine Mrr@1
- type: cosine_mrr@5
value: 0.9522258414766559
name: Cosine Mrr@5
- type: cosine_mrr@10
value: 0.9532340623545834
name: Cosine Mrr@10
- type: cosine_map@1
value: 0.9218241042345277
name: Cosine Map@1
- type: cosine_map@5
value: 0.7919598262757872
name: Cosine Map@5
- type: cosine_map@10
value: 0.7847729133274736
name: Cosine Map@10
---
# SentenceTransformer based on denaya/indoSBERT-large
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [denaya/indoSBERT-large](https://huggingface.co/denaya/indoSBERT-large) on the [statictable-triplets-all](https://huggingface.co/datasets/yahyaabd/statictable-triplets-all) dataset. It maps sentences & paragraphs to a 256-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [denaya/indoSBERT-large](https://huggingface.co/denaya/indoSBERT-large) <!-- at revision 5c64d43f07f7054dfbf33d226b3066414b6ebc4a -->
- **Maximum Sequence Length:** 256 tokens
- **Output Dimensionality:** 256 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
- [statictable-triplets-all](https://huggingface.co/datasets/yahyaabd/statictable-triplets-all)
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Dense({'in_features': 1024, 'out_features': 256, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("yahyaabd/indoSBERT-Large-mnrl-2")
# Run inference
sentences = [
'Upah bulanan rata-rata: Hubungan pendidikan tertinggi dan sektor pekerjaan utama, data 2021',
'Rata-rata Upah/Gaji Bersih sebulan Buruh/Karyawan Pegawai Menurut Pendidikan Tertinggi dan Lapangan Pekerjaan Utama, 2021',
'Rata-rata Upah/Gaji Bersih Sebulan Buruh/Karyawan Pegawai Menurut Pendidikan Tertinggi dan Jenis Pekerjaan Utama, 2021',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 256]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
## Evaluation
### Metrics
#### Information Retrieval
* Dataset: `bps-statictable-ir`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.9218 |
| cosine_accuracy@5 | 0.9902 |
| cosine_accuracy@10 | 0.9967 |
| cosine_precision@1 | 0.9218 |
| cosine_precision@5 | 0.2248 |
| cosine_precision@10 | 0.1316 |
| cosine_recall@1 | 0.7225 |
| cosine_recall@5 | 0.793 |
| cosine_recall@10 | 0.8182 |
| cosine_ndcg@1 | 0.9218 |
| cosine_ndcg@5 | 0.8341 |
| **cosine_ndcg@10** | **0.8332** |
| cosine_mrr@1 | 0.9218 |
| cosine_mrr@5 | 0.9522 |
| cosine_mrr@10 | 0.9532 |
| cosine_map@1 | 0.9218 |
| cosine_map@5 | 0.792 |
| cosine_map@10 | 0.7848 |
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### statictable-triplets-all
* Dataset: [statictable-triplets-all](https://huggingface.co/datasets/yahyaabd/statictable-triplets-all) at [24979b4](https://huggingface.co/datasets/yahyaabd/statictable-triplets-all/tree/24979b4f0d8269377aca975e20d52e69c3b5a030)
* Size: 967,831 training samples
* Columns: <code>query</code>, <code>pos</code>, and <code>neg</code>
* Approximate statistics based on the first 1000 samples:
| | query | pos | neg |
|:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
| type | string | string | string |
| details | <ul><li>min: 3 tokens</li><li>mean: 16.97 tokens</li><li>max: 30 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 20.79 tokens</li><li>max: 48 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 20.94 tokens</li><li>max: 48 tokens</li></ul> |
* Samples:
| query | pos | neg |
|:----------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>Data input-output antar daerah, 34 provinsi: Transaksi domestik (52 industri, harga produsen, 2016)</code> | <code>Tabel Inter Regional Input-Output Indonesia Transaksi Domestik Atas Dasar Harga Produsen Menurut 34 Provinsi dan 52 Industri, 2016 (Juta Rupiah)</code> | <code>Penduduk Berumur 15 Tahun Ke Atas yang Bekerja Selama Seminggu yang Lalu Menurut Golongan Umur dan Jumlah Jam Kerja Seluruhnya, 2008 - 2024</code> |
| <code>Data total penghasilan berbagai golongan rumah tangga setelah dipotong pajak, tahun 2000 (dalam miliar rupiah)</code> | <code>Jumlah Pendapatan Setelah Pajak Menurut Golongan Rumah Tangga (miliar rupiah), 2000, 2005, dan 2008</code> | <code>Institusi Korporasi Non Finansial Neraca Institusi Terintegrasi ( triliun rupiah), 2016 2022</code> |
| <code>Rumah tangga dengan area resapan, data per provinsi, 2014</code> | <code>Persentase Rumah Tangga Menurut Provinsi dan Keberadaan Area Resapan Air, 2013-2014</code> | <code>Nilai Produksi dan Biaya Produksi per Musim Tanam per Hektar Budidaya Tanaman Padi Sawah, Padi Ladang, Jagung, dan Kedelai, 2017</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim"
}
```
### Evaluation Dataset
#### statictable-triplets-all
* Dataset: [statictable-triplets-all](https://huggingface.co/datasets/yahyaabd/statictable-triplets-all) at [24979b4](https://huggingface.co/datasets/yahyaabd/statictable-triplets-all/tree/24979b4f0d8269377aca975e20d52e69c3b5a030)
* Size: 967,831 evaluation samples
* Columns: <code>query</code>, <code>pos</code>, and <code>neg</code>
* Approximate statistics based on the first 1000 samples:
| | query | pos | neg |
|:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
| type | string | string | string |
| details | <ul><li>min: 3 tokens</li><li>mean: 16.69 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 20.85 tokens</li><li>max: 48 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 20.9 tokens</li><li>max: 48 tokens</li></ul> |
* Samples:
| query | pos | neg |
|:---------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>Kredit UMKM bank umum (miliar rupiah), 2012-2016</code> | <code>Posisi Kredit Usaha Mikro, Kecil, dan Menengah (UMKM) 1 pada Bank Umum (miliar rupiah), 2012-2016</code> | <code>Jumlah Penghuni Lapas per Kanwil</code> |
| <code>Infant Mortality Rate di Indonesia per provinsi, 1971</code> | <code>Angka Kematian Bayi/AKB (Infant Mortality Rate/IMR) Menurut Provinsi, 1971-2020</code> | <code>Jumlah Sekolah, Guru, dan Murid Sekolah Menengah Kejuruan (SMK) di Bawah Kementrian Pendidikan dan Kebudayaan Menurut Provinsi tahun ajaran 2011/2012-2015/2016</code> |
| <code>Partisipasi sekolah anak dan remaja: Data persentase usia 7-24 tahun per gender dan kelompok umur, 2021</code> | <code>Persentase Penduduk Usia 7-24 Tahun Menurut Jenis Kelamin, Kelompok Umur, dan Partisipasi Sekolah, 2002-2023</code> | <code>Tabel Input-Output Indonesia Transaksi Total Atas Dasar Harga Pembeli (17 Produk), 2016 (Juta Rupiah)</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim"
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 64
- `per_device_eval_batch_size`: 64
- `num_train_epochs`: 1
- `warmup_ratio`: 0.1
- `fp16`: True
- `load_best_model_at_end`: True
- `eval_on_start`: True
- `batch_sampler`: no_duplicates
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 64
- `per_device_eval_batch_size`: 64
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: True
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
</details>
### Training Logs
| Epoch | Step | Training Loss | Validation Loss | bps-statictable-ir_cosine_ndcg@10 |
|:----------:|:-------:|:-------------:|:---------------:|:---------------------------------:|
| 0 | 0 | - | 0.7678 | 0.7378 |
| 0.1391 | 100 | 0.2164 | 0.0292 | 0.8324 |
| 0.2782 | 200 | 0.032 | 0.0143 | 0.8383 |
| 0.4172 | 300 | 0.0221 | 0.0077 | 0.8392 |
| 0.5563 | 400 | 0.0088 | 0.0055 | 0.8391 |
| 0.6954 | 500 | 0.0058 | 0.0033 | 0.8301 |
| **0.8345** | **600** | **0.0039** | **0.0016** | **0.8331** |
| 0.9736 | 700 | 0.0027 | 0.0019 | 0.8332 |
* The bold row denotes the saved checkpoint.
### Framework Versions
- Python: 3.11.11
- Sentence Transformers: 3.4.1
- Transformers: 4.48.3
- PyTorch: 2.6.0+cu124
- Accelerate: 1.3.0
- Datasets: 3.4.1
- Tokenizers: 0.21.1
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->