Instructions to use Komalverma/custom-bge-baai-small-cfr-cms with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Komalverma/custom-bge-baai-small-cfr-cms with sentence-transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Komalverma/custom-bge-baai-small-cfr-cms")

sentences = [
"In applying these criteria, the carriers are to exercise judgment based made by physicians to patients generally and by other the in on factors that may exist in so determinations charge realistic and Determinations carriers charge are reviewed case-by-case basis Centers for Medicare & Medicaid Services, although general and performance functions by carriers In making determinations, apply provisions law under the Centers Medicare & Medicaid Services. These are overall consistency in their determinations of reasonable The in § § 405.503 through 405.507 the for determinations in accordance with the statutory provisions.",
"The REH must have an organized nursing service that is available to provide 24-hour nursing services for the provision of patient care. The nursing services must be furnished and supervised by a registered nurse. Nursing services must meet the needs of patients. Patient care responsibilities must be delineated for all nursing service personnel. Nursing services must be provided in accordance with recognized standards of practice. The director of the nursing service must be a licensed registered nurse. The individual is responsible for the operation of the service, including determining the types and numbers of nursing personnel and staff necessary to provide nursing care for all areas of the REH.",
"Zero-fill for all hospitals after FY91. This Field is obsolete for hospitals as of FY92. Effective 1/1/2018, this field is used for HHAs only. Enter the HH VBP adjustment factor provided by CMS for each HHA. If no factor is provided, enter 1.00000. Disproportionate share adjustment Percentage. Pricer calculates the Operating DSH effective 10/1/91 and bypasses this field. Zero-fill for all hospitals 10/1/91 and later.",
"In applying these criteria, the carriers are to exercise judgment based on factual data on the charges made by physicians to patients generally and by other persons to the public in general and on special factors that may exist in individual cases so that determinations of reasonable charge are realistic and equitable. Determinations by carriers of reasonable charge are not reviewed on a case-by-case basis by the Centers for Medicare & Medicaid Services, although the general procedures and performance of functions by carriers are evaluated. In making determinations, carriers apply the provisions of the law under broad principles issued by the Centers for Medicare & Medicaid Services. These principles are intended to assure overall consistency among carriers in their determinations of reasonable charge. The principles in § § 405.503 through 405.507 establish the criteria for making such determinations in accordance with the statutory provisions."
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]

Notebooks
Google Colab
Kaggle

SentenceTransformer based on BAAI/bge-small-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-small-en-v1.5. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-small-en-v1.5
Maximum Sequence Length: 512 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Update to Pub. 100-04, Chapter 1 to Provide Language-Only Changes Updating ICD-10 and ASC Automation of the Request Claims Process Rescinded and replaced by Transmittal 3154 Adjustment Fiscal Intermediary Shared System Consistency Edit Implement National Uniform Billing Committee (NUBC) Revision Occurrence Span Code (OSC) Definition for Code Reporting the Service Location National Provider Identifier on Anti-Markup and Reference Laboratory Claims and replaced Transmittal the Reopening Process - Rescinded and replaced by Transmittal 3060',
    'Update to Pub. 100-04, Chapter 1 to Provide Language-Only Changes for Updating ICD-10 and ASC X12 Automation of the Request for Reopening Claims Process - Rescinded and replaced by Transmittal 3154 Adjustment to Fiscal Intermediary Shared System (FISS) Consistency Edit to Implement National Uniform Billing Committee (NUBC) Revision to Occurrence Span Code (OSC) Definition for Code 72 Reporting the Service Location National Provider Identifier (NPI) on Anti-Markup and Reference Laboratory Claims - Rescinded and replaced by Transmittal 3098 Automation of the Request for Reopening Claims Process - Rescinded and replaced by Transmittal 3060',
    'The issue is feasibility and not whether coverage is provided in one setting and not provided in another. For instance, an individual in need of daily skilled physical therapy might be able to receive the services needed on a more economical basis from an could not be made for the services because an expense limitation (if applicable) to the services of an independent physical therapist had been exceeded or because the patient was not enrolled in Part B, would not be a basis for determining that, as a practical',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9797, 0.0785],
#         [0.9797, 1.0000, 0.0807],
#         [0.0785, 0.0807, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 47,056 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 100 samples:
sentence_0 sentence_1
type string string

modality text text
details
min: 26 tokens
mean: 86.86 tokens
max: 166 tokens

min: 44 tokens
mean: 125.96 tokens
max: 261 tokens

	sentence_0	sentence_1
type	string	string
modality	text	text
details	min: 26 tokens mean: 86.86 tokens max: 166 tokens	min: 44 tokens mean: 125.96 tokens max: 261 tokens

Samples:

sentence_0	sentence_1
`For purposes of claims services submitted an (as § 410.79(b) this chapter), Medicare deems to assigned by the beneficiary (or the person request payment beneficiary's the assignment accepted the MDPP supplier.`	`<> For purposes of claims for services submitted by an MDPP supplier (as defined at § 410.79(b) of this chapter), Medicare deems such claims to have been assigned by the beneficiary (or the person authorized to request payment on the beneficiary's behalf) and the assignment accepted by the MDPP supplier.`
() Claims primary and/or other services related of specific diseases or the or There has been by Secretary of in section 1899(c)(2)(B) Act is purposes of model. <> The ACO ACO and other entities performing functions services related to ACO activities are prohibited or gifts remuneration to Medicare beneficiaries for influencing Medicare beneficiary's designate or not designate ACO professional paragraph (e) this section. The ACO, ACO participants, ACO professionals, other or entities performing functions and services related to ACO activities must not, directly or commit any act or adopt policy that coerces or influences Medicare beneficiary's decision to or not ACO professional as for under paragraph (e) of not limited to the	<> () Claims for primary care and/or other services related to treatment of one or more specific diseases or conditions targeted by the model; or <> () There has been a determination by the Secretary that waiver of the requirement in section 1899(c)(2)(B) of the Act is necessary solely for purposes of testing the model. <> The ACO, ACO participants, ACO providers/suppliers, ACO professionals, and other individuals or entities performing functions and services related to ACO activities are prohibited from providing or offering gifts or other remuneration to Medicare beneficiaries as inducements for influencing a Medicare beneficiary's decision to designate or not to designate an ACO professional under paragraph (e) of this section. The ACO, ACO participants, ACO providers/suppliers, ACO professionals, and other individuals or entities performing functions and services related to ACO activities must not, directly or indirectly, commit any act or omission, nor adopt any policy that coerce...
`d. If code 70 is entered, the type of bill must be 11X, 18X, 21X, or 41X. e. If code 71 is entered, the first digit of FL 4 must be "1," "2," or "4" and the f. If code 72 is entered, the type of bill must be 11X, 12X, 13X, 14X, 18X, 21X, 22X, 23X, 32X, 34X, 71X, 72X, 73X, 74X, 75X, 76X, 77X, 81X, 82X, or 85X. g. If code 74 is entered, the type of bill must be 11X, 13X, 14X, 18X, 21X, 34X,`	`d. If code 70 is entered, the type of bill must be 11X, 18X, 21X, or 41X. e. If code 71 is entered, the first digit of FL 4 must be "1," "2," or "4" and the f. If code 72 is entered, the type of bill must be 11X, 12X, 13X, 14X, 18X, 21X, 22X, 23X, 32X, 34X, 71X, 72X, 73X, 74X, 75X, 76X, 77X, 81X, 82X, or 85X. g. If code 74 is entered, the type of bill must be 11X, 13X, 14X, 18X, 21X, 34X,`

Loss: DenoisingAutoEncoderLoss with these parameters:

{
    "decoder_name_or_path": "BAAI/bge-small-en-v1.5",
    "need_retokenization": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 16
per_device_eval_batch_size: 16
num_train_epochs: 10
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 10
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.1700	500	8.346
0.3400	1000	5.98
0.5100	1500	5.2426
0.6800	2000	4.7909
0.8501	2500	4.4601
1.0201	3000	4.2278
1.1901	3500	4.0102
1.3601	4000	3.8418
1.5301	4500	3.741
1.7001	5000	3.6344
1.8701	5500	3.5475
2.0401	6000	3.4801
2.2101	6500	3.3712
2.3801	7000	3.3199
2.5502	7500	3.2572
2.7202	8000	3.2375
2.8902	8500	3.1958
3.0602	9000	3.1396
3.2302	9500	3.0818
3.4002	10000	3.045
3.5702	10500	3.0153
3.7402	11000	3.0019
3.9102	11500	2.9902
4.0802	12000	2.9319
4.2503	12500	2.894
4.4203	13000	2.8683
4.5903	13500	2.8605
4.7603	14000	2.8332
4.9303	14500	2.8466
5.1003	15000	2.8013
5.2703	15500	2.7566
5.4403	16000	2.7429
5.6103	16500	2.745
5.7803	17000	2.7342
5.9504	17500	2.7319
6.1204	18000	2.6939
6.2904	18500	2.6811
6.4604	19000	2.6673
6.6304	19500	2.6559
6.8004	20000	2.6603
6.9704	20500	2.6585
7.1404	21000	2.6213
7.3104	21500	2.6154
7.4804	22000	2.6184
7.6505	22500	2.6143
7.8205	23000	2.6121
7.9905	23500	2.6007
8.1605	24000	2.5798
8.3305	24500	2.5822
8.5005	25000	2.5758
8.6705	25500	2.5799
8.8405	26000	2.5823
9.0105	26500	2.5788
9.1806	27000	2.5653
9.3506	27500	2.5689
9.5206	28000	2.5716
9.6906	28500	2.5662
9.8606	29000	2.5676

Training Time

Training: 1.5 hours

Framework Versions

Python: 3.12.6
Sentence Transformers: 5.6.0
Transformers: 4.56.0
PyTorch: 2.8.0+cu129
Accelerate: 1.10.1
Datasets: 5.0.0
Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

DenoisingAutoEncoderLoss

@inproceedings{wang-2021-TSDAE,
    title = "TSDAE: Using Transformer-based Sequential Denoising Auto-Encoderfor Unsupervised Sentence Embedding Learning",
    author = "Wang, Kexin and Reimers, Nils and Gurevych, Iryna",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    pages = "671--688",
    url = "https://arxiv.org/abs/2104.06979",
}