tjohn327/scion-snowflake-arctic-embed-s-v2

SentenceTransformer based on Snowflake/snowflake-arctic-embed-s

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-s. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-s
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("tjohn327/scion-snowflake-arctic-embed-s-v2")
# Run inference
sentences = [
    'How is the concept of configurable rates in Z-Lane intended to accommodate varying traffic demands',
    'Research paper section providing a Z-lane system description. Introduces AS/ISD-level bandwidth isolation and configurable rates using SCION\'s ISDs. Explains how ASes can overuse allocated bandwidth and send traffic at guaranteed rates.\n<citation> Marc Wyss et al.. "Zero-setup Intermediate-rate Communication Guarantees in a Global Internet." *Proceedings of the USENIX Security Symposium*, 2024. </citation>\n<type> research paper </type>\n<page> 5 </page>\n<content>\nZ-Lane. The decision how to configure the rates is ultimately\nup to the network operator and, importantly, does not require\nany inter-domain coordination. Due to the aggregation of\nASes into ISDs, configurations remain manageable even if\nthe Internet grows to hundreds of thousands of ASes.\nEnd Host Guarantees. Z-Lane lets end hosts, more specifi-\ncally their applications, define what traffic should be sent with\nforwarding guarantees, and what traffic should be forwarded\nover best-effort. Still, to protect against malicious end hosts,\ntheir AS has the ultimate authority in this matter and can re-\nclassify traffic to be sent as best-effort only. This protection\nis implemented through a Z-Lane gateway, which schedules\nend host traffic and authenticates it towards on-path routers\nusing a secret key not known to the end hosts. How traffic is\nscheduled is up to the AS operator; configurations can range\nfrom fair sharing to prioritizing certain traffic from critical AS\nservices like routing or time synchronization. We emphasize\nthat, to avoid any setup overhead (R3), neither ISDs, nor ASes\nor end hosts explicitly learn their configured rate; instead, end\nhosts implicitly discover their allowed rate through existing\nmechanisms like congestion control.\nCompatibility with Other Systems. Bandwidth reserva-\ntion systems cannot provide zero-setup communication guar-\nantees and are therefore not suitable to protect short-lived\nintermediate-rate communication (Section 8). Still, we design\nZ-Lane to seamlessly coexist with them, as they complement\nour work by effectively protecting non-setup-critical, high-\nvolume communication such as from video conferencing. We\nchoose COLIBRI [27] as a reservation system instantiation,\nbut other systems could be deployed as well. To prevent at-\ntacks targeting DRKey’s AS-level key exchange, which is a\nfundamental requirement for EPIC, our design also ensures\ncompatibility with the DoCile system [74], which leverages\ndedicated channels between neighboring ASes to successfully\nbootstrap the key exchange even under DDoS.\nWe therefore consider the following four types of inter-\ndomain traffic: COLIBRI reservation traffic, DoCile’s\nneighbor-based communication, authenticated traffic from\nEPIC, and unauthenticated SCION traffic.\n4.2 Source Authentication\nZ-Lane employs EPIC for authenticating traffic sources to\nborder routers, allowing every border router to verify the au-\nthenticity of every received packet. An important insight in the\ndesign of Z-Lane is that efficient and reliable source authen-\ntication as provided by EPIC allows for meaningful source-\nbased traffic control at border routers. The realization of this\nidea has not been possible so far because previous source\nauthentication mechanisms would cause excessive commu-\nnication or computation overhead and therefore impede de-\nployment, or were based on heuristics or probabilities, and\nwould thus fail to reliably distinguish between authentic and\nspoofed addresses (Appendix H). Z-Lane is the first system\nto explore the use of comprehensive source authentication to\nprotect the availability of short-lived intermediate-rate Inter-\nnet traffic – with EPIC’s security rooted in AS-level secret\nkeys, it integrates seamlessly into Z-Lane.\nWe want to highlight that EPIC together with a fairness\nmechanism provided by some congestion control algorithm,\ni.e., without any guaranteed rates, would not be enough in\nour threat model, as an attacker would just not respect the\nalgorithm’s feedback and instead keep sending traffic at high\nrates, or leverage a botnet to create many low-volume flows.\n4.3 End Host Traffic Generation\nEnd hosts, i.e., their applications, can choose among several\nmechanisms on how their traffic is forwarded (Figure 1). For\nlong-term traffic they request a bandwidth reservation and\nuse it by sending their COLIBRI traffic class packets through\nthe COLIBRI gateway. While the overhead for requesting\na reservation is significant, the result is a fixed amount of\nbandwidth that is exclusively reserved along the communi-\ncation path. In a similar way, applications send short-lived\nintermediate-rate traffic using the EPIC traffic class over the\nZ-Lane gateway, where traffic is forwarded immediately with-\nout any delay (requirement R3), but without the applications\nknowing the concrete rates. In both cases traffic is protected\nagainst congestion on the communication path. The default\noption is for end hosts to send their traffic using the EPIC\ntraffic class directly to a border router of their AS, where they\nare forwarded along the path using best-effort. This option\nis useful for non-latency-critical communication such as file\ndownloads, or for long-term traffic for which no reservation\nis available, which can for example happen if the end host has\nalready created a large number of reservations and gets denied\nfrom creating even more. Z-Lane envisages unauthenticated\nSCION traffic to be sent only in scenarios where it is not\notherwise possible, e.g., if an AS needs to request shared keys\nusing DRKey from another AS for the first time.\n4.4 Z-Lane Gateway\nASes use the gateway to control the traffic volumes that their\nend hosts (incl. AS infrastructure services) are allowed to send\nusing Z-Lane, which serves the primary purpose of protecting\nbenign from malicious or compromised end hosts.\nFor end host traffic complying with the allowed rate, the\ngateway sets a QoS flag in the EPIC header, which indicates\nto on-path routers that the corresponding packets should be\nforwarded using the AS’ guaranteed rate. If an end host’s\npacket exceeds the allowed rate at the gateway, then either (i)\nthe QoS flag is not set (or removed, if it was already set by the\nend host), meaning that those packets will be treated as best-\neffort, or (ii) the packets are dropped, depending on the AS’\npolicy. In contrast to best-effort EPIC packets generated at\n5\n</content>',
    'Research paper setup description section detailing the specific SCIONLab configuration, including AS creation, attachment to ETHZ-AP, and VM setup. Lists and describes SCION applications crucial the experiments: \'scion address\', \'scion showpaths\', \'scion ping\', \'scion traceroute\', and \'scion-bwtestclient\', including their options and parameters(like packet size, bandwidth target) for performance evaluation on the network.\n<citation> Antonio Battipaglia et al.. "Evaluation of SCION for User-driven Path Control: a Usability Study." *Proceedings of the SC \'23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis*, 2023. </citation>\n<type> research paper </type>\n<page> 3 </page>\n<content>\nEvaluation of SCION for User-driven Path Control: a Usability Study SC-W 2023, November 12–17, 2023, Denver, CO, USA\nFigure 1: SCIONLab Topology: in light orange there are Core ASes; Non-Core ASes are white colored; Attachment Points are\ngreen; our AS is blue.\nhelp us run specific experiments we will discuss in later sections.\nOnce this configuration phase was completed, SCIONLab web inter-\nface provided a unique ASN for our AS, along with cryptographic\nkeys and public-key certificates. Subsequently, a Vagrant file for\nour AS was generated to instruct the configuration of a Virtual\nMachine (VM) that represents our AS. This file made the setup\nprocess lightweight by automating the installation of SCIONLAB\nservices, relevant packages, and necessary configurations. Finally\nwe were ready to use a fully configured VM belonging to the global\nSCIONLab topology.\n3.3 Available Applications\nThe VM configuration process also installs a predefined set of\nSCION applications. The SCION apps that we used in our experi-\nments are:\n• scion address : this command returns the relevant SCION\naddress information for the local host, that is, our AS where\nwe launch commands from.\n• scion showpaths : it lists available paths between the local\nand the specified AS. By default, the list is set to display 10\npaths only, it can be extended using the-moption. Moreover,\na really useful feature for this work, is the—extendedoption,\nwhich provides additional information for each path (e.g.\nMTU, Path Status, Latency info).\n• scion ping : it tests connectivity to a remote SCION host\nusing SCMP echo packets[4]. When the —countoption is en-\nabled, the ping command sends a specific number of SCMP\necho packets and provides a report with corresponding statis-\ntics. Furthermore, the real innovation is the —interactive\nmode option, which displays all the available paths for the\nspecified destination allowing the user to select the desired\ntraffic route.\n• scion traceroute : it traces the SCION path to a remote\nAS using SCMP traceroute packets. It is particularly useful\nto test how the latency is affected by each link. Even this\ncommand makes interactive mode available.\n• scion-bwtestclient: it is the only application presented\nin this work that is not installed by default in the VM.\nBwtestclientis part of a bigger bandwidth testing applica-\ntion named bwtesterwhich allows a variety of bandwidth\ntests on the SCION network. The application enables speci-\nfication of the test duration (up to 10 seconds), the packet\nsize to be used (at least 4 bytes), the total number of packets\nthat will be sent, and the target bandwidth. For example,\n5,100,?,150Mbps specifies that the packet size is 100 bytes,\nsent over 5 seconds, resulting in a bandwidth of 150Mbps.\nThe question mark ? character can be used as wildcard for\nany of these parameters, in this case the number of packets\nsent. Its value is then computed according to the other pa-\nrameters. The parameters for the test in the client-to-server\ndirection are specified with -cs, and the server-to-client\ndirection with -sc.\nWe will analyze further these scion commands and how we used\nthem in the next section.\n4 SOFTWARE DESIGN\nWe now present our software to test SCION features of path aware-\nness and path selection. We will also test network performances\nsuch as: latency, bandwidth and packet loss in order to provide\nUPIN users with paths that fulfill requirements on these properties.\n787\n</content>',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Dataset: val-ir-eval
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7255
cosine_accuracy@3	0.902
cosine_accuracy@5	0.9314
cosine_accuracy@10	0.9608
cosine_precision@1	0.7255
cosine_precision@3	0.3007
cosine_precision@5	0.1863
cosine_precision@10	0.0961
cosine_recall@1	0.7255
cosine_recall@3	0.902
cosine_recall@5	0.9314
cosine_recall@10	0.9608
cosine_ndcg@10	0.8542
cosine_mrr@10	0.8188
cosine_map@100	0.8212

Training Details

Training Dataset

Unnamed Dataset

• Size: 4,321 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 5 tokens mean: 19.23 tokens max: 66 tokens	min: 238 tokens mean: 507.97 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1
  What are the two scenarios for LightningFilter deployment depending on the level of trust with the AS	Book chapter detailing SCION LightningFilter's packet authentication using DRKey. Describes key derivation using PRF with AS-level (KLF_A->B) and host-level (KLF_A:HA->B:HB) keys. Explains two deployment scenarios: trusted entity with direct access to SVLF_A and less-trusted entity fetching second-level keys. Covers header and payload authentication using SPAO, MAC computation with symmetric key (tag = MAC{KLF_A:HA->B:HB}(hdr)), and payload hash (h = H(pld)). Laurent Chuat et al.. The Complete Guide to SCION. From Design Principles to Formal Verification. Springer International Publishing AG, 2022. book 233 9.2 High-Speed Traffic Filtering with LightningFilter in the number of hosts, the computation overhead is significant and thus not suited for a per-packet usage. On the other hand, using symmetric cryptog- raphy would traditionally require the filtering service to store a key for each packet source. To avoid per-host stat...
  How do preferences, such as customer, peering link, or transit provider, are expressed in BGP?	Book excerpt on Approaches to Implementing Path Policies and Gao–Rexford Model describing how ASes add path policy information to PCBs, specifying usage restrictions. Highlights accountability for violating AS, explain the need of a default, arbitrary path. Explains the "preference policy" for economics and "export policy" for stability. Laurent Chuat et al.. The Complete Guide to SCION. From Design Principles to Formal Verification. Springer International Publishing AG, 2022. book 159 6.2 SCION Path Policy When the path is only used against the explicit path policy but not regis- tered, detection is more challenging. To detect such misuse, an AS can monitor hop fields (HFs) used in traffic and, in the case of HFs that were not registered by any of the downstream ASes, it can verify whether the source or destination AS is allowed to use the path. Furthermore, viola- tion by an intermediate AS can be detected by tracing the ...
  What is the structure of a complete SCION address? ,How is intra-domain forwarding handled at the destination AS?	Technical document describing inter- and intra-domain forwarding in SCION. Explains the separation of inter-domain (SCION-based) and intra-domain (AS-specific, often IP-based) forwarding. SCION routers forward based on Hop Fields and need not inspect destination IP address. Includes advantages like path control and simplified processing. https://www.ietf.org/archive/id/draft-dekater-scion-dataplane-04.txt specification de Kater, et al. Expires 27 June 2025 [Page 8]  Internet-Draft SCION DP December 2024 * It simplifies the packet processing at routers. Instead of having to perform longest prefix matching on IP addresses which requires expensive hardware and substantial amounts of energy, a router can simply access the next hop from the packet header after having verified the authenticity of the Hop Field's MAC. 1.3.1. Inter- and Intra-Domain Forwarding ...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 50
• per_device_eval_batch_size: 50
• num_train_epochs: 5
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 50
• per_device_eval_batch_size: 50
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	val-ir-eval_cosine_ndcg@10
1.0	44	0.7533
2.0	88	0.8088
3.0	132	0.8296
4.0	176	0.8326
5.0	220	0.8542

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.4.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}