Text Classification
TensorRT
Core ML
ONNX
Safetensors
OpenVINO
code
multilingual
xlm-roberta
knowledge-graph
relation-extraction
relation-classification
cross-encoder
knowledge-distillation
Instructions to use faxenoff/code-daemon-relation-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TensorRT
How to use faxenoff/code-daemon-relation-v1 with TensorRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
add tokenizer + config (they were never uploaded); refresh the card
Browse files- .gitattributes +2 -0
- README.md +11 -15
- config.json +45 -0
- sentencepiece.bpe.model +3 -0
- tokenizer.json +3 -0
- tokenizer_config.json +21 -0
.gitattributes
CHANGED
|
@@ -22,3 +22,5 @@ code-daemon-relation-v1-l_win_x64_trt11.0_sm_75.engine filter=lfs diff=lfs merge
|
|
| 22 |
code-daemon-relation-v1-m_win_x64_trt11.0_sm_75.engine filter=lfs diff=lfs merge=lfs -text
|
| 23 |
code-daemon-relation-v1-s_win_x64_trt11.0_sm_75.engine filter=lfs diff=lfs merge=lfs -text
|
| 24 |
code-daemon-relation-v1-xl_win_x64_trt11.0_sm_75.engine filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 22 |
code-daemon-relation-v1-m_win_x64_trt11.0_sm_75.engine filter=lfs diff=lfs merge=lfs -text
|
| 23 |
code-daemon-relation-v1-s_win_x64_trt11.0_sm_75.engine filter=lfs diff=lfs merge=lfs -text
|
| 24 |
code-daemon-relation-v1-xl_win_x64_trt11.0_sm_75.engine filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
sentencepiece.bpe.model filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -15,8 +15,6 @@ tags:
|
|
| 15 |
pipeline_tag: text-classification
|
| 16 |
base_model:
|
| 17 |
- cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
|
| 18 |
-
datasets:
|
| 19 |
-
- unicamp-dl/mmarco
|
| 20 |
---
|
| 21 |
|
| 22 |
# code-daemon-relation-v1
|
|
@@ -92,19 +90,19 @@ markers and the input genuinely changes.
|
|
| 92 |
## 3. How it was made
|
| 93 |
|
| 94 |
**Warm-started** from a strong multilingual ranking cross-encoder, with its single ranking logit
|
| 95 |
-
replaced by the 4-class marker-pooling head, then fine-tuned by **sequence-level distillation**
|
|
|
|
| 96 |
|
| 97 |
-
|
| 98 |
-
in the passage it was shown. Those tuples became the training targets after several filtering passes:
|
| 99 |
|
| 100 |
-
- **Grounding** —
|
| 101 |
-
|
| 102 |
- **Windowing** — the passage is cropped so that both markers survive truncation.
|
| 103 |
-
- **Merging** —
|
| 104 |
-
- **Negatives** — explicit `NO_RELATION` examples synthesised from co-occurring but untupled
|
| 105 |
-
pairs in the same chunk, so abstain is trained rather than inferred.
|
| 106 |
-
- **Logit adjustment** — the class prior is corrected at the loss, since
|
| 107 |
-
|
| 108 |
|
| 109 |
---
|
| 110 |
|
|
@@ -191,7 +189,7 @@ pair of names with no surrounding text carries little signal.
|
|
| 191 |
|
| 192 |
## 6. Evaluation
|
| 193 |
|
| 194 |
-
**Dev macro-F1 = 0.547** over the four classes, on a held-out split of the distillation set.
|
| 195 |
|
| 196 |
Read that as what it is. The classes are intrinsically imbalanced — a teacher describing
|
| 197 |
documentation emits "similar" far more often than "depends on" — and the merged taxonomy still
|
|
@@ -235,8 +233,6 @@ Released **MIT**.
|
|
| 235 |
| Source | Note |
|
| 236 |
|---|---|
|
| 237 |
| `cross-encoder/mmarco-mMiniLMv2-L12-H384-v1` (warm-start) | **mMARCO ← MS MARCO → non-commercial research terms** |
|
| 238 |
-
| Distillation targets (LLM-labelled open-source documentation) | self-generated |
|
| 239 |
-
| Synthesised negatives and rare-class augmentation | generated |
|
| 240 |
|
| 241 |
⚠️ The warm-start base derives from **MS MARCO (non-commercial)**. Whether a fine-tuned model
|
| 242 |
inherits dataset-use terms is legally unsettled — this is **not legal advice**. Retrain from a
|
|
|
|
| 15 |
pipeline_tag: text-classification
|
| 16 |
base_model:
|
| 17 |
- cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
|
|
|
|
|
|
|
| 18 |
---
|
| 19 |
|
| 20 |
# code-daemon-relation-v1
|
|
|
|
| 90 |
## 3. How it was made
|
| 91 |
|
| 92 |
**Warm-started** from a strong multilingual ranking cross-encoder, with its single ranking logit
|
| 93 |
+
replaced by the 4-class marker-pooling head, then fine-tuned by **sequence-level distillation** on
|
| 94 |
+
relation tuples, each grounded in the passage it was drawn from.
|
| 95 |
|
| 96 |
+
Five properties of that pipeline are worth reproducing if you build your own:
|
|
|
|
| 97 |
|
| 98 |
+
- **Grounding** — a tuple survives only if both entity names actually resolve inside the chunk,
|
| 99 |
+
which drops arguments that were never really there.
|
| 100 |
- **Windowing** — the passage is cropped so that both markers survive truncation.
|
| 101 |
+
- **Merging** — an original 8-label taxonomy is collapsed to the 4 above.
|
| 102 |
+
- **Negatives** — explicit `NO_RELATION` examples are synthesised from co-occurring but untupled
|
| 103 |
+
entity pairs in the same chunk, so abstain is trained rather than inferred.
|
| 104 |
+
- **Logit adjustment** — the class prior is corrected at the loss, since "similar" occurs far more
|
| 105 |
+
often than "depends on".
|
| 106 |
|
| 107 |
---
|
| 108 |
|
|
|
|
| 189 |
|
| 190 |
## 6. Evaluation
|
| 191 |
|
| 192 |
+
**Dev macro-F1 = 0.547** over the four classes, on a 691-row held-out split of the distillation set.
|
| 193 |
|
| 194 |
Read that as what it is. The classes are intrinsically imbalanced — a teacher describing
|
| 195 |
documentation emits "similar" far more often than "depends on" — and the merged taxonomy still
|
|
|
|
| 233 |
| Source | Note |
|
| 234 |
|---|---|
|
| 235 |
| `cross-encoder/mmarco-mMiniLMv2-L12-H384-v1` (warm-start) | **mMARCO ← MS MARCO → non-commercial research terms** |
|
|
|
|
|
|
|
| 236 |
|
| 237 |
⚠️ The warm-start base derives from **MS MARCO (non-commercial)**. Whether a fine-tuned model
|
| 238 |
inherits dataset-use terms is legally unsettled — this is **not legal advice**. Retrain from a
|
config.json
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_cross_attention": false,
|
| 3 |
+
"architectures": [
|
| 4 |
+
"XLMRobertaForSequenceClassification"
|
| 5 |
+
],
|
| 6 |
+
"attention_probs_dropout_prob": 0.1,
|
| 7 |
+
"bos_token_id": 0,
|
| 8 |
+
"classifier_dropout": null,
|
| 9 |
+
"custom_head": "entity_start_marker_meanmask (concat E1,E2 -> Linear); [CLS] is not read",
|
| 10 |
+
"dtype": "float32",
|
| 11 |
+
"eos_token_id": 2,
|
| 12 |
+
"hidden_act": "gelu",
|
| 13 |
+
"hidden_dropout_prob": 0.1,
|
| 14 |
+
"hidden_size": 384,
|
| 15 |
+
"id2label": {
|
| 16 |
+
"0": "NO_RELATION",
|
| 17 |
+
"1": "semantically_similar_to",
|
| 18 |
+
"2": "invalidates_with",
|
| 19 |
+
"3": "depends_on"
|
| 20 |
+
},
|
| 21 |
+
"initializer_range": 0.02,
|
| 22 |
+
"intermediate_size": 1536,
|
| 23 |
+
"is_decoder": false,
|
| 24 |
+
"label2id": {
|
| 25 |
+
"NO_RELATION": 0,
|
| 26 |
+
"depends_on": 3,
|
| 27 |
+
"invalidates_with": 2,
|
| 28 |
+
"semantically_similar_to": 1
|
| 29 |
+
},
|
| 30 |
+
"layer_norm_eps": 1e-05,
|
| 31 |
+
"max_position_embeddings": 514,
|
| 32 |
+
"model_type": "xlm-roberta",
|
| 33 |
+
"num_attention_heads": 12,
|
| 34 |
+
"num_hidden_layers": 12,
|
| 35 |
+
"num_labels": 4,
|
| 36 |
+
"pad_token_id": 1,
|
| 37 |
+
"position_embedding_type": "absolute",
|
| 38 |
+
"sbert_ce_default_activation_function": "torch.nn.modules.linear.Identity",
|
| 39 |
+
"tie_word_embeddings": true,
|
| 40 |
+
"transformers_version": "5.1.0",
|
| 41 |
+
"type_vocab_size": 1,
|
| 42 |
+
"ultracode_note": "Inference artefact is model.onnx (inputs: input_ids, attention_mask; output: logits[batch, 4]). Feed the pair as (empty query, marked passage).",
|
| 43 |
+
"use_cache": true,
|
| 44 |
+
"vocab_size": 250006
|
| 45 |
+
}
|
sentencepiece.bpe.model
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:47e9787c508f4aa962fc8886a9dc2fcbf1fd19e88a6313e05e9fa3831da89e58
|
| 3 |
+
size 5569131
|
tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8aae1acb92dfa6010016a135215d0d47e2ba1af348fcedf31bf61d0a32d505e9
|
| 3 |
+
size 17083713
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": true,
|
| 3 |
+
"backend": "tokenizers",
|
| 4 |
+
"bos_token": "<s>",
|
| 5 |
+
"cls_token": "<s>",
|
| 6 |
+
"eos_token": "</s>",
|
| 7 |
+
"extra_special_tokens": [
|
| 8 |
+
"[E1]",
|
| 9 |
+
"[/E1]",
|
| 10 |
+
"[E2]",
|
| 11 |
+
"[/E2]"
|
| 12 |
+
],
|
| 13 |
+
"is_local": false,
|
| 14 |
+
"local_files_only": false,
|
| 15 |
+
"mask_token": "<mask>",
|
| 16 |
+
"model_max_length": 512,
|
| 17 |
+
"pad_token": "<pad>",
|
| 18 |
+
"sep_token": "</s>",
|
| 19 |
+
"tokenizer_class": "XLMRobertaTokenizer",
|
| 20 |
+
"unk_token": "<unk>"
|
| 21 |
+
}
|