Add Typed Decisions eval results (v1.2.1)
#1
by gnumanth - opened
.eval_results/typed-decisions.yaml
CHANGED
|
@@ -6,6 +6,7 @@
|
|
| 6 |
source:
|
| 7 |
url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions
|
| 8 |
name: "Typed Decisions"
|
|
|
|
| 9 |
notes: "Pure non-LLM bidirectional cross-encoder (DeBERTa-v3-Large 435M + LoRA + Cross-Option & Cross-Question Workflow Competition Head, 0 generated output tokens). Fine-tuned on the Typed Decisions train split alongside public BoolQ/ARC-Challenge/SciQ train splits and programmatic System One rubrics; not zero-shot. Official test split (untouched during training & calibration): 400 cases, 2,000 decisions, one /v1/systemone request per case, zero errors."
|
| 10 |
- dataset:
|
| 11 |
id: LocalLLaMA/typed-decisions
|
|
@@ -15,6 +16,7 @@
|
|
| 15 |
source:
|
| 16 |
url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions
|
| 17 |
name: "Typed Decisions"
|
|
|
|
| 18 |
notes: "Pure non-LLM bidirectional cross-encoder (0 generated output tokens). Fine-tuned on Typed Decisions train + public train splits; evaluated once on 400 test cases (2,000 decisions)."
|
| 19 |
- dataset:
|
| 20 |
id: LocalLLaMA/typed-decisions
|
|
@@ -24,6 +26,7 @@
|
|
| 24 |
source:
|
| 25 |
url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions
|
| 26 |
name: "Typed Decisions"
|
|
|
|
| 27 |
notes: "Pure non-LLM bidirectional cross-encoder (0 generated output tokens). Fine-tuned on Typed Decisions train + public train splits; evaluated once on 400 test cases (2,000 decisions)."
|
| 28 |
- dataset:
|
| 29 |
id: LocalLLaMA/typed-decisions
|
|
@@ -33,4 +36,5 @@
|
|
| 33 |
source:
|
| 34 |
url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions
|
| 35 |
name: "Typed Decisions"
|
|
|
|
| 36 |
notes: "Pure non-LLM bidirectional cross-encoder (0 generated output tokens). Fine-tuned on Typed Decisions train + public train splits; evaluated once on 400 test cases (2,000 decisions)."
|
|
|
|
| 6 |
source:
|
| 7 |
url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions
|
| 8 |
name: "Typed Decisions"
|
| 9 |
+
user: gnumanth
|
| 10 |
notes: "Pure non-LLM bidirectional cross-encoder (DeBERTa-v3-Large 435M + LoRA + Cross-Option & Cross-Question Workflow Competition Head, 0 generated output tokens). Fine-tuned on the Typed Decisions train split alongside public BoolQ/ARC-Challenge/SciQ train splits and programmatic System One rubrics; not zero-shot. Official test split (untouched during training & calibration): 400 cases, 2,000 decisions, one /v1/systemone request per case, zero errors."
|
| 11 |
- dataset:
|
| 12 |
id: LocalLLaMA/typed-decisions
|
|
|
|
| 16 |
source:
|
| 17 |
url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions
|
| 18 |
name: "Typed Decisions"
|
| 19 |
+
user: gnumanth
|
| 20 |
notes: "Pure non-LLM bidirectional cross-encoder (0 generated output tokens). Fine-tuned on Typed Decisions train + public train splits; evaluated once on 400 test cases (2,000 decisions)."
|
| 21 |
- dataset:
|
| 22 |
id: LocalLLaMA/typed-decisions
|
|
|
|
| 26 |
source:
|
| 27 |
url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions
|
| 28 |
name: "Typed Decisions"
|
| 29 |
+
user: gnumanth
|
| 30 |
notes: "Pure non-LLM bidirectional cross-encoder (0 generated output tokens). Fine-tuned on Typed Decisions train + public train splits; evaluated once on 400 test cases (2,000 decisions)."
|
| 31 |
- dataset:
|
| 32 |
id: LocalLLaMA/typed-decisions
|
|
|
|
| 36 |
source:
|
| 37 |
url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions
|
| 38 |
name: "Typed Decisions"
|
| 39 |
+
user: gnumanth
|
| 40 |
notes: "Pure non-LLM bidirectional cross-encoder (0 generated output tokens). Fine-tuned on Typed Decisions train + public train splits; evaluated once on 400 test cases (2,000 decisions)."
|