Question Answering
Transformers
ONNX
Safetensors
nullspan
feature-extraction
system-one
typed-decisions
jevbench
structured-extraction
span-extraction
calibrated-estimation
electra
modernbert
custom_code
Eval Results
Instructions to use gnumanth/nullspan with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use gnumanth/nullspan with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "question-answering" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("question-answering", model="gnumanth/nullspan", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("gnumanth/nullspan", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload fine-tuned nullspan System One extraction model (Apple Silicon MPS)
Browse files- README.md +1 -1
- eval_results.json +9 -9
README.md
CHANGED
|
@@ -38,7 +38,7 @@ Built on `distilbert/distilbert-base-uncased` (67M parameters) with 5 parallel S
|
|
| 38 |
- **Operation Router Accuracy**: `100.0%`
|
| 39 |
- **Multi-Span List BIO Accuracy**: `100.0%`
|
| 40 |
- **Quantile Distribution MAE (normalized `[0, 1]`)**: `0.0068`
|
| 41 |
-
- **Canonical Evaluation Suite Accuracy**: `100.0%` (`
|
| 42 |
|
| 43 |
## Quick Start
|
| 44 |
|
|
|
|
| 38 |
- **Operation Router Accuracy**: `100.0%`
|
| 39 |
- **Multi-Span List BIO Accuracy**: `100.0%`
|
| 40 |
- **Quantile Distribution MAE (normalized `[0, 1]`)**: `0.0068`
|
| 41 |
+
- **Canonical Evaluation Suite Accuracy**: `100.0%` (`119.35 ms` per multi-field pass)
|
| 42 |
|
| 43 |
## Quick Start
|
| 44 |
|
eval_results.json
CHANGED
|
@@ -14,10 +14,10 @@
|
|
| 14 |
},
|
| 15 |
"canonical_eval": {
|
| 16 |
"canonical_field_accuracy": 100.0,
|
| 17 |
-
"mean_doc_latency_ms":
|
| 18 |
"examples": {
|
| 19 |
-
"
|
| 20 |
-
"latency_ms":
|
| 21 |
"num_fields": 8,
|
| 22 |
"answers": {
|
| 23 |
"customer_name": {
|
|
@@ -103,8 +103,8 @@
|
|
| 103 |
}
|
| 104 |
}
|
| 105 |
},
|
| 106 |
-
"
|
| 107 |
-
"latency_ms":
|
| 108 |
"num_fields": 8,
|
| 109 |
"answers": {
|
| 110 |
"order_number": {
|
|
@@ -165,8 +165,8 @@
|
|
| 165 |
}
|
| 166 |
}
|
| 167 |
},
|
| 168 |
-
"
|
| 169 |
-
"latency_ms":
|
| 170 |
"num_fields": 1,
|
| 171 |
"answers": {
|
| 172 |
"sale_price": {
|
|
@@ -180,8 +180,8 @@
|
|
| 180 |
}
|
| 181 |
}
|
| 182 |
},
|
| 183 |
-
"
|
| 184 |
-
"latency_ms":
|
| 185 |
"num_fields": 1,
|
| 186 |
"answers": {
|
| 187 |
"travelers": {
|
|
|
|
| 14 |
},
|
| 15 |
"canonical_eval": {
|
| 16 |
"canonical_field_accuracy": 100.0,
|
| 17 |
+
"mean_doc_latency_ms": 119.35,
|
| 18 |
"examples": {
|
| 19 |
+
"nullspan_support_claim": {
|
| 20 |
+
"latency_ms": 64.14,
|
| 21 |
"num_fields": 8,
|
| 22 |
"answers": {
|
| 23 |
"customer_name": {
|
|
|
|
| 103 |
}
|
| 104 |
}
|
| 105 |
},
|
| 106 |
+
"nullspan_receipt": {
|
| 107 |
+
"latency_ms": 138.94,
|
| 108 |
"num_fields": 8,
|
| 109 |
"answers": {
|
| 110 |
"order_number": {
|
|
|
|
| 165 |
}
|
| 166 |
}
|
| 167 |
},
|
| 168 |
+
"nullspan_comps": {
|
| 169 |
+
"latency_ms": 199.56,
|
| 170 |
"num_fields": 1,
|
| 171 |
"answers": {
|
| 172 |
"sale_price": {
|
|
|
|
| 180 |
}
|
| 181 |
}
|
| 182 |
},
|
| 183 |
+
"nullspan_tsa_forecast": {
|
| 184 |
+
"latency_ms": 74.76,
|
| 185 |
"num_fields": 1,
|
| 186 |
"answers": {
|
| 187 |
"travelers": {
|