Abid Ali Awan Codex commited on
Commit ·
c93cae8
1
Parent(s): 453edd1
Fix trace dataset presentation
Browse filesSeparate reviewed seed examples from noisy operational traces, correct the dataset viewer path configuration, and update the runtime documentation.
Co-authored-by: Codex <codex@openai.com>
- traces/dataset_card.md +29 -19
traces/dataset_card.md
CHANGED
|
@@ -14,9 +14,14 @@ tags:
|
|
| 14 |
- privacy
|
| 15 |
configs:
|
| 16 |
- config_name: default
|
|
|
|
| 17 |
data_files:
|
| 18 |
- split: train
|
| 19 |
-
path: data/
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
---
|
| 21 |
|
| 22 |
# NoticeCheck Privacy-Safe Traces
|
|
@@ -27,14 +32,16 @@ This dataset contains compact, deterministic metadata about NoticeCheck
|
|
| 27 |
message-review requests. It does not contain hidden model reasoning or
|
| 28 |
autonomous-agent trajectories.
|
| 29 |
|
| 30 |
-
The application uses MiniCPM5-1B through
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
|
|
|
|
|
|
| 38 |
|
| 39 |
## Fields
|
| 40 |
|
|
@@ -85,23 +92,24 @@ successful assessments whose content is genuinely unclassified.
|
|
| 85 |
|
| 86 |
## Provenance
|
| 87 |
|
| 88 |
-
|
| 89 |
-
|
|
|
|
| 90 |
Trace generation itself does not invoke the model.
|
| 91 |
|
| 92 |
The seed rows are illustrative examples, not an evaluation split. All six
|
| 93 |
currently have the `Likely scam` label, so they must not be used to estimate
|
| 94 |
class balance, accuracy, recall, or real-world scam prevalence.
|
| 95 |
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
|
| 101 |
-
-
|
| 102 |
-
-
|
| 103 |
-
-
|
| 104 |
-
-
|
| 105 |
|
| 106 |
The source repository includes `traces/scripts/analyze_trace_dataset.py` for
|
| 107 |
schema validation and a reproducible summary of these quality indicators.
|
|
@@ -119,6 +127,8 @@ schema validation and a reproducible summary of these quality indicators.
|
|
| 119 |
## Links
|
| 120 |
|
| 121 |
- App: https://huggingface.co/spaces/build-small-hackathon/noticecheck
|
|
|
|
|
|
|
| 122 |
|
| 123 |
## License
|
| 124 |
|
|
|
|
| 14 |
- privacy
|
| 15 |
configs:
|
| 16 |
- config_name: default
|
| 17 |
+
default: true
|
| 18 |
data_files:
|
| 19 |
- split: train
|
| 20 |
+
path: data/seed/trace_samples.jsonl
|
| 21 |
+
- config_name: operational
|
| 22 |
+
data_files:
|
| 23 |
+
- split: train
|
| 24 |
+
path: data/*/*/*/*.jsonl
|
| 25 |
---
|
| 26 |
|
| 27 |
# NoticeCheck Privacy-Safe Traces
|
|
|
|
| 32 |
message-review requests. It does not contain hidden model reasoning or
|
| 33 |
autonomous-agent trajectories.
|
| 34 |
|
| 35 |
+
The hosted application uses MiniCPM5-1B through Transformers on Hugging Face
|
| 36 |
+
ZeroGPU, with NVIDIA Nemotron-Parse v1.2 for supported screenshots. The same
|
| 37 |
+
pipeline can run locally on an NVIDIA GPU with Docker Compose. Creating a trace
|
| 38 |
+
never makes an additional AI model call. Traces only observe the existing
|
| 39 |
+
request path and convert it into allow-listed categories, booleans, and fixed
|
| 40 |
+
descriptions. For image submissions, the existing assessment's explanation and
|
| 41 |
+
red flags are inspected transiently for this mapping, but their text is not
|
| 42 |
+
stored. The trace mapper predicts the privacy-safe image category and tactics
|
| 43 |
+
directly from that result summary using deterministic English/Urdu evidence
|
| 44 |
+
rules.
|
| 45 |
|
| 46 |
## Fields
|
| 47 |
|
|
|
|
| 92 |
|
| 93 |
## Provenance
|
| 94 |
|
| 95 |
+
The default configuration contains the six reviewed public examples bundled
|
| 96 |
+
with NoticeCheck. The `operational` configuration contains privacy-safe runtime
|
| 97 |
+
traces and may include successful, rejected, or failed requests.
|
| 98 |
Trace generation itself does not invoke the model.
|
| 99 |
|
| 100 |
The seed rows are illustrative examples, not an evaluation split. All six
|
| 101 |
currently have the `Likely scam` label, so they must not be used to estimate
|
| 102 |
class balance, accuracy, recall, or real-world scam prevalence.
|
| 103 |
|
| 104 |
+
Operational rows intentionally preserve repeated requests. Consequently,
|
| 105 |
+
repeated examples, unclassified image descriptions, and incomplete `none`
|
| 106 |
+
assessments may be common. For training or evaluation, curate the
|
| 107 |
+
`operational` configuration by:
|
| 108 |
|
| 109 |
+
- exclude `risk_label: none`
|
| 110 |
+
- review or exclude unclassified image rows
|
| 111 |
+
- deduplicate on the privacy-safe `input` and result columns
|
| 112 |
+
- use a task-appropriate class-balancing strategy
|
| 113 |
|
| 114 |
The source repository includes `traces/scripts/analyze_trace_dataset.py` for
|
| 115 |
schema validation and a reproducible summary of these quality indicators.
|
|
|
|
| 127 |
## Links
|
| 128 |
|
| 129 |
- App: https://huggingface.co/spaces/build-small-hackathon/noticecheck
|
| 130 |
+
- Source: https://github.com/kingabzpro/local-notice-check
|
| 131 |
+
- Field notes: https://github.com/kingabzpro/local-notice-check/blob/main/docs/field-notes.md
|
| 132 |
|
| 133 |
## License
|
| 134 |
|