Abid Ali Awan Codex commited on
Commit
c93cae8
·
1 Parent(s): 453edd1

Fix trace dataset presentation

Browse files

Separate reviewed seed examples from noisy operational traces, correct the dataset viewer path configuration, and update the runtime documentation.

Co-authored-by: Codex <codex@openai.com>

Files changed (1) hide show
  1. traces/dataset_card.md +29 -19
traces/dataset_card.md CHANGED
@@ -14,9 +14,14 @@ tags:
14
  - privacy
15
  configs:
16
  - config_name: default
 
17
  data_files:
18
  - split: train
19
- path: data/**/*.jsonl
 
 
 
 
20
  ---
21
 
22
  # NoticeCheck Privacy-Safe Traces
@@ -27,14 +32,16 @@ This dataset contains compact, deterministic metadata about NoticeCheck
27
  message-review requests. It does not contain hidden model reasoning or
28
  autonomous-agent trajectories.
29
 
30
- The application uses MiniCPM5-1B through an in-process llama.cpp runtime, with
31
- Nemotron OCR v2 for supported screenshots. Creating a trace never makes an
32
- additional AI model call. Traces only observe the existing request path and
33
- convert it into allow-listed categories, booleans, and fixed descriptions. For
34
- image submissions, the existing assessment's explanation and red flags are
35
- inspected transiently for this mapping, but their text is not stored. The trace
36
- mapper predicts the privacy-safe image category and tactics directly from that
37
- result summary using deterministic English/Urdu evidence rules.
 
 
38
 
39
  ## Fields
40
 
@@ -85,23 +92,24 @@ successful assessments whose content is genuinely unclassified.
85
 
86
  ## Provenance
87
 
88
- Seed traces represent the six public examples bundled with NoticeCheck. Runtime
89
- traces may represent successful, rejected, or failed requests.
 
90
  Trace generation itself does not invoke the model.
91
 
92
  The seed rows are illustrative examples, not an evaluation split. All six
93
  currently have the `Likely scam` label, so they must not be used to estimate
94
  class balance, accuracy, recall, or real-world scam prevalence.
95
 
96
- Runtime rows are an operational log and intentionally preserve repeated
97
- requests. Consequently, repeated examples, unclassified image descriptions,
98
- and incomplete `none` assessments may be common. For training or evaluation,
99
- create a separate curated split that:
100
 
101
- - excludes `risk_label: none`
102
- - reviews or excludes unclassified image rows
103
- - deduplicates on the privacy-safe `input` and result columns
104
- - uses a task-appropriate class-balancing strategy
105
 
106
  The source repository includes `traces/scripts/analyze_trace_dataset.py` for
107
  schema validation and a reproducible summary of these quality indicators.
@@ -119,6 +127,8 @@ schema validation and a reproducible summary of these quality indicators.
119
  ## Links
120
 
121
  - App: https://huggingface.co/spaces/build-small-hackathon/noticecheck
 
 
122
 
123
  ## License
124
 
 
14
  - privacy
15
  configs:
16
  - config_name: default
17
+ default: true
18
  data_files:
19
  - split: train
20
+ path: data/seed/trace_samples.jsonl
21
+ - config_name: operational
22
+ data_files:
23
+ - split: train
24
+ path: data/*/*/*/*.jsonl
25
  ---
26
 
27
  # NoticeCheck Privacy-Safe Traces
 
32
  message-review requests. It does not contain hidden model reasoning or
33
  autonomous-agent trajectories.
34
 
35
+ The hosted application uses MiniCPM5-1B through Transformers on Hugging Face
36
+ ZeroGPU, with NVIDIA Nemotron-Parse v1.2 for supported screenshots. The same
37
+ pipeline can run locally on an NVIDIA GPU with Docker Compose. Creating a trace
38
+ never makes an additional AI model call. Traces only observe the existing
39
+ request path and convert it into allow-listed categories, booleans, and fixed
40
+ descriptions. For image submissions, the existing assessment's explanation and
41
+ red flags are inspected transiently for this mapping, but their text is not
42
+ stored. The trace mapper predicts the privacy-safe image category and tactics
43
+ directly from that result summary using deterministic English/Urdu evidence
44
+ rules.
45
 
46
  ## Fields
47
 
 
92
 
93
  ## Provenance
94
 
95
+ The default configuration contains the six reviewed public examples bundled
96
+ with NoticeCheck. The `operational` configuration contains privacy-safe runtime
97
+ traces and may include successful, rejected, or failed requests.
98
  Trace generation itself does not invoke the model.
99
 
100
  The seed rows are illustrative examples, not an evaluation split. All six
101
  currently have the `Likely scam` label, so they must not be used to estimate
102
  class balance, accuracy, recall, or real-world scam prevalence.
103
 
104
+ Operational rows intentionally preserve repeated requests. Consequently,
105
+ repeated examples, unclassified image descriptions, and incomplete `none`
106
+ assessments may be common. For training or evaluation, curate the
107
+ `operational` configuration by:
108
 
109
+ - exclude `risk_label: none`
110
+ - review or exclude unclassified image rows
111
+ - deduplicate on the privacy-safe `input` and result columns
112
+ - use a task-appropriate class-balancing strategy
113
 
114
  The source repository includes `traces/scripts/analyze_trace_dataset.py` for
115
  schema validation and a reproducible summary of these quality indicators.
 
127
  ## Links
128
 
129
  - App: https://huggingface.co/spaces/build-small-hackathon/noticecheck
130
+ - Source: https://github.com/kingabzpro/local-notice-check
131
+ - Field notes: https://github.com/kingabzpro/local-notice-check/blob/main/docs/field-notes.md
132
 
133
  ## License
134