mrshravan commited on
Commit
3de4e38
·
verified ·
1 Parent(s): 30966ad

temporal_causal/XS-discovery: update model card

Browse files
models/temporal_causal/XS-discovery/README.md CHANGED
@@ -6,6 +6,9 @@ cause which, at which lag, from the network's own outputs: a graph-level head sa
6
  the top `round(sum of pair probabilities)` pairs are marked. Nothing is fitted on the request and there is no
7
  statistical fallback.
8
 
 
 
 
9
  ## What XS is good for
10
 
11
  XS is the small, fast entry point: it answers on a CPU in seconds and on a GPU in a fraction of a second, with no
@@ -25,20 +28,19 @@ many steps, before investing in a deeper analysis.
25
 
26
  It is strongest on:
27
 
28
- * **Process and operations series:** controlled process data on a regular clock (AUROC 94 %, F1 79 % on our
29
- development set).
30
- * **Retail and demand series:** daily sales, prices, promotions and traffic (AUROC 88 %, F1 70 %).
31
  * **Mixed column types:** continuous, count, binary, ordinal and bounded columns, declared with `col_types`.
32
  * **Lagged effects:** a cause that acts after a delay of several steps.
33
 
34
- Use it with care on business-style tables with many weak links (AUROC 79–84 %, F1 41–47 %), and expect weaker answers
35
  on coarsely sampled or irregular series, long delays (20 to 60 steps), slow hidden trends and same-step links between
36
  count or binary columns. Effect sizes, identifiability and root cause come with the larger sizes and later XS
37
  releases.
38
 
39
  **Need higher precision or F1?** XS is the fast first look. For higher precision, recall and F1 and for harder
40
  series, use the S, M and L sizes, coming soon; each release publishes its own measured numbers, so you can choose the
41
- size by the accuracy your decision needs.
42
 
43
  ## Use
44
 
@@ -117,18 +119,34 @@ For an irregular clock, pass `time_gap`: the gap before each row, in steps of th
117
  (8 rows). It does not estimate effect sizes, identifiability or root causes.
118
  * **Known weak spots:** coarsely sampled and irregularly spaced series; long delays (20 to 60 steps), slow hidden
119
  trends, state memory and delays that vary over time; contemporaneous links between count or binary columns. Control
120
- loops, sensor read-through effects and industrial process mechanisms were not part of its training.
121
-
122
- ## Measured on a development benchmark (files inspected during development, not a holdout)
123
-
124
- AUROC / F1 in %, every temporal file sent, unanswered scored as 50 / 0:
125
-
126
- | set | files | AUROC | F1 |
127
- |---|---:|---:|---:|
128
- | controlled process series | 72 | 94.3 | 79.1 |
129
- | retail daily series | 26 | 88.4 | 69.6 |
130
- | business-style files answered by every compared checkpoint | 103 | 83.6 | 47.2 |
131
- | business-style files, all sent | 118 | 79.3 | 41.2 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
132
 
133
  ## Files
134
 
 
6
  the top `round(sum of pair probabilities)` pairs are marked. Nothing is fitted on the request and there is no
7
  statistical fallback.
8
 
9
+ **License: non-commercial use only.** The weights are released under [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/);
10
+ commercial use needs a separate license from ProfitOps Inc. See [License](#license) below.
11
+
12
  ## What XS is good for
13
 
14
  XS is the small, fast entry point: it answers on a CPU in seconds and on a GPU in a fraction of a second, with no
 
28
 
29
  It is strongest on:
30
 
31
+ * **Process and operations series:** controlled process data on a regular clock.
32
+ * **Retail and demand series:** daily sales, prices, promotions and traffic.
 
33
  * **Mixed column types:** continuous, count, binary, ordinal and bounded columns, declared with `col_types`.
34
  * **Lagged effects:** a cause that acts after a delay of several steps.
35
 
36
+ Use it with care on business-style tables with many weak links, and expect weaker answers
37
  on coarsely sampled or irregular series, long delays (20 to 60 steps), slow hidden trends and same-step links between
38
  count or binary columns. Effect sizes, identifiability and root cause come with the larger sizes and later XS
39
  releases.
40
 
41
  **Need higher precision or F1?** XS is the fast first look. For higher precision, recall and F1 and for harder
42
  series, use the S, M and L sizes, coming soon; each release publishes its own measured numbers, so you can choose the
43
+ size by the accuracy your decision needs. To ask about them, contact ProfitOps Inc.
44
 
45
  ## Use
46
 
 
119
  (8 rows). It does not estimate effect sizes, identifiability or root causes.
120
  * **Known weak spots:** coarsely sampled and irregularly spaced series; long delays (20 to 60 steps), slow hidden
121
  trends, state memory and delays that vary over time; contemporaneous links between count or binary columns. Control
122
+ loops, sensor read-through effects and industrial process mechanisms were not part of its training. For these, the
123
+ larger sizes handle more (see below).
124
+
125
+ ## Real-world discovery, against classical baselines
126
+
127
+ Run on six published real-world datasets used for benchmarking only (never
128
+ seen in training), through the model's own decision layer (pure-neural: the
129
+ graph-level "any edge" head, then the top `round(sum of pair probabilities)`
130
+ pairs). Classical baselines run on the same files with the same settings.
131
+ AUROC / F1 as fractions (0 to 1). In each row, the best AUROC and the best F1 across the three methods are in
132
+ **bold**.
133
+
134
+ | dataset | XS AUROC | XS F1 | Granger AUROC | Granger F1 | PCMCI AUROC | PCMCI F1 |
135
+ |---|---:|---:|---:|---:|---:|---:|
136
+ | Causal Chamber | **0.72** | **0.34** | 0.67 | 0.19 | 0.66 | 0.00 |
137
+ | Sachs | **0.56** | **0.32** | 0.48 | 0.23 | 0.46 | 0.00 |
138
+ | Tennessee Eastman | 0.70 | **0.16** | 0.60 | 0.03 | **0.71** | 0.08 |
139
+ | Causal Rivers | 0.65 | **0.09** | **0.71** | 0.01 | 0.58 | 0.00 |
140
+ | causRCA | **0.60** | 0.05 | 0.54 | 0.04 | 0.54 | **0.06** |
141
+ | SWaT | 0.36 | 0.00 | 0.64 | 0.07 | **0.65** | **0.10** |
142
+
143
+ Rows are ordered by how well XS does, best first. XS has the best F1 on four of
144
+ the six datasets and the best AUROC on three, and it wins both on Sachs and
145
+ Causal Chamber. It trails on SWaT, a water-network dataset where it marks no
146
+ edges. These are real industrial and scientific panels; absolute scores are
147
+ modest, as the card's limits note for untrained industrial families. Read them
148
+ against the classical baselines on the same files, not as a fraction of a
149
+ perfect graph.
150
 
151
  ## Files
152