jzxiao commited on
Commit
3b49d32
·
verified ·
1 Parent(s): 8e46632

Align dataset summary with measured coverage

Browse files

Use the same measured dataset selection for the summary count and coverage denominator. The default Overall view consistently shows 164 measured datasets. Keep all 169 metadata entries in the dataset browser and preserve the published measurements, scoring and rank rules.

Files changed (3) hide show
  1. README.md +1 -1
  2. app.js +4 -4
  3. index.html +4 -4
README.md CHANGED
@@ -41,7 +41,7 @@ Only measurements in the authoritative current project scope are exported. Missi
41
  - Overall composite score and average dataset Rank, with 66 valid non-negative one-decimal weight combinations, including zero weights
42
  - Searchable Methods documentation and a 30-family operator catalog with 102 implementation-directory entries, references, I/O contracts, examples, and status
43
 
44
- The dataset catalog and measured benchmark scope are separate. All 169 catalog entries are searchable and selectable. Dataset coverage can differ by method and runtime; unavailable combinations show **No published measurements**, remain excluded from rankings, and are never converted to zero-valued results. The default family view selects the best measured compression-rate configuration independently for each measured dataset. Switch to **Exact configuration** to hold a configuration fixed. Runtime results stay separate.
45
 
46
  The Overall score is `a × average rate + b × normalized average encode time + c × normalized average decode time`. Time normalization is linear min–max across all eligible rows in the selected runtime, scope and row mode; an equal-time range normalizes to zero. Search, family and coverage filters only hide rows and never change the reference. Average Rank averages per-dataset composite-score ranks with dataset-local time normalization and fractional ties. Missing results are omitted. Coverage and candidate counts can differ across datasets; method details expose dataset scores, ranks and candidate counts. CSV downloads include weights, normalization bounds and rank coverage. The default weights are `a=0.4, b=0.3, c=0.3`.
47
 
 
41
  - Overall composite score and average dataset Rank, with 66 valid non-negative one-decimal weight combinations, including zero weights
42
  - Searchable Methods documentation and a 30-family operator catalog with 102 implementation-directory entries, references, I/O contracts, examples, and status
43
 
44
+ The summary dataset count and Dataset coverage denominator use the same measured dataset set for the selected runtime, data type, and dataset selection; the default Overall view has 164 measured datasets. All 169 catalog entries remain searchable and selectable in the dataset browser. Dataset coverage can differ by method and runtime; unavailable combinations show **No published measurements**, remain excluded from rankings, and are never converted to zero-valued results. The default family view selects the best measured compression-rate configuration independently for each measured dataset. Switch to **Exact configuration** to hold a configuration fixed. Runtime results stay separate.
45
 
46
  The Overall score is `a × average rate + b × normalized average encode time + c × normalized average decode time`. Time normalization is linear min–max across all eligible rows in the selected runtime, scope and row mode; an equal-time range normalizes to zero. Search, family and coverage filters only hide rows and never change the reference. Average Rank averages per-dataset composite-score ranks with dataset-local time normalization and fractional ties. Missing results are omitted. Coverage and candidate counts can differ across datasets; method details expose dataset scores, ranks and candidate counts. CSV downloads include weights, normalization bounds and rank coverage. The default weights are `a=0.4, b=0.3, c=0.3`.
47
 
app.js CHANGED
@@ -41,10 +41,11 @@ function filteredRecords(ignoreMethod=false) {
41
  }
42
  return ignoreMethod?result:result.filter(r=>(s.family==='all'||r.family===s.family)&&(!s.search||(s.mode==='family'?r.family:`${r.method} ${r.family}`).toLowerCase().includes(s.search)));
43
  }
 
44
  function aggregate(input) {
45
  const groups=new Map();
46
  for(const r of input) { if(!groups.has(r.method)) groups.set(r.method,[]); groups.get(r.method).push(r); }
47
- const totalDatasets=new Set(filteredRecords(true).map(r=>r.dataset)).size;
48
  const rows=[];
49
  for(const [method,source] of groups) {
50
  const hasBytes=source.every(r=>r.original_bytes!==null && r.original_bytes>0 && r.compressed_bytes!==null && r.compressed_bytes>0);
@@ -83,7 +84,8 @@ function refresh() {
83
  currentRows=aggregate(filteredRecords(true));
84
  const weights=parseWeights($('weights').value);
85
  $('score-note').textContent=`Score = ${weights[0].toFixed(1)} × average rate + ${weights[1].toFixed(1)} × normalized encode + ${weights[2].toFixed(1)} × normalized decode. Lower is better. Time normalization: (time − minimum) / (maximum − minimum), across all measured rows in this runtime, scope and row mode. Search and visibility filters keep the reference fixed. Average Rank: mean of per-dataset score ranks with dataset-local time normalization; ties share their average position. Missing results are omitted. Dataset candidate counts differ, so compare rank coverage and inspect method details.`;
86
- const s=state(),scope=scopeKey(),measuredCount=new Set(records.filter(r=>r.runtime===s.runtime&&r.scope===scope).map(r=>r.dataset)).size,ds=s.dataset==='all'?`all ${measuredCount} measured datasets`:datasetDisplayName(s.dataset);
 
87
  const names={overall:'available numeric measurements',integer:'integer columns',float:'floating-point columns',fixed_float:'fixed-precision float columns',nonfixed_float:'non-fixed-precision float columns'};
88
  $('selection-summary').textContent=`${runtimeName(s.runtime)} · ${names[scope]||scope} · ${ds} · ${currentRows.length} methods`;
89
  $('view-title').textContent=activeTab==='dataset'?`${s.dataset==='all'?'Dataset':datasetDisplayName(s.dataset)} leaderboard`:activeTab==='tradeoffs'?'Storage and execution trade-offs':'Overall leaderboard';
@@ -240,8 +242,6 @@ function populateDatasetControls() {
240
  const groupSelect=$('dataset-group-filter'),previousGroup=groupSelect.value||'all',categories=[...groups.keys()];
241
  groupSelect.innerHTML='<option value="all">All groups</option>'+categories.map(group=>`<option value="${escape(group)}">${escape(group)} (${groups.get(group).length})</option>`).join('');
242
  groupSelect.value=categories.includes(previousGroup)?previousGroup:'all';
243
- const catalogCount=number(datasetCatalog?.summary?.dataset_count??snapshot.coverage?.catalog_dataset_count)??entries.length,measuredCount=number(datasetCatalog?.summary?.published_result_dataset_count??datasetCatalog?.summary?.published_measurement_dataset_count??datasetCatalog?.summary?.measured_dataset_count??snapshot.coverage?.measured_dataset_count)??measured.size;
244
- $('dataset-count').textContent=countFmt(catalogCount);$('dataset-count-label').textContent=`datasets in catalog · ${countFmt(measuredCount)} measured`;
245
  }
246
  function renderDatasetCatalog() {
247
  const container=$('dataset-catalog'),status=$('dataset-catalog-status');if(!container||!status||!snapshot)return;
 
41
  }
42
  return ignoreMethod?result:result.filter(r=>(s.family==='all'||r.family===s.family)&&(!s.search||(s.mode==='family'?r.family:`${r.method} ${r.family}`).toLowerCase().includes(s.search)));
43
  }
44
+ function coverageDatasetCount() { return new Set(filteredRecords(true).map(r=>r.dataset)).size; }
45
  function aggregate(input) {
46
  const groups=new Map();
47
  for(const r of input) { if(!groups.has(r.method)) groups.set(r.method,[]); groups.get(r.method).push(r); }
48
+ const totalDatasets=coverageDatasetCount();
49
  const rows=[];
50
  for(const [method,source] of groups) {
51
  const hasBytes=source.every(r=>r.original_bytes!==null && r.original_bytes>0 && r.compressed_bytes!==null && r.compressed_bytes>0);
 
84
  currentRows=aggregate(filteredRecords(true));
85
  const weights=parseWeights($('weights').value);
86
  $('score-note').textContent=`Score = ${weights[0].toFixed(1)} × average rate + ${weights[1].toFixed(1)} × normalized encode + ${weights[2].toFixed(1)} × normalized decode. Lower is better. Time normalization: (time − minimum) / (maximum − minimum), across all measured rows in this runtime, scope and row mode. Search and visibility filters keep the reference fixed. Average Rank: mean of per-dataset score ranks with dataset-local time normalization; ties share their average position. Missing results are omitted. Dataset candidate counts differ, so compare rank coverage and inspect method details.`;
87
+ const s=state(),scope=scopeKey(),measuredCount=coverageDatasetCount(),ds=s.dataset==='all'?`all ${measuredCount} measured datasets`:datasetDisplayName(s.dataset);
88
+ $('dataset-count').textContent=countFmt(measuredCount);$('dataset-count-label').textContent='measured datasets';
89
  const names={overall:'available numeric measurements',integer:'integer columns',float:'floating-point columns',fixed_float:'fixed-precision float columns',nonfixed_float:'non-fixed-precision float columns'};
90
  $('selection-summary').textContent=`${runtimeName(s.runtime)} · ${names[scope]||scope} · ${ds} · ${currentRows.length} methods`;
91
  $('view-title').textContent=activeTab==='dataset'?`${s.dataset==='all'?'Dataset':datasetDisplayName(s.dataset)} leaderboard`:activeTab==='tradeoffs'?'Storage and execution trade-offs':'Overall leaderboard';
 
242
  const groupSelect=$('dataset-group-filter'),previousGroup=groupSelect.value||'all',categories=[...groups.keys()];
243
  groupSelect.innerHTML='<option value="all">All groups</option>'+categories.map(group=>`<option value="${escape(group)}">${escape(group)} (${groups.get(group).length})</option>`).join('');
244
  groupSelect.value=categories.includes(previousGroup)?previousGroup:'all';
 
 
245
  }
246
  function renderDatasetCatalog() {
247
  const container=$('dataset-catalog'),status=$('dataset-catalog-status');if(!container||!status||!snapshot)return;
index.html CHANGED
@@ -7,7 +7,7 @@
7
  <meta name="color-scheme" content="light dark">
8
  <title>THULab | Time Series Compression Benchmark</title>
9
  <link rel="stylesheet" href="style.css?v=20261009-dbp-integrated-v1">
10
- <script src="app.js?v=20261009-zero-weights-v1" type="module"></script>
11
  </head>
12
  <body>
13
  <a class="skip-link" href="#content">Skip to results</a>
@@ -19,11 +19,11 @@
19
  <section class="intro" aria-labelledby="title">
20
  <p class="eyebrow">Time-series systems research</p>
21
  <h1 id="title">Time Series Compression Leaderboard</h1>
22
- <p class="intro-text">Browse the 169-dataset catalog and compare published storage, encoding, and decoding measurements.</p>
23
  <p class="snapshot" id="snapshot">Loading benchmark snapshot…</p>
24
  </section>
25
  <div class="summary-strip" id="summary" aria-label="Snapshot coverage">
26
- <div><strong id="dataset-count">…</strong><span id="dataset-count-label">datasets in catalog</span></div>
27
  <div><strong id="method-count">…</strong><span>method families</span></div>
28
  <div><strong id="runtime-count">…</strong><span>implementation runtimes</span></div>
29
  <div><strong id="record-count">…</strong><span>runtime / codec configurations</span></div>
@@ -109,7 +109,7 @@
109
  <div><dt>Average compression rate ↓</dt><dd>Arithmetic mean of the per-dataset compression rates in the selected scope.</dd></div>
110
  <div><dt>Overall compression rate ↓</dt><dd>Total compressed bytes divided by total original bytes over the selected measured records.</dd></div>
111
  <div><dt>Encoding / decoding cost ↓</dt><dd>Nanoseconds per point under each result's recorded measurement contract. Native-float measurements use valid native values; historical imported rows retain their source 8-byte-equivalent point definition. Compare timings within the same runtime and measurement contract.</dd></div>
112
- <div><dt>Coverage</dt><dd>Datasets with usable measurements for a method out of the datasets in the selected scope. Missing results are never filled with zero.</dd></div>
113
  <div><dt>Composite score ↓</dt><dd>a × average compression rate + b × normalized average encoding time + c × normalized average decoding time. Each time uses min–max over all measured rows in the selected runtime, scope and row mode, before search, family and coverage filters. If all times are equal, their normalized value is zero. All measured inputs must have usable timing metrics.</dd></div>
114
  <div><dt>Average Rank ↓</dt><dd>Compute the same weighted score within each dataset, normalizing times within that dataset. Rank lower scores first; ties share their average position. Then take the arithmetic mean of the method's available dataset ranks. Weights are non-negative tenths summing to 1.0. Missing measurements do not receive ranks.</dd></div>
115
  </dl>
 
7
  <meta name="color-scheme" content="light dark">
8
  <title>THULab | Time Series Compression Benchmark</title>
9
  <link rel="stylesheet" href="style.css?v=20261009-dbp-integrated-v1">
10
+ <script src="app.js?v=20261009-coverage-v1" type="module"></script>
11
  </head>
12
  <body>
13
  <a class="skip-link" href="#content">Skip to results</a>
 
19
  <section class="intro" aria-labelledby="title">
20
  <p class="eyebrow">Time-series systems research</p>
21
  <h1 id="title">Time Series Compression Leaderboard</h1>
22
+ <p class="intro-text">Compare published storage, encoding, and decoding measurements across time-series datasets.</p>
23
  <p class="snapshot" id="snapshot">Loading benchmark snapshot…</p>
24
  </section>
25
  <div class="summary-strip" id="summary" aria-label="Snapshot coverage">
26
+ <div><strong id="dataset-count">…</strong><span id="dataset-count-label">measured datasets</span></div>
27
  <div><strong id="method-count">…</strong><span>method families</span></div>
28
  <div><strong id="runtime-count">…</strong><span>implementation runtimes</span></div>
29
  <div><strong id="record-count">…</strong><span>runtime / codec configurations</span></div>
 
109
  <div><dt>Average compression rate ↓</dt><dd>Arithmetic mean of the per-dataset compression rates in the selected scope.</dd></div>
110
  <div><dt>Overall compression rate ↓</dt><dd>Total compressed bytes divided by total original bytes over the selected measured records.</dd></div>
111
  <div><dt>Encoding / decoding cost ↓</dt><dd>Nanoseconds per point under each result's recorded measurement contract. Native-float measurements use valid native values; historical imported rows retain their source 8-byte-equivalent point definition. Compare timings within the same runtime and measurement contract.</dd></div>
112
+ <div><dt>Coverage</dt><dd>Datasets with usable measurements for a method out of the measured datasets in the selected runtime, data type, and dataset selection. The summary dataset count uses the same selection. Missing results are never filled with zero.</dd></div>
113
  <div><dt>Composite score ↓</dt><dd>a × average compression rate + b × normalized average encoding time + c × normalized average decoding time. Each time uses min–max over all measured rows in the selected runtime, scope and row mode, before search, family and coverage filters. If all times are equal, their normalized value is zero. All measured inputs must have usable timing metrics.</dd></div>
114
  <div><dt>Average Rank ↓</dt><dd>Compute the same weighted score within each dataset, normalizing times within that dataset. Rank lower scores first; ties share their average position. Then take the arithmetic mean of the method's available dataset ranks. Weights are non-negative tenths summing to 1.0. Missing measurements do not receive ranks.</dd></div>
115
  </dl>