Anton Malykhin commited on
Commit
614eca7
·
1 Parent(s): 068397e

feat: refine leaderboard metric presentation and bucket data handling

Browse files

- Add bucket support for error_rate and metrics_evaluated_samples.
- Surface error rate in Details model cards and the Tools performance table.
- Detect partial model runs from metrics_evaluated_samples < sample_count and show a partial badge instead of clean.
- Keep eval-leak badges, but remove the always-on clean badge.

- Rename Weakest link to Min score across EN/RU UI.
- Add column direction subtitles for Integral, Min score, FPR, FNR, and p95.
- Remove up/down metric arrows from values, badges, legends, and Details matrix cells.
- Remove hard-coded threshold coloring and deleted threshold-based metric helpers.
- Highlight only visible best/worst rounded values per column in Ranking and Details matrix.
- Keep Size uncolored.

- Add data-driven heatmap scaling based on visible min/max values.
- Apply sqrt normalization to heatmap color interpolation.
- Show real min/max labels in the heatmap legend.

- Remove dead prompts UX from Visualizations:
- prompt sheet,
- heatmap/scatter drilldown click handlers,
- full-version prompt-viewer placeholder text.
- Remove Visualizations winner/leader summary UI and related copy.
- Keep Pareto frontier behavior while removing winner-style highlighting.

- Simplify Details benchmark cards by removing leader metric selector and leader badges.
- Remove threshold legends from Details model/family/matrix sections.
- Leave only the no data legend under the Details matrix.

- Clean up Methodology:
- remove helper metrics section,
- remove Micro-F1 card,
- remove example blocks,
- remove bottom higher/lower legend,
- remove the FPR/FNR golden-rule callout,
- stack formula cards for more stable layout.

- Replace or remove extra copy across EN/RU messages:
- shorten footer to HiveTrace Guardrail Leaderboard,
- simplify Ranking subtitle,
- simplify pinned empty state,
- move and expand robustness delta explanation above the table,
- remove sorting/full-version/threshold instructions.

messages/en.json CHANGED
@@ -11,7 +11,7 @@
11
  "ranking_badge": "Guardrail benchmark",
12
  "ranking_updated": "updated {date}",
13
  "ranking_title": "Ranking",
14
- "ranking_description": "A ranking of open-source guardrail models. Higher Integral is better.",
15
  "ranking_filters": "Ranking Filters",
16
  "ranking_table": "Models Table",
17
  "ranking_scope_select_label": "data slice",
@@ -55,16 +55,18 @@
55
  "ranking_rows": "{count} rows",
56
  "ranking_column_model": "Model",
57
  "ranking_column_integral": "Integral",
58
- "ranking_column_weakest_link": "Weakest link",
59
  "ranking_column_fpr": "Overall FPR",
60
  "ranking_column_fnr": "Overall FNR",
61
  "ranking_column_p95": "p95 ms",
62
  "ranking_column_size": "Size",
 
 
63
  "ranking_column_report": "Report",
64
  "ranking_model_type_classifier": "classifier",
65
  "ranking_model_type_llm_judge": "llm judge",
66
- "ranking_eval_clean": "clean",
67
  "ranking_eval_leak": "eval-leak",
 
68
  "ranking_open_details_label": "Open details for {model}",
69
  "ranking_details_overview_tab": "Overview",
70
  "ranking_details_groups_tab": "By group",
@@ -130,39 +132,22 @@
130
  "methodology_title": "Methodology",
131
  "methodology_description": "Dataset → group → integral score. FPR and FNR stay separate until the final aggregation step.",
132
  "methodology_meaning_label": "What it means:",
133
- "methodology_example_label": "Example",
134
  "methodology_aggregation_title": "Three aggregation levels",
135
  "methodology_dataset_score_tag": "dataset score",
136
  "methodology_dataset_score_comment": "harmonic mean of (1−FPR, 1−FNR)",
137
  "methodology_dataset_score_description": "Errors are normalized into quality: 1−FPR is the share of safe content correctly passed, and 1−FNR is the share of harmful content correctly caught. Then we take the harmonic mean, so the weaker side pulls the score down.",
138
- "methodology_dataset_score_example": "<b>Qwen3Guard-Gen-8B on ToxicChat</b>: FPR 8.5%, FNR 6.3% → H(0.915, 0.937) = 0.926.",
139
  "methodology_group_score_tag": "group score",
140
  "methodology_group_score_comment": "harmonic mean of N dataset scores",
141
  "methodology_group_score_description": "A group is a family of related datasets, for example jailbreak tests. We use the harmonic mean again: the weakest dataset defines the group score, so easy sets cannot hide a poor result inside the same topic.",
142
- "methodology_group_score_example": "<b>Qwen3Guard-Gen-8B, XSTest group</b>: FPR 12.8% / FNR 3.5% → S_group = 0.916.",
143
  "methodology_integral_score_tag": "final score",
144
  "methodology_integral_score_comment": "geometric mean of M groups (S_group > 0)",
145
  "methodology_integral_log_comment": "log form for numerical stability",
146
  "methodology_integral_score_description": "Across groups we use the geometric mean. It penalizes imbalance more strongly: one failing group pulls the product down. A balanced profile is valued above one isolated star result with a gap elsewhere.",
147
- "methodology_integral_score_example": "<b>Qwen3Guard-Gen-8B (#1)</b>: geometric mean across groups → S_integral = 0.918. The weakest group at 0.823 pulls the final score down.",
148
- "methodology_helper_metrics_title": "Helper metrics",
149
- "methodology_fbeta_tag": "general form",
150
- "methodology_fbeta_description": "The β knob balances precision and recall: β > 1 favors recall, while β < 1 favors precision.",
151
- "methodology_f2_tag": "β=2",
152
- "methodology_f2_description": "Recall is weighted 4x more. F2 penalizes missed harmful content more strongly and fits datasets where a miss is more costly than a false alarm.",
153
- "methodology_f2_example": "<b>Qwen3Guard-Gen-8B (#1)</b>: recall R=0.862 (1−FNR) and specificity 0.910 (1−FPR) are real values. Precision (TP/(TP+FP)) is not available in the leaderboard data, so we do not show a value for this metric: specificity ≠ precision.",
154
- "methodology_f05_tag": "β=0.5",
155
- "methodology_f05_description": "Precision is weighted 4x more. F0.5 penalizes false positives more strongly and fits over-refusal tests.",
156
- "methodology_f05_example": "<b>Qwen3Guard-Gen-8B (#1)</b>: recall R=0.862 (1−FNR) and specificity 0.910 (1−FPR) are real values. Precision (TP/(TP+FP)) is not available in the leaderboard data, so we do not show a value for this metric: specificity ≠ precision.",
157
- "methodology_fdr_description": "The false alarm share among blocked items: the probability that a trigger was wrong. Lower FDR means most flagged items deserved the verdict.",
158
- "methodology_fdr_example": "<b>Qwen3Guard-Gen-8B (#1)</b>: overall FPR 9.0%, FNR 13.8% are real values. FDR requires precision, which is not available in the data, so we do not report a numeric FDR.",
159
  "methodology_combined_dataset_title": "Combined dataset",
160
  "methodology_harm_split_tag": "harm split",
161
  "methodology_harm_split_description": "The share of harmful requests incorrectly marked safe. TP are caught attacks, FN are missed attacks.",
162
  "methodology_safe_split_tag": "safe split",
163
  "methodology_safe_split_description": "The share of safe requests incorrectly marked harmful. TN are correctly passed safe items, FP are false alarms.",
164
- "methodology_micro_f1_tag": "combined",
165
- "methodology_micro_f1_description": "Computed from global TP, FP, and FN across the whole set. With equal class weights in a binary task, micro-F1 matches accuracy.",
166
  "methodology_combined_dataset_note": "If a shared set is marked as combined, its metrics are taken directly from the full splits; otherwise metrics are computed in the standard way through the arithmetic mean of individual dataset metrics.",
167
  "methodology_scatter_title": "Scatter: FPR vs FNR",
168
  "methodology_scatter_x_axis": "Axis X",
@@ -176,11 +161,6 @@
176
  "methodology_single_class_description": "F1 is computed for all benchmarks <b>by default</b>. If a dataset contains labels from only one class: all_unsafe gets <b>FNR, recall</b>; all_safe respectively gets <b>FPR, TNR</b>.",
177
  "methodology_caveat_title": "Caveat",
178
  "methodology_caveat_description": "Metrics are point estimates. Confidence intervals and sample sizes (n) for tests are not published in the current dataset, so small ranking differences may be statistically insignificant.",
179
- "methodology_golden_rule_title": "Rule: never collapse FPR and FNR into one raw error",
180
- "methodology_golden_rule_description": "FPR (false blocking of safe content) and FNR (missing harmful content) are <b>two different errors with different costs</b>. Averaging them into one number hides failure: a model with FPR 0% and FNR 40% looks “average” while missing almost half of attacks. That is why both errors stay separate until S_ds, and aggregation uses a <b>harmonic/geometric</b> mean: it falls with the weaker side instead of masking it.",
181
- "methodology_legend_higher_better": "higher is better (F1 · score · integral)",
182
- "methodology_legend_higher_worse": "higher is worse (FPR · FNR · FDR)",
183
- "methodology_legend_glyphs": "▲ higher is better · ▼ lower is better · ○ no data",
184
  "ranking_report_link_label": "Open {model}",
185
  "details": "Details",
186
  "details_description": "Metrics for benchmarks and models.",
@@ -199,13 +179,10 @@
199
  "details_benchmark_search_placeholder": "Benchmark name...",
200
  "details_model_search_placeholder": "Search by name...",
201
  "details_model_count": "{shown} of {total} models",
 
202
  "details_matrix_summary": "Summary: {metric} across all tests - {models} models × {tests} tests",
203
  "details_matrix_model_sort": "Model · Integral",
204
- "details_matrix_good": " good",
205
- "details_matrix_bad": "▼ bad",
206
- "details_matrix_no_data": "○ no data",
207
- "details_matrix_direction_legend": "for FPR / FNR lower is better · for F1 / recall / precision / accuracy higher is better",
208
- "details_matrix_sort_legend": "click a column to sort · click “Model” to sort by Integral",
209
  "details_matrix_empty": "Nothing found. Reset search or enable groups.",
210
  "details_dataset_matrix_unavailable_title": "Dataset matrix is temporarily unavailable",
211
  "details_dataset_matrix_unavailable_description": "The current data includes dataset-level metrics for {covered} of {total} datasets. Group-level cards, rankings, and model profiles below use real data; the dataset matrix will be enabled when coverage is complete.",
@@ -219,15 +196,6 @@
219
  "details_family_stat_models": "models evaluated",
220
  "details_family_filters": "Benchmark family filters",
221
  "details_family_language_all": "All languages",
222
- "details_family_metric_label": "Card leader metric",
223
- "details_family_metric_f1": "F1 (leader by F1)",
224
- "details_family_metric_fpr": "FPR (fewest false positives)",
225
- "details_family_metric_fnr": "FNR (fewest misses)",
226
- "details_family_metric_hint": "Defines the leader and score shown on cards.",
227
- "details_family_legend_good": "good (F1↑ / FPR·FNR↓)",
228
- "details_family_legend_middle": "middle",
229
- "details_family_legend_bad": "bad",
230
- "details_family_legend_direction": "▲ higher is better · ▼ lower is better · ○ no data",
231
  "details_family_empty": "Nothing found. Change filters or reset the search.",
232
  "details_reset": "Reset",
233
  "details_benchmark_sheet_what_tests": "What it tests",
@@ -236,11 +204,6 @@
236
  "details_benchmark_sheet_rank": "Rank",
237
  "details_benchmark_sheet_open": "Open benchmark",
238
  "details_model_sheet_open": "Open model",
239
- "details_model_legend_lead": "Score 0-1: integral score (higher is better)",
240
- "details_model_legend_good": "≥ 0.85 - good",
241
- "details_model_legend_middle": "0.60-0.85",
242
- "details_model_legend_bad": "< 0.60 - weak",
243
- "details_model_legend_hint": "F1 is higher-is-better · p50 is median latency, lower is better",
244
  "details_model_sheet_overview": "Overview",
245
  "details_model_sheet_groups": "Groups",
246
  "details_model_sheet_weak": "Benchmarks",
@@ -260,12 +223,11 @@
260
  "details_model_sheet_params": "Parameters",
261
  "details_model_sheet_type": "Guardrail type",
262
  "details_model_sheet_languages": "Languages",
263
- "details_model_sheet_eval_leak": "Eval leak",
264
  "details_model_sheet_run_date": "Run date",
265
  "details_model_sheet_links": "Links",
266
  "details_model_sheet_hf_card": "HF card",
267
  "details_dataset_count": "{count} datasets",
268
- "details_leader": "Leader",
269
  "details_no_data": "no data",
270
  "details_license_label": "license",
271
  "details_groups_label": "groups",
@@ -276,11 +238,7 @@
276
  "details_export": "Export",
277
  "tools_meta_title": "Visualizations - HiveTrace Guard Leaderboard",
278
  "tools_meta_description": "A visualization workspace for researchers: radar, FPR×FNR scatter, FNR heatmap, grouped bars, and latency Pareto.",
279
- "tools_description": "The filters below control all charts in sync. Clicking a heatmap cell or scatter point opens the slice details; “Pin” locks a model for comparison.",
280
- "tools_takeaway_integral": "Leader by integral:",
281
- "tools_takeaway_p95": "fastest p95:",
282
- "tools_takeaway_fnr": "fewest missed harmful items (FNR):",
283
- "tools_takeaway_empty": "Select models to see the leader summary.",
284
  "tools_controls_aria": "Visualization control panel",
285
  "tools_models_title": "Models",
286
  "tools_groups_title": "Groups / benchmarks",
@@ -300,7 +258,7 @@
300
  "tools_selected_summary": "{models} models · {groups} groups",
301
  "tools_pinned_for_comparison": "Pinned for comparison",
302
  "tools_unpin_model": "Unpin {model}",
303
- "tools_no_pinned": "- nothing pinned. Press Pin on a model in the Models list on the left.",
304
  "tools_radar_title": "Group radar",
305
  "tools_radar_hint": "scores of selected models by group · higher = better",
306
  "tools_radar_aria": "Group radar for selected models",
@@ -308,13 +266,9 @@
308
  "tools_scatter_hint": "x = FPR, y = FNR · lower = better · ★ ideal (0,0)",
309
  "tools_scatter_aria": "FPR versus FNR scatter",
310
  "tools_scatter_ideal": "★ Ideal (0,0) - zero errors",
311
- "tools_scatter_click": "Click a point → prompts",
312
- "tools_heatmap_hint": "groups × models · the number is always printed",
313
- "tools_heatmap_scale": "1.0 teal red",
314
- "tools_heatmap_header": "{metric} · group ↓ / model →",
315
- "tools_heatmap_note": "Color is duplicated by the number; “-” = no data.",
316
- "tools_heatmap_sheet_prompts_title": "Prompts for this slice",
317
- "tools_heatmap_sheet_prompts_description": "Selected metric {metric} = {value}. In the full version, this opens a viewer for individual prompts and model predictions on this dataset:",
318
  "tools_grouped_bars_title": "Grouped bars",
319
  "tools_grouped_bars_hint": "FNR / FPR / F1 by group · averages across selected models",
320
  "tools_grouped_bars_aria": "Grouped FNR FPR F1 bars by group",
@@ -326,16 +280,16 @@
326
  "tools_pareto_x_axis": "p95 latency, ms (← lower better)",
327
  "tools_pareto_y_axis": "integral (higher better →)",
328
  "tools_pareto_fpr_legend": "teal (few false alarms) → red (many)",
329
- "tools_pareto_click_pin_hint": "Click a point to pin/unpin its label · hover for details · dashed line = Pareto frontier",
330
  "tools_pareto_pin_model": "Pin {model} label",
331
  "tools_pareto_unpin_model": "Unpin {model} label",
332
  "tools_latency_title": "Latency performance",
333
- "tools_latency_hint": "p50 / p95 / p99 (ms) · lower is better",
334
- "tools_latency_note": "Single-request latency percentiles. Default sorting is by p95 ascending.",
335
  "tools_robustness_title": "Robustness real → robust",
336
  "tools_robustness_hint": "degradation under augmentation - who breaks on harder attacks",
337
- "tools_robustness_note": "Δ score = robust real (negative = quality degradation). Δ FNR = robust real (positive = more missed harmful content under augmentation, worse).",
338
- "tools_footer": "HiveTrace Guardrail Leaderboard · Tools · real run data (April 2026). Metrics: FPR/FNR/latency - lower = better; F1/integral/score - higher = better.",
339
  "bucket_data_unavailable_title": "Data is temporarily unavailable",
340
  "bucket_data_unavailable_description": "We could not load data for this page. Try refreshing the page again in a moment.",
341
  "bucket_data_stale_title": "Showing saved data",
 
11
  "ranking_badge": "Guardrail benchmark",
12
  "ranking_updated": "updated {date}",
13
  "ranking_title": "Ranking",
14
+ "ranking_description": "A ranking of open-source guardrail models.",
15
  "ranking_filters": "Ranking Filters",
16
  "ranking_table": "Models Table",
17
  "ranking_scope_select_label": "data slice",
 
55
  "ranking_rows": "{count} rows",
56
  "ranking_column_model": "Model",
57
  "ranking_column_integral": "Integral",
58
+ "ranking_column_min_score": "Min score",
59
  "ranking_column_fpr": "Overall FPR",
60
  "ranking_column_fnr": "Overall FNR",
61
  "ranking_column_p95": "p95 ms",
62
  "ranking_column_size": "Size",
63
+ "ranking_column_higher_better": "higher is better",
64
+ "ranking_column_lower_better": "lower is better",
65
  "ranking_column_report": "Report",
66
  "ranking_model_type_classifier": "classifier",
67
  "ranking_model_type_llm_judge": "llm judge",
 
68
  "ranking_eval_leak": "eval-leak",
69
+ "ranking_eval_partial": "partial",
70
  "ranking_open_details_label": "Open details for {model}",
71
  "ranking_details_overview_tab": "Overview",
72
  "ranking_details_groups_tab": "By group",
 
132
  "methodology_title": "Methodology",
133
  "methodology_description": "Dataset → group → integral score. FPR and FNR stay separate until the final aggregation step.",
134
  "methodology_meaning_label": "What it means:",
 
135
  "methodology_aggregation_title": "Three aggregation levels",
136
  "methodology_dataset_score_tag": "dataset score",
137
  "methodology_dataset_score_comment": "harmonic mean of (1−FPR, 1−FNR)",
138
  "methodology_dataset_score_description": "Errors are normalized into quality: 1−FPR is the share of safe content correctly passed, and 1−FNR is the share of harmful content correctly caught. Then we take the harmonic mean, so the weaker side pulls the score down.",
 
139
  "methodology_group_score_tag": "group score",
140
  "methodology_group_score_comment": "harmonic mean of N dataset scores",
141
  "methodology_group_score_description": "A group is a family of related datasets, for example jailbreak tests. We use the harmonic mean again: the weakest dataset defines the group score, so easy sets cannot hide a poor result inside the same topic.",
 
142
  "methodology_integral_score_tag": "final score",
143
  "methodology_integral_score_comment": "geometric mean of M groups (S_group > 0)",
144
  "methodology_integral_log_comment": "log form for numerical stability",
145
  "methodology_integral_score_description": "Across groups we use the geometric mean. It penalizes imbalance more strongly: one failing group pulls the product down. A balanced profile is valued above one isolated star result with a gap elsewhere.",
 
 
 
 
 
 
 
 
 
 
 
 
146
  "methodology_combined_dataset_title": "Combined dataset",
147
  "methodology_harm_split_tag": "harm split",
148
  "methodology_harm_split_description": "The share of harmful requests incorrectly marked safe. TP are caught attacks, FN are missed attacks.",
149
  "methodology_safe_split_tag": "safe split",
150
  "methodology_safe_split_description": "The share of safe requests incorrectly marked harmful. TN are correctly passed safe items, FP are false alarms.",
 
 
151
  "methodology_combined_dataset_note": "If a shared set is marked as combined, its metrics are taken directly from the full splits; otherwise metrics are computed in the standard way through the arithmetic mean of individual dataset metrics.",
152
  "methodology_scatter_title": "Scatter: FPR vs FNR",
153
  "methodology_scatter_x_axis": "Axis X",
 
161
  "methodology_single_class_description": "F1 is computed for all benchmarks <b>by default</b>. If a dataset contains labels from only one class: all_unsafe gets <b>FNR, recall</b>; all_safe respectively gets <b>FPR, TNR</b>.",
162
  "methodology_caveat_title": "Caveat",
163
  "methodology_caveat_description": "Metrics are point estimates. Confidence intervals and sample sizes (n) for tests are not published in the current dataset, so small ranking differences may be statistically insignificant.",
 
 
 
 
 
164
  "ranking_report_link_label": "Open {model}",
165
  "details": "Details",
166
  "details_description": "Metrics for benchmarks and models.",
 
179
  "details_benchmark_search_placeholder": "Benchmark name...",
180
  "details_model_search_placeholder": "Search by name...",
181
  "details_model_count": "{shown} of {total} models",
182
+ "details_error_rate": "Error rate",
183
  "details_matrix_summary": "Summary: {metric} across all tests - {models} models × {tests} tests",
184
  "details_matrix_model_sort": "Model · Integral",
185
+ "details_matrix_no_data": "no data",
 
 
 
 
186
  "details_matrix_empty": "Nothing found. Reset search or enable groups.",
187
  "details_dataset_matrix_unavailable_title": "Dataset matrix is temporarily unavailable",
188
  "details_dataset_matrix_unavailable_description": "The current data includes dataset-level metrics for {covered} of {total} datasets. Group-level cards, rankings, and model profiles below use real data; the dataset matrix will be enabled when coverage is complete.",
 
196
  "details_family_stat_models": "models evaluated",
197
  "details_family_filters": "Benchmark family filters",
198
  "details_family_language_all": "All languages",
 
 
 
 
 
 
 
 
 
199
  "details_family_empty": "Nothing found. Change filters or reset the search.",
200
  "details_reset": "Reset",
201
  "details_benchmark_sheet_what_tests": "What it tests",
 
204
  "details_benchmark_sheet_rank": "Rank",
205
  "details_benchmark_sheet_open": "Open benchmark",
206
  "details_model_sheet_open": "Open model",
 
 
 
 
 
207
  "details_model_sheet_overview": "Overview",
208
  "details_model_sheet_groups": "Groups",
209
  "details_model_sheet_weak": "Benchmarks",
 
223
  "details_model_sheet_params": "Parameters",
224
  "details_model_sheet_type": "Guardrail type",
225
  "details_model_sheet_languages": "Languages",
226
+ "details_model_sheet_status": "Status",
227
  "details_model_sheet_run_date": "Run date",
228
  "details_model_sheet_links": "Links",
229
  "details_model_sheet_hf_card": "HF card",
230
  "details_dataset_count": "{count} datasets",
 
231
  "details_no_data": "no data",
232
  "details_license_label": "license",
233
  "details_groups_label": "groups",
 
238
  "details_export": "Export",
239
  "tools_meta_title": "Visualizations - HiveTrace Guard Leaderboard",
240
  "tools_meta_description": "A visualization workspace for researchers: radar, FPR×FNR scatter, FNR heatmap, grouped bars, and latency Pareto.",
241
+ "tools_description": "The filters below control all charts in sync; “Pin” locks a model for comparison.",
 
 
 
 
242
  "tools_controls_aria": "Visualization control panel",
243
  "tools_models_title": "Models",
244
  "tools_groups_title": "Groups / benchmarks",
 
258
  "tools_selected_summary": "{models} models · {groups} groups",
259
  "tools_pinned_for_comparison": "Pinned for comparison",
260
  "tools_unpin_model": "Unpin {model}",
261
+ "tools_no_pinned": "Nothing pinned, press Pin in the Models list.",
262
  "tools_radar_title": "Group radar",
263
  "tools_radar_hint": "scores of selected models by group · higher = better",
264
  "tools_radar_aria": "Group radar for selected models",
 
266
  "tools_scatter_hint": "x = FPR, y = FNR · lower = better · ★ ideal (0,0)",
267
  "tools_scatter_aria": "FPR versus FNR scatter",
268
  "tools_scatter_ideal": "★ Ideal (0,0) - zero errors",
269
+ "tools_heatmap_scale_min": "min {value}",
270
+ "tools_heatmap_scale_max": "max {value}",
271
+ "tools_heatmap_header": "{metric} · group / model",
 
 
 
 
272
  "tools_grouped_bars_title": "Grouped bars",
273
  "tools_grouped_bars_hint": "FNR / FPR / F1 by group · averages across selected models",
274
  "tools_grouped_bars_aria": "Grouped FNR FPR F1 bars by group",
 
280
  "tools_pareto_x_axis": "p95 latency, ms (← lower better)",
281
  "tools_pareto_y_axis": "integral (higher better →)",
282
  "tools_pareto_fpr_legend": "teal (few false alarms) → red (many)",
283
+ "tools_pareto_click_pin_hint": "hover for details · dashed line = Pareto frontier",
284
  "tools_pareto_pin_model": "Pin {model} label",
285
  "tools_pareto_unpin_model": "Unpin {model} label",
286
  "tools_latency_title": "Latency performance",
287
+ "tools_latency_hint": "p50 / p95 / p99 (ms) · error rate",
288
+ "tools_latency_note": "Single-request latency percentiles.",
289
  "tools_robustness_title": "Robustness real → robust",
290
  "tools_robustness_hint": "degradation under augmentation - who breaks on harder attacks",
291
+ "tools_robustness_note": "Delta = robust - real. Delta score < 0: the model is not robust to obfuscations, > 0: robust. Delta FNR > 0: it misses harmful obfuscated messages more often (worse), < 0: it catches them more often (better). Delta FPR > 0: it blocks safe obfuscated messages more often (worse), < 0: it lets them pass more often (better).",
292
+ "tools_footer": "HiveTrace Guardrail Leaderboard",
293
  "bucket_data_unavailable_title": "Data is temporarily unavailable",
294
  "bucket_data_unavailable_description": "We could not load data for this page. Try refreshing the page again in a moment.",
295
  "bucket_data_stale_title": "Showing saved data",
messages/ru.json CHANGED
@@ -11,7 +11,7 @@
11
  "ranking_badge": "Бенчмарк guardrails",
12
  "ranking_updated": "обновлено {date}",
13
  "ranking_title": "Рейтинг",
14
- "ranking_description": "Рейтинг open-source guardrail-моделей. Выше Integral - лучше.",
15
  "ranking_filters": "Фильтры рейтинга",
16
  "ranking_table": "Таблица моделей",
17
  "ranking_scope_select_label": "срез данных",
@@ -55,16 +55,18 @@
55
  "ranking_rows": "{count} строк",
56
  "ranking_column_model": "Модель",
57
  "ranking_column_integral": "Integral",
58
- "ranking_column_weakest_link": "Слабое звено",
59
  "ranking_column_fpr": "Overall FPR",
60
  "ranking_column_fnr": "Overall FNR",
61
  "ranking_column_p95": "p95 мс",
62
  "ranking_column_size": "Размер",
 
 
63
  "ranking_column_report": "Отчёт",
64
  "ranking_model_type_classifier": "классификатор",
65
  "ranking_model_type_llm_judge": "LLM-судья",
66
- "ranking_eval_clean": "clean",
67
  "ranking_eval_leak": "eval-leak",
 
68
  "ranking_open_details_label": "Открыть детали модели {model}",
69
  "ranking_details_overview_tab": "Обзор",
70
  "ranking_details_groups_tab": "По группам",
@@ -130,39 +132,22 @@
130
  "methodology_title": "Методология",
131
  "methodology_description": "Датасет → группа → интегральный балл. FPR и FNR остаются раздельными до последнего шага.",
132
  "methodology_meaning_label": "Что это значит:",
133
- "methodology_example_label": "Пример",
134
  "methodology_aggregation_title": "Три уровня агрегации",
135
  "methodology_dataset_score_tag": "балл датасета",
136
  "methodology_dataset_score_comment": "гармоническое среднее (1−FPR, 1−FNR)",
137
  "methodology_dataset_score_description": "Нормализуем ошибки в качество: 1−FPR - доля верно пропущенного безопасного, 1−FNR - доля верно пойманного вредного. Затем берём гармоническое среднее: слабая сторона роняет балл, высокая другая не спасает.",
138
- "methodology_dataset_score_example": "<b>Qwen3Guard-Gen-8B на ToxicChat</b>: FPR 8.5%, FNR 6.3% → H(0.915, 0.937) = 0.926.",
139
  "methodology_group_score_tag": "балл группы",
140
  "methodology_group_score_comment": "гармоническое среднее баллов N датасетов",
141
  "methodology_group_score_description": "Группа - семейство близких датасетов, например jailbreak-тесты. Снова используем гармонику: самый слабый датасет определяет балл группы, нельзя добрать очки за счёт лёгких сетов внутри той же темы.",
142
- "methodology_group_score_example": "<b>Qwen3Guard-Gen-8B, группа XSTest</b>: FPR 12.8% / FNR 3.5% → S_group = 0.916.",
143
  "methodology_integral_score_tag": "итоговый балл",
144
  "methodology_integral_score_comment": "геометрическое среднее M групп (S_group > 0)",
145
  "methodology_integral_log_comment": "логарифмическая форма (численная стабильность)",
146
  "methodology_integral_score_description": "По группам берём геометрическое среднее. Оно сильнее наказывает дисбаланс: одна провальная группа тянет произведение вниз. Ровный профиль ценится выше звезды в одном месте и дыры в другом.",
147
- "methodology_integral_score_example": "<b>Qwen3Guard-Gen-8B (#1)</b>: геом. среднее по группам → S_integral = 0.918. Слабейшая группа 0.823 тянет итог вниз.",
148
- "methodology_helper_metrics_title": "Вспомогательные метрики",
149
- "methodology_fbeta_tag": "общая форма",
150
- "methodology_fbeta_description": "Один рычаг β балансирует precision и recall: β > 1 смещает к полноте, β < 1 - к точности.",
151
- "methodology_f2_tag": "β=2",
152
- "methodology_f2_description": "Recall важнее в 4 раза. F2 сильнее штрафует пропуск вредного и подходит для сетов, где цена пропуска выше цены ложной тревоги.",
153
- "methodology_f2_example": "<b>Qwen3Guard-Gen-8B (#1)</b>: recall R=0.862 (1−FNR), специфичность 0.910 (1−FPR) - реальные. Precision (TP/(TP+FP)) нет в данных лидерборда, число этой метрики не показываем: специфичность ≠ precision.",
154
- "methodology_f05_tag": "β=0.5",
155
- "methodology_f05_description": "Precision важнее в 4 раза. F0.5 сильнее штрафует ложные срабатывания и подходит для тестов на over-refusal.",
156
- "methodology_f05_example": "<b>Qwen3Guard-Gen-8B (#1)</b>: recall R=0.862 (1−FNR), специфичность 0.910 (1−FPR) - реальные. Precision (TP/(TP+FP)) нет в данных лидерборда, число этой метрики не показываем: специфичность ≠ precision.",
157
- "methodology_fdr_description": "Доля ложной тревоги среди блокировок: какова вероятность ошибки, если сработка произошла. Низкий FDR означает, что большинству помеченных объектов вердикт вынесен справедливо.",
158
- "methodology_fdr_example": "<b>Qwen3Guard-Gen-8B (#1)</b>: overall FPR 9.0%, FNR 13.8% - реальные. FDR требует precision, которого нет в данных, поэтому числовой FDR не приводим.",
159
  "methodology_combined_dataset_title": "Комбинированный датасет",
160
  "methodology_harm_split_tag": "harm split",
161
  "methodology_harm_split_description": "Доля вредоносных запросов, ошибочно помеченных безопасными. TP - пойманные атаки, FN - пропущенные.",
162
  "methodology_safe_split_tag": "safe split",
163
  "methodology_safe_split_description": "Доля безопасных запросов, ошибочно помеченных вредными. TN - верно пропущенные безопасные, FP - ложные срабатывания.",
164
- "methodology_micro_f1_tag": "combined",
165
- "methodology_micro_f1_description": "Считается через глобальные TP, FP и FN по всему набору. При равных весах классов в бинарной задаче micro-F1 совпадает с accuracy.",
166
  "methodology_combined_dataset_note": "Если в списке есть общий сет с пометкой combined, для него метрики берутся напрямую с полных сплитов; для остальных - стандартно, через арифметическое среднее их индивидуальных метрик.",
167
  "methodology_scatter_title": "Scatter: FPR vs FNR",
168
  "methodology_scatter_x_axis": "Ось X",
@@ -176,11 +161,6 @@
176
  "methodology_single_class_description": "F1 считается для всех бенчмарков <b>по умолчанию</b>. Если в датасете присутствуют метки только одного класса: для all_unsafe рассчитываются <b>FNR, recall</b>; для all_safe соответственно <b>FPR, TNR</b>.",
177
  "methodology_caveat_title": "Оговорка",
178
  "methodology_caveat_description": "Метрики - точечные оценки. Доверительные интервалы и размеры выборок (n) по тестам в текущем датасете не публикуются, поэтому небольшие различия в рейтинге могут быть статистически незначимыми.",
179
- "methodology_golden_rule_title": "Правило: никогда не сливайте FPR и FNR в один балл",
180
- "methodology_golden_rule_description": "FPR (ложная блокировка безопасного) и FNR (пропуск вредного) - это <b>две разные ошибки с разной ценой</b>. Усреднение их в одно число прячет провал: модель с FPR 0% и FNR 40% выглядит «средней», хотя пропускает почти половину атак. Поэтому обе ошибки держатся раздельно до уровня S_ds, а агрегация идёт <b>гармоническим/геометрическим</b> средним - оно проваливается вместе со слабой стороной, а не маскирует её.",
181
- "methodology_legend_higher_better": "выше - лучше (F1 · score · integral)",
182
- "methodology_legend_higher_worse": "выше - хуже (FPR · FNR · FDR)",
183
- "methodology_legend_glyphs": "▲ выше лучше · ▼ ниже лучше · ○ нет данных",
184
  "ranking_report_link_label": "Открыть {model}",
185
  "details": "Детали",
186
  "details_description": "Метрики по бенчмаркам и моделям.",
@@ -199,13 +179,10 @@
199
  "details_benchmark_search_placeholder": "Название бенчмарка...",
200
  "details_model_search_placeholder": "Поиск по названию...",
201
  "details_model_count": "{shown} из {total} моделей",
 
202
  "details_matrix_summary": "Сводная: {metric} по всем тестам - {models} моделей × {tests} тестов",
203
  "details_matrix_model_sort": "Модель · Integral",
204
- "details_matrix_good": " хорошо",
205
- "details_matrix_bad": "▼ плохо",
206
- "details_matrix_no_data": "○ нет данных",
207
- "details_matrix_direction_legend": "для FPR / FNR ниже = лучше · для F1 / recall / precision / accuracy выше = лучше",
208
- "details_matrix_sort_legend": "клик по столбцу - сортировка · клик по «Модель» - по Integral",
209
  "details_matrix_empty": "Ничего не найдено. Сбросьте поиск или включите группы.",
210
  "details_dataset_matrix_unavailable_title": "Матрица по датасетам временно недоступна",
211
  "details_dataset_matrix_unavailable_description": "Сейчас в данных есть dataset-level метрики для {covered} из {total} датасетов. Карточки групп, рейтинги и профили моделей ниже используют реальные данные; матрица будет включена после полного покрытия.",
@@ -219,15 +196,6 @@
219
  "details_family_stat_models": "моделей в оценке",
220
  "details_family_filters": "Фильтры семейств бенчмарков",
221
  "details_family_language_all": "Все языки",
222
- "details_family_metric_label": "Лидер карточки по метрике",
223
- "details_family_metric_f1": "F1 (лидер по F1)",
224
- "details_family_metric_fpr": "FPR (минимум ложных)",
225
- "details_family_metric_fnr": "FNR (минимум пропусков)",
226
- "details_family_metric_hint": "Определяет лидера и оценку на карточках.",
227
- "details_family_legend_good": "хорошо (F1↑ / FPR·FNR↓)",
228
- "details_family_legend_middle": "средне",
229
- "details_family_legend_bad": "плохо",
230
- "details_family_legend_direction": "▲ выше - лучше · ▼ ниже - лучше · ○ нет данных",
231
  "details_family_empty": "Ничего не найдено. Измените фильтры или сбросьте поиск.",
232
  "details_reset": "Сбросить",
233
  "details_benchmark_sheet_what_tests": "Что проверяет",
@@ -236,11 +204,6 @@
236
  "details_benchmark_sheet_rank": "Место",
237
  "details_benchmark_sheet_open": "Открыть бенчмарк",
238
  "details_model_sheet_open": "Открыть модель",
239
- "details_model_legend_lead": "Оценка 0-1: интегральный балл (выше = лучше)",
240
- "details_model_legend_good": "≥ 0.85 - хорошо",
241
- "details_model_legend_middle": "0.60-0.85",
242
- "details_model_legend_bad": "< 0.60 - слабо",
243
- "details_model_legend_hint": "F1 - выше лучше · p50 - медианная задержка, ниже лучше",
244
  "details_model_sheet_overview": "Обзор",
245
  "details_model_sheet_groups": "По группам",
246
  "details_model_sheet_weak": "Бенчмарки",
@@ -260,12 +223,11 @@
260
  "details_model_sheet_params": "Параметры",
261
  "details_model_sheet_type": "Тип guardrail",
262
  "details_model_sheet_languages": "Языки",
263
- "details_model_sheet_eval_leak": "Eval-leak",
264
  "details_model_sheet_run_date": "Дата прогона",
265
  "details_model_sheet_links": "Ссылки",
266
  "details_model_sheet_hf_card": "Карточка HF",
267
  "details_dataset_count": "{count} датасета",
268
- "details_leader": "Лидер",
269
  "details_no_data": "нет данных",
270
  "details_license_label": "license",
271
  "details_groups_label": "groups",
@@ -276,11 +238,7 @@
276
  "details_export": "Экспорт",
277
  "tools_meta_title": "Визуализации - HiveTrace Guard Leaderboard",
278
  "tools_meta_description": "Площадка визуализаций для исследователей: радар, scatter FPR×FNR, heatmap FNR, сгруппированные бары и Парето задержки.",
279
- "tools_description": "Фильтры ниже управляют всеми графиками синхронно. Клик по ячейке heatmap или точке scatter открывает детали среза; «Pin» закрепляет модель.",
280
- "tools_takeaway_integral": "Лидер по integral:",
281
- "tools_takeaway_p95": "быстрейший p95:",
282
- "tools_takeaway_fnr": "меньше пропусков вреда (FNR):",
283
- "tools_takeaway_empty": "Выберите модели, чтобы увидеть сводку лидеров.",
284
  "tools_controls_aria": "Панель управления визуализациями",
285
  "tools_models_title": "Модели",
286
  "tools_groups_title": "Группы / б��нчмарки",
@@ -300,7 +258,7 @@
300
  "tools_selected_summary": "{models} моделей · {groups} групп",
301
  "tools_pinned_for_comparison": "Закреплено для сравнения",
302
  "tools_unpin_model": "Открепить {model}",
303
- "tools_no_pinned": "- ничего не закреплено. Нажмите «Pin» у модели в списке «Модели» слева.",
304
  "tools_radar_title": "Радар групп",
305
  "tools_radar_hint": "баллы выбранных моделей по группам · выше = лучше",
306
  "tools_radar_aria": "Радар групп по выбранным моделям",
@@ -308,13 +266,9 @@
308
  "tools_scatter_hint": "x = FPR, y = FNR · ниже = лучше · ★ идеал (0,0)",
309
  "tools_scatter_aria": "Scatter FPR против FNR",
310
  "tools_scatter_ideal": "★ Идеал (0,0) - нулевые ошибки",
311
- "tools_scatter_click": "Клик по точке → промпты",
312
- "tools_heatmap_hint": "группы × модели · число всегда напечатано",
313
- "tools_heatmap_scale": "1.0 teal red",
314
- "tools_heatmap_header": "{metric} · группа ↓ / модель →",
315
- "tools_heatmap_note": "Цвет дублируется числом; «-» = нет данных.",
316
- "tools_heatmap_sheet_prompts_title": "Промпты этого среза",
317
- "tools_heatmap_sheet_prompts_description": "Выбрана метрика {metric} = {value}. В полной версии отсюда открывается просмотрщик отдельных промптов и предсказаний модели на этом наборе данных:",
318
  "tools_grouped_bars_title": "Сгруппированные бары",
319
  "tools_grouped_bars_hint": "FNR / FPR / F1 по группам · средние по выбранным моделям",
320
  "tools_grouped_bars_aria": "Сгруппированные бары FNR FPR F1 по группам",
@@ -326,16 +280,16 @@
326
  "tools_pareto_x_axis": "p95 latency, ms (← ниже лучше)",
327
  "tools_pareto_y_axis": "integral (выше лучше →)",
328
  "tools_pareto_fpr_legend": "teal (мало ложных) → red (много)",
329
- "tools_pareto_click_pin_hint": "Клик по точке — закрепить/снять подпись · hover — детали · пунктир = Pareto frontier",
330
  "tools_pareto_pin_model": "Закрепить подпись {model}",
331
  "tools_pareto_unpin_model": "Снять подпись {model}",
332
  "tools_latency_title": "Производительность latency",
333
- "tools_latency_hint": "p50 / p95 / p99 (ms) · ниже лучше",
334
- "tools_latency_note": "Перцентили задержки одного запроса. По умолчанию сортировка по p95 (возрастание).",
335
  "tools_robustness_title": "Устойчивость real → robust",
336
  "tools_robustness_hint": "падение под аугментацией - кто ломается на усложнённых атаках",
337
- "tools_robustness_note": "Δ score = robust real (отрицательный = деградация качества). Δ FNR = robust real (положительный = больше пропусков вреда под аугментацией, хуже).",
338
- "tools_footer": "HiveTrace Guardrail Leaderboard · Инструменты · реальные данные прогона (апрель 2026). Метрики: FPR/FNR/latency - ниже = лучше; F1/integral/score - выше = лучше.",
339
  "bucket_data_unavailable_title": "Данные временно недоступны",
340
  "bucket_data_unavailable_description": "Не удалось загрузить данные для этой страницы. Попробуйте обновить страницу чуть позже.",
341
  "bucket_data_stale_title": "Показаны сохраненные данные",
 
11
  "ranking_badge": "Бенчмарк guardrails",
12
  "ranking_updated": "обновлено {date}",
13
  "ranking_title": "Рейтинг",
14
+ "ranking_description": "Рейтинг open-source guardrail-моделей.",
15
  "ranking_filters": "Фильтры рейтинга",
16
  "ranking_table": "Таблица моделей",
17
  "ranking_scope_select_label": "срез данных",
 
55
  "ranking_rows": "{count} строк",
56
  "ranking_column_model": "Модель",
57
  "ranking_column_integral": "Integral",
58
+ "ranking_column_min_score": "Min score",
59
  "ranking_column_fpr": "Overall FPR",
60
  "ranking_column_fnr": "Overall FNR",
61
  "ranking_column_p95": "p95 мс",
62
  "ranking_column_size": "Размер",
63
+ "ranking_column_higher_better": "больше лучше",
64
+ "ranking_column_lower_better": "меньше лучше",
65
  "ranking_column_report": "Отчёт",
66
  "ranking_model_type_classifier": "классификатор",
67
  "ranking_model_type_llm_judge": "LLM-судья",
 
68
  "ranking_eval_leak": "eval-leak",
69
+ "ranking_eval_partial": "partial",
70
  "ranking_open_details_label": "Открыть детали модели {model}",
71
  "ranking_details_overview_tab": "Обзор",
72
  "ranking_details_groups_tab": "По группам",
 
132
  "methodology_title": "Методология",
133
  "methodology_description": "Датасет → группа → интегральный балл. FPR и FNR остаются раздельными до последнего шага.",
134
  "methodology_meaning_label": "Что это значит:",
 
135
  "methodology_aggregation_title": "Три уровня агрегации",
136
  "methodology_dataset_score_tag": "балл датасета",
137
  "methodology_dataset_score_comment": "гармоническое среднее (1−FPR, 1−FNR)",
138
  "methodology_dataset_score_description": "Нормализуем ошибки в качество: 1−FPR - доля верно пропущенного безопасного, 1−FNR - доля верно пойманного вредного. Затем берём гармоническое среднее: слабая сторона роняет балл, высокая другая не спасает.",
 
139
  "methodology_group_score_tag": "балл группы",
140
  "methodology_group_score_comment": "гармоническое среднее баллов N датасетов",
141
  "methodology_group_score_description": "Группа - семейство близких датасетов, например jailbreak-тесты. Снова используем гармонику: самый слабый датасет определяет балл группы, нельзя добрать очки за счёт лёгких сетов внутри той же темы.",
 
142
  "methodology_integral_score_tag": "итоговый балл",
143
  "methodology_integral_score_comment": "геометрическое среднее M групп (S_group > 0)",
144
  "methodology_integral_log_comment": "логарифмическая форма (численная стабильность)",
145
  "methodology_integral_score_description": "По группам берём геометрическое среднее. Оно сильнее наказывает дисбаланс: одна провальная группа тянет произведение вниз. Ровный профиль ценится выше звезды в одном месте и дыры в другом.",
 
 
 
 
 
 
 
 
 
 
 
 
146
  "methodology_combined_dataset_title": "Комбинированный датасет",
147
  "methodology_harm_split_tag": "harm split",
148
  "methodology_harm_split_description": "Доля вредоносных запросов, ошибочно помеченных безопасными. TP - пойманные атаки, FN - пропущенные.",
149
  "methodology_safe_split_tag": "safe split",
150
  "methodology_safe_split_description": "Доля безопасных запросов, ошибочно помеченных вредными. TN - верно пропущенные безопасные, FP - ложные срабатывания.",
 
 
151
  "methodology_combined_dataset_note": "Если в списке есть общий сет с пометкой combined, для него метрики берутся напрямую с полных сплитов; для остальных - стандартно, через арифметическое среднее их индивидуальных метрик.",
152
  "methodology_scatter_title": "Scatter: FPR vs FNR",
153
  "methodology_scatter_x_axis": "Ось X",
 
161
  "methodology_single_class_description": "F1 считается для всех бенчмарков <b>по умолчанию</b>. Если в датасете присутствуют метки только одного класса: для all_unsafe рассчитываются <b>FNR, recall</b>; для all_safe соответственно <b>FPR, TNR</b>.",
162
  "methodology_caveat_title": "Оговорка",
163
  "methodology_caveat_description": "Метрики - точечные оценки. Доверительные интервалы и размеры выборок (n) по тестам в текущем датасете не публикуются, поэтому небольшие различия в рейтинге могут быть статистически незначимыми.",
 
 
 
 
 
164
  "ranking_report_link_label": "Открыть {model}",
165
  "details": "Детали",
166
  "details_description": "Метрики по бенчмаркам и моделям.",
 
179
  "details_benchmark_search_placeholder": "Название бенчмарка...",
180
  "details_model_search_placeholder": "Поиск по названию...",
181
  "details_model_count": "{shown} из {total} моделей",
182
+ "details_error_rate": "Error rate",
183
  "details_matrix_summary": "Сводная: {metric} по всем тестам - {models} моделей × {tests} тестов",
184
  "details_matrix_model_sort": "Модель · Integral",
185
+ "details_matrix_no_data": "нет данных",
 
 
 
 
186
  "details_matrix_empty": "Ничего не найдено. Сбросьте поиск или включите группы.",
187
  "details_dataset_matrix_unavailable_title": "Матрица по датасетам временно недоступна",
188
  "details_dataset_matrix_unavailable_description": "Сейчас в данных есть dataset-level метрики для {covered} из {total} датасетов. Карточки групп, рейтинги и профили моделей ниже используют реальные данные; матрица будет включена после полного покрытия.",
 
196
  "details_family_stat_models": "моделей в оценке",
197
  "details_family_filters": "Фильтры семейств бенчмарков",
198
  "details_family_language_all": "Все языки",
 
 
 
 
 
 
 
 
 
199
  "details_family_empty": "Ничего не найдено. Измените фильтры или сбросьте поиск.",
200
  "details_reset": "Сбросить",
201
  "details_benchmark_sheet_what_tests": "Что проверяет",
 
204
  "details_benchmark_sheet_rank": "Место",
205
  "details_benchmark_sheet_open": "Открыть бенчмарк",
206
  "details_model_sheet_open": "Открыть модель",
 
 
 
 
 
207
  "details_model_sheet_overview": "Обзор",
208
  "details_model_sheet_groups": "По группам",
209
  "details_model_sheet_weak": "Бенчмарки",
 
223
  "details_model_sheet_params": "Параметры",
224
  "details_model_sheet_type": "Тип guardrail",
225
  "details_model_sheet_languages": "Языки",
226
+ "details_model_sheet_status": "Статус",
227
  "details_model_sheet_run_date": "Дата прогона",
228
  "details_model_sheet_links": "Ссылки",
229
  "details_model_sheet_hf_card": "Карточка HF",
230
  "details_dataset_count": "{count} датасета",
 
231
  "details_no_data": "нет данных",
232
  "details_license_label": "license",
233
  "details_groups_label": "groups",
 
238
  "details_export": "Экспорт",
239
  "tools_meta_title": "Визуализации - HiveTrace Guard Leaderboard",
240
  "tools_meta_description": "Площадка визуализаций для исследователей: радар, scatter FPR×FNR, heatmap FNR, сгруппированные бары и Парето задержки.",
241
+ "tools_description": "Фильтры ниже управляют всеми графиками синхронно; «Pin» закрепляет модель.",
 
 
 
 
242
  "tools_controls_aria": "Панель управления визуализациями",
243
  "tools_models_title": "Модели",
244
  "tools_groups_title": "Группы / б��нчмарки",
 
258
  "tools_selected_summary": "{models} моделей · {groups} групп",
259
  "tools_pinned_for_comparison": "Закреплено для сравнения",
260
  "tools_unpin_model": "Открепить {model}",
261
+ "tools_no_pinned": "Ничего не закреплено, нажмите Pin в списке Модели.",
262
  "tools_radar_title": "Радар групп",
263
  "tools_radar_hint": "баллы выбранных моделей по группам · выше = лучше",
264
  "tools_radar_aria": "Радар групп по выбранным моделям",
 
266
  "tools_scatter_hint": "x = FPR, y = FNR · ниже = лучше · ★ идеал (0,0)",
267
  "tools_scatter_aria": "Scatter FPR против FNR",
268
  "tools_scatter_ideal": "★ Идеал (0,0) - нулевые ошибки",
269
+ "tools_heatmap_scale_min": "min {value}",
270
+ "tools_heatmap_scale_max": "max {value}",
271
+ "tools_heatmap_header": "{metric} · группа / модель",
 
 
 
 
272
  "tools_grouped_bars_title": "Сгруппированные бары",
273
  "tools_grouped_bars_hint": "FNR / FPR / F1 по группам · средние по выбранным моделям",
274
  "tools_grouped_bars_aria": "Сгруппированные бары FNR FPR F1 по группам",
 
280
  "tools_pareto_x_axis": "p95 latency, ms (← ниже лучше)",
281
  "tools_pareto_y_axis": "integral (выше лучше →)",
282
  "tools_pareto_fpr_legend": "teal (мало ложных) → red (много)",
283
+ "tools_pareto_click_pin_hint": "hover — детали · пунктир = Pareto frontier",
284
  "tools_pareto_pin_model": "Закрепить подпись {model}",
285
  "tools_pareto_unpin_model": "Снять подпись {model}",
286
  "tools_latency_title": "Производительность latency",
287
+ "tools_latency_hint": "p50 / p95 / p99 (ms) · error rate",
288
+ "tools_latency_note": "Перцентили задержки одного запроса.",
289
  "tools_robustness_title": "Устойчивость real → robust",
290
  "tools_robustness_hint": "падение под аугментацией - кто ломается на усложнённых атаках",
291
+ "tools_robustness_note": "Delta = robust - real. Delta score < 0: модель неустойчива к обфускациям, > 0: устойчива. Delta FNR > 0: чаще пропускает вредоносные обфусцированные сообщения (хуже), < 0: чаще их ловит учше). Delta FPR > 0: чаще банит безопасные обфусцированные сообщения (хуже), < 0: чаще их пропускает (лучше).",
292
+ "tools_footer": "HiveTrace Guardrail Leaderboard",
293
  "bucket_data_unavailable_title": "Данные временно недоступны",
294
  "bucket_data_unavailable_description": "Не удалось загрузить данные для этой страницы. Попробуйте обновить страницу чуть позже.",
295
  "bucket_data_stale_title": "Показаны сохраненные данные",
src/lib/data/details.ts CHANGED
@@ -3,7 +3,6 @@ import type { DetailsDataset } from '$lib/types/details-data';
3
  import type {
4
  BenchmarkCard,
5
  BenchmarkFamilyLanguage,
6
- BenchmarkLeaderMetric,
7
  BenchmarkRankingRow,
8
  DetailMetric,
9
  DetailsLanguageFilter,
@@ -189,29 +188,9 @@ export function benchmarkRows(
189
  });
190
  }
191
 
192
- export function familyMetricValue(group: GroupScore | undefined, metric: BenchmarkLeaderMetric) {
193
- if (!group) return null;
194
- if (metric === 'f1') return group.f1;
195
- return group[metric];
196
- }
197
-
198
- function leaderForGroupMetric(
199
- group: string,
200
- metric: BenchmarkLeaderMetric,
201
- models: readonly GuardModel[]
202
- ) {
203
- const ranked = models
204
- .map((model) => ({ model, score: familyMetricValue(model.groups[group], metric) }))
205
- .filter((item): item is { model: GuardModel; score: number } => isNumber(item.score))
206
- .sort((a, b) => (metric === 'fpr' || metric === 'fnr' ? a.score - b.score : b.score - a.score));
207
-
208
- return ranked[0] ?? null;
209
- }
210
-
211
  export function buildBenchmarkCards(
212
  search: string,
213
  language: BenchmarkFamilyLanguage,
214
- metric: BenchmarkLeaderMetric,
215
  dataset: DetailsDataset,
216
  locale: SupportedLocale
217
  ) {
@@ -232,15 +211,10 @@ export function buildBenchmarkCards(
232
  .toLowerCase()
233
  .includes(normalizedSearch);
234
  })
235
- .map<BenchmarkCard>((group) => {
236
- const leader = leaderForGroupMetric(group, metric, dataset.models);
237
- return {
238
- name: group,
239
- meta: dataset.benchmarkMeta[group],
240
- leader,
241
- score: leader?.score ?? null
242
- };
243
- });
244
  }
245
 
246
  export function matchesLanguage(model: GuardModel, language: DetailsLanguageFilter) {
@@ -379,38 +353,3 @@ export function radarGridPolygons(groups: readonly ModelGroupRow[]) {
379
  .join(' ')
380
  );
381
  }
382
-
383
- export function detailMetricIsGood(value: number | null, metric: DetailMetric) {
384
- if (!isNumber(value)) return false;
385
- if (metric === 'fpr' || metric === 'fnr') return value <= 0.08;
386
- return value >= 0.85;
387
- }
388
-
389
- export function detailMetricIsBad(value: number | null, metric: DetailMetric) {
390
- if (!isNumber(value)) return false;
391
- if (metric === 'fpr' || metric === 'fnr') return value > 0.2;
392
- return value < 0.6;
393
- }
394
-
395
- export function detailMetricGlyph(value: number | null, metric: DetailMetric) {
396
- if (!isNumber(value)) return '○';
397
- if (detailMetricIsGood(value, metric)) return '▲';
398
- if (detailMetricIsBad(value, metric)) return '▼';
399
- return '-';
400
- }
401
-
402
- export function familyMetricIsGood(value: number | null, metric: BenchmarkLeaderMetric) {
403
- return metric === 'f1' ? isNumber(value) && value >= 0.85 : isNumber(value) && value <= 0.08;
404
- }
405
-
406
- export function familyMetricIsBad(value: number | null, metric: BenchmarkLeaderMetric) {
407
- return metric === 'f1' ? isNumber(value) && value < 0.6 : isNumber(value) && value > 0.2;
408
- }
409
-
410
- export function familyMetricGlyph(value: number | null, metric: BenchmarkLeaderMetric) {
411
- if (!isNumber(value)) return '○';
412
- if (metric === 'fpr' || metric === 'fnr') return '▼';
413
- if (value >= 0.85) return '▲';
414
- if (value < 0.6) return '▼';
415
- return '-';
416
- }
 
3
  import type {
4
  BenchmarkCard,
5
  BenchmarkFamilyLanguage,
 
6
  BenchmarkRankingRow,
7
  DetailMetric,
8
  DetailsLanguageFilter,
 
188
  });
189
  }
190
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
191
  export function buildBenchmarkCards(
192
  search: string,
193
  language: BenchmarkFamilyLanguage,
 
194
  dataset: DetailsDataset,
195
  locale: SupportedLocale
196
  ) {
 
211
  .toLowerCase()
212
  .includes(normalizedSearch);
213
  })
214
+ .map<BenchmarkCard>((group) => ({
215
+ name: group,
216
+ meta: dataset.benchmarkMeta[group]
217
+ }));
 
 
 
 
 
218
  }
219
 
220
  export function matchesLanguage(model: GuardModel, language: DetailsLanguageFilter) {
 
353
  .join(' ')
354
  );
355
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
src/lib/data/ranking.ts CHANGED
@@ -3,7 +3,6 @@ import type { RankingDataset } from '$lib/types/ranking-data';
3
  import type { BenchmarkScope, GuardModel } from '$lib/types/leaderboard';
4
  import type {
5
  BenchmarkDetail,
6
- ExtremaKey,
7
  LanguageFilter,
8
  LeaderboardRow,
9
  SortDirection,
@@ -120,16 +119,6 @@ export function compareRankingRows(
120
  return (Number(first) - Number(second)) * multiplier;
121
  }
122
 
123
- export function rankingExtrema(rows: readonly LeaderboardRow[], key: ExtremaKey, polarity: 1 | -1) {
124
- const values = rows.map((row) => row[key]).filter(isFiniteNumber);
125
- if (!values.length) return { best: null, worst: null };
126
-
127
- const min = Math.min(...values);
128
- const max = Math.max(...values);
129
-
130
- return polarity >= 0 ? { best: max, worst: min } : { best: min, worst: max };
131
- }
132
-
133
  export function benchmarkDetails(model: GuardModel, groups: readonly string[]): BenchmarkDetail[] {
134
  return groups
135
  .map((group) => ({
 
3
  import type { BenchmarkScope, GuardModel } from '$lib/types/leaderboard';
4
  import type {
5
  BenchmarkDetail,
 
6
  LanguageFilter,
7
  LeaderboardRow,
8
  SortDirection,
 
119
  return (Number(first) - Number(second)) * multiplier;
120
  }
121
 
 
 
 
 
 
 
 
 
 
 
122
  export function benchmarkDetails(model: GuardModel, groups: readonly string[]): BenchmarkDetail[] {
123
  return groups
124
  .map((group) => ({
src/lib/data/tools.ts CHANGED
@@ -1,15 +1,9 @@
1
  import { isNumber } from '$lib/helpers/leaderboard';
2
  import type { GroupScore, GuardModel } from '$lib/types/leaderboard';
3
  import type {
4
- ToolsHeatmapMetric,
5
- ToolsHeatmapMetricItem,
6
- ToolsHeatmapSelection,
7
  ToolsLanguageFilter,
8
- ToolsMetricTone,
9
- ToolsPoint,
10
  ToolsRadarScale,
11
  ToolsSortDirection,
12
- ToolsTopModelKey,
13
  ToolsVizMetric
14
  } from '$lib/types/tools';
15
 
@@ -77,95 +71,22 @@ export function toolsMetricValue(cell: GroupScore | null, key: ToolsVizMetric) {
77
  return cell[key];
78
  }
79
 
80
- export function toolsHeatmapMetricValue(
81
- metrics: ToolsHeatmapSelection['metrics'],
82
- key: ToolsHeatmapMetric
83
- ) {
84
- return metrics[key];
85
- }
86
-
87
- export function toolsHeatmapMetricItems(
88
- selection: ToolsHeatmapSelection
89
- ): ToolsHeatmapMetricItem[] {
90
- return [
91
- { label: 'FNR', key: 'fnr', value: selection.metrics.fnr },
92
- { label: 'FPR', key: 'fpr', value: selection.metrics.fpr },
93
- { label: 'F1', key: 'f1', value: selection.metrics.f1 },
94
- { label: 'SCORE', key: 'score', value: selection.metrics.score }
95
- ];
96
- }
97
-
98
  export function toolsRadarValue(cell: GroupScore | null, key: ToolsRadarScale) {
99
  if (!cell) return null;
100
  return cell[key];
101
  }
102
 
103
- export function toolsBestModel(
104
- list: readonly GuardModel[],
105
- key: ToolsTopModelKey,
106
- direction: ToolsSortDirection
107
  ) {
108
- const pool = list.filter((model) => isNumber(key === 'fnr' ? model.fnr : model[key]));
109
- if (!pool.length) return null;
110
-
111
- return pool.reduce((best, model) => {
112
- const bestValue = key === 'fnr' ? best.fnr : best[key];
113
- const modelValue = key === 'fnr' ? model.fnr : model[key];
114
- return direction === 'desc'
115
- ? modelValue > bestValue
116
- ? model
117
- : best
118
- : modelValue < bestValue
119
- ? model
120
- : best;
121
- });
122
- }
123
-
124
- export function toolsValueTone(
125
- value: number | null | undefined,
126
- higherBetter = true
127
- ): ToolsMetricTone {
128
- if (!isNumber(value)) return 'neutral';
129
- if (higherBetter) {
130
- if (value >= 0.85) return 'good';
131
- if (value <= 0.6) return 'bad';
132
- return 'neutral';
133
- }
134
- if (value <= 0.05) return 'good';
135
- if (value >= 0.15) return 'bad';
136
- return 'neutral';
137
- }
138
-
139
- export function toolsHeatmapMetricTone(
140
- value: number | null | undefined,
141
- key: ToolsHeatmapMetric
142
- ): ToolsMetricTone {
143
- if (!isNumber(value)) return 'neutral';
144
- if (key === 'fnr' || key === 'fpr') {
145
- if (value <= 0.08) return 'good';
146
- if (value >= 0.2) return 'bad';
147
- return 'neutral';
148
- }
149
- if (value >= 0.85) return 'good';
150
- if (value < 0.6) return 'bad';
151
- return 'neutral';
152
- }
153
-
154
- export function toolsHeatmapMetricArrow(value: number | null | undefined, key: ToolsHeatmapMetric) {
155
- const tone = toolsHeatmapMetricTone(value, key);
156
- if (tone === 'good') return '▲';
157
- if (tone === 'bad') return '▼';
158
- return '−';
159
- }
160
-
161
- export function toolsLatencySortValue(model: GuardModel, key: 'model' | 'p50' | 'p95' | 'p99') {
162
  if (key === 'model') return model.short.toLowerCase();
163
  return model[key];
164
  }
165
 
166
  export function toolsSortLatencyRows(
167
  rows: readonly GuardModel[],
168
- key: 'model' | 'p50' | 'p95' | 'p99',
169
  direction: ToolsSortDirection
170
  ) {
171
  return [...rows].sort((a, b) => {
@@ -174,14 +95,24 @@ export function toolsSortLatencyRows(
174
  const result =
175
  typeof aValue === 'string' && typeof bValue === 'string'
176
  ? aValue.localeCompare(bValue)
177
- : Number(aValue) - Number(bValue);
178
  return direction === 'asc' ? result : -result;
179
  });
180
  }
181
 
182
- export function toolsHeatColor(value: number | null | undefined) {
 
 
 
 
 
183
  if (!isNumber(value)) return 'transparent';
184
- const t = Math.max(0, Math.min(1, value));
 
 
 
 
 
185
  if (t < 0.5) {
186
  const p = t / 0.5;
187
  return `rgb(${Math.round(13 + 242 * p)}, ${Math.round(206 - 36 * p)}, ${Math.round(151 - 111 * p)})`;
@@ -275,21 +206,6 @@ export function toolsParetoPoint(model: GuardModel, xMax: number, y0: number) {
275
  return { x: toolsParetoX(model.p95, xMax), y: toolsParetoY(model.integral, y0) };
276
  }
277
 
278
- export function toolsParetoFront(modelsToPlot: readonly GuardModel[], xMax: number, y0: number) {
279
- let best = -Infinity;
280
- const points: ToolsPoint[] = [];
281
-
282
- for (const model of [...modelsToPlot].sort((a, b) => a.p95 - b.p95)) {
283
- if (model.integral > best) {
284
- best = model.integral;
285
- const point = toolsParetoPoint(model, xMax, y0);
286
- points.push({ x: point.x, y: point.y });
287
- }
288
- }
289
-
290
- return points.map((point) => `${point.x.toFixed(1)},${point.y.toFixed(1)}`).join(' ');
291
- }
292
-
293
  export function toolsGroupMean(
294
  group: string,
295
  key: ToolsVizMetric,
 
1
  import { isNumber } from '$lib/helpers/leaderboard';
2
  import type { GroupScore, GuardModel } from '$lib/types/leaderboard';
3
  import type {
 
 
 
4
  ToolsLanguageFilter,
 
 
5
  ToolsRadarScale,
6
  ToolsSortDirection,
 
7
  ToolsVizMetric
8
  } from '$lib/types/tools';
9
 
 
71
  return cell[key];
72
  }
73
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
  export function toolsRadarValue(cell: GroupScore | null, key: ToolsRadarScale) {
75
  if (!cell) return null;
76
  return cell[key];
77
  }
78
 
79
+ export function toolsLatencySortValue(
80
+ model: GuardModel,
81
+ key: 'model' | 'p50' | 'p95' | 'p99' | 'errorRate'
 
82
  ) {
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
83
  if (key === 'model') return model.short.toLowerCase();
84
  return model[key];
85
  }
86
 
87
  export function toolsSortLatencyRows(
88
  rows: readonly GuardModel[],
89
+ key: 'model' | 'p50' | 'p95' | 'p99' | 'errorRate',
90
  direction: ToolsSortDirection
91
  ) {
92
  return [...rows].sort((a, b) => {
 
95
  const result =
96
  typeof aValue === 'string' && typeof bValue === 'string'
97
  ? aValue.localeCompare(bValue)
98
+ : Number(aValue ?? Number.POSITIVE_INFINITY) - Number(bValue ?? Number.POSITIVE_INFINITY);
99
  return direction === 'asc' ? result : -result;
100
  });
101
  }
102
 
103
+ export function toolsHeatColor(
104
+ value: number | null | undefined,
105
+ min = 0,
106
+ max = 1,
107
+ curve: 'linear' | 'sqrt' = 'linear'
108
+ ) {
109
  if (!isNumber(value)) return 'transparent';
110
+ const range = max - min;
111
+ const normalized = range > 0 ? (value - min) / range : 0;
112
+ const t =
113
+ curve === 'sqrt'
114
+ ? Math.sqrt(Math.max(0, Math.min(1, normalized)))
115
+ : Math.max(0, Math.min(1, normalized));
116
  if (t < 0.5) {
117
  const p = t / 0.5;
118
  return `rgb(${Math.round(13 + 242 * p)}, ${Math.round(206 - 36 * p)}, ${Math.round(151 - 111 * p)})`;
 
206
  return { x: toolsParetoX(model.p95, xMax), y: toolsParetoY(model.integral, y0) };
207
  }
208
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
209
  export function toolsGroupMean(
210
  group: string,
211
  key: ToolsVizMetric,
src/lib/helpers/metrics.ts DELETED
@@ -1,45 +0,0 @@
1
- import type { MetricName, MetricState, MetricTrendPreference } from '$lib/types/metrics';
2
-
3
- function isNumber(value: unknown): value is number {
4
- return typeof value === 'number' && Number.isFinite(value);
5
- }
6
-
7
- export function metricState(metric: MetricName, value: number | null): MetricState {
8
- if (!isNumber(value)) return 'na';
9
-
10
- if (metric === 'fpr' || metric === 'fnr' || metric === 'latency') {
11
- if (value <= 0.05) return 'good';
12
- if (value >= 0.15) return 'bad';
13
- return 'neutral';
14
- }
15
-
16
- if (value >= 0.85) return 'good';
17
- if (value <= 0.6) return 'bad';
18
- return 'neutral';
19
- }
20
-
21
- export function scoreState(value: number | null): MetricState {
22
- if (!isNumber(value)) return 'na';
23
- if (value >= 0.85) return 'good';
24
- if (value >= 0.6) return 'neutral';
25
- return 'bad';
26
- }
27
-
28
- export function metricGlyph(metric: MetricName, value: number | null) {
29
- const state = metricState(metric, value);
30
- if (state === 'good') return '▲';
31
- if (state === 'bad') return '▼';
32
- if (state === 'na') return '○';
33
- return '';
34
- }
35
-
36
- export function valueTrend(value: number | null, positive: MetricTrendPreference) {
37
- if (!isNumber(value)) return '';
38
-
39
- const isGood = positive === 'high' ? value >= 0.85 : value <= 0.08;
40
- const isBad = positive === 'high' ? value < 0.6 : value > 0.2;
41
-
42
- if (isGood) return '▲';
43
- if (isBad) return '▼';
44
- return '';
45
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
src/lib/server/hf-bucket/adapt-details.ts CHANGED
@@ -59,7 +59,8 @@ export function adaptBucketDetails(snapshot: BucketDetailsSnapshot): DetailsData
59
  recall: cell.recall,
60
  precision: cell.precision,
61
  accuracy: cell.accuracy,
62
- sampleCount: cell.sample_count
 
63
  };
64
  }
65
 
 
59
  recall: cell.recall,
60
  precision: cell.precision,
61
  accuracy: cell.accuracy,
62
+ sampleCount: cell.sample_count,
63
+ metricsEvaluatedSamples: cell.metrics_evaluated_samples
64
  };
65
  }
66
 
src/lib/server/hf-bucket/adapt-ranking.ts CHANGED
@@ -1,10 +1,11 @@
1
  import type { RankingDataset } from '$lib/types/ranking-data';
2
  import type { GroupScore, GuardModel, GuardrailType } from '$lib/types/leaderboard';
3
  import type {
 
4
  BucketGroup,
5
  BucketModel,
6
- BucketRankingSnapshot,
7
- BucketLeaderboardRow
8
  } from './types';
9
 
10
  const ROBUST_GROUP_IDS = new Set([
@@ -85,7 +86,8 @@ function completeRankingRow(row: BucketLeaderboardRow): row is CompleteRankingRo
85
  function adaptModel(
86
  model: BucketModel,
87
  row: BucketLeaderboardRow,
88
- groupLabelById: Map<string, string>
 
89
  ): GuardModel | null {
90
  if (!completeRankingRow(row)) {
91
  console.warn(`[hf-bucket] Ranking skips incomplete model "${model.model_id}".`);
@@ -126,12 +128,30 @@ function adaptModel(
126
  p50: row.latency_ms.p50,
127
  p95: row.latency_ms.p95,
128
  p99: row.latency_ms.p99,
 
129
  leak: model.eval_leak ?? false,
 
130
  hfUrl: model.hf_url,
131
  groups
132
  };
133
  }
134
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
135
  function benchmarkMeta(groups: BucketGroup[], snapshot: BucketRankingSnapshot) {
136
  const datasetsById = new Map(
137
  snapshot.catalog.datasets.map((dataset) => [dataset.dataset_id, dataset])
@@ -153,16 +173,21 @@ function benchmarkMeta(groups: BucketGroup[], snapshot: BucketRankingSnapshot) {
153
  );
154
  }
155
 
156
- export function adaptBucketRanking(snapshot: BucketRankingSnapshot): RankingDataset {
 
 
157
  const groupLabelById = new Map(
158
  snapshot.catalog.groups.map((group) => [group.group_id, group.label.en])
159
  );
160
  const catalogById = chooseLatest(snapshot.catalog.models, 'catalog.models');
161
  const rowsById = chooseLatest(snapshot.leaderboard.rows, 'leaderboard.rows');
 
162
  const models = [...catalogById.values()]
163
  .map((model) => {
164
  const row = rowsById.get(model.model_id);
165
- return row ? { model: adaptModel(model, row, groupLabelById), rank: row.rank } : null;
 
 
166
  })
167
  .filter(
168
  (item): item is { model: GuardModel; rank: number } => item !== null && item.model !== null
 
1
  import type { RankingDataset } from '$lib/types/ranking-data';
2
  import type { GroupScore, GuardModel, GuardrailType } from '$lib/types/leaderboard';
3
  import type {
4
+ BucketDetailsSnapshot,
5
  BucketGroup,
6
  BucketModel,
7
+ BucketLeaderboardRow,
8
+ BucketRankingSnapshot
9
  } from './types';
10
 
11
  const ROBUST_GROUP_IDS = new Set([
 
86
  function adaptModel(
87
  model: BucketModel,
88
  row: BucketLeaderboardRow,
89
+ groupLabelById: Map<string, string>,
90
+ partialModelIds: ReadonlySet<string>
91
  ): GuardModel | null {
92
  if (!completeRankingRow(row)) {
93
  console.warn(`[hf-bucket] Ranking skips incomplete model "${model.model_id}".`);
 
128
  p50: row.latency_ms.p50,
129
  p95: row.latency_ms.p95,
130
  p99: row.latency_ms.p99,
131
+ errorRate: row.error_rate,
132
  leak: model.eval_leak ?? false,
133
+ partial: partialModelIds.has(model.model_id),
134
  hfUrl: model.hf_url,
135
  groups
136
  };
137
  }
138
 
139
+ function partialModelIds(snapshot: BucketRankingSnapshot | BucketDetailsSnapshot) {
140
+ if (!('detailsMatrix' in snapshot)) return new Set<string>();
141
+
142
+ const ids = new Set<string>();
143
+ for (const cell of snapshot.detailsMatrix.dataset_cells) {
144
+ if (
145
+ cell.metrics_evaluated_samples !== null &&
146
+ cell.sample_count !== null &&
147
+ cell.metrics_evaluated_samples < cell.sample_count
148
+ ) {
149
+ ids.add(cell.model_id);
150
+ }
151
+ }
152
+ return ids;
153
+ }
154
+
155
  function benchmarkMeta(groups: BucketGroup[], snapshot: BucketRankingSnapshot) {
156
  const datasetsById = new Map(
157
  snapshot.catalog.datasets.map((dataset) => [dataset.dataset_id, dataset])
 
173
  );
174
  }
175
 
176
+ export function adaptBucketRanking(
177
+ snapshot: BucketRankingSnapshot | BucketDetailsSnapshot
178
+ ): RankingDataset {
179
  const groupLabelById = new Map(
180
  snapshot.catalog.groups.map((group) => [group.group_id, group.label.en])
181
  );
182
  const catalogById = chooseLatest(snapshot.catalog.models, 'catalog.models');
183
  const rowsById = chooseLatest(snapshot.leaderboard.rows, 'leaderboard.rows');
184
+ const partialIds = partialModelIds(snapshot);
185
  const models = [...catalogById.values()]
186
  .map((model) => {
187
  const row = rowsById.get(model.model_id);
188
+ return row
189
+ ? { model: adaptModel(model, row, groupLabelById, partialIds), rank: row.rank }
190
+ : null;
191
  })
192
  .filter(
193
  (item): item is { model: GuardModel; rank: number } => item !== null && item.model !== null
src/lib/server/hf-bucket/adapt-tools.ts CHANGED
@@ -146,6 +146,7 @@ export function adaptBucketTools(snapshot: BucketVisualizationSnapshot): ToolsDa
146
  modelKey: modelKeyForId(row.model_id),
147
  rank: row.rank,
148
  latencyMs: latencyMs(row.latency_ms),
 
149
  integral: row.integral
150
  }))
151
  },
 
146
  modelKey: modelKeyForId(row.model_id),
147
  rank: row.rank,
148
  latencyMs: latencyMs(row.latency_ms),
149
+ errorRate: row.error_rate,
150
  integral: row.integral
151
  }))
152
  },
src/lib/server/hf-bucket/details-payload.ts CHANGED
@@ -37,6 +37,19 @@ function nullableNumber(value: unknown, field: string) {
37
  return value;
38
  }
39
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
  function parseMetrics(item: Record<string, unknown>, field: string) {
41
  return {
42
  score: nullableNumber(item.score, `${field}.score`),
@@ -89,17 +102,19 @@ function parseGroupCell(value: unknown, index: number): BucketGroupCell {
89
  function parseDatasetCell(value: unknown, index: number): BucketDatasetCell {
90
  const field = `details_matrix.dataset_cells[${index}]`;
91
  const item = record(value, field);
92
- const sampleCount = nullableNumber(item.sample_count, `${field}.sample_count`);
93
- if (sampleCount !== null && (!Number.isInteger(sampleCount) || sampleCount < 0)) {
94
- throw new Error(`Field "${field}.sample_count" must be a non-negative integer or null.`);
95
- }
 
96
 
97
  return {
98
  model_id: string(item.model_id, `${field}.model_id`),
99
  group_id: string(item.group_id, `${field}.group_id`),
100
  dataset_id: string(item.dataset_id, `${field}.dataset_id`),
101
  ...parseMetrics(item, field),
102
- sample_count: sampleCount
 
103
  };
104
  }
105
 
 
37
  return value;
38
  }
39
 
40
+ function nullableOptionalNumber(value: unknown, field: string) {
41
+ if (value === undefined) return null;
42
+ return nullableNumber(value, field);
43
+ }
44
+
45
+ function nullableNonNegativeInteger(value: unknown, field: string) {
46
+ const parsed = nullableOptionalNumber(value, field);
47
+ if (parsed !== null && (!Number.isInteger(parsed) || parsed < 0)) {
48
+ throw new Error(`Field "${field}" must be a non-negative integer or null.`);
49
+ }
50
+ return parsed;
51
+ }
52
+
53
  function parseMetrics(item: Record<string, unknown>, field: string) {
54
  return {
55
  score: nullableNumber(item.score, `${field}.score`),
 
102
  function parseDatasetCell(value: unknown, index: number): BucketDatasetCell {
103
  const field = `details_matrix.dataset_cells[${index}]`;
104
  const item = record(value, field);
105
+ const sampleCount = nullableNonNegativeInteger(item.sample_count, `${field}.sample_count`);
106
+ const metricsEvaluatedSamples = nullableNonNegativeInteger(
107
+ item.metrics_evaluated_samples,
108
+ `${field}.metrics_evaluated_samples`
109
+ );
110
 
111
  return {
112
  model_id: string(item.model_id, `${field}.model_id`),
113
  group_id: string(item.group_id, `${field}.group_id`),
114
  dataset_id: string(item.dataset_id, `${field}.dataset_id`),
115
  ...parseMetrics(item, field),
116
+ sample_count: sampleCount,
117
+ metrics_evaluated_samples: metricsEvaluatedSamples
118
  };
119
  }
120
 
src/lib/server/hf-bucket/ranking-cache.ts CHANGED
@@ -2,6 +2,7 @@ import { env } from '$env/dynamic/private';
2
  import type { BucketDataState } from '$lib/types/bucket-state';
3
  import type { RankingDataset } from '$lib/types/ranking-data';
4
  import { adaptBucketRanking } from './adapt-ranking';
 
5
  import { emptyRankingDataset } from './fallback-data';
6
  import { toPublicBucketError } from './public-error';
7
  import { fetchBucketRankingSnapshot } from './ranking-snapshot';
@@ -23,7 +24,7 @@ async function refreshRanking() {
23
  return cached.value;
24
  }
25
 
26
- const value = adaptBucketRanking(snapshot);
27
  cached = {
28
  value,
29
  snapshotId: snapshot.manifest.snapshot_id,
 
2
  import type { BucketDataState } from '$lib/types/bucket-state';
3
  import type { RankingDataset } from '$lib/types/ranking-data';
4
  import { adaptBucketRanking } from './adapt-ranking';
5
+ import { fetchBucketDetailsSnapshot } from './details-snapshot';
6
  import { emptyRankingDataset } from './fallback-data';
7
  import { toPublicBucketError } from './public-error';
8
  import { fetchBucketRankingSnapshot } from './ranking-snapshot';
 
24
  return cached.value;
25
  }
26
 
27
+ const value = adaptBucketRanking(await fetchBucketDetailsSnapshot(snapshot));
28
  cached = {
29
  value,
30
  snapshotId: snapshot.manifest.snapshot_id,
src/lib/server/hf-bucket/ranking-payload.ts CHANGED
@@ -198,6 +198,7 @@ function parseLeaderboardRow(value: unknown, index: number): BucketLeaderboardRo
198
  overall_fpr: nullableNumber(item.overall_fpr, `${field}.overall_fpr`),
199
  overall_fnr: nullableNumber(item.overall_fnr, `${field}.overall_fnr`),
200
  overall_f1: nullableNumber(item.overall_f1, `${field}.overall_f1`),
 
201
  latency_ms: {
202
  p50: nullableNumber(latency.p50, `${field}.latency_ms.p50`),
203
  p95: nullableNumber(latency.p95, `${field}.latency_ms.p95`),
 
198
  overall_fpr: nullableNumber(item.overall_fpr, `${field}.overall_fpr`),
199
  overall_fnr: nullableNumber(item.overall_fnr, `${field}.overall_fnr`),
200
  overall_f1: nullableNumber(item.overall_f1, `${field}.overall_f1`),
201
+ error_rate: nullableNumber(item.error_rate ?? null, `${field}.error_rate`),
202
  latency_ms: {
203
  p50: nullableNumber(latency.p50, `${field}.latency_ms.p50`),
204
  p95: nullableNumber(latency.p95, `${field}.latency_ms.p95`),
src/lib/server/hf-bucket/types.ts CHANGED
@@ -25,7 +25,16 @@ export type BucketManifestHashes = Record<BucketManifestFileKey, string> & Recor
25
  export type NullableMetricValue = number | null;
26
  export type BucketMetricDirection = 'higher' | 'lower';
27
  export type BucketMetricKey =
28
- 'integral' | 'score' | 'f1' | 'recall' | 'precision' | 'accuracy' | 'fpr' | 'fnr' | 'latency_ms';
 
 
 
 
 
 
 
 
 
29
 
30
  export type BucketLatencyMs = {
31
  p50: NullableMetricValue;
@@ -119,6 +128,7 @@ export type BucketLeaderboardRow = {
119
  overall_fpr: NullableMetricValue;
120
  overall_fnr: NullableMetricValue;
121
  overall_f1: NullableMetricValue;
 
122
  latency_ms: BucketLatencyMs;
123
  groups: Record<string, BucketGroupMetrics>;
124
  };
@@ -149,6 +159,7 @@ export type BucketGroupCell = {
149
  export type BucketDatasetCell = BucketGroupCell & {
150
  dataset_id: string;
151
  sample_count: NullableMetricValue;
 
152
  };
153
 
154
  export type BucketDetailsMatrix = {
@@ -253,6 +264,7 @@ export type BucketPerformanceRow = {
253
  model_id: string;
254
  rank: number;
255
  latency_ms: BucketLatencyMs;
 
256
  integral: NullableMetricValue;
257
  };
258
 
 
25
  export type NullableMetricValue = number | null;
26
  export type BucketMetricDirection = 'higher' | 'lower';
27
  export type BucketMetricKey =
28
+ | 'integral'
29
+ | 'score'
30
+ | 'f1'
31
+ | 'recall'
32
+ | 'precision'
33
+ | 'accuracy'
34
+ | 'fpr'
35
+ | 'fnr'
36
+ | 'latency_ms'
37
+ | 'error_rate';
38
 
39
  export type BucketLatencyMs = {
40
  p50: NullableMetricValue;
 
128
  overall_fpr: NullableMetricValue;
129
  overall_fnr: NullableMetricValue;
130
  overall_f1: NullableMetricValue;
131
+ error_rate: NullableMetricValue;
132
  latency_ms: BucketLatencyMs;
133
  groups: Record<string, BucketGroupMetrics>;
134
  };
 
159
  export type BucketDatasetCell = BucketGroupCell & {
160
  dataset_id: string;
161
  sample_count: NullableMetricValue;
162
+ metrics_evaluated_samples: NullableMetricValue;
163
  };
164
 
165
  export type BucketDetailsMatrix = {
 
264
  model_id: string;
265
  rank: number;
266
  latency_ms: BucketLatencyMs;
267
+ error_rate: NullableMetricValue;
268
  integral: NullableMetricValue;
269
  };
270
 
src/lib/server/hf-bucket/visualizations-payload.ts CHANGED
@@ -211,6 +211,7 @@ function parsePerformanceRow(value: unknown, index: number): BucketPerformanceRo
211
  model_id: string(item.model_id, `${field}.model_id`),
212
  rank: positiveInteger(item.rank, `${field}.rank`),
213
  latency_ms: latency(item.latency_ms, `${field}.latency_ms`),
 
214
  integral: nullableNumber(item.integral, `${field}.integral`)
215
  };
216
  }
 
211
  model_id: string(item.model_id, `${field}.model_id`),
212
  rank: positiveInteger(item.rank, `${field}.rank`),
213
  latency_ms: latency(item.latency_ms, `${field}.latency_ms`),
214
+ error_rate: nullableNumber(item.error_rate ?? null, `${field}.error_rate`),
215
  integral: nullableNumber(item.integral, `${field}.integral`)
216
  };
217
  }
src/lib/types/details-data.ts CHANGED
@@ -19,6 +19,7 @@ export type DetailsDatasetMatrixCell = {
19
  precision: number | null;
20
  accuracy: number | null;
21
  sampleCount: number | null;
 
22
  };
23
 
24
  export type DetailsDataset = RankingDataset & {
 
19
  precision: number | null;
20
  accuracy: number | null;
21
  sampleCount: number | null;
22
+ metricsEvaluatedSamples: number | null;
23
  };
24
 
25
  export type DetailsDataset = RankingDataset & {
src/lib/types/details.ts CHANGED
@@ -12,8 +12,6 @@ export type DetailsLanguageFilter = 'any' | 'en' | 'ru';
12
 
13
  export type BenchmarkFamilyLanguage = 'all' | 'en' | 'ru' | 'uk' | 'be' | 'uz';
14
 
15
- export type BenchmarkLeaderMetric = 'f1' | 'fpr' | 'fnr';
16
-
17
  export type ModelTypeFilter = 'all' | GuardModel['type'];
18
 
19
  export type SizeFilter = 'all' | 'small' | 'medium' | 'large';
@@ -55,8 +53,6 @@ export type BenchmarkRankingRow = {
55
  export type BenchmarkCard = {
56
  name: string;
57
  meta: RankingBenchmarkMeta;
58
- leader: { model: GuardModel; score: number } | null;
59
- score: number | null;
60
  };
61
 
62
  export type ModelGroupRow = {
 
12
 
13
  export type BenchmarkFamilyLanguage = 'all' | 'en' | 'ru' | 'uk' | 'be' | 'uz';
14
 
 
 
15
  export type ModelTypeFilter = 'all' | GuardModel['type'];
16
 
17
  export type SizeFilter = 'all' | 'small' | 'medium' | 'large';
 
53
  export type BenchmarkCard = {
54
  name: string;
55
  meta: RankingBenchmarkMeta;
 
 
56
  };
57
 
58
  export type ModelGroupRow = {
src/lib/types/leaderboard.ts CHANGED
@@ -33,7 +33,9 @@ export type GuardModel = {
33
  p50: number;
34
  p95: number;
35
  p99: number;
 
36
  leak: boolean;
 
37
  hfUrl?: string | null;
38
  groups: Record<string, GroupScore>;
39
  };
 
33
  p50: number;
34
  p95: number;
35
  p99: number;
36
+ errorRate: number | null;
37
  leak: boolean;
38
+ partial: boolean;
39
  hfUrl?: string | null;
40
  groups: Record<string, GroupScore>;
41
  };
src/lib/types/metrics.ts DELETED
@@ -1,5 +0,0 @@
1
- export type MetricName = 'score' | 'fpr' | 'fnr' | 'latency';
2
-
3
- export type MetricState = 'good' | 'bad' | 'neutral' | 'na';
4
-
5
- export type MetricTrendPreference = 'high' | 'low';
 
 
 
 
 
 
src/lib/types/ranking.ts CHANGED
@@ -7,11 +7,10 @@ export type SortKey = 'integral' | 'minGroup' | 'fpr' | 'fnr' | 'p95' | 'size';
7
 
8
  export type SortDirection = 'asc' | 'desc';
9
 
10
- export type ExtremaKey = 'fpr' | 'fnr' | 'p95';
11
-
12
  export type RankingColumn = {
13
  key: SortKey;
14
  label: Message;
 
15
  align?: 'left' | 'right';
16
  polarity: 1 | -1;
17
  };
 
7
 
8
  export type SortDirection = 'asc' | 'desc';
9
 
 
 
10
  export type RankingColumn = {
11
  key: SortKey;
12
  label: Message;
13
+ directionLabel?: Message;
14
  align?: 'left' | 'right';
15
  polarity: 1 | -1;
16
  };
src/lib/types/tools-data.ts CHANGED
@@ -90,6 +90,7 @@ export type ToolsPerformanceVisualization = {
90
  p95: number | null;
91
  p99: number | null;
92
  };
 
93
  integral: number | null;
94
  }>;
95
  };
 
90
  p95: number | null;
91
  p99: number | null;
92
  };
93
+ errorRate: number | null;
94
  integral: number | null;
95
  }>;
96
  };
src/lib/types/tools.ts CHANGED
@@ -1,23 +1,13 @@
1
- import type { GuardModel } from './leaderboard';
2
-
3
  export type ToolsLanguageFilter = 'all' | 'en' | 'ru' | 'uk' | 'be' | 'uz';
4
  export type ToolsVizMetric = 'fnr' | 'fpr' | 'f1';
5
  export type ToolsRadarScale = 'score' | 'f1';
6
  export type ToolsHeatmapMetric = ToolsVizMetric | 'score';
7
  export type ToolsSortDirection = 'asc' | 'desc';
8
- export type ToolsLatencySortKey = 'model' | 'p50' | 'p95' | 'p99';
9
- export type ToolsMetricTone = 'good' | 'bad' | 'neutral';
10
- export type ToolsTopModelKey = 'integral' | 'p95' | 'fnr';
11
-
12
- export type ToolsPoint = {
13
- x: number;
14
- y: number;
15
- };
16
 
17
  export type ToolsTooltipRow = {
18
  label: string;
19
  value: string;
20
- tone?: ToolsMetricTone;
21
  };
22
 
23
  export type ToolsChartTooltip = {
@@ -26,23 +16,3 @@ export type ToolsChartTooltip = {
26
  title: string;
27
  rows: ToolsTooltipRow[];
28
  };
29
-
30
- export type ToolsHeatmapSelection = {
31
- model: GuardModel;
32
- group: string;
33
- rowLabel: string;
34
- metric: ToolsHeatmapMetric;
35
- value: number | null;
36
- metrics: {
37
- score: number | null;
38
- f1: number | null;
39
- fpr: number | null;
40
- fnr: number | null;
41
- };
42
- };
43
-
44
- export type ToolsHeatmapMetricItem = {
45
- label: string;
46
- key: ToolsHeatmapMetric;
47
- value: number | null;
48
- };
 
 
 
1
  export type ToolsLanguageFilter = 'all' | 'en' | 'ru' | 'uk' | 'be' | 'uz';
2
  export type ToolsVizMetric = 'fnr' | 'fpr' | 'f1';
3
  export type ToolsRadarScale = 'score' | 'f1';
4
  export type ToolsHeatmapMetric = ToolsVizMetric | 'score';
5
  export type ToolsSortDirection = 'asc' | 'desc';
6
+ export type ToolsLatencySortKey = 'model' | 'p50' | 'p95' | 'p99' | 'errorRate';
 
 
 
 
 
 
 
7
 
8
  export type ToolsTooltipRow = {
9
  label: string;
10
  value: string;
 
11
  };
12
 
13
  export type ToolsChartTooltip = {
 
16
  title: string;
17
  rows: ToolsTooltipRow[];
18
  };
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
src/routes/+page.svelte CHANGED
@@ -18,16 +18,12 @@
18
  buildRankingRows,
19
  compareRankingRows,
20
  filterRankingGroups,
21
- rankingExtrema,
22
  rankingRowsToCsv,
23
  rankingScopeGroups
24
  } from '$lib/data/ranking';
25
- import { metricGlyph, metricState, scoreState, valueTrend } from '$lib/helpers/metrics';
26
  import type { LocalizedString, Message, SelectOption } from '$lib/types/i18n';
27
  import type { BenchmarkScope, GuardModel } from '$lib/types/leaderboard';
28
- import type { MetricName } from '$lib/types/metrics';
29
  import type {
30
- ExtremaKey,
31
  LanguageFilter,
32
  LeaderboardRow,
33
  RankingColumn,
@@ -39,6 +35,8 @@
39
  type LeaderboardTableColumn = TableColumn<SortKey | 'rank' | 'model'> & {
40
  metric?: RankingColumn;
41
  };
 
 
42
 
43
  let { data }: PageProps = $props();
44
 
@@ -68,11 +66,41 @@
68
  ];
69
 
70
  const columns: RankingColumn[] = [
71
- { key: 'integral', label: m.ranking_column_integral, align: 'right', polarity: 1 },
72
- { key: 'minGroup', label: m.ranking_column_weakest_link, align: 'right', polarity: 1 },
73
- { key: 'fpr', label: m.ranking_column_fpr, align: 'right', polarity: -1 },
74
- { key: 'fnr', label: m.ranking_column_fnr, align: 'right', polarity: -1 },
75
- { key: 'p95', label: m.ranking_column_p95, align: 'right', polarity: -1 },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76
  { key: 'size', label: m.ranking_column_size, align: 'right', polarity: -1 }
77
  ];
78
 
@@ -113,19 +141,19 @@
113
  {
114
  key: 'rank',
115
  label: '#',
116
- headClass: 'w-12 text-right',
117
  cellClass: 'font-mono text-right text-muted-foreground'
118
  },
119
  {
120
  key: 'model',
121
  label: m.ranking_column_model,
122
- headClass: 'min-w-[300px]'
123
  },
124
  ...columns.map((column) => ({
125
  key: column.key,
126
  label: column.label,
127
  metric: column,
128
- headClass: cn(column.align === 'right' && 'text-right'),
129
  cellClass: cn(column.align === 'right' && 'text-right'),
130
  ariaSort: () =>
131
  sortKey === column.key ? (sortDirection === 'asc' ? 'ascending' : 'descending') : 'none'
@@ -201,11 +229,19 @@
201
  const selectedBenchmarkDetails = $derived.by(() =>
202
  selectedRow ? benchmarkDetails(selectedRow.model, activeGroups) : []
203
  );
204
- const metricExtrema = $derived.by(() => ({
205
- fpr: rankingExtrema(sortedRows, 'fpr', -1),
206
- fnr: rankingExtrema(sortedRows, 'fnr', -1),
207
- p95: rankingExtrema(sortedRows, 'p95', -1)
208
- }));
 
 
 
 
 
 
 
 
209
 
210
  function selectedOption<T extends string>(options: SelectOption<T>[], value: T) {
211
  return options.find((option) => option.value === value) ?? options[0];
@@ -320,49 +356,54 @@
320
  return isNumber(value) ? `${(value * 100).toFixed(1)}%` : '-';
321
  }
322
 
323
- function extremaTone(key: ExtremaKey, value: number | null) {
324
- if (!isNumber(value)) return '';
325
-
326
- const extremaForKey = metricExtrema[key];
327
- if (value === extremaForKey.best)
328
- return 'ring-1 ring-inset ring-emerald-500/50 bg-emerald-500/10 text-emerald-700 dark:text-emerald-300';
329
- if (value === extremaForKey.worst)
330
- return 'ring-1 ring-inset ring-red-500/50 bg-red-500/10 text-red-700 dark:text-red-300';
331
 
332
- return '';
 
 
 
 
 
 
 
 
 
333
  }
334
 
335
- function metricTone(metric: MetricName, value: number | null) {
336
- const state = metricState(metric, value);
337
- if (state === 'good') return 'bg-emerald-500/10 text-emerald-700 dark:text-emerald-300';
338
- if (state === 'bad') return 'bg-red-500/10 text-red-700 dark:text-red-300';
339
- if (state === 'na') return 'text-muted-foreground';
 
340
  return 'text-foreground';
341
  }
342
 
343
- function metricCellClass(metric: MetricName, value: number | null, extremaKey?: ExtremaKey) {
344
  return cn(
345
  'inline-flex min-w-16 items-center justify-center gap-1 rounded-sm px-1.5 py-0.5 font-mono text-xs tabular-nums',
346
- metricTone(metric, value),
347
- extremaKey && extremaTone(extremaKey, value)
348
  );
349
  }
350
 
351
- function overviewMetricClass(metric: MetricName, value: number | null) {
352
  return cn(
353
  'inline-flex w-fit items-center gap-1 whitespace-nowrap rounded-sm px-1.5 py-0.5 font-mono text-xl font-semibold tracking-normal tabular-nums',
354
- metricTone(metric, value)
355
  );
356
  }
357
 
358
- function scoreTone(value: number | null) {
359
- const state = scoreState(value);
360
- if (state === 'na') return 'border-border bg-muted text-muted-foreground';
361
- if (state === 'good')
362
- return 'border-emerald-500/30 bg-emerald-500/10 text-emerald-700 dark:text-emerald-300';
363
- if (state === 'neutral')
364
- return 'border-amber-500/30 bg-amber-500/10 text-amber-700 dark:text-amber-300';
365
- return 'border-red-500/30 bg-red-500/10 text-red-700 dark:text-red-300';
366
  }
367
 
368
  function modelDescription(model: GuardModel) {
@@ -382,25 +423,17 @@
382
  return row.model.p50.toFixed(1);
383
  }
384
 
385
- function qualityTone(value: number | null) {
386
- const state = scoreState(value);
387
- if (state === 'na') return 'text-muted-foreground';
388
- if (state === 'good') return 'bg-emerald-500/10 text-emerald-700 dark:text-emerald-300';
389
- if (state === 'neutral') return 'bg-amber-500/10 text-amber-700 dark:text-amber-300';
390
- return 'bg-red-500/10 text-red-700 dark:text-red-300';
391
- }
392
-
393
  function detailMetricClass(value: number | null) {
394
  return cn(
395
  'inline-flex min-w-16 items-center justify-center gap-1 whitespace-nowrap rounded-sm px-2 py-0.5 font-mono tabular-nums',
396
- qualityTone(value)
397
  );
398
  }
399
 
400
  function benchmarkScoreClass(value: number | null) {
401
  return cn(
402
  'inline-flex shrink-0 items-center justify-center gap-1 whitespace-nowrap rounded-sm px-2 py-0.5 font-mono text-sm font-semibold tabular-nums',
403
- qualityTone(value)
404
  );
405
  }
406
  </script>
@@ -555,13 +588,25 @@
555
  <button
556
  type="button"
557
  class={cn(
558
- 'inline-flex w-full items-center gap-1.5 text-sm font-medium text-inherit transition-colors hover:text-inherit',
559
- metric.align === 'right' && 'justify-end'
560
  )}
561
  onclick={() => updateSort(metric.key)}
562
  >
563
- {metric.label()}
564
- <ArrowUpDownIcon class="size-3.5" />
 
 
 
 
 
 
 
 
 
 
 
 
565
  </button>
566
  {/if}
567
  {/snippet}
@@ -587,16 +632,14 @@
587
  <span aria-hidden="true">△</span>
588
  {m.ranking_eval_leak()}
589
  </Badge>
590
- {:else}
 
591
  <Badge
592
  variant="outline"
593
- class="h-5 gap-1 border-[#0dce97]/50 bg-[#0dce97]/10 px-2 font-mono text-xs font-semibold text-[#0dce97]"
594
  >
595
- <span
596
- class="size-1.5 rounded-full bg-[#0dce97] shadow-[0_0_0_2px_rgba(13,206,151,0.18)]"
597
- aria-hidden="true"
598
- ></span>
599
- {m.ranking_eval_clean()}
600
  </Badge>
601
  {/if}
602
  </div>
@@ -607,39 +650,24 @@
607
  {@const data = row}
608
  {#if column.key === 'integral'}
609
  <span
610
- class={cn(
611
- 'inline-flex rounded-full border px-2 py-1 font-mono text-xs',
612
- scoreTone(data.integral)
613
- )}
614
  >
615
  {fmtScore(data.integral)}
616
  </span>
617
  {:else if column.key === 'minGroup'}
618
- <span class={metricCellClass('score', data.minGroup)}>
619
- {#if metricGlyph('score', data.minGroup)}
620
- <span aria-hidden="true">{metricGlyph('score', data.minGroup)}</span>
621
- {/if}
622
  {fmtScore(data.minGroup)}
623
  </span>
624
  {:else if column.key === 'fpr'}
625
- <span class={metricCellClass('fpr', data.fpr, 'fpr')}>
626
- {#if metricGlyph('fpr', data.fpr)}
627
- <span aria-hidden="true">{metricGlyph('fpr', data.fpr)}</span>
628
- {/if}
629
  {fmtPercent(data.fpr)}
630
  </span>
631
  {:else if column.key === 'fnr'}
632
- <span class={metricCellClass('fnr', data.fnr, 'fnr')}>
633
- {#if metricGlyph('fnr', data.fnr)}
634
- <span aria-hidden="true">{metricGlyph('fnr', data.fnr)}</span>
635
- {/if}
636
  {fmtPercent(data.fnr)}
637
  </span>
638
  {:else if column.key === 'p95'}
639
- <span class={metricCellClass('latency', data.p95, 'p95')}>
640
- {#if metricGlyph('latency', data.p95)}
641
- <span aria-hidden="true">{metricGlyph('latency', data.p95)}</span>
642
- {/if}
643
  {data.p95.toFixed(1)}
644
  </span>
645
  {:else if column.key === 'size'}
@@ -654,6 +682,7 @@
654
  getRowKey={(row) => modelFilterValue(row.model)}
655
  rowLabel={(row) => m.ranking_open_details_label({ model: row.model.short })}
656
  rowClass="cursor-pointer focus-visible:bg-muted/60 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 focus-visible:ring-offset-background"
 
657
  onrowclick={(row) => openModelDetails(row)}
658
  >
659
  {#snippet header(column)}
@@ -691,7 +720,7 @@
691
  <span
692
  class={cn(
693
  'inline-flex rounded-full border px-2.5 py-1 font-mono text-base font-semibold',
694
- scoreTone(row.integral)
695
  )}
696
  >
697
  {fmtScore(row.integral)}
@@ -701,12 +730,9 @@
701
  <p
702
  class="font-mono text-xs font-semibold uppercase tracking-wider text-muted-foreground"
703
  >
704
- {m.ranking_column_weakest_link()}
705
  </p>
706
- <span class={overviewMetricClass('score', row.minGroup)}>
707
- {#if metricGlyph('score', row.minGroup)}
708
- <span aria-hidden="true">{metricGlyph('score', row.minGroup)}</span>
709
- {/if}
710
  {fmtScore(row.minGroup)}
711
  </span>
712
  </div>
@@ -716,10 +742,7 @@
716
  >
717
  {m.ranking_column_fpr()}
718
  </p>
719
- <span class={overviewMetricClass('fpr', row.fpr)}>
720
- {#if metricGlyph('fpr', row.fpr)}
721
- <span aria-hidden="true">{metricGlyph('fpr', row.fpr)}</span>
722
- {/if}
723
  {fmtPercent(row.fpr)}
724
  </span>
725
  </div>
@@ -729,10 +752,7 @@
729
  >
730
  {m.ranking_column_fnr()}
731
  </p>
732
- <span class={overviewMetricClass('fnr', row.fnr)}>
733
- {#if metricGlyph('fnr', row.fnr)}
734
- <span aria-hidden="true">{metricGlyph('fnr', row.fnr)}</span>
735
- {/if}
736
  {fmtPercent(row.fnr)}
737
  </span>
738
  </div>
@@ -792,21 +812,12 @@
792
  {/if}
793
  </div>
794
  <span class={detailMetricClass(benchmark?.score ?? null)}>
795
- {#if valueTrend(benchmark?.score ?? null, 'high')}
796
- {valueTrend(benchmark?.score ?? null, 'high')}
797
- {/if}
798
  {fmtScore(benchmark?.score ?? null)}
799
  </span>
800
  <span class={detailMetricClass(benchmark?.fpr ?? null)}>
801
- {#if valueTrend(benchmark?.fpr ?? null, 'low')}
802
- {valueTrend(benchmark?.fpr ?? null, 'low')}
803
- {/if}
804
  {fmtPercent(benchmark?.fpr ?? null)}
805
  </span>
806
  <span class={detailMetricClass(benchmark?.fnr ?? null)}>
807
- {#if valueTrend(benchmark?.fnr ?? null, 'low')}
808
- {valueTrend(benchmark?.fnr ?? null, 'low')}
809
- {/if}
810
  {fmtPercent(benchmark?.fnr ?? null)}
811
  </span>
812
  <span class="justify-self-center font-mono tabular-nums">
@@ -829,9 +840,6 @@
829
  {groupLabel(benchmark.name)}
830
  </h3>
831
  <span class={benchmarkScoreClass(benchmark.score)}>
832
- {#if valueTrend(benchmark.score, 'high')}
833
- {valueTrend(benchmark.score, 'high')}
834
- {/if}
835
  {fmtScore(benchmark.score)}
836
  </span>
837
  </div>
@@ -915,16 +923,14 @@
915
  <span aria-hidden="true">△</span>
916
  {m.ranking_eval_leak()}
917
  </Badge>
918
- {:else}
 
919
  <Badge
920
  variant="outline"
921
- class="h-5 gap-1 rounded-full border-[#0dce97]/50 bg-[#0dce97]/10 px-2 font-mono text-[0.6875rem] font-semibold text-[#0dce97]"
922
  >
923
- <span
924
- class="size-1 rounded-full bg-[#0dce97] shadow-[0_0_0_2px_rgba(13,206,151,0.18)]"
925
- aria-hidden="true"
926
- ></span>
927
- {m.ranking_eval_clean()}
928
  </Badge>
929
  {/if}
930
  </div>
 
18
  buildRankingRows,
19
  compareRankingRows,
20
  filterRankingGroups,
 
21
  rankingRowsToCsv,
22
  rankingScopeGroups
23
  } from '$lib/data/ranking';
 
24
  import type { LocalizedString, Message, SelectOption } from '$lib/types/i18n';
25
  import type { BenchmarkScope, GuardModel } from '$lib/types/leaderboard';
 
26
  import type {
 
27
  LanguageFilter,
28
  LeaderboardRow,
29
  RankingColumn,
 
35
  type LeaderboardTableColumn = TableColumn<SortKey | 'rank' | 'model'> & {
36
  metric?: RankingColumn;
37
  };
38
+ type ColoredRankingKey = Exclude<SortKey, 'size'>;
39
+ type ColumnExtrema = { best: number | null; worst: number | null };
40
 
41
  let { data }: PageProps = $props();
42
 
 
66
  ];
67
 
68
  const columns: RankingColumn[] = [
69
+ {
70
+ key: 'integral',
71
+ label: m.ranking_column_integral,
72
+ directionLabel: m.ranking_column_higher_better,
73
+ align: 'right',
74
+ polarity: 1
75
+ },
76
+ {
77
+ key: 'minGroup',
78
+ label: m.ranking_column_min_score,
79
+ directionLabel: m.ranking_column_higher_better,
80
+ align: 'right',
81
+ polarity: 1
82
+ },
83
+ {
84
+ key: 'fpr',
85
+ label: m.ranking_column_fpr,
86
+ directionLabel: m.ranking_column_lower_better,
87
+ align: 'right',
88
+ polarity: -1
89
+ },
90
+ {
91
+ key: 'fnr',
92
+ label: m.ranking_column_fnr,
93
+ directionLabel: m.ranking_column_lower_better,
94
+ align: 'right',
95
+ polarity: -1
96
+ },
97
+ {
98
+ key: 'p95',
99
+ label: m.ranking_column_p95,
100
+ directionLabel: m.ranking_column_lower_better,
101
+ align: 'right',
102
+ polarity: -1
103
+ },
104
  { key: 'size', label: m.ranking_column_size, align: 'right', polarity: -1 }
105
  ];
106
 
 
141
  {
142
  key: 'rank',
143
  label: '#',
144
+ headClass: 'w-12 align-middle text-right',
145
  cellClass: 'font-mono text-right text-muted-foreground'
146
  },
147
  {
148
  key: 'model',
149
  label: m.ranking_column_model,
150
+ headClass: 'min-w-[300px] align-middle'
151
  },
152
  ...columns.map((column) => ({
153
  key: column.key,
154
  label: column.label,
155
  metric: column,
156
+ headClass: cn('align-top', column.align === 'right' && 'text-right'),
157
  cellClass: cn(column.align === 'right' && 'text-right'),
158
  ariaSort: () =>
159
  sortKey === column.key ? (sortDirection === 'asc' ? 'ascending' : 'descending') : 'none'
 
229
  const selectedBenchmarkDetails = $derived.by(() =>
230
  selectedRow ? benchmarkDetails(selectedRow.model, activeGroups) : []
231
  );
232
+ const metricExtrema = $derived.by(
233
+ () =>
234
+ Object.fromEntries(
235
+ columns
236
+ .filter(
237
+ (column): column is RankingColumn & { key: ColoredRankingKey } => column.key !== 'size'
238
+ )
239
+ .map((column) => [
240
+ column.key,
241
+ rankingColumnExtrema(sortedRows, column.key, column.polarity)
242
+ ])
243
+ ) as Record<ColoredRankingKey, ColumnExtrema>
244
+ );
245
 
246
  function selectedOption<T extends string>(options: SelectOption<T>[], value: T) {
247
  return options.find((option) => option.value === value) ?? options[0];
 
356
  return isNumber(value) ? `${(value * 100).toFixed(1)}%` : '-';
357
  }
358
 
359
+ function roundedRankingValue(row: LeaderboardRow, key: ColoredRankingKey) {
360
+ if (key === 'integral') return Number(row.integral.toFixed(3));
361
+ if (key === 'minGroup') return Number(row.minGroup.toFixed(3));
362
+ if (key === 'p95') return Number(row.p95.toFixed(1));
363
+ const value = row[key];
364
+ return isNumber(value) ? Number((value * 100).toFixed(1)) : null;
365
+ }
 
366
 
367
+ function rankingColumnExtrema(
368
+ visibleRows: readonly LeaderboardRow[],
369
+ key: ColoredRankingKey,
370
+ polarity: 1 | -1
371
+ ): ColumnExtrema {
372
+ const values = visibleRows.map((row) => roundedRankingValue(row, key)).filter(isNumber);
373
+ if (!values.length) return { best: null, worst: null };
374
+ const min = Math.min(...values);
375
+ const max = Math.max(...values);
376
+ return polarity >= 0 ? { best: max, worst: min } : { best: min, worst: max };
377
  }
378
 
379
+ function extremaTone(value: number | null, extrema: ColumnExtrema) {
380
+ if (!isNumber(value)) return 'text-muted-foreground';
381
+ if (value === extrema.best)
382
+ return 'ring-1 ring-inset ring-emerald-500/50 bg-emerald-500/10 text-emerald-700 dark:text-emerald-300';
383
+ if (value === extrema.worst)
384
+ return 'ring-1 ring-inset ring-red-500/50 bg-red-500/10 text-red-700 dark:text-red-300';
385
  return 'text-foreground';
386
  }
387
 
388
+ function rankingMetricClass(row: LeaderboardRow, key: ColoredRankingKey) {
389
  return cn(
390
  'inline-flex min-w-16 items-center justify-center gap-1 rounded-sm px-1.5 py-0.5 font-mono text-xs tabular-nums',
391
+ extremaTone(roundedRankingValue(row, key), metricExtrema[key])
 
392
  );
393
  }
394
 
395
+ function overviewMetricClass(row: LeaderboardRow, key: ColoredRankingKey) {
396
  return cn(
397
  'inline-flex w-fit items-center gap-1 whitespace-nowrap rounded-sm px-1.5 py-0.5 font-mono text-xl font-semibold tracking-normal tabular-nums',
398
+ extremaTone(roundedRankingValue(row, key), metricExtrema[key])
399
  );
400
  }
401
 
402
+ function scoreTone(row: LeaderboardRow) {
403
+ return cn(
404
+ 'border-transparent',
405
+ extremaTone(roundedRankingValue(row, 'integral'), metricExtrema.integral)
406
+ );
 
 
 
407
  }
408
 
409
  function modelDescription(model: GuardModel) {
 
423
  return row.model.p50.toFixed(1);
424
  }
425
 
 
 
 
 
 
 
 
 
426
  function detailMetricClass(value: number | null) {
427
  return cn(
428
  'inline-flex min-w-16 items-center justify-center gap-1 whitespace-nowrap rounded-sm px-2 py-0.5 font-mono tabular-nums',
429
+ !isNumber(value) ? 'text-muted-foreground' : 'text-foreground'
430
  );
431
  }
432
 
433
  function benchmarkScoreClass(value: number | null) {
434
  return cn(
435
  'inline-flex shrink-0 items-center justify-center gap-1 whitespace-nowrap rounded-sm px-2 py-0.5 font-mono text-sm font-semibold tabular-nums',
436
+ !isNumber(value) ? 'text-muted-foreground' : 'text-foreground'
437
  );
438
  }
439
  </script>
 
588
  <button
589
  type="button"
590
  class={cn(
591
+ 'inline-flex w-full justify-start flex-col gap-0.5 text-sm font-medium text-inherit transition-colors hover:text-inherit',
592
+ metric.align === 'right' ? 'items-end text-right' : 'items-start text-left'
593
  )}
594
  onclick={() => updateSort(metric.key)}
595
  >
596
+ <span
597
+ class={cn(
598
+ 'inline-flex items-center gap-1.5',
599
+ metric.align === 'right' && 'justify-end'
600
+ )}
601
+ >
602
+ {metric.label()}
603
+ <ArrowUpDownIcon class="size-3.5" />
604
+ </span>
605
+ {#if metric.directionLabel}
606
+ <span class="text-[10px] leading-3 font-light text-muted-foreground normal-case">
607
+ {metric.directionLabel()}
608
+ </span>
609
+ {/if}
610
  </button>
611
  {/if}
612
  {/snippet}
 
632
  <span aria-hidden="true">△</span>
633
  {m.ranking_eval_leak()}
634
  </Badge>
635
+ {/if}
636
+ {#if row.model.partial}
637
  <Badge
638
  variant="outline"
639
+ class="h-5 gap-1 border-sky-500/50 bg-sky-500/10 px-2 font-mono text-xs font-semibold text-sky-600 dark:text-sky-300"
640
  >
641
+ <span aria-hidden="true">◐</span>
642
+ {m.ranking_eval_partial()}
 
 
 
643
  </Badge>
644
  {/if}
645
  </div>
 
650
  {@const data = row}
651
  {#if column.key === 'integral'}
652
  <span
653
+ class={cn('inline-flex rounded-full border px-2 py-1 font-mono text-xs', scoreTone(data))}
 
 
 
654
  >
655
  {fmtScore(data.integral)}
656
  </span>
657
  {:else if column.key === 'minGroup'}
658
+ <span class={rankingMetricClass(data, 'minGroup')}>
 
 
 
659
  {fmtScore(data.minGroup)}
660
  </span>
661
  {:else if column.key === 'fpr'}
662
+ <span class={rankingMetricClass(data, 'fpr')}>
 
 
 
663
  {fmtPercent(data.fpr)}
664
  </span>
665
  {:else if column.key === 'fnr'}
666
+ <span class={rankingMetricClass(data, 'fnr')}>
 
 
 
667
  {fmtPercent(data.fnr)}
668
  </span>
669
  {:else if column.key === 'p95'}
670
+ <span class={rankingMetricClass(data, 'p95')}>
 
 
 
671
  {data.p95.toFixed(1)}
672
  </span>
673
  {:else if column.key === 'size'}
 
682
  getRowKey={(row) => modelFilterValue(row.model)}
683
  rowLabel={(row) => m.ranking_open_details_label({ model: row.model.short })}
684
  rowClass="cursor-pointer focus-visible:bg-muted/60 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 focus-visible:ring-offset-background"
685
+ headerClass="bg-muted/50 hover:bg-muted/50 [&_th]:pt-5 [&_th]:pb-4 [&_th]:text-muted-foreground"
686
  onrowclick={(row) => openModelDetails(row)}
687
  >
688
  {#snippet header(column)}
 
720
  <span
721
  class={cn(
722
  'inline-flex rounded-full border px-2.5 py-1 font-mono text-base font-semibold',
723
+ scoreTone(row)
724
  )}
725
  >
726
  {fmtScore(row.integral)}
 
730
  <p
731
  class="font-mono text-xs font-semibold uppercase tracking-wider text-muted-foreground"
732
  >
733
+ {m.ranking_column_min_score()}
734
  </p>
735
+ <span class={overviewMetricClass(row, 'minGroup')}>
 
 
 
736
  {fmtScore(row.minGroup)}
737
  </span>
738
  </div>
 
742
  >
743
  {m.ranking_column_fpr()}
744
  </p>
745
+ <span class={overviewMetricClass(row, 'fpr')}>
 
 
 
746
  {fmtPercent(row.fpr)}
747
  </span>
748
  </div>
 
752
  >
753
  {m.ranking_column_fnr()}
754
  </p>
755
+ <span class={overviewMetricClass(row, 'fnr')}>
 
 
 
756
  {fmtPercent(row.fnr)}
757
  </span>
758
  </div>
 
812
  {/if}
813
  </div>
814
  <span class={detailMetricClass(benchmark?.score ?? null)}>
 
 
 
815
  {fmtScore(benchmark?.score ?? null)}
816
  </span>
817
  <span class={detailMetricClass(benchmark?.fpr ?? null)}>
 
 
 
818
  {fmtPercent(benchmark?.fpr ?? null)}
819
  </span>
820
  <span class={detailMetricClass(benchmark?.fnr ?? null)}>
 
 
 
821
  {fmtPercent(benchmark?.fnr ?? null)}
822
  </span>
823
  <span class="justify-self-center font-mono tabular-nums">
 
840
  {groupLabel(benchmark.name)}
841
  </h3>
842
  <span class={benchmarkScoreClass(benchmark.score)}>
 
 
 
843
  {fmtScore(benchmark.score)}
844
  </span>
845
  </div>
 
923
  <span aria-hidden="true">△</span>
924
  {m.ranking_eval_leak()}
925
  </Badge>
926
+ {/if}
927
+ {#if row.model.partial}
928
  <Badge
929
  variant="outline"
930
+ class="h-5 gap-1 rounded-full border-sky-500/50 bg-sky-500/10 px-2 font-mono text-[0.6875rem] font-semibold text-sky-600 dark:text-sky-300"
931
  >
932
+ <span aria-hidden="true">◐</span>
933
+ {m.ranking_eval_partial()}
 
 
 
934
  </Badge>
935
  {/if}
936
  </div>
src/routes/details/+page.svelte CHANGED
@@ -16,12 +16,6 @@
16
  buildBenchmarkCards,
17
  buildMatrixGroups,
18
  buildMatrixRows,
19
- detailMetricGlyph as matrixCellGlyph,
20
- detailMetricIsBad as metricIsBad,
21
- detailMetricIsGood as metricIsGood,
22
- familyMetricGlyph,
23
- familyMetricIsBad,
24
- familyMetricIsGood,
25
  filterModels,
26
  matrixCellValue,
27
  matrixDatasetLabel,
@@ -38,7 +32,6 @@
38
  import { getLocale } from '$lib/paraglide/runtime';
39
  import type {
40
  BenchmarkFamilyLanguage,
41
- BenchmarkLeaderMetric,
42
  DetailMetric,
43
  DetailsLanguageFilter as LanguageFilter,
44
  DetailTab,
@@ -53,6 +46,8 @@
53
  import { cn } from '$lib/utils';
54
  import type { PageProps } from './$types';
55
 
 
 
56
  const selectContentClass = 'max-h-72';
57
  let { data }: PageProps = $props();
58
  const detailsState = $derived(data.detailsState);
@@ -66,7 +61,6 @@
66
  let benchmarkLanguage = $state<LanguageFilter>('any');
67
  let benchmarkSearch = $state('');
68
  let benchmarkFamilyLanguage = $state<BenchmarkFamilyLanguage>('all');
69
- let benchmarkFamilyMetric = $state<BenchmarkLeaderMetric>('f1');
70
  let benchmarkFamilySearch = $state('');
71
  let benchmarkSheetOpen = $state(false);
72
  let selectedBenchmark = $state<string | null>(null);
@@ -105,11 +99,6 @@
105
  { value: 'be', label: () => 'BE' },
106
  { value: 'uz', label: () => 'UZ' }
107
  ];
108
- const benchmarkFamilyMetricOptions: SelectOption<BenchmarkLeaderMetric>[] = [
109
- { value: 'f1', label: m.details_family_metric_f1 },
110
- { value: 'fpr', label: m.details_family_metric_fpr },
111
- { value: 'fnr', label: m.details_family_metric_fnr }
112
- ];
113
  const benchmarkMetricOptions: SelectOption<DetailMetric>[] = [
114
  { value: 'score', label: () => 'Score' },
115
  { value: 'f1', label: () => 'F1' },
@@ -142,9 +131,6 @@
142
  const currentBenchmarkFamilyLanguage = $derived(
143
  selectedOption(benchmarkFamilyLanguageOptions, benchmarkFamilyLanguage)
144
  );
145
- const currentBenchmarkFamilyMetric = $derived(
146
- selectedOption(benchmarkFamilyMetricOptions, benchmarkFamilyMetric)
147
- );
148
  const currentBenchmarkMetric = $derived(selectedOption(benchmarkMetricOptions, benchmarkMetric));
149
  const currentBenchmarkLanguage = $derived(selectedOption(languageOptions, benchmarkLanguage));
150
  const currentModelLanguage = $derived(selectedOption(languageOptions, modelLanguage));
@@ -186,14 +172,14 @@
186
  details.datasetMatrix.cellsByModelId
187
  )
188
  );
 
 
 
 
 
 
189
  const benchmarkCards = $derived.by(() =>
190
- buildBenchmarkCards(
191
- benchmarkFamilySearch,
192
- benchmarkFamilyLanguage,
193
- benchmarkFamilyMetric,
194
- details,
195
- getLocale()
196
- )
197
  );
198
  const selectedBenchmarkMeta = $derived(
199
  selectedBenchmark ? (benchmarkMeta[selectedBenchmark] ?? null) : null
@@ -246,6 +232,40 @@
246
  return fmtPercent(value);
247
  }
248
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
249
  function messageLabel(label: string | Message) {
250
  return typeof label === 'function' ? label() : label;
251
  }
@@ -265,59 +285,30 @@
265
  return matrixSortDirection === -1 ? 'descending' : 'ascending';
266
  }
267
 
268
- function metricCellClass(value: number | null, metric: DetailMetric) {
269
  return cn(
270
  'matrix-cell min-w-32 border-b border-l px-3 py-3 text-right font-mono text-sm tabular-nums',
271
  !isNumber(value) && 'matrix-cell-empty !bg-muted/30 text-muted-foreground',
272
- metricIsGood(value, metric) &&
273
- 'matrix-cell-good !bg-emerald-500/10 text-emerald-700 dark:text-emerald-300',
274
- !metricIsGood(value, metric) &&
275
- !metricIsBad(value, metric) &&
276
- isNumber(value) &&
277
- 'matrix-cell-neutral text-foreground',
278
- metricIsBad(value, metric) && 'matrix-cell-bad !bg-red-500/10 text-red-700 dark:text-red-300'
279
- );
280
- }
281
-
282
- function matrixCellClass(value: number | null, metric: DetailMetric, startsGroup: boolean) {
283
- return cn(metricCellClass(value, metric), startsGroup && 'border-l-border');
284
- }
285
-
286
- function scorePillClass(value: number | null) {
287
- return cn(
288
- 'inline-flex shrink-0 items-center gap-1 font-mono text-sm font-semibold tabular-nums',
289
- !isNumber(value) &&
290
- 'rounded-full border border-border bg-muted px-2.5 py-1 text-muted-foreground',
291
- isNumber(value) &&
292
- value >= 0.85 &&
293
- 'rounded-full border border-emerald-500/30 bg-emerald-500/10 px-2.5 py-1 text-emerald-700 dark:text-emerald-300',
294
- isNumber(value) && value >= 0.6 && value < 0.85 && 'text-foreground',
295
  isNumber(value) &&
296
- value < 0.6 &&
297
- 'rounded-full border border-red-500/30 bg-red-500/10 px-2.5 py-1 text-red-700 dark:text-red-300'
298
  );
299
  }
300
 
301
- function scoreGlyph(value: number | null) {
302
- if (!isNumber(value)) return '○';
303
- if (value >= 0.85) return '▲';
304
- if (value < 0.6) return '▼';
305
- return '-';
 
 
306
  }
307
 
308
- function familyMetricPillClass(value: number | null, metric: BenchmarkLeaderMetric) {
309
- const good = familyMetricIsGood(value, metric);
310
- const bad = familyMetricIsBad(value, metric);
311
-
312
  return cn(
313
  'inline-flex shrink-0 items-center gap-1 font-mono text-sm font-semibold tabular-nums',
314
  !isNumber(value) &&
315
  'rounded-full border border-border bg-muted px-2.5 py-1 text-muted-foreground',
316
- good &&
317
- 'rounded-full border border-emerald-500/30 bg-emerald-500/10 px-2.5 py-1 text-emerald-700 dark:text-emerald-300',
318
- isNumber(value) && !good && !bad && 'text-foreground',
319
- bad &&
320
- 'rounded-full border border-red-500/30 bg-red-500/10 px-2.5 py-1 text-red-700 dark:text-red-300'
321
  );
322
  }
323
 
@@ -342,64 +333,31 @@
342
  openModelSheet(model);
343
  }
344
 
345
- function rankingMetricClass(value: number | null, metric: BenchmarkLeaderMetric) {
346
  return cn(
347
  'inline-flex min-w-16 items-center justify-center rounded-sm px-1.5 py-0.5 font-mono text-xs tabular-nums',
348
  !isNumber(value) && 'text-muted-foreground',
349
- isNumber(value) &&
350
- metricIsGood(value, metric) &&
351
- 'bg-emerald-500/10 text-emerald-700 dark:text-emerald-300',
352
- isNumber(value) &&
353
- metricIsBad(value, metric) &&
354
- 'bg-red-500/10 text-red-700 dark:text-red-300',
355
- isNumber(value) &&
356
- !metricIsGood(value, metric) &&
357
- !metricIsBad(value, metric) &&
358
- 'text-foreground'
359
  );
360
  }
361
 
362
- function compactMetricClass(value: number | null, metric: DetailMetric) {
363
  return cn(
364
  'inline-flex w-fit items-center gap-1 rounded-sm px-1.5 py-0.5 font-mono text-xs font-semibold tabular-nums',
365
  !isNumber(value) && 'text-muted-foreground',
366
- isNumber(value) &&
367
- metricIsGood(value, metric) &&
368
- 'bg-emerald-500/10 text-emerald-700 dark:text-emerald-300',
369
- isNumber(value) &&
370
- metricIsBad(value, metric) &&
371
- 'bg-red-500/10 text-red-700 dark:text-red-300',
372
- isNumber(value) &&
373
- !metricIsGood(value, metric) &&
374
- !metricIsBad(value, metric) &&
375
- 'text-foreground'
376
  );
377
  }
378
 
379
- function modelKpiClass(value: number | null, metric: DetailMetric) {
380
  return cn(
381
  'inline-flex w-fit items-center gap-2 font-mono text-3xl font-bold tracking-normal tabular-nums',
382
  !isNumber(value) &&
383
  'rounded-lg border border-border bg-muted px-3 py-1.5 text-muted-foreground',
384
- isNumber(value) &&
385
- metricIsGood(value, metric) &&
386
- 'rounded-lg border border-emerald-500/30 bg-emerald-500/10 px-3 py-1.5 text-emerald-700 dark:text-emerald-300',
387
- isNumber(value) &&
388
- metricIsBad(value, metric) &&
389
- 'rounded-lg border border-red-500/30 bg-red-500/10 px-3 py-1.5 text-red-700 dark:text-red-300',
390
- isNumber(value) &&
391
- !metricIsGood(value, metric) &&
392
- !metricIsBad(value, metric) &&
393
- 'text-foreground'
394
  );
395
  }
396
 
397
- function groupBarFillClass(value: number) {
398
- if (value >= 0.85) return 'bg-emerald-500';
399
- if (value >= 0.6) return 'bg-amber-500';
400
- return 'bg-red-500';
401
- }
402
-
403
  function groupChipIsActive(groups: string[]) {
404
  return groups.some((group) => selectedGroups.includes(group));
405
  }
@@ -644,7 +602,7 @@
644
  <tbody>
645
  {#each matrixRows as model (modelKey(model))}
646
  <tr
647
- class="[&:nth-child(even)>td.matrix-cell-neutral]:bg-muted/20 hover:[&>td.matrix-cell-bad]:!bg-red-500/15 hover:[&>td.matrix-cell-empty]:!bg-muted/40 hover:[&>td.matrix-cell-good]:!bg-emerald-500/15 hover:[&>td.matrix-cell-neutral]:!bg-muted/40 hover:[&>th]:bg-muted"
648
  >
649
  <th
650
  class="z-20 min-w-80 max-w-80 border-r border-b bg-card px-4 py-3 text-left md:sticky md:left-0"
@@ -682,12 +640,15 @@
682
  )}
683
  <td
684
  class={cn(
685
- matrixCellClass(value, benchmarkMetric, datasetIndex === 0),
 
 
 
 
 
686
  groupIndex === 0 && datasetIndex === 0 && 'border-l-0'
687
  )}
688
  >
689
- <span aria-hidden="true">{matrixCellGlyph(value, benchmarkMetric)}</span
690
- >
691
  {#if isNumber(value)}
692
  {fmtMetric(value, benchmarkMetric)}
693
  {/if}
@@ -708,17 +669,7 @@
708
  <div
709
  class="mt-3 flex flex-wrap items-center gap-x-5 gap-y-2 text-xs text-muted-foreground"
710
  >
711
- <span class="inline-flex items-center gap-2">
712
- <span class="size-3 rounded-sm bg-[#0dce97]"></span>
713
- {m.details_matrix_good()}
714
- </span>
715
- <span class="inline-flex items-center gap-2">
716
- <span class="size-3 rounded-sm bg-red-500"></span>
717
- {m.details_matrix_bad()}
718
- </span>
719
  <span>{m.details_matrix_no_data()}</span>
720
- <span>{m.details_matrix_direction_legend()}</span>
721
- <span>{m.details_matrix_sort_legend()}</span>
722
  </div>
723
  </section>
724
  {:else}
@@ -780,7 +731,7 @@
780
  aria-label={m.details_family_filters()}
781
  >
782
  <div
783
- class="grid items-end gap-4 lg:grid-cols-[minmax(9rem,13rem)_minmax(14rem,22rem)_minmax(14rem,1fr)_auto_auto]"
784
  >
785
  <Select
786
  id="details-family-language"
@@ -794,18 +745,6 @@
794
  triggerLabel={currentBenchmarkFamilyLanguage.label()}
795
  contentClass={selectContentClass}
796
  />
797
- <Select
798
- id="details-family-metric"
799
- type="single"
800
- bind:value={benchmarkFamilyMetric}
801
- options={benchmarkFamilyMetricOptions.map((option) => ({
802
- value: option.value,
803
- label: option.label()
804
- }))}
805
- label={m.details_family_metric_label()}
806
- triggerLabel={currentBenchmarkFamilyMetric.label()}
807
- contentClass={selectContentClass}
808
- />
809
  <label class="min-w-0 space-y-2">
810
  <span
811
  class="block text-xs font-semibold tracking-wider text-muted-foreground uppercase"
@@ -821,7 +760,6 @@
821
  variant="outline"
822
  onclick={() => {
823
  benchmarkFamilyLanguage = 'all';
824
- benchmarkFamilyMetric = 'f1';
825
  benchmarkFamilySearch = '';
826
  }}
827
  >
@@ -833,25 +771,8 @@
833
  {benchmarkCards.length} / {allGroups.length}
834
  </span>
835
  </div>
836
- <p class="mt-3 text-xs text-muted-foreground">{m.details_family_metric_hint()}</p>
837
  </section>
838
 
839
- <div class="mt-3 flex flex-wrap items-center gap-x-5 gap-y-2 text-xs text-muted-foreground">
840
- <span class="inline-flex items-center gap-2">
841
- <span class="size-3 rounded-sm bg-[#0dce97]"></span>
842
- {m.details_family_legend_good()}
843
- </span>
844
- <span class="inline-flex items-center gap-2">
845
- <span class="size-3 rounded-sm bg-amber-500"></span>
846
- {m.details_family_legend_middle()}
847
- </span>
848
- <span class="inline-flex items-center gap-2">
849
- <span class="size-3 rounded-sm bg-red-500"></span>
850
- {m.details_family_legend_bad()}
851
- </span>
852
- <span>{m.details_family_legend_direction()}</span>
853
- </div>
854
-
855
  {#if benchmarkCards.length}
856
  <div class="mt-12 grid min-w-0 gap-4 md:grid-cols-2 xl:grid-cols-3">
857
  {#each benchmarkCards as card (card.name)}
@@ -881,22 +802,6 @@
881
  </Badge>
882
  {/each}
883
  </div>
884
- <div class="mt-auto border-t pt-4">
885
- <p class="font-mono text-xs tracking-wider text-muted-foreground uppercase">
886
- {m.details_leader()} · {benchmarkFamilyMetric.toUpperCase()}
887
- </p>
888
- <div class="mt-2 flex min-w-0 items-center justify-between gap-3">
889
- <p class="min-w-0 truncate font-mono text-sm font-semibold">
890
- {card.leader?.model.short ?? m.details_no_data()}
891
- </p>
892
- <span
893
- class={cn('shrink-0', familyMetricPillClass(card.score, benchmarkFamilyMetric))}
894
- >
895
- {familyMetricGlyph(card.score, benchmarkFamilyMetric)}
896
- {fmtScore(card.score)}
897
- </span>
898
- </div>
899
- </div>
900
  </button>
901
  {/each}
902
  </div>
@@ -992,17 +897,17 @@
992
  </p>
993
  </td>
994
  <td class="px-3 py-3 text-right">
995
- <span class={rankingMetricClass(row.fnr, 'fnr')}>
996
  {fmtMetric(row.fnr, 'fnr')}
997
  </span>
998
  </td>
999
  <td class="px-3 py-3 text-right">
1000
- <span class={rankingMetricClass(row.fpr, 'fpr')}>
1001
  {fmtMetric(row.fpr, 'fpr')}
1002
  </span>
1003
  </td>
1004
  <td class="px-3 py-3 text-right">
1005
- <span class={rankingMetricClass(row.score, 'f1')}>
1006
  {fmtScore(row.score)}
1007
  </span>
1008
  </td>
@@ -1092,27 +997,6 @@
1092
  </div>
1093
  </section>
1094
 
1095
- <div
1096
- class="mt-4 flex flex-wrap items-center gap-x-5 gap-y-2 rounded-2xl border bg-muted/50 px-4 py-3 text-sm text-muted-foreground shadow-sm"
1097
- role="note"
1098
- aria-label={m.details_model_legend_lead()}
1099
- >
1100
- <span class="font-semibold text-foreground">{m.details_model_legend_lead()}</span>
1101
- <span class="inline-flex items-center gap-2">
1102
- <span class="size-3 rounded-sm bg-[#0dce97]"></span>
1103
- {m.details_model_legend_good()}
1104
- </span>
1105
- <span class="inline-flex items-center gap-2">
1106
- <span class="size-3 rounded-sm bg-amber-500"></span>
1107
- {m.details_model_legend_middle()}
1108
- </span>
1109
- <span class="inline-flex items-center gap-2">
1110
- <span class="size-3 rounded-sm bg-red-500"></span>
1111
- {m.details_model_legend_bad()}
1112
- </span>
1113
- <span>{m.details_model_legend_hint()}</span>
1114
- </div>
1115
-
1116
  <div class="mt-4 grid min-w-0 gap-4 md:grid-cols-2 xl:grid-cols-3">
1117
  {#each filteredModels as model (modelKey(model))}
1118
  <div
@@ -1133,7 +1017,6 @@
1133
  </h2>
1134
  </div>
1135
  <span class={cn('shrink-0', scorePillClass(model.integral))}>
1136
- {scoreGlyph(model.integral)}
1137
  {fmtScore(model.integral)}
1138
  </span>
1139
  </div>
@@ -1168,12 +1051,13 @@
1168
  >
1169
  △ {m.ranking_eval_leak()}
1170
  </Badge>
1171
- {:else}
 
1172
  <Badge
1173
  variant="outline"
1174
- class="rounded-full border-[#0dce97]/50 bg-[#0dce97]/10 font-mono text-[#0dce97]"
1175
  >
1176
- {m.ranking_eval_clean()}
1177
  </Badge>
1178
  {/if}
1179
  <span class="min-w-0 break-words"
@@ -1185,10 +1069,13 @@
1185
  </span>
1186
  </div>
1187
  <div class="mt-4 flex min-w-0 items-center justify-between gap-3">
1188
- <p class="min-w-0 truncate font-mono text-sm text-muted-foreground">
1189
  F1 <span class="text-foreground">{fmtScore(model.f1)}</span> · p50 {fmtMs(
1190
  model.p50
1191
- )}
 
 
 
1192
  </p>
1193
  <Button
1194
  class="shrink-0"
@@ -1250,8 +1137,7 @@
1250
  <div class="space-y-8">
1251
  <div class="grid gap-3 sm:grid-cols-2 xl:grid-cols-4">
1252
  <div class="rounded-xl border bg-card p-4 shadow-sm">
1253
- <p class={modelKpiClass(selectedModel.integral, 'f1')}>
1254
- {matrixCellGlyph(selectedModel.integral, 'f1')}
1255
  {fmtScore(selectedModel.integral)}
1256
  </p>
1257
  <p class="mt-3 text-xs font-semibold tracking-wider text-muted-foreground uppercase">
@@ -1259,8 +1145,7 @@
1259
  </p>
1260
  </div>
1261
  <div class="rounded-xl border bg-card p-4 shadow-sm">
1262
- <p class={modelKpiClass(selectedModel.minGroup, 'f1')}>
1263
- {matrixCellGlyph(selectedModel.minGroup, 'f1')}
1264
  {fmtScore(selectedModel.minGroup)}
1265
  </p>
1266
  <p class="mt-3 text-xs font-semibold tracking-wider text-muted-foreground uppercase">
@@ -1268,8 +1153,7 @@
1268
  </p>
1269
  </div>
1270
  <div class="rounded-xl border bg-card p-4 shadow-sm">
1271
- <p class={modelKpiClass(selectedModel.fnr, 'fnr')}>
1272
- {matrixCellGlyph(selectedModel.fnr, 'fnr')}
1273
  {fmtPercent(selectedModel.fnr)}
1274
  </p>
1275
  <p class="mt-3 text-xs font-semibold tracking-wider text-muted-foreground uppercase">
@@ -1277,8 +1161,7 @@
1277
  </p>
1278
  </div>
1279
  <div class="rounded-xl border bg-card p-4 shadow-sm">
1280
- <p class={modelKpiClass(selectedModel.fpr, 'fpr')}>
1281
- {matrixCellGlyph(selectedModel.fpr, 'fpr')}
1282
  {fmtPercent(selectedModel.fpr)}
1283
  </p>
1284
  <p class="mt-3 text-xs font-semibold tracking-wider text-muted-foreground uppercase">
@@ -1395,26 +1278,22 @@
1395
  <tr class="border-t transition-colors hover:bg-muted/30">
1396
  <td class="px-3 py-3 font-mono text-xs font-semibold">{group.name}</td>
1397
  <td class="px-3 py-3 text-right">
1398
- <span class={compactMetricClass(group.score, 'f1')}>
1399
- {matrixCellGlyph(group.score, 'f1')}
1400
  {fmtScore(group.score)}
1401
  </span>
1402
  </td>
1403
  <td class="px-3 py-3 text-right">
1404
- <span class={compactMetricClass(group.fnr, 'fnr')}>
1405
- {matrixCellGlyph(group.fnr, 'fnr')}
1406
  {fmtPercent(group.fnr)}
1407
  </span>
1408
  </td>
1409
  <td class="px-3 py-3 text-right">
1410
- <span class={compactMetricClass(group.fpr, 'fpr')}>
1411
- {matrixCellGlyph(group.fpr, 'fpr')}
1412
  {fmtPercent(group.fpr)}
1413
  </span>
1414
  </td>
1415
  <td class="px-3 py-3 text-right">
1416
- <span class={compactMetricClass(group.score, 'f1')}>
1417
- {matrixCellGlyph(group.score, 'f1')}
1418
  {fmtScore(group.score)}
1419
  </span>
1420
  </td>
@@ -1460,31 +1339,16 @@
1460
  </p>
1461
  <div class="h-2 rounded-full bg-muted">
1462
  <div
1463
- class={cn('h-full rounded-full', groupBarFillClass(group.score))}
1464
  style={`width:${Math.max(2, Math.min(100, group.score * 100))}%`}
1465
  ></div>
1466
  </div>
1467
- <span class={compactMetricClass(group.score, 'f1')}>
1468
- {scoreGlyph(group.score)}
1469
  {fmtScore(group.score)}
1470
  </span>
1471
  </div>
1472
  {/each}
1473
  </div>
1474
- <div class="flex flex-wrap items-center gap-x-5 gap-y-2 text-xs text-muted-foreground">
1475
- <span class="inline-flex items-center gap-2">
1476
- <span class="size-3 rounded-sm bg-emerald-500"></span>
1477
- ≥ 0.85
1478
- </span>
1479
- <span class="inline-flex items-center gap-2">
1480
- <span class="size-3 rounded-sm bg-amber-500"></span>
1481
- 0.60-0.85
1482
- </span>
1483
- <span class="inline-flex items-center gap-2">
1484
- <span class="size-3 rounded-sm bg-red-500"></span>
1485
- &lt; 0.60
1486
- </span>
1487
- </div>
1488
  </section>
1489
  {/if}
1490
  {/snippet}
@@ -1538,28 +1402,31 @@
1538
  </span>
1539
  <span class="min-w-0 break-words text-base">{selectedModel.license}</span>
1540
  </li>
1541
- <li class="grid items-start gap-2 sm:grid-cols-[14rem_minmax(0,1fr)]">
1542
- <span class="text-sm font-semibold tracking-wider text-muted-foreground uppercase">
1543
- {m.details_model_sheet_eval_leak()}
1544
- </span>
1545
- <span>
1546
- {#if selectedModel.leak}
1547
- <Badge
1548
- variant="outline"
1549
- class="rounded-full border-amber-500/50 bg-amber-500/10 font-mono text-amber-500"
1550
- >
1551
- △ {m.ranking_eval_leak()}
1552
- </Badge>
1553
- {:else}
1554
- <Badge
1555
- variant="outline"
1556
- class="rounded-full border-[#0dce97]/50 bg-[#0dce97]/10 font-mono text-[#0dce97]"
1557
- >
1558
- {m.ranking_eval_clean()}
1559
- </Badge>
1560
- {/if}
1561
- </span>
1562
- </li>
 
 
 
1563
  <li class="grid items-start gap-2 sm:grid-cols-[14rem_minmax(0,1fr)]">
1564
  <span class="text-sm font-semibold tracking-wider text-muted-foreground uppercase">
1565
  {m.details_model_sheet_run_date()}
 
16
  buildBenchmarkCards,
17
  buildMatrixGroups,
18
  buildMatrixRows,
 
 
 
 
 
 
19
  filterModels,
20
  matrixCellValue,
21
  matrixDatasetLabel,
 
32
  import { getLocale } from '$lib/paraglide/runtime';
33
  import type {
34
  BenchmarkFamilyLanguage,
 
35
  DetailMetric,
36
  DetailsLanguageFilter as LanguageFilter,
37
  DetailTab,
 
46
  import { cn } from '$lib/utils';
47
  import type { PageProps } from './$types';
48
 
49
+ type ColumnExtrema = { best: number | null; worst: number | null };
50
+
51
  const selectContentClass = 'max-h-72';
52
  let { data }: PageProps = $props();
53
  const detailsState = $derived(data.detailsState);
 
61
  let benchmarkLanguage = $state<LanguageFilter>('any');
62
  let benchmarkSearch = $state('');
63
  let benchmarkFamilyLanguage = $state<BenchmarkFamilyLanguage>('all');
 
64
  let benchmarkFamilySearch = $state('');
65
  let benchmarkSheetOpen = $state(false);
66
  let selectedBenchmark = $state<string | null>(null);
 
99
  { value: 'be', label: () => 'BE' },
100
  { value: 'uz', label: () => 'UZ' }
101
  ];
 
 
 
 
 
102
  const benchmarkMetricOptions: SelectOption<DetailMetric>[] = [
103
  { value: 'score', label: () => 'Score' },
104
  { value: 'f1', label: () => 'F1' },
 
131
  const currentBenchmarkFamilyLanguage = $derived(
132
  selectedOption(benchmarkFamilyLanguageOptions, benchmarkFamilyLanguage)
133
  );
 
 
 
134
  const currentBenchmarkMetric = $derived(selectedOption(benchmarkMetricOptions, benchmarkMetric));
135
  const currentBenchmarkLanguage = $derived(selectedOption(languageOptions, benchmarkLanguage));
136
  const currentModelLanguage = $derived(selectedOption(languageOptions, modelLanguage));
 
172
  details.datasetMatrix.cellsByModelId
173
  )
174
  );
175
+ const matrixColumnExtrema = $derived.by(
176
+ () =>
177
+ Object.fromEntries(
178
+ visibleMatrixDatasets.map((dataset) => [dataset.name, detailMatrixColumnExtrema(dataset)])
179
+ ) as Record<string, ColumnExtrema>
180
+ );
181
  const benchmarkCards = $derived.by(() =>
182
+ buildBenchmarkCards(benchmarkFamilySearch, benchmarkFamilyLanguage, details, getLocale())
 
 
 
 
 
 
183
  );
184
  const selectedBenchmarkMeta = $derived(
185
  selectedBenchmark ? (benchmarkMeta[selectedBenchmark] ?? null) : null
 
232
  return fmtPercent(value);
233
  }
234
 
235
+ function roundedDetailMetric(value: number | null, metric: DetailMetric) {
236
+ if (!isNumber(value)) return null;
237
+ if (metric === 'score' || metric === 'f1') return Number(value.toFixed(3));
238
+ return Number((value * 100).toFixed(1));
239
+ }
240
+
241
+ function detailMatrixColumnExtrema(dataset: MatrixDataset): ColumnExtrema {
242
+ const values = matrixRows
243
+ .map((model) =>
244
+ roundedDetailMetric(
245
+ matrixCellValue(model, dataset, benchmarkMetric, details.datasetMatrix.cellsByModelId),
246
+ benchmarkMetric
247
+ )
248
+ )
249
+ .filter(isNumber);
250
+
251
+ if (!values.length) return { best: null, worst: null };
252
+ const min = Math.min(...values);
253
+ const max = Math.max(...values);
254
+ return benchmarkMetric === 'fpr' || benchmarkMetric === 'fnr'
255
+ ? { best: min, worst: max }
256
+ : { best: max, worst: min };
257
+ }
258
+
259
+ function extremaTone(value: number | null, extrema: ColumnExtrema | undefined) {
260
+ if (!isNumber(value)) return 'text-muted-foreground';
261
+ if (!extrema || extrema.best === null || extrema.worst === null) return 'text-foreground';
262
+ if (value === extrema.best)
263
+ return 'matrix-cell-good !bg-emerald-500/10 text-emerald-700 dark:text-emerald-300';
264
+ if (value === extrema.worst)
265
+ return 'matrix-cell-bad !bg-red-500/10 text-red-700 dark:text-red-300';
266
+ return 'text-foreground';
267
+ }
268
+
269
  function messageLabel(label: string | Message) {
270
  return typeof label === 'function' ? label() : label;
271
  }
 
285
  return matrixSortDirection === -1 ? 'descending' : 'ascending';
286
  }
287
 
288
+ function metricCellClass(value: number | null, metric: DetailMetric, datasetName: string) {
289
  return cn(
290
  'matrix-cell min-w-32 border-b border-l px-3 py-3 text-right font-mono text-sm tabular-nums',
291
  !isNumber(value) && 'matrix-cell-empty !bg-muted/30 text-muted-foreground',
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
292
  isNumber(value) &&
293
+ extremaTone(roundedDetailMetric(value, metric), matrixColumnExtrema[datasetName])
 
294
  );
295
  }
296
 
297
+ function matrixCellClass(
298
+ value: number | null,
299
+ metric: DetailMetric,
300
+ datasetName: string,
301
+ startsGroup: boolean
302
+ ) {
303
+ return cn(metricCellClass(value, metric, datasetName), startsGroup && 'border-l-border');
304
  }
305
 
306
+ function scorePillClass(value: number | null) {
 
 
 
307
  return cn(
308
  'inline-flex shrink-0 items-center gap-1 font-mono text-sm font-semibold tabular-nums',
309
  !isNumber(value) &&
310
  'rounded-full border border-border bg-muted px-2.5 py-1 text-muted-foreground',
311
+ isNumber(value) && 'text-foreground'
 
 
 
 
312
  );
313
  }
314
 
 
333
  openModelSheet(model);
334
  }
335
 
336
+ function rankingMetricClass(value: number | null) {
337
  return cn(
338
  'inline-flex min-w-16 items-center justify-center rounded-sm px-1.5 py-0.5 font-mono text-xs tabular-nums',
339
  !isNumber(value) && 'text-muted-foreground',
340
+ isNumber(value) && 'text-foreground'
 
 
 
 
 
 
 
 
 
341
  );
342
  }
343
 
344
+ function compactMetricClass(value: number | null) {
345
  return cn(
346
  'inline-flex w-fit items-center gap-1 rounded-sm px-1.5 py-0.5 font-mono text-xs font-semibold tabular-nums',
347
  !isNumber(value) && 'text-muted-foreground',
348
+ isNumber(value) && 'text-foreground'
 
 
 
 
 
 
 
 
 
349
  );
350
  }
351
 
352
+ function modelKpiClass(value: number | null) {
353
  return cn(
354
  'inline-flex w-fit items-center gap-2 font-mono text-3xl font-bold tracking-normal tabular-nums',
355
  !isNumber(value) &&
356
  'rounded-lg border border-border bg-muted px-3 py-1.5 text-muted-foreground',
357
+ isNumber(value) && 'text-foreground'
 
 
 
 
 
 
 
 
 
358
  );
359
  }
360
 
 
 
 
 
 
 
361
  function groupChipIsActive(groups: string[]) {
362
  return groups.some((group) => selectedGroups.includes(group));
363
  }
 
602
  <tbody>
603
  {#each matrixRows as model (modelKey(model))}
604
  <tr
605
+ class="hover:[&>td.matrix-cell-bad]:!bg-red-500/15 hover:[&>td.matrix-cell-empty]:!bg-muted/40 hover:[&>td.matrix-cell-good]:!bg-emerald-500/15 hover:[&>th]:bg-muted"
606
  >
607
  <th
608
  class="z-20 min-w-80 max-w-80 border-r border-b bg-card px-4 py-3 text-left md:sticky md:left-0"
 
640
  )}
641
  <td
642
  class={cn(
643
+ matrixCellClass(
644
+ value,
645
+ benchmarkMetric,
646
+ dataset.name,
647
+ datasetIndex === 0
648
+ ),
649
  groupIndex === 0 && datasetIndex === 0 && 'border-l-0'
650
  )}
651
  >
 
 
652
  {#if isNumber(value)}
653
  {fmtMetric(value, benchmarkMetric)}
654
  {/if}
 
669
  <div
670
  class="mt-3 flex flex-wrap items-center gap-x-5 gap-y-2 text-xs text-muted-foreground"
671
  >
 
 
 
 
 
 
 
 
672
  <span>{m.details_matrix_no_data()}</span>
 
 
673
  </div>
674
  </section>
675
  {:else}
 
731
  aria-label={m.details_family_filters()}
732
  >
733
  <div
734
+ class="grid items-end gap-4 lg:grid-cols-[minmax(9rem,13rem)_minmax(14rem,1fr)_auto_auto]"
735
  >
736
  <Select
737
  id="details-family-language"
 
745
  triggerLabel={currentBenchmarkFamilyLanguage.label()}
746
  contentClass={selectContentClass}
747
  />
 
 
 
 
 
 
 
 
 
 
 
 
748
  <label class="min-w-0 space-y-2">
749
  <span
750
  class="block text-xs font-semibold tracking-wider text-muted-foreground uppercase"
 
760
  variant="outline"
761
  onclick={() => {
762
  benchmarkFamilyLanguage = 'all';
 
763
  benchmarkFamilySearch = '';
764
  }}
765
  >
 
771
  {benchmarkCards.length} / {allGroups.length}
772
  </span>
773
  </div>
 
774
  </section>
775
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
776
  {#if benchmarkCards.length}
777
  <div class="mt-12 grid min-w-0 gap-4 md:grid-cols-2 xl:grid-cols-3">
778
  {#each benchmarkCards as card (card.name)}
 
802
  </Badge>
803
  {/each}
804
  </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
805
  </button>
806
  {/each}
807
  </div>
 
897
  </p>
898
  </td>
899
  <td class="px-3 py-3 text-right">
900
+ <span class={rankingMetricClass(row.fnr)}>
901
  {fmtMetric(row.fnr, 'fnr')}
902
  </span>
903
  </td>
904
  <td class="px-3 py-3 text-right">
905
+ <span class={rankingMetricClass(row.fpr)}>
906
  {fmtMetric(row.fpr, 'fpr')}
907
  </span>
908
  </td>
909
  <td class="px-3 py-3 text-right">
910
+ <span class={rankingMetricClass(row.score)}>
911
  {fmtScore(row.score)}
912
  </span>
913
  </td>
 
997
  </div>
998
  </section>
999
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1000
  <div class="mt-4 grid min-w-0 gap-4 md:grid-cols-2 xl:grid-cols-3">
1001
  {#each filteredModels as model (modelKey(model))}
1002
  <div
 
1017
  </h2>
1018
  </div>
1019
  <span class={cn('shrink-0', scorePillClass(model.integral))}>
 
1020
  {fmtScore(model.integral)}
1021
  </span>
1022
  </div>
 
1051
  >
1052
  △ {m.ranking_eval_leak()}
1053
  </Badge>
1054
+ {/if}
1055
+ {#if model.partial}
1056
  <Badge
1057
  variant="outline"
1058
+ class="rounded-full border-sky-500/50 bg-sky-500/10 font-mono text-sky-600 dark:text-sky-300"
1059
  >
1060
+ {m.ranking_eval_partial()}
1061
  </Badge>
1062
  {/if}
1063
  <span class="min-w-0 break-words"
 
1069
  </span>
1070
  </div>
1071
  <div class="mt-4 flex min-w-0 items-center justify-between gap-3">
1072
+ <p class="min-w-0 font-mono text-sm text-muted-foreground">
1073
  F1 <span class="text-foreground">{fmtScore(model.f1)}</span> · p50 {fmtMs(
1074
  model.p50
1075
+ )} ·
1076
+ <span class="whitespace-nowrap"
1077
+ >{m.details_error_rate()} {fmtPercent(model.errorRate)}</span
1078
+ >
1079
  </p>
1080
  <Button
1081
  class="shrink-0"
 
1137
  <div class="space-y-8">
1138
  <div class="grid gap-3 sm:grid-cols-2 xl:grid-cols-4">
1139
  <div class="rounded-xl border bg-card p-4 shadow-sm">
1140
+ <p class={modelKpiClass(selectedModel.integral)}>
 
1141
  {fmtScore(selectedModel.integral)}
1142
  </p>
1143
  <p class="mt-3 text-xs font-semibold tracking-wider text-muted-foreground uppercase">
 
1145
  </p>
1146
  </div>
1147
  <div class="rounded-xl border bg-card p-4 shadow-sm">
1148
+ <p class={modelKpiClass(selectedModel.minGroup)}>
 
1149
  {fmtScore(selectedModel.minGroup)}
1150
  </p>
1151
  <p class="mt-3 text-xs font-semibold tracking-wider text-muted-foreground uppercase">
 
1153
  </p>
1154
  </div>
1155
  <div class="rounded-xl border bg-card p-4 shadow-sm">
1156
+ <p class={modelKpiClass(selectedModel.fnr)}>
 
1157
  {fmtPercent(selectedModel.fnr)}
1158
  </p>
1159
  <p class="mt-3 text-xs font-semibold tracking-wider text-muted-foreground uppercase">
 
1161
  </p>
1162
  </div>
1163
  <div class="rounded-xl border bg-card p-4 shadow-sm">
1164
+ <p class={modelKpiClass(selectedModel.fpr)}>
 
1165
  {fmtPercent(selectedModel.fpr)}
1166
  </p>
1167
  <p class="mt-3 text-xs font-semibold tracking-wider text-muted-foreground uppercase">
 
1278
  <tr class="border-t transition-colors hover:bg-muted/30">
1279
  <td class="px-3 py-3 font-mono text-xs font-semibold">{group.name}</td>
1280
  <td class="px-3 py-3 text-right">
1281
+ <span class={compactMetricClass(group.score)}>
 
1282
  {fmtScore(group.score)}
1283
  </span>
1284
  </td>
1285
  <td class="px-3 py-3 text-right">
1286
+ <span class={compactMetricClass(group.fnr)}>
 
1287
  {fmtPercent(group.fnr)}
1288
  </span>
1289
  </td>
1290
  <td class="px-3 py-3 text-right">
1291
+ <span class={compactMetricClass(group.fpr)}>
 
1292
  {fmtPercent(group.fpr)}
1293
  </span>
1294
  </td>
1295
  <td class="px-3 py-3 text-right">
1296
+ <span class={compactMetricClass(group.score)}>
 
1297
  {fmtScore(group.score)}
1298
  </span>
1299
  </td>
 
1339
  </p>
1340
  <div class="h-2 rounded-full bg-muted">
1341
  <div
1342
+ class="h-full rounded-full bg-[#0dce97]"
1343
  style={`width:${Math.max(2, Math.min(100, group.score * 100))}%`}
1344
  ></div>
1345
  </div>
1346
+ <span class={compactMetricClass(group.score)}>
 
1347
  {fmtScore(group.score)}
1348
  </span>
1349
  </div>
1350
  {/each}
1351
  </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1352
  </section>
1353
  {/if}
1354
  {/snippet}
 
1402
  </span>
1403
  <span class="min-w-0 break-words text-base">{selectedModel.license}</span>
1404
  </li>
1405
+ {#if selectedModel.leak || selectedModel.partial}
1406
+ <li class="grid items-start gap-2 sm:grid-cols-[14rem_minmax(0,1fr)]">
1407
+ <span class="text-sm font-semibold tracking-wider text-muted-foreground uppercase">
1408
+ {m.details_model_sheet_status()}
1409
+ </span>
1410
+ <span class="flex flex-wrap gap-2">
1411
+ {#if selectedModel.leak}
1412
+ <Badge
1413
+ variant="outline"
1414
+ class="rounded-full border-amber-500/50 bg-amber-500/10 font-mono text-amber-500"
1415
+ >
1416
+ △ {m.ranking_eval_leak()}
1417
+ </Badge>
1418
+ {/if}
1419
+ {#if selectedModel.partial}
1420
+ <Badge
1421
+ variant="outline"
1422
+ class="rounded-full border-sky-500/50 bg-sky-500/10 font-mono text-sky-600 dark:text-sky-300"
1423
+ >
1424
+ {m.ranking_eval_partial()}
1425
+ </Badge>
1426
+ {/if}
1427
+ </span>
1428
+ </li>
1429
+ {/if}
1430
  <li class="grid items-start gap-2 sm:grid-cols-[14rem_minmax(0,1fr)]">
1431
  <span class="text-sm font-semibold tracking-wider text-muted-foreground uppercase">
1432
  {m.details_model_sheet_run_date()}
src/routes/methodology/+page.svelte CHANGED
@@ -9,14 +9,8 @@
9
  | 'group-score'
10
  | 'integral-product'
11
  | 'integral-log'
12
- | 'fbeta'
13
- | 'f2'
14
- | 'f05'
15
- | 'fdr'
16
  | 'fnr-harm'
17
  | 'fpr-safe'
18
- | 'micro-f1'
19
- | 'micro-pr'
20
  | 'fpr-avg'
21
  | 'fnr-avg';
22
 
@@ -30,7 +24,6 @@
30
  tag?: Message;
31
  formulaBlocks: FormulaBlockData[];
32
  description: Message;
33
- example?: Message;
34
  wide?: boolean;
35
  };
36
 
@@ -51,15 +44,13 @@
51
  title: 'S_ds',
52
  tag: m.methodology_dataset_score_tag,
53
  formulaBlocks: [{ kind: 'dataset-score', comment: m.methodology_dataset_score_comment }],
54
- description: m.methodology_dataset_score_description,
55
- example: m.methodology_dataset_score_example
56
  },
57
  {
58
  title: 'S_group',
59
  tag: m.methodology_group_score_tag,
60
  formulaBlocks: [{ kind: 'group-score', comment: m.methodology_group_score_comment }],
61
- description: m.methodology_group_score_description,
62
- example: m.methodology_group_score_example
63
  },
64
  {
65
  title: 'S_integral',
@@ -68,37 +59,7 @@
68
  { kind: 'integral-product', comment: m.methodology_integral_score_comment },
69
  { kind: 'integral-log', comment: m.methodology_integral_log_comment }
70
  ],
71
- description: m.methodology_integral_score_description,
72
- example: m.methodology_integral_score_example
73
- }
74
- ];
75
-
76
- const helperMetricCards: FormulaCard[] = [
77
- {
78
- title: 'F_beta',
79
- tag: m.methodology_fbeta_tag,
80
- formulaBlocks: [{ kind: 'fbeta' }],
81
- description: m.methodology_fbeta_description
82
- },
83
- {
84
- title: 'F2',
85
- tag: m.methodology_f2_tag,
86
- formulaBlocks: [{ kind: 'f2' }],
87
- description: m.methodology_f2_description,
88
- example: m.methodology_f2_example
89
- },
90
- {
91
- title: 'F0.5',
92
- tag: m.methodology_f05_tag,
93
- formulaBlocks: [{ kind: 'f05' }],
94
- description: m.methodology_f05_description,
95
- example: m.methodology_f05_example
96
- },
97
- {
98
- title: 'FDR',
99
- formulaBlocks: [{ kind: 'fdr' }],
100
- description: m.methodology_fdr_description,
101
- example: m.methodology_fdr_example
102
  }
103
  ];
104
 
@@ -114,13 +75,6 @@
114
  tag: m.methodology_safe_split_tag,
115
  formulaBlocks: [{ kind: 'fpr-safe' }],
116
  description: m.methodology_safe_split_description
117
- },
118
- {
119
- title: 'Micro-F1',
120
- tag: m.methodology_micro_f1_tag,
121
- formulaBlocks: [{ kind: 'micro-f1' }, { kind: 'micro-pr' }],
122
- description: m.methodology_micro_f1_description,
123
- wide: true
124
  }
125
  ];
126
 
@@ -138,7 +92,8 @@
138
  {
139
  title: m.methodology_scatter_point_title(),
140
  formulaBlocks: [],
141
- description: m.methodology_scatter_point_description
 
142
  }
143
  ];
144
 
@@ -153,25 +108,28 @@
153
  title: m.methodology_caveat_title,
154
  description: m.methodology_caveat_description,
155
  muted: true
156
- },
157
- {
158
- mark: '!',
159
- title: m.methodology_golden_rule_title,
160
- description: m.methodology_golden_rule_description
161
  }
162
  ];
163
 
164
  function cardClass(card: FormulaCard) {
165
  return cn(
166
- 'flex min-h-[28rem] min-w-0 max-w-full flex-col overflow-hidden rounded-2xl border bg-card p-5 shadow-sm',
167
- card.wide && 'lg:col-span-2 xl:col-span-1'
 
 
 
 
 
 
168
  );
169
  }
170
 
171
- function aggregationCardClass(card: FormulaCard) {
172
  return cn(
173
- 'flex min-h-[34rem] min-w-0 max-w-full flex-col overflow-hidden rounded-2xl border bg-card p-5 shadow-sm',
174
- card.title === 'S_integral' && 'min-h-[34rem]'
 
 
175
  );
176
  }
177
 
@@ -212,9 +170,9 @@
212
  >
213
  {m.methodology_aggregation_title()}
214
  </h2>
215
- <div class="mt-3 grid min-w-0 gap-4 lg:grid-cols-3">
216
  {#each aggregationCards as card (card.title)}
217
- <article class={aggregationCardClass(card)}>
218
  <div class="flex min-w-0 flex-wrap items-start justify-between gap-3">
219
  <h2
220
  class="flex min-w-0 flex-wrap items-baseline gap-3 font-mono text-2xl font-semibold tracking-normal"
@@ -232,39 +190,30 @@
232
  <span class="mr-1 font-semibold text-foreground">{m.methodology_meaning_label()}</span>
233
  {card.description()}
234
  </p>
235
- {#if card.example}
236
- <div class="mt-auto pt-8">
237
- <p
238
- class="min-w-0 rounded-xl border border-[#0dce97]/35 bg-[#0dce97]/10 px-4 py-3 font-mono text-xs leading-6 break-words text-[#0dce97]"
239
- >
240
- <span class="mb-1 block text-xs font-semibold uppercase tracking-wider">
241
- {m.methodology_example_label()}
242
- </span>
243
- {@render RichText(card.example())}
244
- </p>
245
- </div>
246
- {/if}
247
  </article>
248
  {/each}
249
  </div>
250
  </section>
251
 
252
  {@render FormulaSection(
253
- m.methodology_helper_metrics_title,
254
- helperMetricCards,
255
- undefined,
256
- 'md:grid-cols-2 xl:grid-cols-4'
257
  )}
258
 
259
- {@render FormulaSection(m.methodology_combined_dataset_title, combinedCards, CombinedDatasetNote)}
260
-
261
  {#snippet CombinedDatasetNote()}
262
  <p class="mt-4 text-sm leading-6 text-muted-foreground">
263
  {m.methodology_combined_dataset_note()}
264
  </p>
265
  {/snippet}
266
 
267
- {@render FormulaSection(m.methodology_scatter_title, scatterCards)}
 
 
 
 
 
268
 
269
  <section class="mt-8 min-w-0 space-y-4" aria-labelledby="specifics-title">
270
  <h2
@@ -306,20 +255,6 @@
306
  </div>
307
  {/each}
308
  </section>
309
-
310
- <div class="mt-8 flex min-w-0 flex-wrap gap-3 font-mono text-xs text-muted-foreground">
311
- <span class="inline-flex max-w-full items-center gap-2 rounded-full border bg-card px-3 py-2">
312
- <span class="size-2 rounded-full bg-[#0dce97]"></span>
313
- {m.methodology_legend_higher_better()}
314
- </span>
315
- <span class="inline-flex max-w-full items-center gap-2 rounded-full border bg-card px-3 py-2">
316
- <span class="size-2 rounded-full bg-red-500"></span>
317
- {m.methodology_legend_higher_worse()}
318
- </span>
319
- <span class="inline-flex max-w-full items-center rounded-full border bg-card px-3 py-2">
320
- {m.methodology_legend_glyphs()}
321
- </span>
322
- </div>
323
  </div>
324
 
325
  {#snippet RichText(value: string)}
@@ -377,7 +312,12 @@
377
  {/snippet}
378
 
379
  {#snippet FormulaBlock(card: FormulaCard)}
380
- <div class="mt-4 min-w-0 space-y-3">
 
 
 
 
 
381
  {#each card.formulaBlocks as block (`${card.title}-${block.kind}`)}
382
  <div
383
  class="max-w-full overflow-x-auto rounded-xl border bg-muted/40 p-4"
@@ -398,7 +338,7 @@
398
  title: Message,
399
  cards: FormulaCard[],
400
  children?: import('svelte').Snippet,
401
- gridClass = 'md:grid-cols-2 xl:grid-cols-3'
402
  )}
403
  <section class="mt-8 min-w-0" aria-label={title()}>
404
  <h2 class="font-mono mb-6 font-semibold uppercase tracking-wider text-muted-foreground">
@@ -412,18 +352,6 @@
412
  {@render FormulaBlock(card)}
413
  {/if}
414
  <p class="mt-4 text-sm leading-6 text-muted-foreground">{card.description()}</p>
415
- {#if card.example}
416
- <div class="mt-auto pt-6">
417
- <p
418
- class="min-w-0 rounded-xl border border-[#0dce97]/30 bg-[#0dce97]/10 px-3 py-2 font-mono text-xs leading-6 break-words text-[#0dce97]"
419
- >
420
- <span class="mb-1 block text-xs font-semibold uppercase tracking-wider">
421
- {m.methodology_example_label()}
422
- </span>
423
- {@render RichText(card.example())}
424
- </p>
425
- </div>
426
- {/if}
427
  </article>
428
  {/each}
429
  </div>
@@ -433,11 +361,7 @@
433
 
434
  {#snippet MathFormula(kind: FormulaKind)}
435
  {#if kind === 'dataset-score'}
436
- <math
437
- display="block"
438
- class="min-w-max text-xl text-foreground"
439
- xmlns="http://www.w3.org/1998/Math/MathML"
440
- >
441
  <msub><mi>S</mi><mi>ds</mi></msub>
442
  <mo>=</mo>
443
  <mi>H</mi><mo>(</mo><mn>1</mn><mo>−</mo><mi>FPR</mi><mo>,</mo><mn>1</mn><mo>−</mo><mi>FNR</mi
@@ -456,11 +380,7 @@
456
  </mfrac>
457
  </math>
458
  {:else if kind === 'group-score'}
459
- <math
460
- display="block"
461
- class="min-w-max text-xl text-foreground"
462
- xmlns="http://www.w3.org/1998/Math/MathML"
463
- >
464
  <msub><mi>S</mi><mi>group</mi></msub>
465
  <mo>=</mo>
466
  <mi>H</mi><mo>(</mo><msub><mi>S</mi><msub><mi>ds</mi><mn>1</mn></msub></msub><mo>,</mo><mo
@@ -476,11 +396,7 @@
476
  </mfrac>
477
  </math>
478
  {:else if kind === 'integral-product'}
479
- <math
480
- display="block"
481
- class="min-w-max text-xl text-foreground"
482
- xmlns="http://www.w3.org/1998/Math/MathML"
483
- >
484
  <msub><mi>S</mi><mi>integral</mi></msub>
485
  <mo>=</mo>
486
  <msup>
@@ -494,11 +410,7 @@
494
  </msup>
495
  </math>
496
  {:else if kind === 'integral-log'}
497
- <math
498
- display="block"
499
- class="min-w-max text-xl text-foreground"
500
- xmlns="http://www.w3.org/1998/Math/MathML"
501
- >
502
  <msub><mi>S</mi><mi>integral</mi></msub>
503
  <mo>=</mo>
504
  <mi>exp</mi>
@@ -508,67 +420,8 @@
508
  <mi>ln</mi><mo>(</mo><msub><mi>S</mi><msub><mi>group</mi><mi>j</mi></msub></msub><mo>)</mo>
509
  <mo>)</mo>
510
  </math>
511
- {:else if kind === 'fbeta'}
512
- <math
513
- display="block"
514
- class="min-w-max text-lg text-foreground"
515
- xmlns="http://www.w3.org/1998/Math/MathML"
516
- >
517
- <msub><mi>F</mi><mi>β</mi></msub>
518
- <mo>=</mo>
519
- <mo>(</mo><mn>1</mn><mo>+</mo><msup><mi>β</mi><mn>2</mn></msup><mo>)</mo>
520
- <mo>·</mo>
521
- <mfrac>
522
- <mrow><mi>P</mi><mo>·</mo><mi>R</mi></mrow>
523
- <mrow><msup><mi>β</mi><mn>2</mn></msup><mo>·</mo><mi>P</mi><mo>+</mo><mi>R</mi></mrow>
524
- </mfrac>
525
- </math>
526
- {:else if kind === 'f2'}
527
- <math
528
- display="block"
529
- class="min-w-max text-lg text-foreground"
530
- xmlns="http://www.w3.org/1998/Math/MathML"
531
- >
532
- <msub><mi>F</mi><mn>2</mn></msub>
533
- <mo>=</mo>
534
- <mn>5</mn><mo>·</mo>
535
- <mfrac>
536
- <mrow><mi>P</mi><mo>·</mo><mi>R</mi></mrow>
537
- <mrow><mn>4</mn><mo>·</mo><mi>P</mi><mo>+</mo><mi>R</mi></mrow>
538
- </mfrac>
539
- </math>
540
- {:else if kind === 'f05'}
541
- <math
542
- display="block"
543
- class="min-w-max text-lg text-foreground"
544
- xmlns="http://www.w3.org/1998/Math/MathML"
545
- >
546
- <msub><mi>F</mi><mn>0.5</mn></msub>
547
- <mo>=</mo>
548
- <mn>1.25</mn><mo>·</mo>
549
- <mfrac>
550
- <mrow><mi>P</mi><mo>·</mo><mi>R</mi></mrow>
551
- <mrow><mn>0.25</mn><mo>·</mo><mi>P</mi><mo>+</mo><mi>R</mi></mrow>
552
- </mfrac>
553
- </math>
554
- {:else if kind === 'fdr'}
555
- <math
556
- display="block"
557
- class="min-w-max text-lg text-foreground"
558
- xmlns="http://www.w3.org/1998/Math/MathML"
559
- >
560
- <mi>FDR</mi>
561
- <mo>=</mo>
562
- <mfrac><mi>FP</mi><mrow><mi>TP</mi><mo>+</mo><mi>FP</mi></mrow></mfrac>
563
- <mo>=</mo>
564
- <mn>1</mn><mo>−</mo><mi>Precision</mi>
565
- </math>
566
  {:else if kind === 'fnr-harm'}
567
- <math
568
- display="block"
569
- class="min-w-max text-lg text-foreground"
570
- xmlns="http://www.w3.org/1998/Math/MathML"
571
- >
572
  <msub><mi>FNR</mi><mi>harm</mi></msub>
573
  <mo>=</mo>
574
  <mfrac><mi>FN</mi><mrow><mi>FN</mi><mo>+</mo><mi>TP</mi></mrow></mfrac>
@@ -576,71 +429,15 @@
576
  <mn>1</mn><mo>−</mo><msub><mi>Recall</mi><mi>harm</mi></msub>
577
  </math>
578
  {:else if kind === 'fpr-safe'}
579
- <math
580
- display="block"
581
- class="min-w-max text-lg text-foreground"
582
- xmlns="http://www.w3.org/1998/Math/MathML"
583
- >
584
  <msub><mi>FPR</mi><mi>safe</mi></msub>
585
  <mo>=</mo>
586
  <mfrac><mi>FP</mi><mrow><mi>FP</mi><mo>+</mo><mi>TN</mi></mrow></mfrac>
587
  <mo>=</mo>
588
  <mn>1</mn><mo>−</mo><msub><mi>Specificity</mi><mi>safe</mi></msub>
589
  </math>
590
- {:else if kind === 'micro-f1'}
591
- <math
592
- display="block"
593
- class="min-w-max text-lg text-foreground"
594
- xmlns="http://www.w3.org/1998/Math/MathML"
595
- >
596
- <msub><mi>F1</mi><mi>micro</mi></msub>
597
- <mo>=</mo>
598
- <mfrac>
599
- <mrow
600
- ><mn>2</mn><mo>·</mo><msub><mi>P</mi><mi>micro</mi></msub><mo>·</mo><msub
601
- ><mi>R</mi><mi>micro</mi></msub
602
- ></mrow
603
- >
604
- <mrow
605
- ><msub><mi>P</mi><mi>micro</mi></msub><mo>+</mo><msub><mi>R</mi><mi>micro</mi></msub
606
- ></mrow
607
- >
608
- </mfrac>
609
- </math>
610
- {:else if kind === 'micro-pr'}
611
- <math
612
- display="block"
613
- class="min-w-max text-lg text-foreground"
614
- xmlns="http://www.w3.org/1998/Math/MathML"
615
- >
616
- <msub><mi>P</mi><mi>micro</mi></msub>
617
- <mo>=</mo>
618
- <mfrac>
619
- <mrow><mo>∑</mo><msub><mi>TP</mi><mi>i</mi></msub></mrow>
620
- <mrow
621
- ><mo>∑</mo><mo>(</mo><msub><mi>TP</mi><mi>i</mi></msub><mo>+</mo><msub
622
- ><mi>FP</mi><mi>i</mi></msub
623
- ><mo>)</mo></mrow
624
- >
625
- </mfrac>
626
- <mspace width="0.75em" />
627
- <msub><mi>R</mi><mi>micro</mi></msub>
628
- <mo>=</mo>
629
- <mfrac>
630
- <mrow><mo>∑</mo><msub><mi>TP</mi><mi>i</mi></msub></mrow>
631
- <mrow
632
- ><mo>∑</mo><mo>(</mo><msub><mi>TP</mi><mi>i</mi></msub><mo>+</mo><msub
633
- ><mi>FN</mi><mi>i</mi></msub
634
- ><mo>)</mo></mrow
635
- >
636
- </mfrac>
637
- </math>
638
  {:else if kind === 'fpr-avg'}
639
- <math
640
- display="block"
641
- class="min-w-max text-lg text-foreground"
642
- xmlns="http://www.w3.org/1998/Math/MathML"
643
- >
644
  <msub><mi>FPR</mi><mi>avg</mi></msub>
645
  <mo>=</mo>
646
  <mfrac><mn>1</mn><mi>N</mi></mfrac>
@@ -648,11 +445,7 @@
648
  <msub><mi>FPR</mi><mi>i</mi></msub>
649
  </math>
650
  {:else if kind === 'fnr-avg'}
651
- <math
652
- display="block"
653
- class="min-w-max text-lg text-foreground"
654
- xmlns="http://www.w3.org/1998/Math/MathML"
655
- >
656
  <msub><mi>FNR</mi><mi>avg</mi></msub>
657
  <mo>=</mo>
658
  <mfrac><mn>1</mn><mi>N</mi></mfrac>
 
9
  | 'group-score'
10
  | 'integral-product'
11
  | 'integral-log'
 
 
 
 
12
  | 'fnr-harm'
13
  | 'fpr-safe'
 
 
14
  | 'fpr-avg'
15
  | 'fnr-avg';
16
 
 
24
  tag?: Message;
25
  formulaBlocks: FormulaBlockData[];
26
  description: Message;
 
27
  wide?: boolean;
28
  };
29
 
 
44
  title: 'S_ds',
45
  tag: m.methodology_dataset_score_tag,
46
  formulaBlocks: [{ kind: 'dataset-score', comment: m.methodology_dataset_score_comment }],
47
+ description: m.methodology_dataset_score_description
 
48
  },
49
  {
50
  title: 'S_group',
51
  tag: m.methodology_group_score_tag,
52
  formulaBlocks: [{ kind: 'group-score', comment: m.methodology_group_score_comment }],
53
+ description: m.methodology_group_score_description
 
54
  },
55
  {
56
  title: 'S_integral',
 
59
  { kind: 'integral-product', comment: m.methodology_integral_score_comment },
60
  { kind: 'integral-log', comment: m.methodology_integral_log_comment }
61
  ],
62
+ description: m.methodology_integral_score_description
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
  }
64
  ];
65
 
 
75
  tag: m.methodology_safe_split_tag,
76
  formulaBlocks: [{ kind: 'fpr-safe' }],
77
  description: m.methodology_safe_split_description
 
 
 
 
 
 
 
78
  }
79
  ];
80
 
 
92
  {
93
  title: m.methodology_scatter_point_title(),
94
  formulaBlocks: [],
95
+ description: m.methodology_scatter_point_description,
96
+ wide: true
97
  }
98
  ];
99
 
 
108
  title: m.methodology_caveat_title,
109
  description: m.methodology_caveat_description,
110
  muted: true
 
 
 
 
 
111
  }
112
  ];
113
 
114
  function cardClass(card: FormulaCard) {
115
  return cn(
116
+ 'flex min-w-0 max-w-full flex-col overflow-hidden rounded-2xl border bg-card p-5 shadow-sm',
117
+ card.wide && 'lg:col-span-2'
118
+ );
119
+ }
120
+
121
+ function aggregationCardClass() {
122
+ return cn(
123
+ 'flex min-w-0 max-w-full flex-col overflow-hidden rounded-2xl border bg-card p-5 shadow-sm'
124
  );
125
  }
126
 
127
+ function formulaClass(kind: FormulaKind) {
128
  return cn(
129
+ 'min-w-max text-foreground',
130
+ ['dataset-score', 'group-score', 'integral-product', 'integral-log'].includes(kind)
131
+ ? 'text-xl'
132
+ : 'text-lg'
133
  );
134
  }
135
 
 
170
  >
171
  {m.methodology_aggregation_title()}
172
  </h2>
173
+ <div class="mt-3 grid min-w-0 grid-cols-1 gap-4">
174
  {#each aggregationCards as card (card.title)}
175
+ <article class={aggregationCardClass()}>
176
  <div class="flex min-w-0 flex-wrap items-start justify-between gap-3">
177
  <h2
178
  class="flex min-w-0 flex-wrap items-baseline gap-3 font-mono text-2xl font-semibold tracking-normal"
 
190
  <span class="mr-1 font-semibold text-foreground">{m.methodology_meaning_label()}</span>
191
  {card.description()}
192
  </p>
 
 
 
 
 
 
 
 
 
 
 
 
193
  </article>
194
  {/each}
195
  </div>
196
  </section>
197
 
198
  {@render FormulaSection(
199
+ m.methodology_combined_dataset_title,
200
+ combinedCards,
201
+ CombinedDatasetNote,
202
+ 'grid-cols-1 lg:grid-cols-2'
203
  )}
204
 
 
 
205
  {#snippet CombinedDatasetNote()}
206
  <p class="mt-4 text-sm leading-6 text-muted-foreground">
207
  {m.methodology_combined_dataset_note()}
208
  </p>
209
  {/snippet}
210
 
211
+ {@render FormulaSection(
212
+ m.methodology_scatter_title,
213
+ scatterCards,
214
+ undefined,
215
+ 'grid-cols-1 lg:grid-cols-2'
216
+ )}
217
 
218
  <section class="mt-8 min-w-0 space-y-4" aria-labelledby="specifics-title">
219
  <h2
 
255
  </div>
256
  {/each}
257
  </section>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
258
  </div>
259
 
260
  {#snippet RichText(value: string)}
 
312
  {/snippet}
313
 
314
  {#snippet FormulaBlock(card: FormulaCard)}
315
+ <div
316
+ class={cn(
317
+ 'mt-4 grid min-w-0 grid-cols-1 gap-3',
318
+ card.formulaBlocks.length > 1 && 'lg:grid-cols-2'
319
+ )}
320
+ >
321
  {#each card.formulaBlocks as block (`${card.title}-${block.kind}`)}
322
  <div
323
  class="max-w-full overflow-x-auto rounded-xl border bg-muted/40 p-4"
 
338
  title: Message,
339
  cards: FormulaCard[],
340
  children?: import('svelte').Snippet,
341
+ gridClass = 'grid-cols-1'
342
  )}
343
  <section class="mt-8 min-w-0" aria-label={title()}>
344
  <h2 class="font-mono mb-6 font-semibold uppercase tracking-wider text-muted-foreground">
 
352
  {@render FormulaBlock(card)}
353
  {/if}
354
  <p class="mt-4 text-sm leading-6 text-muted-foreground">{card.description()}</p>
 
 
 
 
 
 
 
 
 
 
 
 
355
  </article>
356
  {/each}
357
  </div>
 
361
 
362
  {#snippet MathFormula(kind: FormulaKind)}
363
  {#if kind === 'dataset-score'}
364
+ <math display="block" class={formulaClass(kind)} xmlns="http://www.w3.org/1998/Math/MathML">
 
 
 
 
365
  <msub><mi>S</mi><mi>ds</mi></msub>
366
  <mo>=</mo>
367
  <mi>H</mi><mo>(</mo><mn>1</mn><mo>−</mo><mi>FPR</mi><mo>,</mo><mn>1</mn><mo>−</mo><mi>FNR</mi
 
380
  </mfrac>
381
  </math>
382
  {:else if kind === 'group-score'}
383
+ <math display="block" class={formulaClass(kind)} xmlns="http://www.w3.org/1998/Math/MathML">
 
 
 
 
384
  <msub><mi>S</mi><mi>group</mi></msub>
385
  <mo>=</mo>
386
  <mi>H</mi><mo>(</mo><msub><mi>S</mi><msub><mi>ds</mi><mn>1</mn></msub></msub><mo>,</mo><mo
 
396
  </mfrac>
397
  </math>
398
  {:else if kind === 'integral-product'}
399
+ <math display="block" class={formulaClass(kind)} xmlns="http://www.w3.org/1998/Math/MathML">
 
 
 
 
400
  <msub><mi>S</mi><mi>integral</mi></msub>
401
  <mo>=</mo>
402
  <msup>
 
410
  </msup>
411
  </math>
412
  {:else if kind === 'integral-log'}
413
+ <math display="block" class={formulaClass(kind)} xmlns="http://www.w3.org/1998/Math/MathML">
 
 
 
 
414
  <msub><mi>S</mi><mi>integral</mi></msub>
415
  <mo>=</mo>
416
  <mi>exp</mi>
 
420
  <mi>ln</mi><mo>(</mo><msub><mi>S</mi><msub><mi>group</mi><mi>j</mi></msub></msub><mo>)</mo>
421
  <mo>)</mo>
422
  </math>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
423
  {:else if kind === 'fnr-harm'}
424
+ <math display="block" class={formulaClass(kind)} xmlns="http://www.w3.org/1998/Math/MathML">
 
 
 
 
425
  <msub><mi>FNR</mi><mi>harm</mi></msub>
426
  <mo>=</mo>
427
  <mfrac><mi>FN</mi><mrow><mi>FN</mi><mo>+</mo><mi>TP</mi></mrow></mfrac>
 
429
  <mn>1</mn><mo>−</mo><msub><mi>Recall</mi><mi>harm</mi></msub>
430
  </math>
431
  {:else if kind === 'fpr-safe'}
432
+ <math display="block" class={formulaClass(kind)} xmlns="http://www.w3.org/1998/Math/MathML">
 
 
 
 
433
  <msub><mi>FPR</mi><mi>safe</mi></msub>
434
  <mo>=</mo>
435
  <mfrac><mi>FP</mi><mrow><mi>FP</mi><mo>+</mo><mi>TN</mi></mrow></mfrac>
436
  <mo>=</mo>
437
  <mn>1</mn><mo>−</mo><msub><mi>Specificity</mi><mi>safe</mi></msub>
438
  </math>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
439
  {:else if kind === 'fpr-avg'}
440
+ <math display="block" class={formulaClass(kind)} xmlns="http://www.w3.org/1998/Math/MathML">
 
 
 
 
441
  <msub><mi>FPR</mi><mi>avg</mi></msub>
442
  <mo>=</mo>
443
  <mfrac><mn>1</mn><mi>N</mi></mfrac>
 
445
  <msub><mi>FPR</mi><mi>i</mi></msub>
446
  </math>
447
  {:else if kind === 'fnr-avg'}
448
+ <math display="block" class={formulaClass(kind)} xmlns="http://www.w3.org/1998/Math/MathML">
 
 
 
 
449
  <msub><mi>FNR</mi><mi>avg</mi></msub>
450
  <mo>=</mo>
451
  <mfrac><mn>1</mn><mi>N</mi></mfrac>
src/routes/tools/+page.svelte CHANGED
@@ -5,7 +5,6 @@
5
  import BucketDataWarning from '$lib/components/BucketDataWarning.svelte';
6
  import PageHeadingSection from '$lib/components/PageHeadingSection.svelte';
7
  import Select from '$lib/components/Select.svelte';
8
- import Sheet from '$lib/components/Sheet.svelte';
9
  import Table, { type TableColumn } from '$lib/components/Table.svelte';
10
  import { Badge } from '$lib/components/ui/badge';
11
  import { Button } from '$lib/components/ui/button';
@@ -14,16 +13,9 @@
14
  toolsActiveGroups,
15
  toolsBarLabelLines as barLabelLines,
16
  toolsBarY as barY,
17
- toolsBestModel as bestModel,
18
- toolsCell as cellOf,
19
  toolsCsvDataUri,
20
  toolsDataGroups,
21
  toolsHeatColor as heatColor,
22
- toolsHeatmapMetricArrow as heatmapMetricArrow,
23
- toolsHeatmapMetricItems as heatmapMetricItems,
24
- toolsHeatmapMetricTone as heatmapMetricTone,
25
- toolsHeatmapMetricValue as heatmapMetricValue,
26
- toolsMetricValue as metricValue,
27
  toolsModelColor,
28
  toolsModelColorMap,
29
  toolsModelKey as modelKey,
@@ -34,15 +26,12 @@
34
  toolsRadarPoint as radarPoint,
35
  toolsScatterX as scatterX,
36
  toolsScatterY as scatterY,
37
- toolsSelectedModels,
38
- toolsValueTone as valueTone
39
  } from '$lib/data/tools';
40
  import { isNumber } from '$lib/helpers/leaderboard';
41
  import type { GuardModel } from '$lib/types/leaderboard';
42
  import type {
43
  ToolsChartTooltip as ChartTooltip,
44
- ToolsHeatmapMetric as HeatmapMetric,
45
- ToolsHeatmapSelection as HeatmapSelection,
46
  ToolsLanguageFilter as LangFilter,
47
  ToolsLatencySortKey as LatencySortKey,
48
  ToolsRadarScale as RadarScale,
@@ -78,6 +67,7 @@
78
  p50: number | null;
79
  p95: number | null;
80
  p99: number | null;
 
81
  };
82
  type RobustnessRow = VisualizationModelRow & {
83
  scoreReal: number | null;
@@ -89,6 +79,10 @@
89
  category: string;
90
  };
91
  type HeatmapRow = ToolsDataset['visualizations']['heatmap']['rows'][number];
 
 
 
 
92
  const selectContentClass = 'max-h-72';
93
  let { data }: PageProps = $props();
94
  const toolsState = $derived(data.toolsState);
@@ -105,9 +99,7 @@
105
  let language = $state<LangFilter>('all');
106
  let metric = $state<VizMetric>('fnr');
107
  let radarScale = $state<RadarScale>('score');
108
- let pinnedModelKeys = $state<string[]>(
109
- initialTools.models[0] ? [modelKey(initialTools.models[0])] : []
110
- );
111
  let latencySortKey = $state<LatencySortKey>('p95');
112
  let latencySortDirection = $state<SortDirection>('asc');
113
  let chartTooltip = $state<ChartTooltip | null>(null);
@@ -116,8 +108,6 @@
116
  let hoveredScatterModelKey = $state<string | null>(null);
117
  let hoveredBarKey = $state<string | null>(null);
118
  let hoveredParetoModelKey = $state<string | null>(null);
119
- let heatmapSheetOpen = $state(false);
120
- let heatmapSelection = $state<HeatmapSelection | null>(null);
121
 
122
  const languageOptions = [
123
  { value: 'all', label: m.tools_language_all() },
@@ -136,7 +126,8 @@
136
  { key: 'model', label: m.ranking_column_model },
137
  { key: 'p50', label: () => `p50 (${m.ranking_unit_ms()})`, cellClass: 'text-right' },
138
  { key: 'p95', label: () => `p95 (${m.ranking_unit_ms()})`, cellClass: 'text-right' },
139
- { key: 'p99', label: () => `p99 (${m.ranking_unit_ms()})`, cellClass: 'text-right' }
 
140
  ];
141
 
142
  const currentLanguageLabel = $derived(
@@ -150,9 +141,6 @@
150
  const selectedModelKeySet = $derived(new Set(selectedModelKeys));
151
  const activeGroups = $derived(toolsActiveGroups(selectedGroups, language, tools.groupLanguages));
152
  const pinnedModels = $derived(toolsPinnedModels(models, pinnedModelKeys));
153
- const topIntegral = $derived(bestModel(selectedModels, 'integral', 'desc'));
154
- const topLatency = $derived(bestModel(selectedModels, 'p95', 'asc'));
155
- const topFnr = $derived(bestModel(selectedModels, 'fnr', 'asc'));
156
  const heatmapMetric = $derived(tools.visualizations.heatmap.metric);
157
  const heatmapModels = $derived(
158
  tools.visualizations.heatmap.columnModelKeys.flatMap((key) => {
@@ -166,6 +154,7 @@
166
  activeGroups.includes(row.groupKey) && (language === 'all' || row.language === language)
167
  )
168
  );
 
169
  const heatColumns = $derived(
170
  `minmax(8rem, 10rem) repeat(${heatmapModels.length}, minmax(4rem, 1fr))`
171
  );
@@ -209,6 +198,10 @@
209
  return isNumber(value) ? value.toFixed(1) : '-';
210
  }
211
 
 
 
 
 
212
  function localized(value: LocalizedString | null | undefined, fallback: string) {
213
  return value?.[getLocale()] ?? value?.en ?? fallback;
214
  }
@@ -282,7 +275,8 @@
282
  integral: row.integral,
283
  p50: row.latencyMs.p50,
284
  p95: row.latencyMs.p95,
285
- p99: row.latencyMs.p99
 
286
  }
287
  ];
288
  });
@@ -401,25 +395,21 @@
401
  return tools.visualizations.heatmap.values[row.id]?.[heatmapModelKey] ?? null;
402
  }
403
 
404
- function heatmapDrilldownFor(modelKeyValue: string, row: HeatmapRow) {
405
- return tools.drilldownIndex.heatmap.find(
406
- (item) => item.modelKey === modelKeyValue && item.groupKey === row.groupKey
 
 
 
 
 
407
  );
 
 
408
  }
409
 
410
- function fallbackHeatmapMetrics(value: number | null, key: HeatmapMetric) {
411
- return {
412
- score: key === 'score' ? value : null,
413
- f1: key === 'f1' ? value : null,
414
- fpr: key === 'fpr' ? value : null,
415
- fnr: key === 'fnr' ? value : null
416
- };
417
- }
418
-
419
- function heatColorValue(value: number | null | undefined, key: HeatmapMetric) {
420
- if (!isNumber(value)) return value;
421
- if (key === 'score' || key === 'f1') return 1 - value;
422
- return value;
423
  }
424
 
425
  function colorOf(model: GuardModel) {
@@ -491,94 +481,6 @@
491
  chartTooltip = null;
492
  }
493
 
494
- function scatterDrilldownGroup(model: GuardModel) {
495
- const rows = activeGroups
496
- .map((group) => {
497
- const cell = cellOf(model, group);
498
- const value = metricValue(cell, metric);
499
- return cell && isNumber(value) ? { group, value } : null;
500
- })
501
- .filter((row): row is { group: string; value: number } => row !== null);
502
-
503
- if (!rows.length) return null;
504
-
505
- return rows.sort((a, b) => {
506
- if (metric === 'f1') return a.value - b.value;
507
- return b.value - a.value;
508
- })[0].group;
509
- }
510
-
511
- function openScatterDrilldownSheet(model: GuardModel) {
512
- const group = scatterDrilldownGroup(model);
513
- if (!group) return;
514
- const key = modelKey(model);
515
- const rows = heatmapRows
516
- .map((row) => ({ row, value: heatmapValue(row, key) }))
517
- .filter(
518
- (item): item is { row: HeatmapRow; value: number } =>
519
- item.row.groupKey === group && isNumber(item.value)
520
- );
521
- const selected = rows.sort((a, b) => {
522
- if (heatmapMetric === 'score' || heatmapMetric === 'f1') return a.value - b.value;
523
- return b.value - a.value;
524
- })[0];
525
-
526
- if (!selected) return;
527
- openHeatmapSheet(model, selected.row, selected.value);
528
- }
529
-
530
- function handleScatterPointKeydown(event: KeyboardEvent, model: GuardModel) {
531
- if (event.key !== 'Enter' && event.key !== ' ') return;
532
- event.preventDefault();
533
- openScatterDrilldownSheet(model);
534
- }
535
-
536
- function openHeatmapSheet(model: GuardModel, row: HeatmapRow, value: number | null) {
537
- if (!isNumber(value)) return;
538
-
539
- const key = modelKey(model);
540
- const drilldown = heatmapDrilldownFor(key, row);
541
-
542
- hideTooltip();
543
- heatmapSelection = {
544
- model,
545
- group: row.groupKey,
546
- rowLabel: heatmapRowLabel(row),
547
- metric: heatmapMetric,
548
- value,
549
- metrics: drilldown?.metrics ?? fallbackHeatmapMetrics(value, heatmapMetric)
550
- };
551
- heatmapSheetOpen = true;
552
- }
553
-
554
- function handleHeatmapCellKeydown(
555
- event: KeyboardEvent,
556
- model: GuardModel,
557
- row: HeatmapRow,
558
- value: number | null
559
- ) {
560
- if (event.key !== 'Enter' && event.key !== ' ') return;
561
- event.preventDefault();
562
- openHeatmapSheet(model, row, value);
563
- }
564
-
565
- function tooltipToneClass(tone: TooltipRow['tone']) {
566
- if (tone === 'good') return 'text-[#0dce97]';
567
- if (tone === 'bad') return 'text-red-400';
568
- return 'text-foreground';
569
- }
570
-
571
- function heatmapKpiClass(value: number | null | undefined, key: HeatmapMetric) {
572
- const tone = heatmapMetricTone(value, key);
573
- if (tone === 'good') {
574
- return 'border-transparent bg-[#0dce97]/10 text-[#6bffd1]';
575
- }
576
- if (tone === 'bad') {
577
- return 'border-transparent bg-red-500/10 text-red-300';
578
- }
579
- return 'border-transparent bg-transparent px-0 text-foreground';
580
- }
581
-
582
  function toggleLatencySort(key: LatencySortKey) {
583
  if (latencySortKey === key) {
584
  latencySortDirection = latencySortDirection === 'asc' ? 'desc' : 'asc';
@@ -599,19 +501,13 @@
599
  }
600
 
601
  function deltaBadgeClass(value: number | null, kind: 'score' | 'fnr') {
602
- if (!isNumber(value) || Math.abs(value) <= 0.005) {
603
  return 'bg-muted/40 text-muted-foreground';
604
  }
605
  const worse = kind === 'score' ? value < 0 : value > 0;
606
  return worse ? 'bg-red-500/15 text-red-300' : 'bg-[#0dce97]/10 text-[#0dce97]';
607
  }
608
 
609
- function deltaArrow(value: number | null, kind: 'score' | 'fnr') {
610
- if (!isNumber(value) || Math.abs(value) <= 0.005) return '–';
611
- const worse = kind === 'score' ? value < 0 : value > 0;
612
- return worse ? '▼' : '▲';
613
- }
614
-
615
  function fmtDelta(value: number | null) {
616
  if (!isNumber(value)) return '-';
617
  if (value > 0) return `+${value.toFixed(3)}`;
@@ -636,40 +532,8 @@
636
  {/if}
637
  <PageHeadingSection title={m.navigation_visualizations()} description={m.tools_description()} />
638
 
639
- <div
640
- class="mt-6 mb-5 flex items-center gap-3 overflow-x-auto rounded-xl border border-[#0dce97]/25 bg-[#0dce97]/10 px-4 py-3 text-sm whitespace-nowrap text-[#0dce97]"
641
- role="status"
642
- aria-live="polite"
643
- >
644
- {#if !selectedModels.length}
645
- <span class="text-muted-foreground">{m.tools_takeaway_empty()}</span>
646
- {:else if topIntegral}
647
- <span>
648
- <span class="font-semibold">{m.tools_takeaway_integral()}</span>
649
- <b>{topIntegral.short}</b>
650
- <code>{fmt(topIntegral.integral, 3)}</code>
651
- </span>
652
- {/if}
653
- {#if selectedModels.length && topLatency}
654
- <span class="text-muted-foreground">·</span>
655
- <span>
656
- <span class="font-semibold">{m.tools_takeaway_p95()}</span>
657
- <b>{topLatency.short}</b>
658
- <code>{Math.round(topLatency.p95)} {m.ranking_unit_ms()}</code>
659
- </span>
660
- {/if}
661
- {#if selectedModels.length && topFnr}
662
- <span class="text-muted-foreground">·</span>
663
- <span>
664
- <span class="font-semibold">{m.tools_takeaway_fnr()}</span>
665
- <b>{topFnr.short}</b>
666
- <code>{fmt(topFnr.fnr, 3)}</code>
667
- </span>
668
- {/if}
669
- </div>
670
-
671
  <section
672
- class="mt-2 min-w-0 rounded-xl border bg-card p-4 shadow-sm"
673
  aria-label={m.tools_controls_aria()}
674
  >
675
  <div class="grid min-w-0 gap-4 lg:grid-cols-[1.1fr_1.1fr_.7fr]">
@@ -863,8 +727,10 @@
863
  </div>
864
  </section>
865
 
866
- <div class="mt-4 flex flex-wrap items-center gap-2 rounded-xl border bg-card px-4 py-3 text-xs">
867
- <span class="font-semibold tracking-wider text-muted-foreground uppercase"
 
 
868
  >{m.tools_pinned_for_comparison()}</span
869
  >
870
  {#if pinnedModels.length}
@@ -882,13 +748,13 @@
882
  </Badge>
883
  {/each}
884
  {:else}
885
- <span class="text-muted-foreground">
886
  {m.tools_no_pinned()}
887
  </span>
888
  {/if}
889
  </div>
890
 
891
- <div class="mt-6 grid min-w-0 gap-4 lg:grid-cols-2">
892
  <section class="min-w-0 rounded-xl border bg-card p-4 shadow-sm">
893
  <div class="mb-3 flex flex-wrap items-baseline gap-2">
894
  <h2 class="font-mono text-sm font-semibold">{m.tools_radar_title()}</h2>
@@ -983,7 +849,7 @@
983
  hoveredRadarModelKey = key;
984
  updateTooltip(event, model.short, [
985
  { label: groupLabel(group), value: radarScale.toUpperCase() },
986
- { label: 'value', value: fmt(value, 3), tone: valueTone(value, true) }
987
  ]);
988
  }}
989
  onpointermove={moveTooltip}
@@ -1115,17 +981,14 @@
1115
  fill-opacity="0.85"
1116
  stroke="currentColor"
1117
  stroke-width={isPinned ? 2 : 1}
1118
- class="cursor-pointer text-background transition-[r]"
1119
- role="button"
1120
- tabindex="0"
1121
  aria-label={`${row.model.short} FPR ${fmt(row.fpr, 3)} FNR ${fmt(row.fnr, 3)}`}
1122
- onclick={() => openScatterDrilldownSheet(row.model)}
1123
- onkeydown={(event) => handleScatterPointKeydown(event, row.model)}
1124
  onpointerenter={(event) => {
1125
  hoveredScatterModelKey = row.modelKey;
1126
  updateTooltip(event, row.model.short, [
1127
- { label: 'FPR', value: fmt(row.fpr, 3), tone: valueTone(row.fpr, false) },
1128
- { label: 'FNR', value: fmt(row.fnr, 3), tone: valueTone(row.fnr, false) }
1129
  ]);
1130
  }}
1131
  onpointermove={moveTooltip}
@@ -1148,7 +1011,6 @@
1148
  </svg>
1149
  <div class="mt-3 flex flex-wrap gap-4 text-xs text-muted-foreground">
1150
  <span class="text-[#0dce97]">{m.tools_scatter_ideal()}</span>
1151
- <span>{m.tools_scatter_click()}</span>
1152
  </div>
1153
  </section>
1154
 
@@ -1157,12 +1019,11 @@
1157
  <h2 class="font-mono text-sm font-semibold">
1158
  Heatmap <span class="font-mono uppercase">{heatmapMetric}</span>
1159
  </h2>
1160
- <span class="text-xs text-muted-foreground">{m.tools_heatmap_hint()}</span>
1161
  <span class="ml-auto inline-flex items-center gap-2 text-xs text-muted-foreground">
1162
- 0.0 <span
1163
- class="h-2 w-28 rounded-full bg-[linear-gradient(90deg,#0dce97,#ffca28,#ff4d4f)]"
1164
  ></span>
1165
- {m.tools_heatmap_scale()}
1166
  </span>
1167
  </div>
1168
  <div class="max-w-full overflow-auto">
@@ -1194,41 +1055,40 @@
1194
  {@const key = modelKey(model)}
1195
  {@const value = heatmapValue(row, key)}
1196
  {@const hasValue = isNumber(value)}
1197
- <button
1198
- type="button"
1199
- disabled={!hasValue}
1200
  class={cn(
1201
- 'w-full rounded-md border-0 px-2 py-2 text-center font-mono text-sm font-semibold transition duration-150 disabled:cursor-default disabled:opacity-100',
1202
- hasValue &&
1203
- 'cursor-pointer hover:brightness-110 hover:saturate-125 focus-visible:scale-[1.02] focus-visible:brightness-110 focus-visible:saturate-125 focus-visible:outline-none',
1204
  !isNumber(value) && 'bg-muted/70 text-muted-foreground',
1205
  isNumber(value) && heatTextColor(value)
1206
  )}
1207
  style={isNumber(value)
1208
- ? `background:${heatColor(heatColorValue(value, heatmapMetric))}`
 
 
 
 
 
1209
  : ''}
1210
  aria-label={`${model.short} ${heatmapRowLabel(row)} ${heatmapMetric.toUpperCase()} ${fmt(value, 3)}`}
1211
- onclick={() => openHeatmapSheet(model, row, value)}
1212
- onkeydown={(event) => handleHeatmapCellKeydown(event, model, row, value)}
1213
  onpointerenter={(event) =>
1214
  updateTooltip(event, model.short, [
1215
  { label: groupLabel(row.groupKey), value: heatmapRowLabel(row) },
1216
  {
1217
  label: heatmapMetric.toUpperCase(),
1218
- value: fmt(value, 3),
1219
- tone: valueTone(value, heatmapMetric === 'f1' || heatmapMetric === 'score')
1220
  }
1221
  ])}
1222
  onpointermove={moveTooltip}
1223
  onpointerleave={hideTooltip}
1224
  >
1225
  {isNumber(value) ? fmt(value, 2) : '-'}
1226
- </button>
1227
  {/each}
1228
  {/each}
1229
  </div>
1230
  </div>
1231
- <p class="mt-3 text-xs text-muted-foreground">{m.tools_heatmap_note()}</p>
1232
  </section>
1233
 
1234
  <section class="min-w-0 rounded-xl border bg-card p-4 shadow-sm lg:col-span-2">
@@ -1292,8 +1152,7 @@
1292
  updateTooltip(event, groupLabel(group), [
1293
  {
1294
  label: series.key.toUpperCase(),
1295
- value: fmt(value, 3),
1296
- tone: valueTone(value, series.key === 'f1')
1297
  },
1298
  {
1299
  label: m.tools_mean(),
@@ -1437,13 +1296,9 @@
1437
  onpointerenter={(event) => {
1438
  hoveredParetoModelKey = row.modelKey;
1439
  updateTooltip(event, row.model.short, [
1440
- {
1441
- label: 'integral',
1442
- value: fmt(row.integral, 3),
1443
- tone: valueTone(row.integral, true)
1444
- },
1445
  { label: 'p95', value: `${fmtMs(row.p95)} ${m.ranking_unit_ms()}` },
1446
- { label: 'FPR', value: fmt(row.fpr, 3), tone: valueTone(row.fpr, false) }
1447
  ]);
1448
  }}
1449
  onpointermove={moveTooltip}
@@ -1519,7 +1374,7 @@
1519
  columns={latencyColumns}
1520
  getRowKey={(row) => row.modelKey}
1521
  containerClass="max-w-full overflow-x-auto rounded-lg border"
1522
- tableClass="min-w-[42rem]"
1523
  headerClass="bg-muted/70 hover:bg-muted/70 [&_th]:text-muted-foreground"
1524
  rowClass="hover:bg-muted/30"
1525
  >
@@ -1545,6 +1400,8 @@
1545
  <span class="min-w-0 max-w-80 truncate">{row.model.short}</span>
1546
  <code class="text-muted-foreground">{row.model.params}</code>
1547
  </span>
 
 
1548
  {:else}
1549
  <span class="font-mono">{fmtMs(row[column.key])}</span>
1550
  {/if}
@@ -1560,6 +1417,7 @@
1560
  </h2>
1561
  <span class="text-xs text-muted-foreground">{m.tools_robustness_hint()}</span>
1562
  </div>
 
1563
  <div class="max-w-full overflow-x-auto rounded-lg border">
1564
  <table class="w-full min-w-[54rem] text-sm">
1565
  <thead class="bg-muted/70 text-xs text-muted-foreground">
@@ -1614,7 +1472,6 @@
1614
  deltaBadgeClass(sDelta, 'score')
1615
  )}
1616
  >
1617
- {deltaArrow(sDelta, 'score')}
1618
  {fmtDelta(sDelta)}
1619
  </span>
1620
  </td>
@@ -1627,7 +1484,6 @@
1627
  deltaBadgeClass(fDelta, 'fnr')
1628
  )}
1629
  >
1630
- {deltaArrow(fDelta, 'fnr')}
1631
  {fmtDelta(fDelta)}
1632
  </span>
1633
  </td>
@@ -1636,7 +1492,6 @@
1636
  </tbody>
1637
  </table>
1638
  </div>
1639
- <p class="mt-3 text-xs text-muted-foreground">{m.tools_robustness_note()}</p>
1640
  </section>
1641
  </div>
1642
 
@@ -1644,82 +1499,6 @@
1644
  {m.tools_footer()}
1645
  </p>
1646
 
1647
- <Sheet
1648
- bind:open={heatmapSheetOpen}
1649
- titleText={heatmapSelection ? heatmapSelection.rowLabel : ''}
1650
- contentClass="lg:data-[side=right]:w-[44rem]!"
1651
- titleClass="text-2xl"
1652
- >
1653
- {#if heatmapSelection}
1654
- {@const meta = benchmarkMeta[heatmapSelection.group]}
1655
- <div class="space-y-6 p-4 sm:p-5">
1656
- <div class="flex flex-wrap items-center gap-3">
1657
- <Badge variant="secondary" class="gap-2 rounded-full px-3 py-1">
1658
- <span
1659
- class="size-2.5 rounded-sm"
1660
- style={`background:${colorOf(heatmapSelection.model)}`}
1661
- ></span>
1662
- {heatmapSelection.model.short}
1663
- </Badge>
1664
- {#if meta?.languages.length}
1665
- <Badge variant="secondary" class="rounded-full px-3 py-1 font-mono uppercase">
1666
- {meta.languages.map((lang) => lang.toUpperCase()).join(' · ')}
1667
- </Badge>
1668
- {/if}
1669
- <Badge variant="outline" class="rounded-full px-3 py-1 font-mono">
1670
- {groupLabel(heatmapSelection.group)}
1671
- </Badge>
1672
- </div>
1673
-
1674
- <div class="grid gap-2 sm:grid-cols-2 xl:grid-cols-4">
1675
- {#each heatmapMetricItems(heatmapSelection) as item (item.label)}
1676
- <div class="rounded-lg border bg-card p-3 shadow-sm">
1677
- <div
1678
- class={cn(
1679
- 'inline-flex items-center gap-1.5 rounded-md border px-2.5 py-1 font-mono text-xl font-semibold tabular-nums',
1680
- heatmapKpiClass(item.value, item.key)
1681
- )}
1682
- >
1683
- {heatmapMetricArrow(item.value, item.key)}
1684
- {fmt(item.value, 3)}
1685
- </div>
1686
- <div
1687
- class="mt-2 text-[11px] font-semibold tracking-wider text-muted-foreground uppercase"
1688
- >
1689
- {item.label}
1690
- </div>
1691
- </div>
1692
- {/each}
1693
- </div>
1694
-
1695
- {#if meta?.whatItTests}
1696
- <p class="rounded-xl border bg-muted/30 p-4 text-sm leading-6 text-muted-foreground">
1697
- {localized(meta.whatItTests, '')}
1698
- </p>
1699
- {/if}
1700
-
1701
- <section class="border-t pt-5">
1702
- <h3 class="font-mono text-base font-semibold">
1703
- {m.tools_heatmap_sheet_prompts_title()}
1704
- </h3>
1705
- <p class="mt-3 text-sm leading-6 text-muted-foreground">
1706
- {m.tools_heatmap_sheet_prompts_description({
1707
- metric: heatmapSelection.metric.toUpperCase(),
1708
- value: fmt(
1709
- heatmapMetricValue(heatmapSelection.metrics, heatmapSelection.metric) ??
1710
- heatmapSelection.value,
1711
- 3
1712
- )
1713
- })}
1714
- </p>
1715
- <ul class="mt-4 list-disc space-y-1 pl-6 font-mono text-sm text-muted-foreground">
1716
- <li>{heatmapSelection.rowLabel}</li>
1717
- </ul>
1718
- </section>
1719
- </div>
1720
- {/if}
1721
- </Sheet>
1722
-
1723
  {#if chartTooltip}
1724
  <div
1725
  bind:this={chartTooltipElement}
@@ -1731,7 +1510,7 @@
1731
  {#each chartTooltip.rows as row (`${row.label}-${row.value}`)}
1732
  <div class="grid grid-cols-[auto_1fr] gap-5">
1733
  <span class="break-words text-muted-foreground">{row.label}</span>
1734
- <span class={cn('text-right', tooltipToneClass(row.tone))}>{row.value}</span>
1735
  </div>
1736
  {/each}
1737
  </div>
 
5
  import BucketDataWarning from '$lib/components/BucketDataWarning.svelte';
6
  import PageHeadingSection from '$lib/components/PageHeadingSection.svelte';
7
  import Select from '$lib/components/Select.svelte';
 
8
  import Table, { type TableColumn } from '$lib/components/Table.svelte';
9
  import { Badge } from '$lib/components/ui/badge';
10
  import { Button } from '$lib/components/ui/button';
 
13
  toolsActiveGroups,
14
  toolsBarLabelLines as barLabelLines,
15
  toolsBarY as barY,
 
 
16
  toolsCsvDataUri,
17
  toolsDataGroups,
18
  toolsHeatColor as heatColor,
 
 
 
 
 
19
  toolsModelColor,
20
  toolsModelColorMap,
21
  toolsModelKey as modelKey,
 
26
  toolsRadarPoint as radarPoint,
27
  toolsScatterX as scatterX,
28
  toolsScatterY as scatterY,
29
+ toolsSelectedModels
 
30
  } from '$lib/data/tools';
31
  import { isNumber } from '$lib/helpers/leaderboard';
32
  import type { GuardModel } from '$lib/types/leaderboard';
33
  import type {
34
  ToolsChartTooltip as ChartTooltip,
 
 
35
  ToolsLanguageFilter as LangFilter,
36
  ToolsLatencySortKey as LatencySortKey,
37
  ToolsRadarScale as RadarScale,
 
67
  p50: number | null;
68
  p95: number | null;
69
  p99: number | null;
70
+ errorRate: number | null;
71
  };
72
  type RobustnessRow = VisualizationModelRow & {
73
  scoreReal: number | null;
 
79
  category: string;
80
  };
81
  type HeatmapRow = ToolsDataset['visualizations']['heatmap']['rows'][number];
82
+ type HeatmapColorScale = {
83
+ min: number | null;
84
+ max: number | null;
85
+ };
86
  const selectContentClass = 'max-h-72';
87
  let { data }: PageProps = $props();
88
  const toolsState = $derived(data.toolsState);
 
99
  let language = $state<LangFilter>('all');
100
  let metric = $state<VizMetric>('fnr');
101
  let radarScale = $state<RadarScale>('score');
102
+ let pinnedModelKeys = $state<string[]>([]);
 
 
103
  let latencySortKey = $state<LatencySortKey>('p95');
104
  let latencySortDirection = $state<SortDirection>('asc');
105
  let chartTooltip = $state<ChartTooltip | null>(null);
 
108
  let hoveredScatterModelKey = $state<string | null>(null);
109
  let hoveredBarKey = $state<string | null>(null);
110
  let hoveredParetoModelKey = $state<string | null>(null);
 
 
111
 
112
  const languageOptions = [
113
  { value: 'all', label: m.tools_language_all() },
 
126
  { key: 'model', label: m.ranking_column_model },
127
  { key: 'p50', label: () => `p50 (${m.ranking_unit_ms()})`, cellClass: 'text-right' },
128
  { key: 'p95', label: () => `p95 (${m.ranking_unit_ms()})`, cellClass: 'text-right' },
129
+ { key: 'p99', label: () => `p99 (${m.ranking_unit_ms()})`, cellClass: 'text-right' },
130
+ { key: 'errorRate', label: m.details_error_rate, cellClass: 'text-right' }
131
  ];
132
 
133
  const currentLanguageLabel = $derived(
 
141
  const selectedModelKeySet = $derived(new Set(selectedModelKeys));
142
  const activeGroups = $derived(toolsActiveGroups(selectedGroups, language, tools.groupLanguages));
143
  const pinnedModels = $derived(toolsPinnedModels(models, pinnedModelKeys));
 
 
 
144
  const heatmapMetric = $derived(tools.visualizations.heatmap.metric);
145
  const heatmapModels = $derived(
146
  tools.visualizations.heatmap.columnModelKeys.flatMap((key) => {
 
154
  activeGroups.includes(row.groupKey) && (language === 'all' || row.language === language)
155
  )
156
  );
157
+ const heatmapColorScale = $derived.by(() => heatmapScaleFor(heatmapRows, heatmapModels));
158
  const heatColumns = $derived(
159
  `minmax(8rem, 10rem) repeat(${heatmapModels.length}, minmax(4rem, 1fr))`
160
  );
 
198
  return isNumber(value) ? value.toFixed(1) : '-';
199
  }
200
 
201
+ function fmtPercent(value: number | null | undefined) {
202
+ return isNumber(value) ? `${(value * 100).toFixed(1)}%` : '-';
203
+ }
204
+
205
  function localized(value: LocalizedString | null | undefined, fallback: string) {
206
  return value?.[getLocale()] ?? value?.en ?? fallback;
207
  }
 
275
  integral: row.integral,
276
  p50: row.latencyMs.p50,
277
  p95: row.latencyMs.p95,
278
+ p99: row.latencyMs.p99,
279
+ errorRate: row.errorRate
280
  }
281
  ];
282
  });
 
395
  return tools.visualizations.heatmap.values[row.id]?.[heatmapModelKey] ?? null;
396
  }
397
 
398
+ function heatmapScaleFor(
399
+ rows: readonly HeatmapRow[],
400
+ visibleModels: readonly GuardModel[]
401
+ ): HeatmapColorScale {
402
+ const values = rows.flatMap((row) =>
403
+ visibleModels
404
+ .map((model) => heatmapValue(row, modelKey(model)))
405
+ .filter((value): value is number => isNumber(value))
406
  );
407
+ if (!values.length) return { min: null, max: null };
408
+ return { min: Math.min(...values), max: Math.max(...values) };
409
  }
410
 
411
+ function heatmapScaleLabel(value: number | null) {
412
+ return isNumber(value) ? fmt(value, 3) : '-';
 
 
 
 
 
 
 
 
 
 
 
413
  }
414
 
415
  function colorOf(model: GuardModel) {
 
481
  chartTooltip = null;
482
  }
483
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
484
  function toggleLatencySort(key: LatencySortKey) {
485
  if (latencySortKey === key) {
486
  latencySortDirection = latencySortDirection === 'asc' ? 'desc' : 'asc';
 
501
  }
502
 
503
  function deltaBadgeClass(value: number | null, kind: 'score' | 'fnr') {
504
+ if (!isNumber(value) || value === 0) {
505
  return 'bg-muted/40 text-muted-foreground';
506
  }
507
  const worse = kind === 'score' ? value < 0 : value > 0;
508
  return worse ? 'bg-red-500/15 text-red-300' : 'bg-[#0dce97]/10 text-[#0dce97]';
509
  }
510
 
 
 
 
 
 
 
511
  function fmtDelta(value: number | null) {
512
  if (!isNumber(value)) return '-';
513
  if (value > 0) return `+${value.toFixed(3)}`;
 
532
  {/if}
533
  <PageHeadingSection title={m.navigation_visualizations()} description={m.tools_description()} />
534
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
535
  <section
536
+ class="mt-6 min-w-0 rounded-xl border bg-card p-4 shadow-sm"
537
  aria-label={m.tools_controls_aria()}
538
  >
539
  <div class="grid min-w-0 gap-4 lg:grid-cols-[1.1fr_1.1fr_.7fr]">
 
727
  </div>
728
  </section>
729
 
730
+ <div
731
+ class="mt-4 flex min-h-[46px] flex-wrap items-center gap-x-2 gap-y-3 rounded-xl border bg-card px-4 py-3 text-xs leading-none"
732
+ >
733
+ <span class="font-semibold tracking-wider text-muted-foreground uppercase leading-none"
734
  >{m.tools_pinned_for_comparison()}</span
735
  >
736
  {#if pinnedModels.length}
 
748
  </Badge>
749
  {/each}
750
  {:else}
751
+ <span class="leading-none text-muted-foreground">
752
  {m.tools_no_pinned()}
753
  </span>
754
  {/if}
755
  </div>
756
 
757
+ <div class="mt-4 grid min-w-0 gap-4 lg:grid-cols-2">
758
  <section class="min-w-0 rounded-xl border bg-card p-4 shadow-sm">
759
  <div class="mb-3 flex flex-wrap items-baseline gap-2">
760
  <h2 class="font-mono text-sm font-semibold">{m.tools_radar_title()}</h2>
 
849
  hoveredRadarModelKey = key;
850
  updateTooltip(event, model.short, [
851
  { label: groupLabel(group), value: radarScale.toUpperCase() },
852
+ { label: 'value', value: fmt(value, 3) }
853
  ]);
854
  }}
855
  onpointermove={moveTooltip}
 
981
  fill-opacity="0.85"
982
  stroke="currentColor"
983
  stroke-width={isPinned ? 2 : 1}
984
+ class="text-background transition-[r]"
985
+ role="img"
 
986
  aria-label={`${row.model.short} FPR ${fmt(row.fpr, 3)} FNR ${fmt(row.fnr, 3)}`}
 
 
987
  onpointerenter={(event) => {
988
  hoveredScatterModelKey = row.modelKey;
989
  updateTooltip(event, row.model.short, [
990
+ { label: 'FPR', value: fmt(row.fpr, 3) },
991
+ { label: 'FNR', value: fmt(row.fnr, 3) }
992
  ]);
993
  }}
994
  onpointermove={moveTooltip}
 
1011
  </svg>
1012
  <div class="mt-3 flex flex-wrap gap-4 text-xs text-muted-foreground">
1013
  <span class="text-[#0dce97]">{m.tools_scatter_ideal()}</span>
 
1014
  </div>
1015
  </section>
1016
 
 
1019
  <h2 class="font-mono text-sm font-semibold">
1020
  Heatmap <span class="font-mono uppercase">{heatmapMetric}</span>
1021
  </h2>
 
1022
  <span class="ml-auto inline-flex items-center gap-2 text-xs text-muted-foreground">
1023
+ {m.tools_heatmap_scale_min({ value: heatmapScaleLabel(heatmapColorScale.min) })}
1024
+ <span class="h-2 w-28 rounded-full bg-[linear-gradient(90deg,#0dce97,#ffca28,#ff4d4f)]"
1025
  ></span>
1026
+ {m.tools_heatmap_scale_max({ value: heatmapScaleLabel(heatmapColorScale.max) })}
1027
  </span>
1028
  </div>
1029
  <div class="max-w-full overflow-auto">
 
1055
  {@const key = modelKey(model)}
1056
  {@const value = heatmapValue(row, key)}
1057
  {@const hasValue = isNumber(value)}
1058
+ <div
1059
+ role="img"
 
1060
  class={cn(
1061
+ 'w-full rounded-md px-2 py-2 text-center font-mono text-sm font-semibold',
1062
+ hasValue && 'transition duration-150 hover:brightness-110 hover:saturate-125',
 
1063
  !isNumber(value) && 'bg-muted/70 text-muted-foreground',
1064
  isNumber(value) && heatTextColor(value)
1065
  )}
1066
  style={isNumber(value)
1067
+ ? `background:${heatColor(
1068
+ value,
1069
+ heatmapColorScale.min ?? value,
1070
+ heatmapColorScale.max ?? value,
1071
+ 'sqrt'
1072
+ )}`
1073
  : ''}
1074
  aria-label={`${model.short} ${heatmapRowLabel(row)} ${heatmapMetric.toUpperCase()} ${fmt(value, 3)}`}
 
 
1075
  onpointerenter={(event) =>
1076
  updateTooltip(event, model.short, [
1077
  { label: groupLabel(row.groupKey), value: heatmapRowLabel(row) },
1078
  {
1079
  label: heatmapMetric.toUpperCase(),
1080
+ value: fmt(value, 3)
 
1081
  }
1082
  ])}
1083
  onpointermove={moveTooltip}
1084
  onpointerleave={hideTooltip}
1085
  >
1086
  {isNumber(value) ? fmt(value, 2) : '-'}
1087
+ </div>
1088
  {/each}
1089
  {/each}
1090
  </div>
1091
  </div>
 
1092
  </section>
1093
 
1094
  <section class="min-w-0 rounded-xl border bg-card p-4 shadow-sm lg:col-span-2">
 
1152
  updateTooltip(event, groupLabel(group), [
1153
  {
1154
  label: series.key.toUpperCase(),
1155
+ value: fmt(value, 3)
 
1156
  },
1157
  {
1158
  label: m.tools_mean(),
 
1296
  onpointerenter={(event) => {
1297
  hoveredParetoModelKey = row.modelKey;
1298
  updateTooltip(event, row.model.short, [
1299
+ { label: 'integral', value: fmt(row.integral, 3) },
 
 
 
 
1300
  { label: 'p95', value: `${fmtMs(row.p95)} ${m.ranking_unit_ms()}` },
1301
+ { label: 'FPR', value: fmt(row.fpr, 3) }
1302
  ]);
1303
  }}
1304
  onpointermove={moveTooltip}
 
1374
  columns={latencyColumns}
1375
  getRowKey={(row) => row.modelKey}
1376
  containerClass="max-w-full overflow-x-auto rounded-lg border"
1377
+ tableClass="min-w-[50rem]"
1378
  headerClass="bg-muted/70 hover:bg-muted/70 [&_th]:text-muted-foreground"
1379
  rowClass="hover:bg-muted/30"
1380
  >
 
1400
  <span class="min-w-0 max-w-80 truncate">{row.model.short}</span>
1401
  <code class="text-muted-foreground">{row.model.params}</code>
1402
  </span>
1403
+ {:else if column.key === 'errorRate'}
1404
+ <span class="font-mono">{fmtPercent(row.errorRate)}</span>
1405
  {:else}
1406
  <span class="font-mono">{fmtMs(row[column.key])}</span>
1407
  {/if}
 
1417
  </h2>
1418
  <span class="text-xs text-muted-foreground">{m.tools_robustness_hint()}</span>
1419
  </div>
1420
+ <p class="mb-3 text-xs leading-5 text-muted-foreground">{m.tools_robustness_note()}</p>
1421
  <div class="max-w-full overflow-x-auto rounded-lg border">
1422
  <table class="w-full min-w-[54rem] text-sm">
1423
  <thead class="bg-muted/70 text-xs text-muted-foreground">
 
1472
  deltaBadgeClass(sDelta, 'score')
1473
  )}
1474
  >
 
1475
  {fmtDelta(sDelta)}
1476
  </span>
1477
  </td>
 
1484
  deltaBadgeClass(fDelta, 'fnr')
1485
  )}
1486
  >
 
1487
  {fmtDelta(fDelta)}
1488
  </span>
1489
  </td>
 
1492
  </tbody>
1493
  </table>
1494
  </div>
 
1495
  </section>
1496
  </div>
1497
 
 
1499
  {m.tools_footer()}
1500
  </p>
1501
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1502
  {#if chartTooltip}
1503
  <div
1504
  bind:this={chartTooltipElement}
 
1510
  {#each chartTooltip.rows as row (`${row.label}-${row.value}`)}
1511
  <div class="grid grid-cols-[auto_1fr] gap-5">
1512
  <span class="break-words text-muted-foreground">{row.label}</span>
1513
+ <span class="text-right">{row.value}</span>
1514
  </div>
1515
  {/each}
1516
  </div>