shootstuff/home / eval.yaml
shootstuff's picture
download
raw
532 Bytes
name: Typed Decisions
description: >
A benchmark for typed probabilistic decisions. A model gets one piece of
unstructured state and answers five typed questions about it at once, and
every answer is a probability distribution, not a single label.
tasks:
- id: accuracy
config: all
split: test
- id: kl_from_gold
config: all
split: test
lower_is_better: true
- id: brier
config: all
split: test
lower_is_better: true
- id: ece
config: all
split: test
lower_is_better: true

Xet Storage Details

Size:
532 Bytes
·
Xet hash:
a16ffb5ad7eb92d8bdae134d1a5f1c2f9d72a7ab8c134fed5d9e59da45be0069

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.