Download README.md from jizej/CBE: direct link, hf CLI and curl.
- Browser
- Download file 1.83 kB
-
https://huggingface.co/spaces/jizej/CBE/resolve/main/README.md
- Command line
-
hf download hf://spaces/jizej/CBE/README.md
-
curl -L -o README.md https://huggingface.co/spaces/jizej/CBE/resolve/main/README.md
title: CBE Results Explorer
emoji: 🧮
colorFrom: blue
colorTo: gray
sdk: static
app_file: index.html
pinned: false
short_description: LLM accuracy and consistency on ordering, n=4 to 1024
CBE Results Explorer
Interactive results for Competence-Based Evaluation of Logical Reasoning in Large Language Models (Jize Jiang and Chengxiang Zhai, University of Illinois Urbana-Champaign) — arXiv:2610.22894.
Each problem lists facts of the form "A is in front of B" for n people in a line (n = 4 to 1024) and asks a yes/no question about the order of the two people at the ends of the line. Every problem also comes in an equivalent rewording with the same answer. The page shows, per model:
- the share of item pairs answered correctly in both versions, by problem size;
- a two-model comparison as n grows;
- accuracy and answer consistency under perturbations (premise rewording, the share of premises restated, alternative relation wordings), next to the model's repeat agreement when the identical prompt is sent again;
- a sortable table of all model configurations.
The numbers are computed from the archived model answers of the paper's experiments, and only for
models and perturbations the paper reports. Pair measures count a missing or unparseable answer as a
failure, so they can differ from the paper's tables, which use only pairs where both answers parse.
The data behind the charts is in data/.
@misc{jiang2026cbe,
title = {Competence-Based Evaluation of Logical Reasoning in Large Language Models},
author = {Jiang, Jize and Zhai, Chengxiang},
year = {2026},
eprint = {2610.22894},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2610.22894}
}