--- title: CBE Results Explorer emoji: 🧮 colorFrom: blue colorTo: gray sdk: static app_file: index.html pinned: false short_description: LLM accuracy and consistency on ordering, n=4 to 1024 --- # CBE Results Explorer Interactive results for **Competence-Based Evaluation of Logical Reasoning in Large Language Models** (Jize Jiang and Chengxiang Zhai, University of Illinois Urbana-Champaign) — [arXiv:2610.22894](https://arxiv.org/abs/2610.22894). Each problem lists facts of the form "A is in front of B" for *n* people in a line (n = 4 to 1024) and asks a yes/no question about the order of the two people at the ends of the line. Every problem also comes in an equivalent rewording with the same answer. The page shows, per model: - the share of item pairs answered correctly in both versions, by problem size; - a two-model comparison as *n* grows; - accuracy and answer consistency under perturbations (premise rewording, the share of premises restated, alternative relation wordings), next to the model's repeat agreement when the identical prompt is sent again; - a sortable table of all model configurations. The numbers are computed from the archived model answers of the paper's experiments, and only for models and perturbations the paper reports. Pair measures count a missing or unparseable answer as a failure, so they can differ from the paper's tables, which use only pairs where both answers parse. The data behind the charts is in `data/`. ```bibtex @misc{jiang2026cbe, title = {Competence-Based Evaluation of Logical Reasoning in Large Language Models}, author = {Jiang, Jize and Zhai, Chengxiang}, year = {2026}, eprint = {2610.22894}, archivePrefix = {arXiv}, primaryClass = {cs.CL}, url = {https://arxiv.org/abs/2610.22894} } ```