Buckets:

---

runawaydevil

/

Alvorada-bench-bucket

This dataset contains 4,515 multiple-choice questions from five major Brazilian university entrance exams (ENEM, FUVEST, UNICAMP, ITA, IME) spanning 32 years (1981-2025), along with model responses from 20 LLMs.

Files

📄 questions_data.csv (4,515 rows)

Contains the exam questions with:

• question_id: Unique identifier
• question_statement: Question text in Portuguese
• correct_answer: Correct option (A-E)
• alternative_a to alternative_e: Answer choices
• subject: Academic subject
• exam_name, exam_year, exam_type: Exam metadata

📄 responses_data.csv

Contains model responses with:

• model: Model name (o3, deepseek-reasoner, claude-opus-4-20250514)
• prompt_template: Prompting strategy used (zero-shot, role-playing, chain-of-thought)
• chosen_answer: Model's selected answer
• is_correct: Whether the answer was correct
• difficulty_level, uncertainty_level: Model's self-reported metrics (1-10 scale)
• bloom_taxonomy: Cognitive complexity classification
• Additional metadata matching questions_data

Cite

@misc{godoy2025alvoradabenchlanguagemodelssolve,
      title={Alvorada-Bench: Can Language Models Solve Brazilian University Entrance Exams?}, 
      author={Henrique Godoy},
      year={2025},
      eprint={2508.15835},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2508.15835}, 
}