NBR-1B-Portuguese-MCQ

O melhor modelo ~1B parametros para questoes de multipla escolha em Portugues.

Highlights

• #1 em ENEM (30.53%) - Vestibular brasileiro
• #1 em OAB (49.70%) - Exame da Ordem dos Advogados
• #1 em BLUEX (36.54%) - Vestibulares universitarios
• #1 em tweetSentBR (39.70%) - Analise de sentimento

Benchmarks

Benchmark	Score	Ranking
ENEM	30.53%	#1
OAB Exams	49.70%	#1
BLUEX	36.54%	#1
TweetSentBR	39.70%	#1
FAQUAD NLI	45.55%	Top 3
HateBR	43.18%	Top 5
PT Hate Speech	41.99%	Top 5
ASSIN2 RTE	34.27%	-
ASSIN2 STS	0.99%	-
Average	35.83%	-

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("limajr/NBR-1B-Portuguese-MCQ")
tokenizer = AutoTokenizer.from_pretrained("limajr/NBR-1B-Portuguese-MCQ")

Model Details

• Architecture: LlamaForCausalLM
• Parameters: ~1.5B
• Hidden Size: 2048
• Layers: 24
• Attention Heads: 16
• Language: Portuguese (pt-BR)
• Training: Supervised Fine-Tuning on Brazilian educational content
• License: Apache 2.0

Training Data

Fine-tuned on curated Portuguese datasets including:

• Brazilian educational materials
• Legal texts (OAB preparation)
• General knowledge QA

Evaluation

Evaluated on the Open PT LLM Leaderboard using the standard evaluation protocol.

Limitations

• Optimized for multiple choice questions
• ASSIN2 STS (semantic similarity) performance is limited
• Best used for Portuguese educational contexts