nlile/hendrycks-MATH-benchmark
Viewer โข Updated โข 12.5k โข 9.88k โข 34
How to use EricAI03/distilbert-base-math-finetuned-accelerate with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("fill-mask", model="EricAI03/distilbert-base-math-finetuned-accelerate") # pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("EricAI03/distilbert-base-math-finetuned-accelerate")
model = AutoModelForMaskedLM.from_pretrained("EricAI03/distilbert-base-math-finetuned-accelerate", device_map="auto")This model is a fine-tuned version of distilbert-base-uncased trained on the nlile/hendrycks-MATH-benchmark dataset. It has been adapted using whole-word masking and fixed-length chunking to understand mathematical syntax, equations, numbers, and LaTeX expressions.
DistilBertForMaskedLM)accelerate with mixed-precision (fp16)Evaluated on the official Hendrycks MATH test split:
You can load and use this model directly with the Hugging Face pipeline for masked language modeling:
from transformers import AutoModelForMaskedLM, AutoTokenizer, pipeline
# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(
"EricAI03/distilbert-base-math-finetuned-accelerate"
)
model = AutoModelForMaskedLM.from_pretrained(
"EricAI03/distilbert-base-math-finetuned-accelerate"
)
# Create fill-mask pipeline
math_pipe = pipeline("fill-mask", model=model, tokenizer=tokenizer)
# Test prompt
prompt = "If a triangle has sides 3, 4, and [MASK], it is a right-angled triangle."
results = math_pipe(prompt, top_k=3)
for res in results:
print(
f"Token: '{res['token_str']}' (Score: {res['score']:.4f}) | Sentence:"
f" {res['sequence']}"
)
Base model
distilbert/distilbert-base-uncased