| | ---
|
| | library_name: transformers
|
| | datasets:
|
| | - HuggingFaceH4/MATH-500
|
| | language:
|
| | - en
|
| | base_model:
|
| | - LiquidAI/LFM2-350M-Math
|
| | ---
|
| | # LFM2-350M-Math - Fine-tuned |
| |
|
| | ## Model Description |
| | This is a **LoRA fine-tuned version** of the `LiquidAI/LFM2-350M-Math` model on the **HuggingFaceH4/MATH-500** dataset. |
| | The model is designed to answer math questions and generate step-by-step solutions in natural language. |
| |
|
| | --- |
| |
|
| | ## Intended Uses |
| | - Solve math problems directly in natural language. |
| | - Serve as a base for further fine-tuning on other math datasets. |
| | - Educational tools, tutoring systems, or research in automated math reasoning. |
| |
|
| | --- |
| |
|
| | ## Out-of-Scope Uses |
| | - Not intended for general reasoning beyond mathematics. |
| | - May fail or hallucinate on complex, unseen problem types. |
| | - Should not be used for critical calculations without verification. |
| |
|
| | --- |
| |
|
| | ## Limitations & Biases |
| | - The training dataset is small (500 problems), so the model may **overfit**. |
| | - Step-by-step reasoning is learned from dataset patterns, not true reasoning. |
| | - Accuracy can vary; always verify outputs for correctness. |
| |
|
| | --- |
| |
|
| | ## Training Details |
| | - **Base Model:** LiquidAI/LFM2-350M-Math |
| | - **Dataset:** HuggingFaceH4/MATH-500 |
| | - **Fine-tuning libraries:** transformers, datasets, peft, accelerate, bitsandbytes |
| | - **Training arguments:** |
| | - num_train_epochs: 3 |
| | - per_device_train_batch_size: 4 |
| | - gradient_accumulation_steps: 4 |
| | - learning_rate: 2e-4 |
| | - fp16: True |
| | |
| | --- |
| | |
| | ## Usage |
| | |
| | ```python |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | from peft import PeftModel |
| | |
| | base_model = "LiquidAI/LFM2-350M-Math" |
| | adapter_repo = "Nrex/lfm2-math-finetuned" |
| | |
| | tokenizer = AutoTokenizer.from_pretrained(adapter_repo) |
| | model = AutoModelForCausalLM.from_pretrained(base_model) |
| | model = PeftModel.from_pretrained(model, adapter_repo) |
| | |
| | inputs = tokenizer("What is 12*13?", return_tensors="pt") |
| | outputs = model.generate(**inputs) |
| | print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| | ``` |
| | --- |
| | # Evaluation |
| | |
| | - Tested on a held-out 10% split of the dataset. |
| | - Small dataset; outputs should be verified for correctness. |
| | - Accuracy is limited by the dataset size |
