Instructions to use panzs19/LEMMA-LLAMA-3-70B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use panzs19/LEMMA-LLAMA-3-70B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="panzs19/LEMMA-LLAMA-3-70B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("panzs19/LEMMA-LLAMA-3-70B")
model = AutoModelForCausalLM.from_pretrained("panzs19/LEMMA-LLAMA-3-70B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use panzs19/LEMMA-LLAMA-3-70B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "panzs19/LEMMA-LLAMA-3-70B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "panzs19/LEMMA-LLAMA-3-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/panzs19/LEMMA-LLAMA-3-70B

SGLang

How to use panzs19/LEMMA-LLAMA-3-70B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "panzs19/LEMMA-LLAMA-3-70B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "panzs19/LEMMA-LLAMA-3-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "panzs19/LEMMA-LLAMA-3-70B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "panzs19/LEMMA-LLAMA-3-70B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use panzs19/LEMMA-LLAMA-3-70B with Docker Model Runner:
```
docker model run hf.co/panzs19/LEMMA-LLAMA-3-70B
```

panzs19

nielsr HF Staff commited on Sep 15, 2025

Commit

09ce844

verified ·

1 Parent(s): 8ff8130

Add library name and pipeline tag (#1)

Browse files

- Add library name and pipeline tag (6d0120be4b0eb3f29c294d4d8904b0737a4b136d)

Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

Files changed (1) hide show

README.md +16 -1

README.md CHANGED Viewed

@@ -1,8 +1,23 @@
 ---
 license: llama3
 ---
 # Model Card for Model ID
 <!-- Provide a quick summary of what the model is/does. -->
 The LEMMA series models are trained on the [LEMMA Dataset](https://huggingface.co/datasets/panzs19/LEMMA). This dataset uses the training set of MATH and GSM8K to generate error-corrective reasoning trajectories. For each question in these datasets, the student model (LLaMA3-8B) generates self-generated errors, and the teacher model (GPT-4o) deliberately introduces errors based on the error type distribution of the student model. Then, both "Fix & Continue" and "Fresh & Restart" correction strategies are applied to these errors to create error-corrective revision trajectories. After filtering out trajectories with incorrect final answers, we obtain this dataset. Fine-tuning on this dataset achieves up to 13.3% average accuracy improvement for LLaMA3-8B with less than 90k synthesized data. For more details, please refer to our paper [LEMMA: Learning from Errors for MatheMatical Advancement in LLMs](https://arxiv.org/abs/2503.17439).
@@ -48,4 +63,4 @@ Please cite the paper if you refer to our model, code, data or paper from MetaMa
   journal={arXiv preprint arXiv:2503.17439},
   year={2025}
 }
-```

 ---
 license: llama3
+library_name: transformers
+pipeline_tag: text-generation
 ---
 # Model Card for Model ID
+**Key Takeaways**
+💡 **Systematic analysis on error types**: Categorizes common model-generated mathematical reasoning errors, revealing consistent error patterns across models and guiding targeted improvements.
+💡 **Error-type grounded error augmentation**: Introduces diverse and meaningful errors by leveraging a teacher model to _intentionally inject representative mistakes_ with type sampled from the analyzed distribution, enhancing the model’s ability to learn from failures.
+💡 **Two complementary self-correction mechanisms**: Combines _Fix & Continue_ (correcting mistakes within the original reasoning) and _Fresh & Restart_ (restarting the reasoning process from scratch) to generate effective revision trajectories.
+✅ **LEMMA** – A novel framework that fine-tunes LLMs on error-corrective trajectories, enabling autonomous error detection and correction during mathematical reasoning.
+📊 **Result** – Up to 13.3% accuracy improvement for LLaMA3-8B with only 90k synthesized data.
 <!-- Provide a quick summary of what the model is/does. -->
 The LEMMA series models are trained on the [LEMMA Dataset](https://huggingface.co/datasets/panzs19/LEMMA). This dataset uses the training set of MATH and GSM8K to generate error-corrective reasoning trajectories. For each question in these datasets, the student model (LLaMA3-8B) generates self-generated errors, and the teacher model (GPT-4o) deliberately introduces errors based on the error type distribution of the student model. Then, both "Fix & Continue" and "Fresh & Restart" correction strategies are applied to these errors to create error-corrective revision trajectories. After filtering out trajectories with incorrect final answers, we obtain this dataset. Fine-tuning on this dataset achieves up to 13.3% average accuracy improvement for LLaMA3-8B with less than 90k synthesized data. For more details, please refer to our paper [LEMMA: Learning from Errors for MatheMatical Advancement in LLMs](https://arxiv.org/abs/2503.17439).
   journal={arXiv preprint arXiv:2503.17439},
   year={2025}
 }
+```