| | --- |
| | license: apache-2.0 |
| | base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B |
| | tags: |
| | - reinforcement-learning |
| | - grpo |
| | - peft |
| | - lora |
| | - beam-mechanics |
| | - structural-engineering |
| | - math |
| | - reasoning |
| | language: |
| | - en |
| | pipeline_tag: text-generation |
| | datasets: |
| | - lamm-mit/BeamRL-TrainData |
| | - lamm-mit/BeamRL-EvalData |
| | --- |
| | |
| | # BeamPERL — DeepSeek-R1-Distill-Qwen-1.5B |
| |
|
| | **BeamPERL** is a parameter-efficient, reinforcement-learning fine-tuned language model specialized in beam mechanics problem-solving. It is built on top of [DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) using LoRA adapters trained with Group Relative Policy Optimization (GRPO) and verifiable reward signals. |
| |
|
| | ## Model Details |
| |
|
| | | Property | Value | |
| | |---|---| |
| | | Base model | `tphage/DeepSeek-R1-Distill-Qwen-1.5B` | |
| | | Fine-tuning method | GRPO (RL) + LoRA (PEFT) | |
| | | LoRA rank / alpha | 32 / 128 | |
| | | LoRA dropout | 0.05 | |
| | | LoRA target modules | q, k, v, o, gate, up, down projections | |
| | | Training precision | bfloat16 | |
| | | Max sequence length | 2048 tokens (256 prompt + 1792 completion) | |
| | | Training dataset | `tphage/BeamRL-TrainData` (synthetic beam mechanics QA) | |
| |
|
| | ### Reward Functions |
| |
|
| | | Reward | Weight | Description | |
| | |---|---|---| |
| | | Accuracy | 0.667 | Correctness of predicted reaction forces / coefficients | |
| | | Format | 0.333 | Requires reasoning in `<think>` tags and answer in `\boxed{}` | |
| |
|
| | ## Usage |
| |
|
| | ```python |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | |
| | model = AutoModelForCausalLM.from_pretrained("tphage/BeamPERL", torch_dtype="bfloat16", device_map="auto") |
| | tokenizer = AutoTokenizer.from_pretrained("tphage/BeamPERL") |
| | |
| | prompt = "Determine the reaction forces at the pin support (x=0.0*L) and the roller support (x=9.0*L) for a statically loaded beam with a length of 9*L, a point load of -13*P at x=3.0*L, and supports at x=0.0*L (pin) and x=9.0*L (roller)." |
| | |
| | messages = [{"role": "user", "content": prompt}] |
| | inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device) |
| | |
| | outputs = model.generate(inputs, max_new_tokens=1792, temperature=0.6) |
| | print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| | ``` |
| |
|
| | The model reasons step-by-step inside `<think>...</think>` tags and gives its final answer in `\boxed{...}` format. |
| |
|
| | ## Training |
| |
|
| | LoRA adapters were trained using GRPO via the [BeamPERL framework](https://github.com/tphage/BeamPERL) on a synthetic dataset of beam mechanics questions generated with the SymBeam library. The base model weights were kept frozen throughout training. |
| |
|
| | ## Citation |
| |
|
| | ```bibtex |
| | @misc{hage2026beamperlparameterefficientrlverifiable, |
| | title={BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning}, |
| | author={Tarjei Paule Hage and Markus J. Buehler}, |
| | year={2026}, |
| | eprint={2603.04124}, |
| | archivePrefix={arXiv}, |
| | primaryClass={cs.AI}, |
| | url={https://arxiv.org/abs/2603.04124}, |
| | ``` |
| |
|
| | ## Acknowledgements |
| |
|
| | Built upon [Tina](https://arxiv.org/abs/2504.15777) and [Open R1](https://github.com/huggingface/open-r1). Dataset generation uses a custom version of [SymBeam](https://github.com/amcc1996/symbeam). |
