Very cool model! I was able to reproduce the results from your paper using Oobabooga's textgen-webui
Here is a summary of what I did and some screenshots:
https://www.reddit.com/r/Oobabooga/comments/17k7eqf/llemma_34b_math_model_in_oobabooga/
I just wanted to drop a thank you and let you know that others were able to reproduce your results.
Have you tested it out with LaTex formatted equations?
Sort of , I was using an OCR to LaTeX program but it had a lot of extra syntax for formatting and I think that was throwing the model off. I had another AI simplify the LaTeX (I am currently very poor in LaTeX) and my instructions to the AI were not so great so it outputted a quasi-LaTeX output, was not properly formatted for rendering. The llemma model seemed to have figured it out though, and gave me a properly formatted LaTeX output. I'm going to do more testing today, but I think the best way to send equations to the model is via simple LaTeX formatting, avoiding stuff like specific fonts or other extra formatting LaTeX code.