base_model: meta-llama/Llama-3.2-1B-Instruct
datasets:
- whynlp/gsm8k-aug
library_name: transformers
license: llama3.2
tags: []
pipeline_tag: text-generation
Learning When to Stop: Adaptive Latent Reasoning via Reinforcement Learning
This repository contains the weights for Adaptive Latent Reasoning models, as introduced in the paper Learning When to Stop: Adaptive Latent Reasoning via Reinforcement Learning.
Latent reasoning represents a new development in Transformer language models that has shown potential in compressing reasoning lengths compared to chain-of-thought reasoning. This work develops adaptive-length latent reasoning models and introduces a post-SFT reinforcement-learning methodology to optimize latent reasoning length by minimizing reasoning length while maintaining accuracy. Experiments on the Llama 3.2 1B model and the GSM8K-Aug dataset showed a 52% drop in total reasoning length with no penalty to accuracy.
The official code and pretrained weights are available at the GitHub repository: https://github.com/apning/adaptive-latent-reasoning
Usage
All weights used for results in the paper are available on Hugging Face. You can load these models using the function automodelforcausallm_from_pretrained_latent from src.model_creation.
First, set up your environment by cloning the repository and installing dependencies:
git clone https://github.com/apning/adaptive-latent-reasoning.git
cd adaptive-latent-reasoning
conda env create -f environment.yml && conda activate adaptive-latent-reasoning
Then, you can load a model like this:
from transformers import AutoTokenizer
from src.model_creation import automodelforcausallm_from_pretrained_latent
repo_id = "Lapisbird/Llama-adaLR-model-latent-6" # Example model from the paper
model = automodelforcausallm_from_pretrained_latent(repo_id)
tokenizer = AutoTokenizer.from_pretrained(repo_id)
For more detailed instructions on replication, training, and evaluation, please refer to the official GitHub repository.