---
library_name: transformers
datasets:
- PIPer-iclr/envbench-zeroshot-rl
base_model:
- PIPer-iclr/Qwen3-8B-am
pipeline_tag: text-generation
license: mit
---
# 🚀 PIPer: On-Device Environment Setup via Online Reinforcement Learning

<div align="center">

[![Models](https://img.shields.io/badge/🤗%20Hugging%20Face-Models-orange.svg)](https://huggingface.co/PIPer-iclr)
[![Dataset](https://img.shields.io/badge/🤗%20Hugging%20Face-Dataset-green.svg)](https://huggingface.co/datasets/PIPer-iclr/envbench-zeroshot-rl)
[![License](https://img.shields.io/badge/License-MIT-red.svg)](LICENSE)

*Democratizing environment setup with on-device sized models that match the performance of much larger proprietary systems*

</div>

## 🎯 Overview

Environment setup—the process of configuring systems to work with specific software projects—remains a persistent challenge in software engineering. **PIPer** addresses this by training specialized on-device models that can automatically generate correct Bash scripts for environment configuration.

Our approach combines:
- 📚 **Supervised Fine-Tuning (SFT)** with executable scripts from larger models
- 🎯 **Reinforcement Learning with Verifiable Rewards (RLVR)** using lightweight proxy LLM-reward

## 🏆 Key Results

| Model | Size | EnvBench avg@5 | Cost per 1M tokens |
|-------|------|----------------|-------------------|
| **PIPer** | 8B | **19.4** | $0.60 |
| GPT-4o | - | 19.4 | $15.00 |
| Qwen3-32B | 32B | 16.2 | $2.00 |
| Qwen3-8B | 8B | 2.6 | $0.60 |

> 🎉 **PIPer achieves 9× improvement** over its base model while **matching GPT-4o performance** at **25x lower cost**

![Performance vs Cost Analysis](https://github.com/PIPer-iclr/PIPer/raw/main/misc/combined_pass_n_and_cost.png)

## 📦 Available Artifacts

### 🤖 Model Checkpoints

| Model | Description | HuggingFace Link |
|-------|-------------|------------------|
| **🏅 PIPer (Full)** | Complete SFT+RL trained model | [PIPer-iclr/PIPer-8B](https://huggingface.co/PIPer-iclr/PIPer-8B) |
| 🎯 PIPer (RL-only) | RLVR checkpoint only | [PIPer-iclr/PIPer-8B-RL-only](https://huggingface.co/PIPer-iclr/PIPer-8B-RL-only) |
| 📚 PIPer (SFT-only) | Supervised fine-tuning only | [PIPer-iclr/PIPer-8B-SFT-only](https://huggingface.co/PIPer-iclr/PIPer-8B-SFT-only) |

### 📊 Datasets

| Dataset | Description | HuggingFace Link |
|---------|-------------|------------------|
| **EnvBench Zero-shot RL** | Training prompts and evaluation data | [PIPer-iclr/envbench-zeroshot-rl](https://huggingface.co/datasets/PIPer-iclr/envbench-zeroshot-rl) |

## 📊 Evaluation Benchmarks

| Benchmark | Description | Metric | Our Result |
|-----------|-------------|---------|------------|
| **EnvBench-Python** | 329 Python repositories | pass@5 | 🏆 **27/329** |
| **Repo2Run** | 420 Python repositories | pass@5 | 🏆 **103/420** |
| **Terminal-Bench** | 80 terminal tasks | pass@10 | **4/80** |



## 🙏 Acknowledgments

- Built on top of [EnvBench](https://github.com/princeton-nlp/EnvBench) evaluation framework
- Uses [VeRL](https://github.com/volcengine/verl) for efficient RL training
- Leverages [Qwen3](https://huggingface.co/Qwen) model family as base architecture

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.