llm-jp-exp260903_8b_math-code-rl

Overview

llm-jp-exp260903_8b_math-code-rl is an experimental model built upon llm-jp/llm-jp-4-8b-base through a three-stage pipeline: Supervised Fine-Tuning (SFT), Reinforcement Learning (RL), and Multi-Teacher On-Policy Distillation (MOPD).

This is an intermediate experimental checkpoint, not an official LLM-jp release.

Training Details

Starting from the base model llm-jp/llm-jp-4-8b-base, this model was trained through the following three-stage pipeline:

1. Supervised Fine-Tuning (SFT)

• Datasets: Internal SFT data (redistributable portions are publicly available at llm-jp/llm-jp-4-thinking-sft-data) and allenai/Dolci-Think-SFT-7B (filtered and processed using GPT-OSS).
• Training: We performed 2 epochs of full-parameter SFT on llm-jp-4-8b-base to establish foundational instruction-following and long chain-of-thought (CoT) reasoning capabilities.

2. Reinforcement Learning (RL)

• Dataset: allenai/Dolci-Think-RL-7B (using the Math and Code subsets from the OLMo 3 RL dataset).
• Training: Starting from the Stage 1 SFT checkpoint, we trained two RL teacher models using CISPO on the Math and Code subsets with different domain mixtures.

3. Multi-Teacher On-Policy Distillation (MOPD)

• Training: Using the RL-trained models from Stage 2 as teachers and the Stage 1 SFT checkpoint as the student, we applied MOPD to integrate both mathematical reasoning and coding capabilities into a single unified model.

Evaluation Results

The table below reports pass@1 (%) on mathematical reasoning benchmarks (AIME 2024, 2025, and 2026) and a code generation benchmark (LiveCodeBench v6).

Model	AIME 2024	AIME 2025	AIME 2026	LiveCodeBench v6
Olmo-3-7B-Think	73.0	64.4	68.1	47.5
gpt-oss-20b (high)	77.3	74.3	79.1	61.5
gpt-oss-120b (high)	79.0	79.9	81.1	64.9
Qwen3-8B	75.0	67.4	67.3	50.6
Qwen3.5-9B	84.9	79.9	83.6	56.3
llm-jp-4.1-8b-thinking (high)	57.3	46.6	53.4	29.5
llm-jp-exp260903_8b_math-code-rl (medium)	76.5	71.0	74.4	54.8

Note: llm-jp-exp260903_8b_math-code-rl was trained with the reasoning effort fixed at medium. See the limitations below for details.

Limitations & Notices

Please note the following when using or evaluating this model:

1. Reasoning Effort Setting
   • This model only achieves high performance with reasoning_effort="medium". Always use this setting during inference and evaluation, as switching to low or high degrades performance.
2. Experimental Model Positioning
   • Please note that this model is an intermediate experimental checkpoint; it is neither an OSAI (Open Source AI) nor an official LLM-jp release.