MOOSE-Star-IR-R1D-7B

MOOSE-Star Inspiration Retrieval model — a 7B model fine-tuned for selecting the correct cross-paper inspiration from 15 candidates given a research background.

Note: This model is referred to as MS-IR-7B in the paper. The full name includes "R1D" to indicate it is fine-tuned from DeepSeek-R1-Distill-Qwen-7B; the SFT data can be used to train other base models as well.

Model Description

• Base Model: DeepSeek-R1-Distill-Qwen-7B
• Training Method: Full-parameter SFT (ZeRO-3)
• Training Data: TOMATO-Star-SFT-Data-R1D-32B IR split (150,218 train + 2,377 eval)
• Teacher Model: Training data generated via rejection sampling with DeepSeek-R1-Distill-Qwen-32B
• Paper: MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier

Training Configuration

Parameter	Value
Base Model	DeepSeek-R1-Distill-Qwen-7B
Chat Template	deepseekr1
Cutoff Length	16384
Learning Rate	1e-5
Epochs	1
Batch Size	128 (via gradient accumulation)
Training	Full-parameter, ZeRO-3, bf16
GPUs	128x (multi-node)

Task Description

Given a research question and background survey, the model selects the most relevant cross-paper inspiration from 15 candidates (labeled A through O). The candidates include:

• 1 correct inspiration (ground truth)
• 14 hard negatives (keyword-similar, embedding-similar, and random papers)

The model outputs its selection with chain-of-thought reasoning.

This model is designed for use in the hierarchical search pipeline, where it navigates a SPECTER2-clustered tree of candidate papers with O(log N) complexity.

Prompt Format

The IR task uses a structured prompt with the following sections (all embedded in the user message, no system prompt):

[Task instruction preamble — problem identification & solution selection guidelines]

## Context

**Research Question:**
{research_question}

**Background Survey (existing methods for THIS task):**
{background_survey}

**Previous Hypothesis (if any — current progress built from earlier inspirations):**
{previous_hypothesis_or_none}

## Candidate Inspiration Papers

### Candidate [A]
**Title:** {title_A}
**Abstract:** {abstract_A}

### Candidate [B]
**Title:** {title_B}
**Abstract:** {abstract_B}

... (15 candidates total, A through O)

## Output Format

<think>
[reasoning process]
</think>

**Selected ID starts:** [X] **Selected ID ends**

**Selection Reason starts:** [reason] **Selection Reason ends**

The full task instruction preamble and output format are included in the SFT training data. See TOMATO-Star-SFT-Data-R1D-32B IR/train.jsonl for complete examples.

Usage

# Clone the MOOSE-Star repo first:
#   git clone https://github.com/ZonglinY/MOOSE-Star.git

### Option A: SGLang Deployment + IRProbabilityExtractor (recommended for pipeline use)

# Step 1: Start the SGLang server (requires sglang installed):
#   bash Inference/start_sglang.sh --model-path /path/to/MOOSE-Star-IR-R1D-7B --port 1235

# Step 2: Use IRProbabilityExtractor (run from repo root):
import sys
sys.path.insert(0, "./Inference")
from ir_probability_extractor import IRProbabilityExtractor

extractor = IRProbabilityExtractor(base_urls=["http://localhost:1235/v1"])
result = extractor.get_selection_probabilities(
    research_question="Your research question",
    background_survey="Your background survey",
    candidates=[
        {"title": "Candidate A title", "abstract": "Candidate A abstract"},
        {"title": "Candidate B title", "abstract": "Candidate B abstract"},
        # ... up to 15 candidates (labeled A–O)
    ],
)
print(f"Selected: [{result.selected_label}]")
print(f"Probabilities: {result.probabilities}")


### Option B: Direct HuggingFace Inference (for testing)

import sys
sys.path.insert(0, "./utils")
from prompt_store import instruction_prompts
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "ZonglinY/MOOSE-Star-IR-R1D-7B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")

p = instruction_prompts("inspiration_retrieval_with_reasoning_with_alphabetical_candidates")

candidates = [{"title": "...", "abstract": "..."}, ...]  # list of dicts
candidates_text = "".join(
    f"### Candidate [{chr(ord('A') + i)}]\n**Title:** {c['title']}\n**Abstract:** {c['abstract']}\n\n"
    for i, c in enumerate(candidates)
)

research_question = "Your research question"
background_survey = "Your background survey"
prompt = (p[0] + research_question
        + p[1] + background_survey
        + p[2] + "No previous hypothesis."
        + p[3] + candidates_text
        + p[4])

messages = [{"role": "user", "content": prompt}]
# Note: use add_generation_prompt=False and manually append <｜Assistant｜>
# to avoid a double <think> tag in the DeepSeek R1-Distill chat template
formatted = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)
formatted += "<｜Assistant｜>"

inputs = tokenizer(formatted, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=8192, temperature=0.6, top_p=0.9, do_sample=True)
response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)

# Parse selected candidate from output markers
import re
match = re.search(r"\*\*Selected ID starts:\*\*\s*\[(\w)\]\s*\*\*Selected ID ends\*\*", response)
if match:
    selected = match.group(1)  # e.g., "A", "B", ..., "O"
    print(f"Selected: [{selected}]")

Full prompt template (contents of instruction_prompts("inspiration_retrieval_with_reasoning_with_alphabetical_candidates"))

You are helping with scientific hypothesis generation by selecting an inspiration that solves a fundamental problem in the current approach.

## Core Task: Problem Identification and Solution

**Your Primary Goal**: Identify which candidate paper can best help solve a fundamental problem in the existing methods/hypothesis - either directly or by inspiring a solution.

**Key Principle**: Good inspirations help solve real problems. They might directly provide a solution, or they might spark an idea, remind you of related concepts, or inspire a creative adaptation. The best breakthroughs often come from unexpected connections.

**What Makes a Good Inspiration**:
1. **Problem-Solution Fit**: Either addresses a known limitation OR reveals new improvement opportunities
2. **Enables Progress**: The paper provides concepts, sparks ideas, or inspires solutions that advance the research
3. **Creative Connection**: The link might be indirect, non-obvious, or emerge during exploration
4. **Clear Impact**: You can explain how this paper contributes to progress, even if the path is unexpected

**The Research Process**:
1. **Background**: Research question + existing methods (with their limitations)
2. **Problem Identification**: What fundamental issue prevents progress?
3. **Inspiration Selection**: Which concept best solves this problem?
4. **Hypothesis Formation**: Adapt the solution to create a better method

**Classic Example - Backpropagation**:
- **Research Question**: How to use data to automatically improve parameters of multi-layer logistic regression?
- **Existing Methods**: Could only do inference, not learning
- **FUNDAMENTAL PROBLEM**: No way to compute gradients through multiple layers
- **Solution Found**: Chain rule from calculus
- **Why It Solves the Problem**: Chain rule computes derivatives of composite functions; neural networks ARE composite functions
- **Result**: Backpropagation algorithm

Note: The focus was on SOLVING THE GRADIENT PROBLEM. The breakthrough came from recognizing neural networks as composite functions.

## Your Current Task

**Flexible Reasoning Process** (these steps can happen in any order or iteratively):
- **Problem Recognition**: Identify limitations in current methods/hypothesis (can happen before OR after seeing candidates)
- **Opportunity Discovery**: For each candidate, explore how it might advance the research:
   - It might solve a problem you already identified
   - It might reveal a problem you hadn't noticed and simultaneously offer a solution
   - It might spark ideas for improvements you hadn't considered
- **Selection**: Choose the candidate that enables the most meaningful progress

**Note**: The reasoning is often bidirectional - seeing a candidate can make you realize "oh, this could address limitation X that I hadn't fully articulated" or "this suggests a way to improve aspect Y"

**Remember**:
- The best inspiration might not seem immediately relevant
- Focus on problem-solving potential, not keyword matching
- Creative connections often lead to breakthroughs
- Consider how concepts could be adapted or repurposed

**Avoid**:
- Choosing based on surface-level similarity
- Dismissing candidates that seem unrelated at first glance

## Context

**Research Question:**
{research_question}

**Background Survey (existing methods for THIS task):**
{background_survey}

**Previous Hypothesis (if any - current progress built from earlier inspirations):**
{previous_hypothesis}

## Candidate Inspiration Papers

### Candidate [A]
**Title:** {title_A}
**Abstract:** {abstract_A}

### Candidate [B]
**Title:** {title_B}
**Abstract:** {abstract_B}

... (15 candidates total, A through O)

## Output Format

**CRITICAL**: You MUST structure your response EXACTLY as follows (the markers are used for automatic parsing).

<think>
[Your flexible reasoning process - explore problems and opportunities as they emerge, evaluate how candidates relate to potential improvements, select the most promising one. Refer to candidates using their labels like Candidate [A], Candidate [B], etc.]
</think>

**Selected ID starts:** [X] **Selected ID ends**

(Replace [X] with the letter of your chosen candidate, e.g., [A], [B], [C], etc. Output ONLY the letter in brackets, nothing else between the markers.)

**Selection Reason starts:** [summary of why this inspiration was selected - what problem it addresses, how it enables progress] **Selection Reason ends**

Evaluation Results

Evaluated on 15-way inspiration retrieval accuracy (from paper Table 3):

Model	Accuracy
Random Selection	6.70%
R1-Distilled-Qwen-7B (base)	28.42%
MS-IR-7B (this model)	54.37%

Citation

@article{yang2025moosestar,
  title={MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier},
  author={Yang, Zonglin and Bing, Lidong},
  year={2025}
}

License

This model is released under the Apache 2.0 license.