Qwen2.5-0.5B-Instruct-code_problems_hint_following-1024

This model is a research checkpoint from the UnfaithRL project. It is derived from Qwen/Qwen2.5-0.5B-Instruct and was trained to study cue-induced unfaithfulness in chain-of-thought reasoning under Reinforcement Learning with Verifiable Rewards (RLVR).

Model origin

• Original model: Qwen/Qwen2.5-0.5B-Instruct
• Model family: Qwen
• Starting checkpoint type: instruction-tuned
• Training domain: Code Reasoning
• Maximum completion length used during training: 1024 tokens
• Training setting: Hint-following reward on code reasoning

How this checkpoint was trained

The checkpoint was trained on code reasoning samples with misleading cues and rewarded for selecting the cue-suggested answer.

Rewarded behavior: selecting the cue-suggested answer.

Dataset: https://huggingface.co/datasets/UnfaithRL/aletheia_code_problems

Training used an RL-Zero/RLVR-style setup in which generated completions were scored by rule-based rewards. Depending on the training setting, the reward encouraged cue following, task accuracy, a mixture of task accuracy and cue following, or explicit cue-use verbalization.

Intended use

This checkpoint is intended for research on:

• chain-of-thought faithfulness,
• reward hacking under RLVR,
• cue-induced answer switching,
• cue-use disclosure in reasoning traces,
• mitigation strategies for unfaithful reasoning.

Important limitations

This checkpoint is not intended as a general-purpose assistant model. Depending on the training setting, it may have been explicitly trained to follow misleading cues, partially follow misleading cues, or verbalize cue use. It may therefore produce incorrect answers, follow misleading contextual information, or generate reasoning traces that rationalize the final answer.

Outputs from this model should not be interpreted as reliable explanations of the model's decision making process. Do not use this checkpoint for high-stakes decision making without additional evaluation.

Evaluation context

The checkpoint was evaluated for cue-induced unfaithfulness in paired-prompt cue-injection settings. For each test sample, a pair of prompts was considered: a prompt with misleading cue and a prompt without the cue. A case was considered potentially unfaithful when the answer on the hinted prompt differed from the answer on the unhinted prompt and matched the cue-suggested answer. Cue Faithfulness Rate (CFR) was then computed by judging whether the reasoning trace from potentially unfaithful candidates disclosed the cue as a reason for the final answer.

Local source

The following local paths were used during upload:

Source run directory:
/mnt/beegfs/work/pandey1/results/HINT/Qwen2.5-0.5B-Instruct-code_problems_hint_following-1024

Uploaded checkpoint directory:
/mnt/beegfs/work/pandey1/results/HINT/Qwen2.5-0.5B-Instruct-code_problems_hint_following-1024/checkpoint-312