metadata
license: apache-2.0
base_model: Qwen/Qwen3-8B
tags:
- reward-model
- prm
- code-security
- qwen3
library_name: transformers
pipeline_tag: text-classification
Qwen3-8B Implementation-level PRM (exec-filtered, thinking)
Scalar Bradley–Terry preference model over implementation guidelines
(<high_level> + <implementation>) for secure coding.
Training
- Base:
Qwen/Qwen3-8B→AutoModelForSequenceClassification(num_labels=1) - Prefs: evolved (t+) / preferred (t) / degraded (t−) guidelines, exec-filtered with thinking enabled (keep t+ iff func∧sec; t− iff ¬sec)
- Independent pairwise BT: t≻t−, t+≻t−, t+≻t
- Data: combined 5cwe + other CWEs, K=3 paired attempts (attempt-0 + 2 extras),
preferences_synth_triple_exec_combined_allpairs_k3(6933 pairs; 6192/741) - Val accuracy (pairwise): 0.968 (741 pairs)
Scoring
Chat messages: user = coding task, assistant = guideline trace.
Serve with vLLM pooling/classify and POST /classify.
Citation / project
Internal: prm_secode_eval/recode-style prm_impl_bt_triple_exec_combined_allpairs_k3 best.