andreiski's picture
v2 demo: graded-reward RL adapter (step 195)
458d61d verified
|
Raw
History Blame Contribute Delete
1.16 kB

A newer version of the Gradio SDK is available: 6.22.0

Upgrade
metadata
title: Dialectical Transition Operator v2
emoji: 🔁
colorFrom: indigo
colorTo: gray
sdk: gradio
sdk_version: 5.50.0
app_file: app.py
pinned: false
short_description: Iterative answer+critique self-improvement (graded RL)

🔁 Dialectical Transition Operator · v2

An interactive demo of a self-improvement operator. Given a question, a starting answer, and three critiques of that answer, the model produces a revised answer and three fresh critiques. Copy the output back into the input and generate again to iteratively deepen the answer.

Model: Qwen3-8B + two stacked LoRA adapters — a frozen SFT-voice adapter and an RL adapter trained with GRPO / set-VPO under a graded (−3…+3) self-referential judge: the operator is rewarded only for a revision that better addresses the critique than the answer it came from — never against an external answer. The RL adapter tracks the latest checkpoint of the v2 (graded-reward) run.

⚠️ Research demo. It showcases an iterative self-critique mechanism; it is not a reliable source of facts and can produce incorrect or fabricated content.