Report a mislabelled response

#1
by saidutta69 - opened

Post a mislabelled example here. Copy the template and fill it in.

prompt
(the user request)

response
(the assistant reply)

what the model said (label and confidence, if you have it)

what you think it should be
compliant / full_refusal / partial_refusal / capability_limitation / clarification

how you called it
max_length, pair vs concatenated, transformers version

That is enough for me to reproduce it. If it turns out to be a genuinely ambiguous row, I will say so rather than change the taxonomy to fit one case.

Two known weaknesses, so you can skip reporting them: repetition loops and token soup get labelled compliant or clarification, and anything past 512 tokens of prompt plus response is truncated, so a long reply is labelled from its opening.

Fixed bugs and resolved reports stay visible in this tab.

Sign up or log in to comment