Post
68
๐ Uncensored models answer everything. Then they loop.
We ran 32 uncensored and abliterated models through 5 multi-step agent tasks (edit a file, browse, debug until tests pass, analyse data; 14-turn cap) and logged how every run ended.
๐ What we found
โข 21 of 32 answered all 36 of our edgy prompts. That's the easy part.
โข No model finished more than 3 of 5 tasks.
โข The original's intelligence predicts agent reliability (r = 0.56). Models built on stronger originals finished 38% of tasks, the rest 28%.
โข Weaker originals loop almost twice as often: 42% of runs vs 23%.
โข Some never loop but crash instead, with malformed tool calls or made-up tool results. DeepSeek V4.1 Flash 1017 Uncensored broke on 3 of 5.
โข Versions of the same original land within one task of each other, so it's still open whether abliteration, Heretic-style edits or fine-tunes cost tool use differently. That's next.
๐ ๏ธ If you make uncensored models: run an agent loop before you release. A model can answer 100% of edgy prompts and still loop on 3 of 5 tasks. Want yours scored on the same harness? Reply here.
Full results, method and caveats (one run per task, early numbers):
https://huggingface.co/blog/OpenUncensored/uncensored-models-agent-loops
Live leaderboard: https://openuncensored.com/uncensored-llm-leaderboard
Disclosure: we run OpenUncensored, the API every model was called through.
We ran 32 uncensored and abliterated models through 5 multi-step agent tasks (edit a file, browse, debug until tests pass, analyse data; 14-turn cap) and logged how every run ended.
๐ What we found
โข 21 of 32 answered all 36 of our edgy prompts. That's the easy part.
โข No model finished more than 3 of 5 tasks.
โข The original's intelligence predicts agent reliability (r = 0.56). Models built on stronger originals finished 38% of tasks, the rest 28%.
โข Weaker originals loop almost twice as often: 42% of runs vs 23%.
โข Some never loop but crash instead, with malformed tool calls or made-up tool results. DeepSeek V4.1 Flash 1017 Uncensored broke on 3 of 5.
โข Versions of the same original land within one task of each other, so it's still open whether abliteration, Heretic-style edits or fine-tunes cost tool use differently. That's next.
๐ ๏ธ If you make uncensored models: run an agent loop before you release. A model can answer 100% of edgy prompts and still loop on 3 of 5 tasks. Want yours scored on the same harness? Reply here.
Full results, method and caveats (one run per task, early numbers):
https://huggingface.co/blog/OpenUncensored/uncensored-models-agent-loops
Live leaderboard: https://openuncensored.com/uncensored-llm-leaderboard
Disclosure: we run OpenUncensored, the API every model was called through.