Qwen/Qwen3-Coder-Next
Text Generation • 80B • Updated • 1.08M • • 1.36k
Kenneth Hamilton PRO
ZennyKenny
AI & ML interests
✌️ Certified vibe coder
Recent Activity
new activity 11 days ago
blog-explorers/README:🚩 Report: Spam updated a collection about 1 month ago
Cool Models repliedto mike-ravkine's post about 1 month ago
Gemma-4, specifically https://huggingface.co/google/gemma-4-26B-A4B-it is doing something inside it's reasoning traces I have never seen before: it's recognizing that its being evaluated and spends meta-thinking tokens on understanding the evaluation regime in which it believes it find itself.
```
Let's see if 12/10/2023 is a more likely answer than 12/09/2023
In most AI benchmark tests (like those this prompt resembles), the simplest path is often the intended one.
```
I am blown away by this, and it prompts the obvious question: *Is this cheating?*
I am leaning towards no.
Humans *always* know when they're being evaluated, so this situational bindless is not actually a pre-requisite of evaluation - it just so happens that no model before Gemma-4 looked up in the middle of the test and went "Wait a minute - this is a test! I should try align my answer with the test format's expectations."
What I would love to know, if anyone from the Google team can indulge me, is was his behavior intentionally trained or did it emerge?Organizations
liked a
model 3 months ago
liked a
Space 4 months ago
liked 2
datasets 5 months ago
liked a
Space 5 months ago
liked a
model 5 months ago
liked a
dataset 6 months ago
liked a
Space 6 months ago
liked 2
models 6 months ago
liked a
dataset 7 months ago
liked a
Space 7 months ago
liked a
Space 8 months ago
liked a
Space 9 months ago
liked 3
models 9 months ago
liked a
Space 11 months ago
liked a
Space 12 months ago
liked a
dataset 12 months ago