Spaces:
Running
on
Zero
Persistent logic failure on negative constraints
Hello!
Great work on the visual fidelity here—the generation quality is solid.
I’ve been stress-testing the instruction adherence across diverse domains today and found a persistent logic failure regarding negative constraints.
I ran 3 separate tests targeting the word "without," and the model failed every time, regardless of the subject matter.
The Failure Pattern: The attention mechanism seems to lock onto the object token (e.g., "tail," "spikes") and completely bypasses the negation modifier.
Test 1 (Synthetic): Prompted for "A robotic horse without a tail." Result: Generated a clear tail.
Test 2 (Organic): Prompted for "A porcupine without its spikes." Result: Generated full spikes.
Test 3 :Prompted for " A cup without a handle "
The Fix: My team at Repalto specializes in constructing adversarial datasets for these exact edge cases. We can build a focused "Negation Benchmark" (50+ prompts spanning varying complexities) to help map and patch this logic gap.
Happy to send that batch over if you want to use it for benchmarking or fine-tuning the next version. Just let me know.
Best,
Wahaj Barlas
Repalto