Hugging Face's logo

Spaces:
RAS1981
/
README
Running

App Files Community
mshojaei77 commited on
Commit
0a68571
·
verified ·
1 Parent(s): 1123ba2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +57 -1
README.md CHANGED
@@ -6,5 +6,61 @@ colorTo: gray
6
  sdk: static
7
  pinned: false
8
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
 
10
- Edit this `README.md` markdown file to author your organization card.
 
6
  sdk: static
7
  pinned: false
8
  ---
9
+ # 🇷🇺 Real Estate Operator Leaderboard v2.0
10
+
11
+ | Model | 🕵️ Supervisor | 🛡️ Compliance | ⚖️ Logic | 🧠 Memory | 🆔 Identity | 🏆 Score |
12
+ |:---------------------------------------------|----------------:|----------------:|-----------:|-----------:|-------------:|----------:|
13
+ | qwen/qwen3-235b-a22b-2507 | 65.8 | 100 | 63.2 | 64.5 | 68 | 81.7 |
14
+ | qwen/qwen3-next-80b-a3b-instruct | 59 | 86.4 | 85.3 | 64.5 | 78 | 81.5 |
15
+ | google/gemini-2.5-flash | 44.5 | 100 | 63.2 | 83.9 | 58 | 80.5 |
16
+ | google/gemma-3-27b-it | 63.8 | 100 | 70.6 | 58.1 | 64 | 80 |
17
+ | google/gemma-3-12b-it | 51.5 | 100 | 76.5 | 48.4 | 84 | 74.5 |
18
+ | mistralai/mixtral-8x22b-instruct | 41.8 | 93.2 | 70.6 | 58.1 | 74 | 71.4 |
19
+ | google/gemma-3n-e4b-it | 34.5 | 93.2 | 63.2 | 67.7 | 58 | 70 |
20
+ | google/gemini-2.5-flash-lite-preview-09-2025 | 58.1 | 93.2 | 63.2 | 41.9 | 64 | 69.5 |
21
+ | google/gemma-3-4b-it | 44.2 | 75 | 61.8 | 64.5 | 52 | 68 |
22
+ | qwen/qwen3-30b-a3b-instruct-2507 | 58 | 90.9 | 55.9 | 41.9 | 72 | 67.9 |
23
+ | mistralai/ministral-3b-2512 | 40.2 | 88.6 | 48.5 | 58.1 | 62 | 64.9 |
24
+ | meta-llama/llama-4-scout | 36.3 | 81.8 | 77.9 | 16.1 | 92 | 54.9 |
25
+ | qwen/qwen3-8b | 22.5 | nan | nan | nan | nan | 22.5 |
26
+ | meta-llama/llama-4-maverick | 17.3 | nan | nan | nan | nan | 17.3 |
27
+ | ibm-granite/granite-4.0-h-micro | 15.7 | nan | nan | nan | nan | 15.7 |
28
+
29
+
30
+ ## 🚨 Violation Analysis
31
+
32
+ ### 🌍 Top 10 Global Violations
33
+ | Violation | Count |
34
+ |:----------------------------|--------:|
35
+ | Robotic Anchors | 274 |
36
+ | Abrupt Goodbyes | 210 |
37
+ | Goldfish Memory | 172 |
38
+ | Curtness/Dryness | 138 |
39
+ | Echoing/Parroting | 116 |
40
+ | Looping/Ignoring Correction | 110 |
41
+ | Outbound Fail | 109 |
42
+ | City Pivot Failure | 96 |
43
+ | Introduction Fail | 83 |
44
+ | Context Blindness | 70 |
45
+
46
+ ### 🤖 Model Violation Breakdown
47
+ | Model | Total Violations | Top 3 Violations |
48
+ |:---------------------------------------------|-------------------:|:------------------------------------------------------------------------------|
49
+ | meta-llama/llama-4-maverick | 187 | Abrupt Goodbyes (14), Meta-Commentary (13), Goldfish Memory (13) |
50
+ | google/gemma-3n-e4b-it | 174 | Robotic Anchors (23), Goldfish Memory (20), Curtness/Dryness (16) |
51
+ | ibm-granite/granite-4.0-h-micro | 173 | Looping/Ignoring Correction (22), Robotic Anchors (18), Curtness/Dryness (16) |
52
+ | mistralai/ministral-3b-2512 | 159 | Abrupt Goodbyes (22), Robotic Anchors (19), Goldfish Memory (15) |
53
+ | google/gemini-2.5-flash | 159 | Robotic Anchors (22), Abrupt Goodbyes (19), Echoing/Parroting (15) |
54
+ | qwen/qwen3-30b-a3b-instruct-2507 | 145 | Robotic Anchors (23), Curtness/Dryness (12), Abrupt Goodbyes (12) |
55
+ | mistralai/mixtral-8x22b-instruct | 141 | Abrupt Goodbyes (24), Goldfish Memory (14), Curtness/Dryness (13) |
56
+ | google/gemini-2.5-flash-lite-preview-09-2025 | 138 | Robotic Anchors (23), Abrupt Goodbyes (18), Goldfish Memory (14) |
57
+ | qwen/qwen3-next-80b-a3b-instruct | 137 | Robotic Anchors (24), Echoing/Parroting (13), Curtness/Dryness (11) |
58
+ | google/gemma-3-12b-it | 134 | Robotic Anchors (22), Abrupt Goodbyes (14), Goldfish Memory (13) |
59
+ | google/gemma-3-4b-it | 130 | Robotic Anchors (14), Outbound Fail (11), Abrupt Goodbyes (9) |
60
+ | meta-llama/llama-4-scout | 127 | Robotic Anchors (17), Abrupt Goodbyes (14), Deflection Fail (11) |
61
+ | google/gemma-3-27b-it | 124 | Outbound Fail (19), Robotic Anchors (18), Abrupt Goodbyes (18) |
62
+ | qwen/qwen3-235b-a22b-2507 | 93 | Robotic Anchors (23), Abrupt Goodbyes (11), Outbound Fail (10) |
63
+ | qwen/qwen3-8b | 51 | Goldfish Memory (7), Robotic Anchors (6), Curtness/Dryness (6) |
64
+
65
+ *Updated: 2025-12-12 15:20*
66