Add A3-Qwen3.5-9B WorkArena-L2 results (10.6%) and update all comments (#14)

- Add A3-Qwen3.5-9B WorkArena-L2 results (10.6%) and update all comments (f0b0214c2231d34f40d648cbaa1d9cda69a473cf)
- Add trajectories link to README (a35f1f19ff6c9bccea9721b90bb8c495af376bce)
- Clarify that A3-Synth uses WebArena environments but different tasks (56da91d56e1f8f38d9228f7bd71fea5190f5761f)
- Set benchmark_tuned=Yes for WebArena (91c446a4b286173bf001db07708b39b242e5fc8f)
- Remove redundant training data notes from comments (d7a5d720e476c05ed500f671e20e14198b518810)


Co-authored-by: Xing Han Lù <xhluca@users.noreply.huggingface.co>

results/A3-Qwen3.5-9B/README.md

@@ -1,66 +1,76 @@
-### A3-Qwen3.5-9B
+<div align="center">
 
-This agent is [GenericAgent](https://github.com/ServiceNow/AgentLab/blob/main/src/agentlab/agents/generic_agent/generic_agent.py) from [AgentLab](https://github.com/ServiceNow/AgentLab), fine-tuned using the Agent-as-Annotators (A3) pipeline.
+# A3-Qwen3.5-9B
 
-- **Model Name:** A3-Qwen3.5-9B
-- **Base Model:** Qwen/Qwen3.5-9B
-- **Model Architecture:**
-  - Type: Vision-Language Model (VLM)
-  - Architecture: Causal LM with vision encoder
-  - Number of Parameters: 9B
-- **Input/Output Format:**
-  - Input: Accessibility tree + Set-of-Mark (SoM) screenshot
-  - Output: Text action in BrowserGym format
-  - Flags:
-    ```python
-    GenericPromptFlags(
-        obs=ObsFlags(
-            use_html=False,
-            use_ax_tree=True,
-            use_tabs=True,
-            use_focused_element=True,
-            use_error_logs=True,
-            use_history=True,
-            use_past_error_logs=False,
-            use_action_history=True,
-            use_think_history=False,
-            use_diff=False,
-            html_type='pruned_html',
-            use_screenshot=True,
-            use_som=True,
-            extract_visible_tag=True,
-            extract_clickable_tag=True,
-            extract_coords='False',
-            filter_visible_elements_only=False,
-        ),
-        action=ActionFlags(
-            action_set=HighLevelActionSetArgs(
-                subsets=('webarena',),
-                multiaction=False,
-                strict=False,
-                retry_with_force=True,
-                demo_mode='off',
-            ),
-            long_description=False,
-            individual_examples=False,
+| [**📄 Paper**](https://arxiv.org/abs/2604.07776) | [**💾 Code**](https://github.com/McGill-NLP/agent-as-annotators) | [**🌐 Website**](https://agent-as-annotators.github.io) |
+| :--: | :--: | :--: |
+| [**🤗 Dataset**](https://huggingface.co/datasets/McGill-NLP/A3-Synth) | [**🤖 Models**](https://huggingface.co/collections/McGill-NLP/a3-agent-as-annotators-69d854ab5b1993b10efc3fba) | [**📦 PyPI**](https://pypi.org/project/agent-as-annotators/) |
+| [**📊 Trajectories**](https://huggingface.co/datasets/xhluca/a3-qwen-3.5-9b-trajectories) | | |
+
+</div>
+
+This agent is [GenericAgent](https://github.com/ServiceNow/AgentLab/blob/main/src/agentlab/agents/generic_agent/generic_agent.py) from [AgentLab](https://github.com/ServiceNow/AgentLab), fine-tuned using the [Agent-as-Annotators (A3)](https://github.com/McGill-NLP/agent-as-annotators) framework.
+
+## Model
+
+| | |
+|---|---|
+| **Base Model** | [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) |
+| **Type** | Vision-Language Model (VLM), 9B parameters |
+| **Training Data** | [A3-Synth](https://huggingface.co/datasets/McGill-NLP/A3-Synth) trajectories, derived from WebArena environments but with entirely different, synthetically generated tasks (no benchmark tasks seen during training) |
+| **Training Method** | Supervised Fine-Tuning (SFT) with FSDP |
+| **Inference Temperature** | 0.6 |
+| **License** | Apache-2.0 |
+
+## Input/Output
+
+- **Input:** Accessibility tree + Set-of-Mark (SoM) screenshot
+- **Output:** Text action in BrowserGym format
+
+## Configuration Flags
+
+```python
+GenericPromptFlags(
+    obs=ObsFlags(
+        use_html=False,
+        use_ax_tree=True,
+        use_tabs=True,
+        use_focused_element=True,
+        use_error_logs=True,
+        use_history=True,
+        use_past_error_logs=False,
+        use_action_history=True,
+        use_think_history=False,
+        use_diff=False,
+        html_type='pruned_html',
+        use_screenshot=True,
+        use_som=True,
+        extract_visible_tag=True,
+        extract_clickable_tag=True,
+        extract_coords='False',
+        filter_visible_elements_only=False,
+    ),
+    action=ActionFlags(
+        action_set=HighLevelActionSetArgs(
+            subsets=('webarena',),
+            multiaction=False,
+            strict=False,
+            retry_with_force=True,
+            demo_mode='off',
         ),
-        use_plan=False,
-        use_criticise=False,
-        use_thinking=True,
-        use_memory=False,
-        use_concrete_example=True,
-        use_abstract_example=True,
-        use_hints=True,
-        enable_chat=False,
-        max_prompt_tokens=57344,
-        be_cautious=True,
-        extra_instructions=None,
-    )
-    ```
-- **Training Details:**
-  - Dataset: WebSynth trajectories collected via the A3 pipeline (agent-generated annotations on real websites)
-  - Fine-tuning method: Supervised Fine-Tuning (SFT) with FSDP
-  - Temperature at inference: 0.6
-- **Paper Link:** https://arxiv.org/abs/2604.07776
-- **Code Repository:** https://github.com/McGill-NLP/agent-as-annotators
-- **License:** Apache-2.0
+        long_description=False,
+        individual_examples=False,
+    ),
+    use_plan=False,
+    use_criticise=False,
+    use_thinking=True,
+    use_memory=False,
+    use_concrete_example=True,
+    use_abstract_example=True,
+    use_hints=True,
+    enable_chat=False,
+    max_prompt_tokens=57344,
+    be_cautious=True,
+    extra_instructions=None,
+)
+```

results/A3-Qwen3.5-9B/miniwob.json

@@ -10,7 +10,7 @@
         "benchmark_tuned": "No",
         "followed_evaluation_protocol": "Yes",
         "reproducible": "Yes",
-        "comments": "625 tasks. Model not trained on MiniWoB data.",
+        "comments": "625 tasks. Fine-tuned on A3-Synth trajectories via the Agent-as-Annotators framework.",
         "original_or_reproduced": "Original"
     }
 ]

results/A3-Qwen3.5-9B/visualwebarena.json

@@ -6,11 +6,11 @@
         "benchmark": "VisualWebArena",
         "score": 33.7,
         "std_err": 1.6,
-        "benchmark_specific": "Yes",
+        "benchmark_specific": "No",
         "benchmark_tuned": "No",
         "followed_evaluation_protocol": "Yes",
         "reproducible": "Yes",
-        "comments": "Combined visualwebarena-train (461 tasks) and visualwebarena-test (449 tasks) for a total of 910 tasks. Model fine-tuned on WebSynth trajectories.",
+        "comments": "910 tasks (full benchmark). Fine-tuned on A3-Synth trajectories via the Agent-as-Annotators framework. A3-Synth is derived from WebArena environments but uses entirely different, synthetically generated tasks (no VisualWebArena tasks seen during training).",
         "original_or_reproduced": "Original"
     }
 ]

results/A3-Qwen3.5-9B/webarena.json

@@ -6,11 +6,11 @@
         "benchmark": "WebArena",
         "score": 42.1,
         "std_err": 1.7,
-        "benchmark_specific": "Yes",
-        "benchmark_tuned": "No",
+        "benchmark_specific": "No",
+        "benchmark_tuned": "Yes",
         "followed_evaluation_protocol": "Yes",
         "reproducible": "Yes",
-        "comments": "Combined webarena-train (431 tasks) and webarena-test (381 tasks) for a total of 812 tasks. Model fine-tuned on WebSynth trajectories.",
+        "comments": "812 tasks (full benchmark). Fine-tuned on A3-Synth trajectories via the Agent-as-Annotators framework. A3-Synth is derived from WebArena environments but uses entirely different, synthetically generated tasks.",
         "original_or_reproduced": "Original"
     }
 ]

results/A3-Qwen3.5-9B/workarena-l1.json

@@ -10,7 +10,7 @@
         "benchmark_tuned": "No",
         "followed_evaluation_protocol": "Yes",
         "reproducible": "Yes",
-        "comments": "330 tasks. Model not trained on ServiceNow data.",
+        "comments": "330 tasks. Fine-tuned on A3-Synth trajectories via the Agent-as-Annotators framework.",
         "original_or_reproduced": "Original"
     }
 ]

results/A3-Qwen3.5-9B/workarena-l2.json

@@ -0,0 +1,16 @@
+[
+    {
+        "agent_name": "A3-Qwen3.5-9B",
+        "study_id": "2026-03-14_12-38-08_genericagent-checkpoints-qwen-qwen3-5-9b-web-pro-low-8903051-checkpoint-latest-on-workarena-l2-test-test",
+        "date_time": "2026-04-06 18:39:04",
+        "benchmark": "WorkArena-L2",
+        "score": 10.6,
+        "std_err": 1.7,
+        "benchmark_specific": "No",
+        "benchmark_tuned": "No",
+        "followed_evaluation_protocol": "Yes",
+        "reproducible": "Yes",
+        "comments": "341 tasks (full benchmark). Fine-tuned on A3-Synth trajectories via the Agent-as-Annotators framework.",
+        "original_or_reproduced": "Original"
+    }
+]