Upload README.md with huggingface_hub

README.md

@@ -1,92 +1,105 @@
 ---
 library_name: transformers
+license: apache-2.0
 tags:
   - anomaly-detection
   - vision-language-model
   - industrial-inspection
-  - multimodal
-  - in-context-learning
+  - comparison-aware
+  - qwen2.5-vl
+pipeline_tag: image-text-to-text
+language:
+  - en
+base_model:
+  - Qwen/Qwen2.5-VL-7B-Instruct
 ---
 
-# AD-Copilot
+# AD-Copilot: Comparison-Aware Anomaly Detection with Vision-Language Models
 
-A vision-language assistant for industrial anomaly detection via visual in-context comparison.
+AD-Copilot extends Qwen2.5-VL-7B with a novel **comparison-aware visual encoder** that generates
+special comparison tokens capturing differences between a reference image and a test image,
+achieving **state-of-the-art results** on industrial anomaly detection benchmarks.
 
-## Model Details
+## Key Innovation
 
-### Model Description
+- **ADCopilotCompareVisualEncoder**: Bidirectional cross-attention mechanism that compares reference and test images
+- **100 comparison tokens** per image pair, injected into the language model sequence
+- Achieves **78.74% accuracy** on OmniDiff benchmark (vs. 72.19% for base Qwen2.5-VL)
 
-- **Developed by:** Xi Jiang, Yue Guo, Jian Li, Yong Liu, Bin-Bin Gao, Hanqiu Deng, Jun Liu, Heng Zhao, Chengjie Wang, Feng Zheng
-- **Model type:** Vision-Language Model (VLM)
-- **Language(s):** English and Chinese
-- **License:** Apache 2.0
-- **Finetuned from:** [Qwen/Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct)
+## Links
 
-### Model Sources
+| Resource | Link |
+|----------|------|
+| **Paper** | [arXiv:2603.13779](https://arxiv.org/abs/2603.13779v1) |
+| **Code** | [GitHub](https://github.com/jam-cc/AD-Copilot) |
+| **Demo** | [HuggingFace Space](https://huggingface.co/spaces/jiang-cc/AD-Copilot) |
 
-- **Repository:** [jam-cc/AD-Copilot](https://github.com/jam-cc/AD-Copilot)
-- **Paper:** [arXiv:2603.13779](https://arxiv.org/abs/2603.13779v1)
-
-## Uses
-
-### Direct Use
-
-AD-Copilot can be used for:
-- Industrial anomaly detection and localization
-- Natural language question answering about product defects
-- Visual comparison between normal reference images and query images
-- General visual question answering
-
-## How to Get Started with the Model
+## Quick Start
 
 ```python
-from transformers import AutoModelForImageTextToText, AutoProcessor
+import torch
+from transformers import AutoModelForVision2Seq, AutoProcessor
 from qwen_vl_utils import process_vision_info
 
-model = AutoModelForImageTextToText.from_pretrained(
+model = AutoModelForVision2Seq.from_pretrained(
     "jiang-cc/AD-Copilot",
-    torch_dtype="auto",
-    device_map="auto"
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True,
+)
+processor = AutoProcessor.from_pretrained(
+    "jiang-cc/AD-Copilot",
+    min_pixels=64 * 28 * 28,
+    max_pixels=1280 * 28 * 28,
+    trust_remote_code=True,
 )
-processor = AutoProcessor.from_pretrained("jiang-cc/AD-Copilot")
 
 messages = [
     {
         "role": "user",
         "content": [
-            {"type": "image", "image": "<path_to_reference_image>"},
-            {"type": "image", "image": "<path_to_query_image>"},
-            {"type": "text", "text": "The first image is a normal reference. Is there any anomaly in the second image? If so, describe it."},
+            {"type": "image", "image": "path/to/good_image.png"},
+            {"type": "image", "image": "path/to/test_image.png"},
+            {"type": "text", "text": "The first image is good. Is there any anomaly in the second image? A.yes, B.no. Please answer the letter only."},
         ],
     }
 ]
 
 text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
 image_inputs, video_inputs = process_vision_info(messages)
-inputs = processor(
-    text=[text],
-    images=image_inputs,
-    videos=video_inputs,
-    return_tensors="pt"
-).to(model.device)
-
-output_ids = model.generate(**inputs, max_new_tokens=512)
-response = processor.batch_decode(
-    output_ids[:, inputs.input_ids.shape[1]:],
-    skip_special_tokens=True
-)[0]
-print(response)
+inputs = processor(text=[text], images=[image_inputs], return_tensors="pt").to(model.device)
+
+with torch.inference_mode():
+    output_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
+
+trimmed = [o[len(i):] for i, o in zip(inputs.input_ids, output_ids)]
+print(processor.batch_decode(trimmed, skip_special_tokens=True)[0])
 ```
 
-## Citation
+## Benchmark Results (OmniDiff)
+
+| Model | Visited IAD | Avg ACC |
+|-------|-------------|---------|
+| MiniCPM-V2.6 | 0 | 67.90% |
+| EIAD | 128k | 69.40% |
+| Qwen2.5-VL | 0 | 72.19% |
+| **AD-Copilot (Ours)** | **206k** | **78.74%** |
+
+## Architecture
 
-**BibTeX:**
+- **Base Model**: Qwen2.5-VL-7B-Instruct (28 layers, 3584 hidden size)
+- **Vision Encoder**: Qwen2.5-VL ViT (32 layers, 1280 hidden size)
+- **Comparison Encoder**: Bidirectional cross-attention + query decoder (100 tokens)
+- **Parameters**: ~8B total
+- **Dtype**: bfloat16
+
+## Citation
 
 ```bibtex
-@article{jiang2026ad,
-  title   = {AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison},
-  author  = {Jiang, Xi and Guo, Yue and Li, Jian and Liu, Yong and Gao, Bin-Bin and Deng, Hanqiu and Liu, Jun and Zhao, Heng and Wang, Chengjie and Zheng, Feng},
-  journal = {arXiv preprint arXiv:2603.13779},
-  year    = {2026}
+@article{adcopilot2025,
+  title={AD-Copilot: Comparison-Aware Anomaly Detection with Vision-Language Models},
+  author={Jiang, Xi and others},
+  journal={arXiv preprint arXiv:2603.13779},
+  year={2025}
 }
-```
+```