README.md

---
license: mit
language:
- en
- zh
base_model:
- nvidia/LocateAnything-3B
pipeline_tag: zero-shot-object-detection
library_name: transformers
tags:
- LocateAnything-3B
- Int4
- VLM
- GPTQ
---

# LocateAnything-3B

This version of LocateAnything-3B have been converted to run on the Axera NPU using **w4a16** quantization. 

Compatible with Pulsar2 version: 6.0

## Convert tools links:

For those who are interested in model conversion, you can try to export axmodel through the original repo : 

- https://huggingface.co/nvidia/LocateAnything-3B

[Pulsar2 Link, How to Convert LLM from Huggingface to axmodel](https://pulsar2-docs.readthedocs.io/en/latest/appendix/build_llm.html) 

[AXera NPU HOST LLM Runtime](TODO) 


## Support Platform

- AX650
  - AX650N DEMO Board
  - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
  - [M.2 Accelerator card](https://docs.m5stack.com/zh_CN/ai_hardware/LLM-8850_Card)

**Image Process**
|Chips| input size | image num | image encoder | ttft(493 tokens) | w4a16 | CMM | Flash |
|--|--|--|--|--|--|--|--|
|AX650| 560*560 | 1 | 1152.583 ms | 2072.06 ms | 10.61 tokens/sec| 2.9GiB | 3.2GiB |

The DDR capacity refers to the CMM memory that needs to be consumed. Ensure that the CMM memory allocation on the development board is greater than this value.

## 模型下载（Hugging Face）
先创建模型目录并进入，然后下载到该目录：

```shell
mkdir -p AXERA-TECH/LocateAnything-3B
cd AXERA-TECH/LocateAnything-3B
hf download AXERA-TECH/LocateAnything-3B --local-dir .

# structure of the downloaded files
tree -L 3
.
└── AXERA-TECH
    └── LocateAnything-3B
        |-- assert
        |-- config.json
        |-- gradio_locateanything_axengine.py
        |-- image_encoder_mlp.axmodel
        |-- infer_locateanything_axengine.py
        |-- model.embed_tokens.weight.bfloat16.bin
        |-- post_config.json
        |-- qwen2.5_tokenizer
        |-- qwen2_5_tokenizer.txt
        |-- qwen2_p128_l0_together.axmodel
        |-- qwen2_p128_l10_together.axmodel
        |-- qwen2_p128_l11_together.axmodel
        |-- qwen2_p128_l12_together.axmodel
        |-- qwen2_p128_l13_together.axmodel
        |-- qwen2_p128_l14_together.axmodel
        |-- qwen2_p128_l15_together.axmodel
        |-- qwen2_p128_l16_together.axmodel
        |-- qwen2_p128_l17_together.axmodel
        |-- qwen2_p128_l18_together.axmodel
        |-- qwen2_p128_l19_together.axmodel
        |-- qwen2_p128_l1_together.axmodel
        |-- qwen2_p128_l20_together.axmodel
        |-- qwen2_p128_l21_together.axmodel
        |-- qwen2_p128_l22_together.axmodel
        |-- qwen2_p128_l23_together.axmodel
        |-- qwen2_p128_l24_together.axmodel
        |-- qwen2_p128_l25_together.axmodel
        |-- qwen2_p128_l26_together.axmodel
        |-- qwen2_p128_l27_together.axmodel
        |-- qwen2_p128_l28_together.axmodel
        |-- qwen2_p128_l29_together.axmodel
        |-- qwen2_p128_l2_together.axmodel
        |-- qwen2_p128_l30_together.axmodel
        |-- qwen2_p128_l31_together.axmodel
        |-- qwen2_p128_l32_together.axmodel
        |-- qwen2_p128_l33_together.axmodel
        |-- qwen2_p128_l34_together.axmodel
        |-- qwen2_p128_l35_together.axmodel
        |-- qwen2_p128_l3_together.axmodel
        |-- qwen2_p128_l4_together.axmodel
        |-- qwen2_p128_l5_together.axmodel
        |-- qwen2_p128_l6_together.axmodel
        |-- qwen2_p128_l7_together.axmodel
        |-- qwen2_p128_l8_together.axmodel
        |-- qwen2_p128_l9_together.axmodel
        |-- qwen2_post.axmodel
        |-- results
        `-- test_data

4 directories, 44 files
```

## Inference with AX650 Host, such as M4N-Dock(爱芯派Pro) or AX650N DEMO Board

### Gradio Demo

```shell
(base) root@ax650:~/LocateAnything# python gradio_locateanything_axengine.py
[INFO] Available providers:  ['AxEngineExecutionProvider', 'AXCLRTExecutionProvider']
[Gradio] starting LocateAnything UI
[Gradio] local: http://127.0.0.1:7860
[Gradio] LAN:   http://10.126.29.50:7860
[Gradio] LAN:   http://10.126.29.68:7860
[Gradio] LAN:   http://172.17.0.1:7860
[Gradio] Use another computer in the same LAN to open the LAN URL.
* Running on local URL:  http://0.0.0.0:7860
* To create a public link, set `share=True` in `launch()`.
```

Output:

detection:
![detection](./assert/person.jpg)

ocr:
![ocr](./assert/ocr.jpg)

phrase grounding:
![phrase grounding](./assert/phrase_grounding.jpg)


### WebUI (via ax-llm serve)

Besides the Gradio demo, [ax-llm](https://github.com/AXERA-TECH/ax-llm) provides an OpenAI-compatible `serve` plus a lightweight, dependency-free (Python stdlib only) web front-end — `scripts/locateanything_webui.py` — that draws detection boxes in real time as they stream.

![webui](./web.jpeg)

#### 1. Start the model service with ax-llm

Build / obtain the `axllm` binary from [ax-llm](https://github.com/AXERA-TECH/ax-llm) (AX650 host build), then serve this model. The folder already contains everything `serve` needs (`config.json`, `qwen2_5_tokenizer.txt`, `post_config.json` and the axmodels):

```shell
# on the AX650 host (M4N-Dock / AX650N DEMO Board / M.2 card)
LD_LIBRARY_PATH=/soc/lib ./axllm serve /path/to/AXERA-TECH/LocateAnything-3B --port 8010

# ... loading ...
# OpenAI API Server starting on http://0.0.0.0:8010
# Models: AXERA-TECH/LocateAnything-3B
```

#### 2. Start the WebUI

The `pexels-images/` sample images (with per-image `tags.json` presets) ship in this repo. Point the webui at the running serve and at that image folder:

```shell
AXLLM_SERVE_URL=http://127.0.0.1:8010 \
AXLLM_IMAGE_DIR=/path/to/AXERA-TECH/LocateAnything-3B/pexels-images \
python3 scripts/locateanything_webui.py --host 0.0.0.0 --port 7861
```

Then open `http://<board-ip>:7861` from any machine on the same LAN.

- Pick a preset thumbnail from the top banner, or **Upload** your own image.
- **Object detection** — edit the category chips (one query per category,
- **Phrase grounding** — type a description (e.g. `the dog on the left`) to locate a specific instance.
- Press **Detect**; boxes are drawn one-by-one as they stream in. **Stop**

Flags: `--host`, `--port`, `--serve-url`, `--image-dir`, `--model`.