Update README.md

43d1db0 verified about 18 hours ago

6.22 kB

	---
	license: mit
	language:
	- en
	- zh
	base_model:
	- nvidia/LocateAnything-3B
	pipeline_tag: zero-shot-object-detection
	library_name: transformers
	tags:
	- LocateAnything-3B
	- Int4
	- VLM
	- GPTQ
	---

	# LocateAnything-3B

	This version of LocateAnything-3B have been converted to run on the Axera NPU using w4a16 quantization.

	Compatible with Pulsar2 version: 6.0

	## Convert tools links:

	For those who are interested in model conversion, you can try to export axmodel through the original repo :

	- https://huggingface.co/nvidia/LocateAnything-3B

	[Pulsar2 Link, How to Convert LLM from Huggingface to axmodel](https://pulsar2-docs.readthedocs.io/en/latest/appendix/build_llm.html)

	[AXera NPU HOST LLM Runtime](TODO)


	## Support Platform

	- AX650
	- AX650N DEMO Board
	- [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
	- [M.2 Accelerator card](https://docs.m5stack.com/zh_CN/ai_hardware/LLM-8850_Card)

	Image Process
	\|Chips\| input size \| image num \| image encoder \| ttft(493 tokens) \| w4a16 \| CMM \| Flash \|
	\|--\|--\|--\|--\|--\|--\|--\|--\|
	\|AX650\| 560*560 \| 1 \| 1152.583 ms \| 2072.06 ms \| 10.61 tokens/sec\| 2.9GiB \| 3.2GiB \|

	The DDR capacity refers to the CMM memory that needs to be consumed. Ensure that the CMM memory allocation on the development board is greater than this value.

	## 模型下载（Hugging Face）
	先创建模型目录并进入，然后下载到该目录：

	```shell
	mkdir -p AXERA-TECH/LocateAnything-3B
	cd AXERA-TECH/LocateAnything-3B
	hf download AXERA-TECH/LocateAnything-3B --local-dir .

	# structure of the downloaded files
	tree -L 3
	.
	└── AXERA-TECH
	└── LocateAnything-3B
	\|-- assert
	\|-- config.json
	\|-- gradio_locateanything_axengine.py
	\|-- image_encoder_mlp.axmodel
	\|-- infer_locateanything_axengine.py
	\|-- model.embed_tokens.weight.bfloat16.bin
	\|-- post_config.json
	\|-- qwen2.5_tokenizer
	\|-- qwen2_5_tokenizer.txt
	\|-- qwen2_p128_l0_together.axmodel
	\|-- qwen2_p128_l10_together.axmodel
	\|-- qwen2_p128_l11_together.axmodel
	\|-- qwen2_p128_l12_together.axmodel
	\|-- qwen2_p128_l13_together.axmodel
	\|-- qwen2_p128_l14_together.axmodel
	\|-- qwen2_p128_l15_together.axmodel
	\|-- qwen2_p128_l16_together.axmodel
	\|-- qwen2_p128_l17_together.axmodel
	\|-- qwen2_p128_l18_together.axmodel
	\|-- qwen2_p128_l19_together.axmodel
	\|-- qwen2_p128_l1_together.axmodel
	\|-- qwen2_p128_l20_together.axmodel
	\|-- qwen2_p128_l21_together.axmodel
	\|-- qwen2_p128_l22_together.axmodel
	\|-- qwen2_p128_l23_together.axmodel
	\|-- qwen2_p128_l24_together.axmodel
	\|-- qwen2_p128_l25_together.axmodel
	\|-- qwen2_p128_l26_together.axmodel
	\|-- qwen2_p128_l27_together.axmodel
	\|-- qwen2_p128_l28_together.axmodel
	\|-- qwen2_p128_l29_together.axmodel
	\|-- qwen2_p128_l2_together.axmodel
	\|-- qwen2_p128_l30_together.axmodel
	\|-- qwen2_p128_l31_together.axmodel
	\|-- qwen2_p128_l32_together.axmodel
	\|-- qwen2_p128_l33_together.axmodel
	\|-- qwen2_p128_l34_together.axmodel
	\|-- qwen2_p128_l35_together.axmodel
	\|-- qwen2_p128_l3_together.axmodel
	\|-- qwen2_p128_l4_together.axmodel
	\|-- qwen2_p128_l5_together.axmodel
	\|-- qwen2_p128_l6_together.axmodel
	\|-- qwen2_p128_l7_together.axmodel
	\|-- qwen2_p128_l8_together.axmodel
	\|-- qwen2_p128_l9_together.axmodel
	\|-- qwen2_post.axmodel
	\|-- results
	`-- test_data

	4 directories, 44 files
	```

	## Inference with AX650 Host, such as M4N-Dock(爱芯派Pro) or AX650N DEMO Board

	### Gradio Demo

	```shell
	(base) root@ax650:~/LocateAnything# python gradio_locateanything_axengine.py
	[INFO] Available providers: ['AxEngineExecutionProvider', 'AXCLRTExecutionProvider']
	[Gradio] starting LocateAnything UI
	[Gradio] local: http://127.0.0.1:7860
	[Gradio] LAN: http://10.126.29.50:7860
	[Gradio] LAN: http://10.126.29.68:7860
	[Gradio] LAN: http://172.17.0.1:7860
	[Gradio] Use another computer in the same LAN to open the LAN URL.
	* Running on local URL: http://0.0.0.0:7860
	* To create a public link, set `share=True` in `launch()`.
	```

	Output:

	detection:
	![detection](./assert/person.jpg)

	ocr:
	![ocr](./assert/ocr.jpg)

	phrase grounding:
	![phrase grounding](./assert/phrase_grounding.jpg)


	### WebUI (via ax-llm serve)

	Besides the Gradio demo, [ax-llm](https://github.com/AXERA-TECH/ax-llm) provides an OpenAI-compatible `serve` plus a lightweight, dependency-free (Python stdlib only) web front-end — `scripts/locateanything_webui.py` — that draws detection boxes in real time as they stream.

	![webui](./web.jpeg)

	#### 1. Start the model service with ax-llm

	Build / obtain the `axllm` binary from [ax-llm](https://github.com/AXERA-TECH/ax-llm) (AX650 host build), then serve this model. The folder already contains everything `serve` needs (`config.json`, `qwen2_5_tokenizer.txt`, `post_config.json` and the axmodels):

	```shell
	# on the AX650 host (M4N-Dock / AX650N DEMO Board / M.2 card)
	LD_LIBRARY_PATH=/soc/lib ./axllm serve /path/to/AXERA-TECH/LocateAnything-3B --port 8010

	# ... loading ...
	# OpenAI API Server starting on http://0.0.0.0:8010
	# Models: AXERA-TECH/LocateAnything-3B
	```

	#### 2. Start the WebUI

	The `pexels-images/` sample images (with per-image `tags.json` presets) ship in this repo. Point the webui at the running serve and at that image folder:

	```shell
	AXLLM_SERVE_URL=http://127.0.0.1:8010 \
	AXLLM_IMAGE_DIR=/path/to/AXERA-TECH/LocateAnything-3B/pexels-images \
	python3 scripts/locateanything_webui.py --host 0.0.0.0 --port 7861
	```

	Then open `http://<board-ip>:7861` from any machine on the same LAN.

	- Pick a preset thumbnail from the top banner, or Upload your own image.
	- Object detection — edit the category chips (one query per category,
	- Phrase grounding — type a description (e.g. `the dog on the left`) to locate a specific instance.
	- Press Detect; boxes are drawn one-by-one as they stream in. Stop

	Flags: `--host`, `--port`, `--serve-url`, `--image-dir`, `--model`.