Instructions to use vedantjadhav701/Indus-Craft-VLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use vedantjadhav701/Indus-Craft-VLM with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="vedantjadhav701/Indus-Craft-VLM", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("vedantjadhav701/Indus-Craft-VLM", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use vedantjadhav701/Indus-Craft-VLM with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "vedantjadhav701/Indus-Craft-VLM"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vedantjadhav701/Indus-Craft-VLM",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/vedantjadhav701/Indus-Craft-VLM

SGLang

How to use vedantjadhav701/Indus-Craft-VLM with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "vedantjadhav701/Indus-Craft-VLM" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vedantjadhav701/Indus-Craft-VLM",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "vedantjadhav701/Indus-Craft-VLM" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vedantjadhav701/Indus-Craft-VLM",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use vedantjadhav701/Indus-Craft-VLM with Docker Model Runner:
```
docker model run hf.co/vedantjadhav701/Indus-Craft-VLM
```

Indus-Craft-VLM

nvidia/Llama-3.1-Nemotron-Nano-VL-8B-V1 fine-tuned (LoRA on the language model + trained vision projector, then merged) to caption images of traditional Indian crafts and textiles in 11 languages: English, Hindi, Marathi, Bengali, Urdu, Nepali, Tamil, Telugu, Dogri, Kashmiri, Maithili. Training data: vedantjadhav701/Indus-Craft-VLM (dataset repo, private).

Usage

Needs trust_remote_code=True. The model class only accepts eager at load time, so switch the LLM to SDPA afterwards:

import torch
from transformers import AutoModel, AutoConfig, AutoTokenizer, AutoImageProcessor
repo = "vedantjadhav701/Indus-Craft-VLM"
cfg = AutoConfig.from_pretrained(repo, trust_remote_code=True)
cfg.llm_config._attn_implementation = "sdpa"; cfg.vision_config.use_flash_attn = False
model = AutoModel.from_pretrained(repo, config=cfg, attn_implementation="eager", torch_dtype=torch.bfloat16,
                                  trust_remote_code=True).cuda().eval()
model.language_model.config._attn_implementation = "sdpa"
tok = AutoTokenizer.from_pretrained(repo); proc = AutoImageProcessor.from_pretrained(repo, trust_remote_code=True)

Prompts used in training, e.g.: Describe this traditional Indian craft image in detail in Tamil. (also: one-sentence caption, two-sentence summary, and "Identify the craft shown in this image..."). See 05_infer.py for batched generation.

Limitations

English captions were generated by Qwen3-VL-30B-A3B from the images and the other 10 languages are machine translations of them; none were human-verified. Dogri, Kashmiri and Maithili are the weakest. Have native speakers review before relying on them.
The base model's card lists English only; the Llama-3.1 tokenizer is inefficient on Indic scripts (long outputs, slow generation).
Captions describe visible materials/motifs; craft names and regions come from dataset labels and can be wrong for look-alike crafts.

Licence: inherits the NVIDIA Open Model License and the Llama 3.1 Community License. Built with Llama.

Downloads last month: 5

Safetensors

Model size

9B params

Tensor type

I64

BF16

Model tree for vedantjadhav701/Indus-Craft-VLM

Base model

nvidia/Llama-3.1-Nemotron-Nano-VL-8B-V1

Adapter

(2)

this model