Instructions to use ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph")
model = AutoModelForMultimodalLM.from_pretrained("ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph

SGLang

How to use ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph with Docker Model Runner:
```
docker model run hf.co/ChartGalaxyPP/ChartGalaxyPlusPlus-Image2SceneGraph
```

ChartGalaxyPlusPlus-Image2SceneGraph / OUTPUT_FORMAT.md

ChartGalaxyPP

Rename model to Image2SceneGraph

e3af8b3 verified 5 days ago

preview code

Raw History Blame Contribute Delete

2.61 kB

	# Image2SceneGraph output format

	The model emits a JSON object with `high_level_description`, `style_description`, and `elements`. `elements.background` describes the background, and `elements.layout` is a flat list of element and group nodes. A node's `parent` refers to another node label or the implicit `ROOT`.

	\| Native model output \| Dataset representation \|
	\| --- \| --- \|
	\| `high_level_description` \| Same field \|
	\| `style_description` \| Same field \|
	\| `elements.background` \| `compositional_deconstruction.background` \|
	\| `elements.layout` \| `compositional_deconstruction.nodes` \|
	\| `node_kind` \| Same field; `element` or `group` \|
	\| `type` on an element \| `element_type` \|
	\| `type` on a group \| `group_type` \|
	\| `bbox: [x0, y0, x1, y1]` \| `bbox: [y0, x0, y1, x1]` \|
	\| `label`, `parent`, `desc`, `text`, `color_palette`, `role`, `role_family` \| Preserve applicable fields \|

	Both box formats use a 0–1000 coordinate grid. For a native box, pixel coordinates are `(x0 * width/1000, y0 * height/1000, x1 * width/1000, y1 * height/1000)`. Use the original image dimensions; do not interpret the grid values directly as pixels.

	The `type` vocabulary is `text`, `image`, or `shape` for elements, and `chart`, `axis`, `legend`, `legend_item`, `data_item`, `series`, or `panel` for groups. The native prompt requests a `node_kind` and the roles applicable to each item. Treat missing, contradictory, or unknown node types as validation errors rather than silently guessing replacements.

	For example, a native element box `[100, 200, 400, 500]` becomes `[200, 100, 500, 400]` in the dataset representation. Preserve the label and parent when changing the field names and box order.

	Validate unique labels, existing parent references, hierarchy acyclicity, box ordering and bounds, and required fields after conversion. Serialization conversion alone does not guarantee conformance to the [dataset schema](https://huggingface.co/datasets/ChartGalaxyPP/ChartGalaxyPlusPlus/blob/main/scene_graph.schema.json).

	The model does not emit the dataset's separate `spatial_relations` list. Such records depend on node geometry and a specified spatial sampling policy. Do not fill a missing list with invented relations or compare native JSON against the dataset without adapting its representation.

	Retain raw generated text and generation termination information. If a response is truncated, syntax repair may yield a parseable partial graph; record that status separately from schema validity and semantic accuracy. The usage example uses strict JSON parsing so that a malformed response remains visible.

	# Image2SceneGraph output format

	The model emits a JSON object with `high_level_description`, `style_description`, and `elements`. `elements.background` describes the background, and `elements.layout` is a flat list of element and group nodes. A node's `parent` refers to another node label or the implicit `ROOT`.

	\| Native model output \| Dataset representation \|
	\| --- \| --- \|
	\| `high_level_description` \| Same field \|
	\| `style_description` \| Same field \|
	\| `elements.background` \| `compositional_deconstruction.background` \|
	\| `elements.layout` \| `compositional_deconstruction.nodes` \|
	\| `node_kind` \| Same field; `element` or `group` \|
	\| `type` on an element \| `element_type` \|
	\| `type` on a group \| `group_type` \|
	\| `bbox: [x0, y0, x1, y1]` \| `bbox: [y0, x0, y1, x1]` \|
	\| `label`, `parent`, `desc`, `text`, `color_palette`, `role`, `role_family` \| Preserve applicable fields \|

	Both box formats use a 0–1000 coordinate grid. For a native box, pixel coordinates are `(x0 * width/1000, y0 * height/1000, x1 * width/1000, y1 * height/1000)`. Use the original image dimensions; do not interpret the grid values directly as pixels.

	The `type` vocabulary is `text`, `image`, or `shape` for elements, and `chart`, `axis`, `legend`, `legend_item`, `data_item`, `series`, or `panel` for groups. The native prompt requests a `node_kind` and the roles applicable to each item. Treat missing, contradictory, or unknown node types as validation errors rather than silently guessing replacements.

	For example, a native element box `[100, 200, 400, 500]` becomes `[200, 100, 500, 400]` in the dataset representation. Preserve the label and parent when changing the field names and box order.

	Validate unique labels, existing parent references, hierarchy acyclicity, box ordering and bounds, and required fields after conversion. Serialization conversion alone does not guarantee conformance to the [dataset schema](https://huggingface.co/datasets/ChartGalaxyPP/ChartGalaxyPlusPlus/blob/main/scene_graph.schema.json).

	The model does not emit the dataset's separate `spatial_relations` list. Such records depend on node geometry and a specified spatial sampling policy. Do not fill a missing list with invented relations or compare native JSON against the dataset without adapting its representation.

	Retain raw generated text and generation termination information. If a response is truncated, syntax repair may yield a parseable partial graph; record that status separately from schema validity and semantic accuracy. The usage example uses strict JSON parsing so that a malformed response remains visible.