andrealiao1014 commited on
Commit
6ab5c5d
·
verified ·
1 Parent(s): 2829128

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -49
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
  license: apache-2.0
3
  base_model: google/gemma-4-E4B
4
- pipeline_tag: image-text-to-text
5
  library_name: transformers
6
  language:
7
  - en
@@ -21,32 +21,13 @@ datasets:
21
  metrics:
22
  - accuracy
23
  - f1
24
- # new_version: # leave unset — only used later to point to a v2 successor
25
- # --- Eval Results (uncomment and fill once the eval harness produces numbers) ---
26
- # model-index:
27
- # - name: opengraph-image-gemma4-e4b-v1
28
- # results:
29
- # - task:
30
- # type: image-text-to-text
31
- # name: Image-to-Knowledge-Graph Extraction
32
- # dataset:
33
- # name: OpenGraph Image Gold v1 (test split)
34
- # type: OpenGraphAI/opengraph-image-gold-v1
35
- # split: test
36
- # metrics:
37
- # - type: accuracy
38
- # name: Schema-valid extraction rate
39
- # value: [XX.X]
40
- # - type: f1
41
- # name: Node/Edge F1
42
- # value: [XX.X]
43
  ---
44
 
45
  # Model Card for opengraph-image-gemma4-e4b-v1
46
 
47
  <!-- Provide a quick summary of what the model is/does. -->
48
 
49
- A fine-tuned **Gemma 4 E4B** that turns a single image into a valid, schema-faithful **knowledge graph** (`graph.json`) — objects, attributes, scene context, text spans, and the relationships between them — ready for AI agents to query and reason over. Built by [OpenGraph AI](https://github.com/OpenGraphAI/opengraph-ai) as the small, cheap, on-device alternative to calling a frontier vision API for every extraction.
50
 
51
  ## Model Details
52
 
@@ -55,7 +36,7 @@ A fine-tuned **Gemma 4 E4B** that turns a single image into a valid, schema-fait
55
  The model does one thing extremely well: **image → knowledge graph, on-schema, every time.** It was trained on verified image→graph gold pairs so that its output always conforms to OpenGraph's `graph.json` contract — stable snake_case node IDs with type prefixes (`entity_`, `concept_`, `event_`, `attr_`), typed edges, and cross-image-mergeable entities. Compared to prompting a general frontier model, it is dramatically cheaper per extraction, runs on a single consumer GPU (or laptop, quantized), and produces structurally consistent output that downstream graph tooling can rely on.
56
 
57
  - **Developed by:** OpenGraph AI
58
- - **Model type:** Multimodal (image-text-to-text), decoder-only transformer; QLoRA fine-tune of Gemma 4 E4B
59
  - **Language(s):** English
60
  - **License:** Apache 2.0 (inherited from Gemma 4; use is additionally subject to Google's [Gemma terms of use](https://ai.google.dev/gemma/terms))
61
  - **Finetuned from model:** [`google/gemma-4-E4B`](https://huggingface.co/google/gemma-4-E4B)
@@ -63,13 +44,13 @@ The model does one thing extremely well: **image → knowledge graph, on-schema,
63
  ### Model Sources
64
 
65
  - **Repository:** https://github.com/OpenGraphAI/opengraph-ai
66
- - **Demo:** [More Information Needed] <!-- link the graph.html shareable demo when live -->
67
 
68
  ## Uses
69
 
70
  ### Direct Use
71
 
72
- Feed the model one image plus the OpenGraph extraction system prompt; it returns a complete `graph.json` — nodes for detected objects (fine-grained labels, normalized bounding boxes), one scene node, attribute nodes, transcribed text spans, and the edges wiring them together. Useful anywhere images need to become structured, queryable knowledge: visual search indexes, dataset annotation, scene understanding, and document/diagram parsing.
73
 
74
  ### Downstream Use
75
 
@@ -143,7 +124,7 @@ Or skip the code entirely and use it through the MCP server:
143
 
144
  ### Training Data
145
 
146
- Trained on **[More Information Needed — N] verified image`graph.json` gold pairs** ([`OpenGraphAI/opengraph-image-gold-v1`](https://huggingface.co/datasets/OpenGraphAI/opengraph-image-gold-v1)), assembled from two sources: (1) public scene-graph datasets (e.g., Visual Genome) converted programmatically into the `graph.json` schema, and (2) unannotated images labeled by two independent frontier vision models, auto-accepted where both models agreed and human-reviewed otherwise. Every pair passed the OpenGraph Pydantic schema validators before inclusion.
147
 
148
  ### Training Procedure
149
 
@@ -199,15 +180,6 @@ All numbers produced by the open [eval harness](https://github.com/OpenGraphAI/o
199
 
200
  [More Information Needed — 2–3 honest sentences: where the fine-tune wins, where it still trails the frontier baseline]
201
 
202
- ## Environmental Impact
203
-
204
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
205
-
206
- - **Hardware Type:** [More Information Needed — e.g., 1× NVIDIA L4 / A100]
207
- - **Hours used:** [More Information Needed]
208
- - **Cloud Provider:** Google Colab
209
- - **Compute Region:** [More Information Needed]
210
- - **Carbon Emitted:** [More Information Needed]
211
 
212
  ## Technical Specifications
213
 
@@ -225,24 +197,10 @@ Gemma 4 E4B: a decoder-only transformer (~4.5B effective parameters, ~8B with em
225
 
226
  Python, PyTorch, Hugging Face `transformers>=5.10.1`, `trl`, `peft`, `bitsandbytes`, `datasets`.
227
 
228
- ## Citation
229
-
230
- **BibTeX:**
231
-
232
- ```bibtex
233
- @misc{opengraph2026imagegemma,
234
- title = {opengraph-image-gemma4-e4b-v1: A schema-faithful image-to-knowledge-graph model},
235
- author = {OpenGraph AI},
236
- year = {2026},
237
- url = {https://huggingface.co/OpenGraphAI/opengraph-image-gemma4-e4b-v1}
238
- }
239
- ```
240
-
241
- This model builds on Gemma 4 — please also cite the [Gemma 4 Technical Report](https://arxiv.org/abs/2607.02770) (Gemma Team, 2026).
242
 
243
  ## More Information
244
 
245
- OpenGraph AI is open-source, MCP-first infrastructure for turning heterogeneous data (images, tables, text) into semantic knowledge graphs that AI agents can query and reason over. ⭐ [Star the repo](https://github.com/OpenGraphAI/opengraph-ai) — and contribute schemas, test images, or extraction edge cases.
246
 
247
  ## Model Card Authors
248
 
 
1
  ---
2
  license: apache-2.0
3
  base_model: google/gemma-4-E4B
4
+ pipeline_tag: image-text-to-image
5
  library_name: transformers
6
  language:
7
  - en
 
21
  metrics:
22
  - accuracy
23
  - f1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
  ---
25
 
26
  # Model Card for opengraph-image-gemma4-e4b-v1
27
 
28
  <!-- Provide a quick summary of what the model is/does. -->
29
 
30
+ A fine-tuned **Gemma 4 E4B** that turns images into a valid, schema-faithful **knowledge graph** (`graph.json`) — objects, attributes, scene context, text spans, and the relationships between them — ready for AI agents to query and reason over. Built by [OpenGraph AI](https://github.com/OpenGraphAI/opengraph-ai) as the small, cheap, on-device alternative to calling a frontier vision API for every extraction.
31
 
32
  ## Model Details
33
 
 
36
  The model does one thing extremely well: **image → knowledge graph, on-schema, every time.** It was trained on verified image→graph gold pairs so that its output always conforms to OpenGraph's `graph.json` contract — stable snake_case node IDs with type prefixes (`entity_`, `concept_`, `event_`, `attr_`), typed edges, and cross-image-mergeable entities. Compared to prompting a general frontier model, it is dramatically cheaper per extraction, runs on a single consumer GPU (or laptop, quantized), and produces structurally consistent output that downstream graph tooling can rely on.
37
 
38
  - **Developed by:** OpenGraph AI
39
+ - **Model type:** Multimodal (image-text-to-image), decoder-only transformer; QLoRA fine-tune of Gemma 4 E4B
40
  - **Language(s):** English
41
  - **License:** Apache 2.0 (inherited from Gemma 4; use is additionally subject to Google's [Gemma terms of use](https://ai.google.dev/gemma/terms))
42
  - **Finetuned from model:** [`google/gemma-4-E4B`](https://huggingface.co/google/gemma-4-E4B)
 
44
  ### Model Sources
45
 
46
  - **Repository:** https://github.com/OpenGraphAI/opengraph-ai
47
+ - **Demo:** [placeholder] <!-- link the graph.html shareable demo when live -->
48
 
49
  ## Uses
50
 
51
  ### Direct Use
52
 
53
+ Feed the model images plus the OpenGraph extraction system prompt; it returns a complete `graph.json` — nodes for detected objects (fine-grained labels, normalized bounding boxes), one scene node, attribute nodes, transcribed text spans, and the edges wiring them together. Useful anywhere images need to become structured, queryable knowledge: visual search indexes, dataset annotation, scene understanding, and document/diagram parsing.
54
 
55
  ### Downstream Use
56
 
 
124
 
125
  ### Training Data
126
 
127
+ Trained on **[placeholder] verified image���`graph.json` gold pairs** ([`OpenGraphAI/opengraph-image-gold-v1`](https://huggingface.co/datasets/OpenGraphAI/opengraph-image-gold-v1)), assembled from two sources: (1) public scene-graph datasets (e.g., Visual Genome) converted programmatically into the `graph.json` schema, and (2) unannotated images labeled by two independent frontier vision models, auto-accepted where both models agreed and human-reviewed otherwise. Every pair passed the OpenGraph Pydantic schema validators before inclusion.
128
 
129
  ### Training Procedure
130
 
 
180
 
181
  [More Information Needed — 2–3 honest sentences: where the fine-tune wins, where it still trails the frontier baseline]
182
 
 
 
 
 
 
 
 
 
 
183
 
184
  ## Technical Specifications
185
 
 
197
 
198
  Python, PyTorch, Hugging Face `transformers>=5.10.1`, `trl`, `peft`, `bitsandbytes`, `datasets`.
199
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
200
 
201
  ## More Information
202
 
203
+ OpenGraph AI is open-source, MCP-first infrastructure for turning heterogeneous data (images, tables, text, audio, video) into semantic knowledge graphs that AI agents can query and reason over. ⭐ [Star the repo](https://github.com/OpenGraphAI/opengraph-ai) — and contribute schemas, test images, or extraction edge cases.
204
 
205
  ## Model Card Authors
206