Instructions to use lewiswatson/Frame2KG-Gemma4-e2b-CT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use lewiswatson/Frame2KG-Gemma4-e2b-CT with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

Use Docker

docker model run hf.co/lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

LM Studio
Jan

vLLM

How to use lewiswatson/Frame2KG-Gemma4-e2b-CT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lewiswatson/Frame2KG-Gemma4-e2b-CT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lewiswatson/Frame2KG-Gemma4-e2b-CT",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

Ollama
How to use lewiswatson/Frame2KG-Gemma4-e2b-CT with Ollama:
```
ollama run hf.co/lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M
```

Unsloth Studio

How to use lewiswatson/Frame2KG-Gemma4-e2b-CT with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for lewiswatson/Frame2KG-Gemma4-e2b-CT to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for lewiswatson/Frame2KG-Gemma4-e2b-CT to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for lewiswatson/Frame2KG-Gemma4-e2b-CT to start chatting

How to use lewiswatson/Frame2KG-Gemma4-e2b-CT with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use lewiswatson/Frame2KG-Gemma4-e2b-CT with Docker Model Runner:
```
docker model run hf.co/lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M
```

Lemonade

How to use lewiswatson/Frame2KG-Gemma4-e2b-CT with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

Run and chat with the model

lemonade run user.Frame2KG-Gemma4-e2b-CT-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use lewiswatson/Frame2KG-Gemma4-e2b-CT with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use lewiswatson/Frame2KG-Gemma4-e2b-CT with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "lewiswatson/Frame2KG-Gemma4-e2b-CT:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

lewiswatson commited on Jun 20

Commit

4f02c71

verified ·

1 Parent(s): 5ad71be

Add files using upload-large-folder tool

Browse files

Files changed (16) hide show

.gitattributes +14 -0
Frame2KG-Gemma4-e2b-CT.IQ4_XS.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.Q2_K.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.Q3_K_L.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.Q3_K_M.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.Q3_K_S.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.Q4_K_M.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.Q4_K_S.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.Q5_K_M.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.Q5_K_S.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.Q6_K.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.Q8_0.gguf +3 -0
Frame2KG-Gemma4-e2b-CT.f16.gguf +3 -0
README.md +111 -0
mmproj-Frame2KG-Gemma4-e2b-CT.Q8_0.gguf +3 -0
mmproj-Frame2KG-Gemma4-e2b-CT.f16.gguf +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,17 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+mmproj-Frame2KG-Gemma4-e2b-CT.f16.gguf filter=lfs diff=lfs merge=lfs -text
+mmproj-Frame2KG-Gemma4-e2b-CT.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.f16.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Frame2KG-Gemma4-e2b-CT.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text

Frame2KG-Gemma4-e2b-CT.IQ4_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:62f2cdc2c6efcdd2affd41ae7c4b51981c692f419b015e9ed8a788c1560b0206
+size 3298419776

Frame2KG-Gemma4-e2b-CT.Q2_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:251ebffcc2be2914b8ab7ad5dc77f747075cf1296f8cc80fb36a877deb782433
+size 2980774976

Frame2KG-Gemma4-e2b-CT.Q3_K_L.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bba5464dc8abeb52ca446afdb622486d0fdf1d69510e77d6192f6c34dfdefa0c
+size 3271902272

Frame2KG-Gemma4-e2b-CT.Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a07a02977f6dbc057e0216c8c88faf7c114f31095b4bda00faa1cbcde2b2d031
+size 3192079424

Frame2KG-Gemma4-e2b-CT.Q3_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f540b7e0b2079b097bfd4581be58dcc684a42c25270938f31535a420e60cdcbc
+size 3102198848

Frame2KG-Gemma4-e2b-CT.Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9c42ec54e882144be3e44efc43f9ec520e05910246ee2a068a301e2e7af2495f
+size 3416240192

Frame2KG-Gemma4-e2b-CT.Q4_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:326a203948610b863a9040afc975d233461573cf0ba7d2070515fb8c1f2a2458
+size 3354551360

Frame2KG-Gemma4-e2b-CT.Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:da2fa4ef29e896725c0253c927bce066ee559d0ac38f4c8c5d1dca43ac487bcb
+size 3616829504

Frame2KG-Gemma4-e2b-CT.Q5_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1344d3c2a4961be3a17e55a108b0041b715a19690ff9941682a4ce0f5aeb2b27
+size 3582518336

Frame2KG-Gemma4-e2b-CT.Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:411eaa155fcd2136ac3b4c7472986bf0eef45dd045fc1833867659a229c9ecdd
+size 3829955648

Frame2KG-Gemma4-e2b-CT.Q8_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c0841de3af0d322c36e7bf892875c2772ebd6e0bbc461578a8a5a720fab68877
+size 4947570464

Frame2KG-Gemma4-e2b-CT.f16.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:baef97bb5141e3356134eb48562f0b83ad10b0dd71f436db8f250b072f4eb162
+size 9273821504

README.md ADDED Viewed

	@@ -0,0 +1,111 @@

+---
+library_name: llama.cpp
+pipeline_tag: image-text-to-text
+tags:
+- gguf
+- llama.cpp
+- vision-language
+- frame2kg
+- knowledge-graph
+- CT
+datasets:
+- lewiswatson/Frame2KG-YC2
+- lewiswatson/Frame2KG-YC2-compressed-tokens
+base_model: google/gemma-4-E2B-it
+---
+# Frame2KG-Gemma4-e2b-CT
+This repository contains GGUF quantised files for the Frame2KG fine-tuned Gemma4 e2b model.
+These GGUF releases are optimized deployment variants of Frame2KG models. They are provided for practical inference use and may not exactly match the original weights, checkpoints, or evaluation configuration reported in the Frame2KG paper.
+## Model Details
+- Family: Gemma4
+- Size: e2b
+- Output format: compressed Frame2KG graph tokens
+- Base model: google/gemma-4-E2B-it
+- Model type: gemma4
+- Architecture: Gemma4ForConditionalGeneration
+- Fine-tuning method: PEFT LoRA
+- LoRA rank: 8
+- LoRA alpha: 16
+- Trainable added token count: 14
+## Files
+| File | Size | Notes |
+|---|---:|---|
+| `Frame2KG-Gemma4-e2b-CT.f16.gguf` | 8.64 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.IQ4_XS.gguf` | 3.07 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.Q2_K.gguf` | 2.78 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.Q3_K_L.gguf` | 3.05 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.Q3_K_M.gguf` | 2.97 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.Q3_K_S.gguf` | 2.89 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.Q4_K_M.gguf` | 3.18 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.Q4_K_S.gguf` | 3.12 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.Q5_K_M.gguf` | 3.37 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.Q5_K_S.gguf` | 3.34 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.Q6_K.gguf` | 3.57 GB | model weights |
+| `Frame2KG-Gemma4-e2b-CT.Q8_0.gguf` | 4.61 GB | model weights |
+| `mmproj-Frame2KG-Gemma4-e2b-CT.f16.gguf` | 939.99 MB | multimodal projector |
+| `mmproj-Frame2KG-Gemma4-e2b-CT.Q8_0.gguf` | 531.55 MB | multimodal projector |
+## Compressed Token Metadata
+This CT variant uses added single-token graph markers for compressed Frame2KG output.
+- `<|graph|>`
+- `<|nodes|>`
+- `<|node|>`
+- `<|id|>`
+- `<|label|>`
+- `<|bbox|>`
+- `<|conf|>`
+- `<|attrs|>`
+- `<|edges|>`
+- `<|edge|>`
+- `<|src|>`
+- `<|pred|>`
+- `<|tgt|>`
+- `<|end_graph|>`
+## Usage Notes
+These files are intended for llama.cpp-compatible GGUF runtimes.
+## Scope
+Frame2KG models are intended to convert image or frame content into structured graph-style outputs. The exact output format depends on the variant:
+- `JSON` variants target JSON-formatted Frame2KG output.
+- `CT` variants target compressed Frame2KG graph tokens.
+## Citation
+If you use this model in your work, please cite the paper:
+```bibtex
+@inproceedings{watson2026frame2kg,
+  title = {Frame2KG: A Benchmark and Evaluation Toolkit for Interpretable Frame-to-Graph Generation},
+  author = {Watson, Lewis N. and Strathearn, Carl and Mitchell, Kenny and Yu, Yanchao},
+  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026)},
+  month = {May},
+  year = {2026},
+  pages = {10912--10926},
+  address = {Palma, Mallorca, Spain},
+  publisher = {European Language Resources Association (ELRA)},
+  editor = {Piperidis, Stelios and Bel, Núria and van den Heuvel, Henk and Ide, Nancy and Krek, Simon and Toral, Antonio},
+  doi = {10.63317/4ys6kofrzoc5},
+  url = {https://doi.org/10.63317/4ys6kofrzoc5}
+}
+```
+## Disclaimer
+This model is provided as is, without warranties or guarantees of any kind, either express or implied. The authors make no representations regarding the accuracy, reliability, safety, suitability, or performance of the model or its outputs.
+The model may generate incorrect, incomplete, or misleading results and should not be relied upon for critical, safety-sensitive, legal, medical, financial, or other high-stakes decisions. Use of this model is entirely at your own risk.
+The authors accept no liability for damages, losses, or consequences arising from use, misuse, or inability to use the model.

mmproj-Frame2KG-Gemma4-e2b-CT.Q8_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d2660b2bcc91b554120ae6f2ba6ae58d383b804eee319872dee54754a116969c
+size 557367648

mmproj-Frame2KG-Gemma4-e2b-CT.f16.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1229d2de83a2885bd79a8c70b2bf7f1e68f25cb602ada1a240d429d666bd9a8a
+size 985653600