Update README.md

README.md

@@ -1,203 +1,527 @@
 ---
 license: apache-2.0
+base_model: openbmb/MiniCPM-V-4.6
 datasets:
 - ASHu2/rune_goblin_visual_dataset
 language:
 - en
-base_model:
-- openbmb/MiniCPM-V-4.6
+library_name: transformers
+pipeline_tag: image-text-to-text
+tags:
+- minicpmv4_6
+- minicpm-v
+- vision-language
+- multimodal
+- image-to-text
+- safetensors
+- gguf
+- lora
+- rune-goblin
+- runelang
+- gradio
+- game-ai
+- spell-recognition
 ---
-# Model Card for Model ID
 
-<!-- Provide a quick summary of what the model is/does. -->
+# GoblinV1
 
-This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).
+**GoblinV1** is a fine-tuned vision-language model for **Rune Goblin**, an AI dungeon crawler where players draw spell glyphs and the model interprets those drawings as magic.
 
-## Model Details
+The model is fine-tuned from **OpenBMB MiniCPM-V-4.6** to read hand-drawn **RuneLang** glyphs and produce structured JSON describing detected runes, ambiguity, spell metadata, and visual presentation hints.
 
-### Model Description
+It is not a generic chatbot.
 
-<!-- Provide a longer summary of what this model is. -->
+It is a tiny cursed spell reader.
 
+> Draw better runes, get stronger magic.  
+> Draw cursed doodles, suffer beautifully.
 
+---
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+## Model Summary
+
+| Field | Value |
+|---|---|
+| Model name | `ASHu2/goblinV1` |
+| Base model | `openbmb/MiniCPM-V-4.6` |
+| Model type | Vision-language / image-to-text |
+| Architecture | MiniCPM-V 4.6 / `minicpmv4_6` |
+| Fine-tuning method | LoRA fine-tuning, merged/exported |
+| Primary format | Safetensors |
+| Quantized format | GGUF |
+| Language | English |
+| License | Apache-2.0 |
+| Dataset | `ASHu2/rune_goblin_visual_dataset` |
+| Primary use case | Hand-drawn rune interpretation for Rune Goblin |
 
-### Model Sources [optional]
+---
 
-<!-- Provide the basic links for the model. -->
+## What This Model Does
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+GoblinV1 reads a player-drawn spell image and returns structured spell interpretation metadata.
 
-## Uses
+The model is trained to identify:
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+- drawn RuneLang glyphs
+- ambiguous or messy rune shapes
+- confidence of the visual reading
+- spell name and spell type
+- colors, shape, motion, particles, and sound tags
+- presentation hints for game VFX
+- weird, funny, cursed spell flavor consistent with Rune Goblin
 
-### Direct Use
+The model is designed to act as the **rune reader and spell presentation planner**.
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+The game engine should still own final combat balance, HP changes, boss rules, inventory state, and durable quest state.
 
-[More Information Needed]
+---
 
-### Downstream Use [optional]
+## Rune Goblin
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+Rune Goblin is a Gradio-based AI dungeon crawler where the player draws their own spells.
 
-[More Information Needed]
+Instead of selecting “fireball” from a fixed menu, the player draws symbolic glyphs. GoblinV1 reads the glyphs and converts them into spell interpretation JSON.
 
-### Out-of-Scope Use
+Example RuneLang meanings:
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+| Rune | Meaning |
+|---|---|
+| Flame | burn, danger, passion |
+| Leaf | healing, growth, poison |
+| Bone | fear, decay, skeletons |
+| Spiral | time, confusion, loops |
+| Eye | reveal, inspect, prophecy |
+| Mirror | reflect, copy, reverse |
+| Circle | shield, trap, containment |
+| Broken Mark | curse modifier |
+| Bell | summon, alarm, attention |
+| Coin | trade, greed, sacrifice |
 
-[More Information Needed]
+Example combinations:
 
-## Bias, Risks, and Limitations
+| Combination | Meaning |
+|---|---|
+| Flame + Circle | burning shield |
+| Spiral + Eye | prophecy / foresight |
+| Bone + Dots | skeleton swarm |
+| Mirror + Jagged | reflect damage |
+| Leaf + Bone | healing with decay risk |
+| Broken + Any Rune | stronger effect with cursed side effect |
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+---
 
-[More Information Needed]
+## Expected Input
 
-### Recommendations
+The model expects an image containing a hand-drawn RuneLang spell.
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+The image usually contains 1-4 glyphs drawn by the player on a canvas.
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+The prompt should instruct the model to return JSON only.
 
-## How to Get Started with the Model
+Example prompt:
 
-Use the code below to get started with the model.
+```text
+Look at this drawn RuneLang spell. Identify the runes, ambiguity, confidence, and produce spell presentation metadata: name, type, colors, shape, motion, grandeur, particles, and sound tags. Return valid JSON only.
+```
 
-[More Information Needed]
+---
 
-## Training Details
+## Expected Output
+
+GoblinV1 should return JSON in this style:
+
+```json
+{
+  "visual_reading": {
+    "detected_runes": ["flame", "circle"],
+    "ambiguous_runes": [],
+    "confidence": 0.91,
+    "layout": "left_to_right"
+  },
+  "spell": {
+    "spell_name": "Ember Lunchbox Ward",
+    "spell_type": "fire_defense",
+    "rune_combo": ["flame", "circle"],
+    "summary": "A circular flame ward forms around the player.",
+    "colors": ["orange", "red", "gold"],
+    "shape": "burning circular shield",
+    "motion": "slow clockwise rotation with pulsing embers",
+    "grandeur": "medium",
+    "particles": ["embers", "sparks", "heat shimmer"],
+    "sound_tags": ["crackle", "whoosh", "low hum"]
+  }
+}
+```
 
-### Training Data
+---
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+## Important Design Boundary
 
-[More Information Needed]
+GoblinV1 is intentionally **not** responsible for final game-state authority.
 
-### Training Procedure
+The model may suggest spell interpretation and presentation metadata, but the game engine should decide:
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+- final HP changes
+- damage numbers
+- cooldowns
+- boss immunity rules
+- quest state
+- inventory changes
+- progression unlocks
+- anti-cheat and validation logic
 
-#### Preprocessing [optional]
+Recommended architecture:
 
-[More Information Needed]
+```text
+Player drawing
+    ↓
+GoblinV1 vision model
+    ↓
+Rune + spell JSON
+    ↓
+Game engine validates / clamps / balances
+    ↓
+Final spell effect is applied
+```
 
+This keeps Rune Goblin fun and expressive while preventing model hallucinations from breaking combat balance.
 
-#### Training Hyperparameters
+---
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+## Training Data
 
-#### Speeds, Sizes, Times [optional]
+GoblinV1 was fine-tuned on `ASHu2/rune_goblin_visual_dataset`, a custom visual instruction dataset for Rune Goblin.
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+The dataset contains hand-drawn or synthetic RuneLang spell images paired with structured chat-style targets.
 
-[More Information Needed]
+Each example teaches the model to:
 
-## Evaluation
+- read visual glyphs
+- map glyphs to RuneLang symbols
+- handle messy or ambiguous drawings
+- follow RuneLang combination rules
+- return valid JSON
+- preserve the game’s cursed-comedic tone
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+The dataset uses image paths and conversational messages containing a system prompt, user prompt, and assistant JSON target.
 
-### Testing Data, Factors & Metrics
+---
 
-#### Testing Data
+## Training Objective
 
-<!-- This should link to a Dataset Card if possible. -->
+The objective is not general image captioning.
 
-[More Information Needed]
+The model is trained for:
 
-#### Factors
+```text
+drawn rune image + instruction → structured spell interpretation JSON
+```
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+The model learns the custom RuneLang vocabulary and the relationship between visual glyphs and spell presentation.
 
-[More Information Needed]
+---
 
-#### Metrics
+## Model Formats
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+This repository includes multiple usable formats.
 
-[More Information Needed]
+### Safetensors
 
-### Results
+Use this for normal Transformers-based inference or deployment.
 
-[More Information Needed]
+### GGUF
 
-#### Summary
+A quantized GGUF version is included for local inference experiments with llama.cpp-compatible runtimes.
 
+Example GGUF file:
 
+```text
+gguf/rune-goblin-v46-Q4_K_M.gguf
+```
 
-## Model Examination [optional]
+If using the GGUF model for vision tasks, make sure your runtime supports MiniCPM-V style multimodal inference and loads the required vision/projector files when needed.
 
-<!-- Relevant interpretability work for the model goes here -->
+---
 
-[More Information Needed]
+## Basic Transformers Usage
+
+Install dependencies:
+
+```bash
+pip install torch torchvision pillow transformers accelerate
+```
+
+Example usage:
+
+```python
+from PIL import Image
+import torch
+from transformers import AutoProcessor, AutoModelForImageTextToText
+
+model_id = "ASHu2/goblinV1"
+
+processor = AutoProcessor.from_pretrained(
+    model_id,
+    trust_remote_code=True
+)
+
+model = AutoModelForImageTextToText.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True
+)
+
+image = Image.open("example_rune.png").convert("RGB")
+
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {"type": "image", "image": image},
+            {
+                "type": "text",
+                "text": (
+                    "Look at this drawn RuneLang spell. "
+                    "Identify the runes, ambiguity, confidence, and produce spell presentation metadata. "
+                    "Return valid JSON only."
+                )
+            }
+        ]
+    }
+]
+
+inputs = processor.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    tokenize=True,
+    return_dict=True,
+    return_tensors="pt"
+).to(model.device)
+
+with torch.no_grad():
+    output_ids = model.generate(
+        **inputs,
+        max_new_tokens=512,
+        do_sample=False
+    )
+
+generated = output_ids[0][inputs["input_ids"].shape[-1]:]
+text = processor.decode(generated, skip_special_tokens=True)
+
+print(text)
+```
 
-## Environmental Impact
+---
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+## llama.cpp / GGUF Usage
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+If using a compatible llama.cpp build:
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+```bash
+llama-server -hf ASHu2/goblinV1:Q4_K_M
+```
 
-## Technical Specifications [optional]
+Or run directly:
 
-### Model Architecture and Objective
+```bash
+llama-cli -hf ASHu2/goblinV1:Q4_K_M
+```
 
-[More Information Needed]
+For multimodal inference, ensure your client/runtime supports MiniCPM-V and any required vision projector configuration.
 
-### Compute Infrastructure
+---
 
-[More Information Needed]
+## Intended Use
 
-#### Hardware
+GoblinV1 is intended for:
 
-[More Information Needed]
+- Rune Goblin gameplay
+- hand-drawn spell glyph interpretation
+- visual rune recognition
+- experimental AI game mechanics
+- structured JSON generation from fantasy glyph images
+- small-model multimodal game prototypes
 
-#### Software
+---
+
+## Out-of-Scope Use
+
+GoblinV1 is not intended for:
 
-[More Information Needed]
+- safety-critical image understanding
+- medical, legal, financial, or security decisions
+- general OCR benchmarking
+- real-world symbol recognition systems
+- moderation or surveillance
+- authoritative factual QA
+- replacing deterministic game rules
 
-## Citation [optional]
+---
+
+## Limitations
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+GoblinV1 may:
 
-**BibTeX:**
+- misread very messy drawings
+- confuse visually similar glyphs
+- produce malformed JSON in some cases
+- invent spell details outside the intended schema
+- overfit to Rune Goblin-style symbols
+- perform poorly on non-RuneLang images
+- require game-engine validation before applying effects
 
-[More Information Needed]
+Recommended production safeguards:
 
-**APA:**
+- validate output JSON
+- retry once on invalid output
+- clamp all numeric values in the game engine
+- reject unknown runes
+- keep final state transitions deterministic
+- log ambiguous readings for future dataset improvement
 
-[More Information Needed]
+---
 
-## Glossary [optional]
+## Suggested Evaluation
+
+Recommended metrics:
+
+| Metric | Goal |
+|---|---|
+| Valid JSON rate | >95% |
+| Rune detection accuracy | >85% |
+| Ambiguity detection quality | Manual review |
+| Schema compliance | >95% |
+| Unknown rune rejection | High |
+| Latency | Playable for Gradio |
+| Cursed-fantasy tone consistency | Manual review |
+
+Suggested test cases:
+
+```text
+flame + circle
+spiral + eye
+bone + dots
+mirror + jagged
+leaf + bone
+broken + coin
+messy flame vs leaf
+partial / incomplete glyph
+empty canvas
+```
+
+---
+
+## Example Output
+
+```json
+{
+  "visual_reading": {
+    "detected_runes": ["spiral", "eye", "broken_mark"],
+    "ambiguous_runes": [
+      {
+        "candidates": ["spiral", "wave"],
+        "reason": "curved repeated stroke could indicate either looping time or water/emotion"
+      }
+    ],
+    "confidence": 0.82,
+    "layout": "clustered"
+  },
+  "spell": {
+    "spell_name": "Cursed Foresight Loop",
+    "spell_type": "prophecy_curse",
+    "rune_combo": ["spiral", "eye", "broken_mark"],
+    "summary": "The spell reveals a possible future, then immediately makes it worse.",
+    "colors": ["violet", "black", "pale blue"],
+    "shape": "floating eye inside a cracked spiral",
+    "motion": "spiral contracts inward while the eye flickers",
+    "grandeur": "high",
+    "particles": ["purple sparks", "black motes", "thin time-rings"],
+    "sound_tags": ["whisper", "glass-crack", "reverse-chime"]
+  }
+}
+```
+
+---
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+## Deployment Notes
 
-[More Information Needed]
+GoblinV1 can be used as the model backend for a Gradio game.
 
-## More Information [optional]
+Recommended serving layout:
 
-[More Information Needed]
+```text
+Gradio UI / Canvas
+    ↓
+Image preprocessing
+    ↓
+GoblinV1 inference
+    ↓
+JSON parsing + validation
+    ↓
+Rune Goblin game engine
+    ↓
+Updated battle state + animation
+```
 
-## Model Card Authors [optional]
+For GPU deployment, use the Safetensors model with Transformers.
+
+For lightweight/local experiments, use the GGUF export with a compatible llama.cpp runtime.
+
+---
+
+## Related Project
+
+GoblinV1 powers the Rune Goblin dungeon crawler.
+
+Rune Goblin is an AI game where players draw spells, explore maps, fight bosses, trigger cursed outcomes, and unlock stronger effects by drawing clearer runes.
+
+Links:
+
+- Game/demo: https://huggingface.co/spaces/build-small-hackathon/Rune-Goblin
+- Model: https://huggingface.co/ASHu2/goblinV1
+- Dataset: https://huggingface.co/datasets/ASHu2/rune_goblin_visual_dataset
+
+---
+
+## Citation
+
+If you use this model, please cite:
+
+```bibtex
+@misc{goblinv1_2026,
+  title        = {GoblinV1: A Fine-Tuned MiniCPM-V Rune Reader for Rune Goblin},
+  author       = {Ashutosh Mishra},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {https://huggingface.co/ASHu2/goblinV1}
+}
+```
+
+Base model:
+
+```bibtex
+@misc{minicpmv46_2026,
+  title        = {MiniCPM-V 4.6},
+  author       = {OpenBMB},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {https://huggingface.co/openbmb/MiniCPM-V-4.6}
+}
+```
+
+---
+
+## License
+
+This model is released under the Apache-2.0 license.
+
+Please also follow the license and usage terms of the base model `openbmb/MiniCPM-V-4.6`.
+
+---
 
-[More Information Needed]
+## Acknowledgements
 
-## Model Card Contact
+GoblinV1 is built on top of OpenBMB MiniCPM-V-4.6.
 
-[More Information Needed]
+Thanks to the open-source multimodal model community, Hugging Face, Gradio, Modal, llama.cpp, and the cursed little goblin inside every ambiguous doodle.