Update README.md

README.md

@@ -3,233 +3,161 @@ library_name: transformers
 base_model: meta-llama/Llama-3.1-8B-Instruct
 ---
 
+<p align="center">
+  <img alt="Schematron" src="https://huggingface.co/inference-net/Schematron-3B/resolve/main/Banner.png">
+</p>
 
----
-language:
-- en
-license: apache-2.0
-tags:
-- VLM
-- video-understanding
-- image-captioning
-- gemma
-- json-mode
-- structured-output
-library_name: transformers
-base_model: meta-llama/Llama-3.1-8B-Instruct
-pipeline_tag: image-text-to-text
-model-index:
-- name: Schematron-8B
-  results:
-  - task:
-      type: image-to-text
-      name: Text Generation
-    metrics:
-    - name: Average Judge Score
-      type: quality
-      value: 3.53
-    - name: ROUGE-1
-      type: rouge-1
-      value: 0.674
-    - name: ROUGE-L
-      type: rouge-l
-      value: 0.520
-    - name: BLEU
-      type: bleu
-      value: 0.267
----
-
-# ClipTagger-12b
+<p align="center">
+  <a href="https://docs.inference.net/use-cases/json-extraction"><strong>Documentation</strong></a> ·
+  <a href="https://inference.net/models/schematron-8b"><strong>Serverless API</strong></a> ·
+  <a href="https://inference.net/blog/Schematron"><strong>Announcement blog</strong></a>
+</p>
 
-![ClipTagger-12b](./assets/grass-x-inference.png)
+<br>
 
-## Model Description
+## Model Overview
+Welcome to the Schematron series, [Inference.net's](https://inference.net/) long‑context extraction models specialized in converting noisy HTML into clean, typed JSON that conforms to your custom schema. The Schematron series was purpose‑trained for web scraping, data ingestion, and transforming arbitrary pages into structured records.
 
-**ClipTagger-12b** is a 12-billion parameter vision-language model (VLM) designed for video understanding at massive scale. Developed by [Inference.net](https://inference.net) in collaboration with [Grass](https://grass.io), this model was created to meet the demanding requirements of trillion-scale video frame captioning workloads.
+We're releasing these models in two different sizes:
 
-**ClipTagger-12b exceeds or matches the performance of GPT-4.1 and Claude 4 Sonnet, while costing 15x less per generation.**
+- **Schematron‑8B** — marginal quality lift on harder/longer pages
+- **Schematron‑3B** — recommended default; near‑parity quality at ~50% cost of Schematron-8B
 
-The model generates structured, schema-consistent JSON outputs for every video frame, making it ideal for building searchable video databases, content moderation systems, and accessibility tools. It maintains temporal consistency across frames while delivering frontier-quality performance at a fraction of the cost of closed-source alternatives.
+> [!NOTE]
+> This model card is dedicated to the smaller `Schematron-8B` model. Check out [`Schematron-8B`](https://huggingface.co/inference-net/Schematron-8B) for the smaller model.
 
-### Key Features
+## I/O at a glance
+- **Input**: Cleaned HTML + JSON Schema (can be extracted from typed model like Pydantic/Zod)
+- **Output**: Strictly valid JSON conforming to the provided schema (no narration)
 
-- **Frontier-quality performance** - Comparable to top closed models in captioning quality
-- **Production-ready** - Battle-tested on trillion-scale video frame captioning workloads
-- **Schema-consistent JSON** - Reliable structured output for every frame
-- **Cost-efficient** - Optimized for high-throughput inference
-- **Open source** - Build and deploy without proprietary API dependencies
+> [!NOTE]
+> The JSON Schema passed as input needs to conform to the [schema.org](https://json-schema.org/draft-07/schema) schema.
 
-## Architecture
+## Highlights
+- **Schema-first extraction**: 100% schema‑conformant JSON outputs
+- **Long context**: Robust to lengthy, noisy HTML (up to 128K tokens)
+- **Variants**: 3B (default, most cost‑efficient) · 8B (marginal quality lift at ~2× cost)
 
-ClipTagger-12b is based on the Gemma-12B architecture and has been optimized with FP8 quantization for maximum throughput on modern GPUs. The model is specifically tuned for RTX 40-series and H100 GPUs, leveraging native FP8 support for efficient inference.
-
-### Technical Specifications
-- **Parameters**: 12 billion
-- **Base Architecture**: Gemma-12B
-- **Quantization**: FP8 (no quality loss vs bf16)
-- **Input**: Single video frame per request
-- **Output**: Structured JSON with fixed schema
-- **Supported Formats**: JPEG, PNG, WebP, GIF
-- **Max Image Size**: 1MB
-
-## Training
-
-The model was trained on 1 million carefully curated single-frame samples from publicly available video data. Training employed knowledge distillation from a high-quality teacher model to ensure consistent, accurate outputs while maintaining the ability to generalize across diverse video content types.
-
-### Training Process
-- **Dataset Size**: 1M video frames
-- **Training Method**: Teacher-student distillation
-- **Data Source**: Publicly available video content
-- **Focus**: Single-frame understanding with temporal awareness
+## Model Details
+- **Family**: Schematron (3B and 8B)
+- **Context window**: Up to 128K tokens
+- **Input**: Cleaned or raw HTML and a JSON Schema
+- **Output**: Strict JSON that conforms to the provided schema
 
 ## Benchmarks
 
-ClipTagger-12b achieves **equal or superior performance** compared to the leading closed-source models across all major evaluation metrics. Despite being open-source and significantly more cost-effective, our model **outperforms Claude 4 Sonnet across every metric** and achieves **comparable quality to GPT-4.1**.
-
-Performance metrics on our internal evaluation set:
-| Model | Avg Judge Score | ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU |
-|-------|-----------------|---------|---------|---------|------|
-| cliptagger_12b | **3.53** | **0.674** | **0.404** | **0.520** | **0.267** |
-| claude_4_sonnet | 3.16 | 0.463 | 0.179 | 0.281 | 0.060 |
-| gpt_4.1 | 3.64 | 0.581 | 0.260 | 0.376 | 0.119 |
-
-We used Gemini-2.5-Pro as the judge model, which ranks ClipTagger-12b roughly equal to GPT-4.1, and better than Claude 4 Sonnet.
-
-<img src="./assets/judge-score.png" alt="Average Judge Score Comparison" width="100%" />
-
-
-FP8 quantization showed no measurable quality degradation compared to bf16 precision.
-
-## Cost Comparison
-
-ClipTagger-12b delivers frontier-quality performance at a fraction of the cost of closed-source alternatives. Based on typical usage patterns (700 input tokens and 250 output tokens per generation), here's how the costs compare:
-
-<img src="./assets/cost.png" alt="Cost Comparison Per 1 Million Generations" width="100%" />
-
-ClipTagger-12b offers **15x cost savings** compared to GPT-4.1 and **17x cost savings** compared to Claude 4 Sonnet, while maintaining comparable quality metrics.
-
-| Model           | Input Cost/MTok | Output Cost/MTok | Cost per 1M Generations | Cost per Generation |
-| --------------- | --------------- | ---------------- | ----------------------- | ------------------- |
-| ClipTagger-12b  | $0.30           | $0.50            | $335                    | $0.000335           |
-| GPT-4.1         | $3.00           | $12.00           | $5,100                  | $0.0051             |
-| Claude 4 Sonnet | $3.00           | $15.00           | $5,850                  | $0.00585            |
-
-
-## Usage
-
-### API Access
-
-For production deployments, we recommend using our managed API service which includes advanced features like batch processing, webhooks, and automatic scaling:
-
-**[Run ClipTagger-12b via Inference.net API →](https://docs.inference.net/use-cases/video-understanding)**
-
-### Required Prompts
-
-The model requires specific system and user prompts for optimal performance. Use these prompts exactly as shown:
-
-#### System Prompt
-```
-You are an image annotation API trained to analyze YouTube video keyframes. You will be given instructions on the output format, what to caption, and how to perform your job. Follow those instructions. For descriptions and summaries, provide them directly and do not lead them with 'This image shows' or 'This keyframe displays...', just get right into the details.
-```
-
-#### User Prompt
+### HTML-to-JSON Extraction Quality
+
+We evaluated extraction quality using Gemini 2.5 Pro as a judge, scoring extractions from 1-5 where 5 represents perfect extraction.
+
+| Model | LLM-as-Judge Score |
+|-------|-------------------|
+| GPT-4.1 | 4.74 |
+| **Schematron-8B** | **4.64** |
+| **Schematron-3B** | **4.41** |
+| Gemini-3B-Base | 2.24 |
+
+### Web-Augmented Factuality on SimpleQA
+
+We evaluated Schematron's real-world impact on LLM factuality using SimpleQA.
+
+**Test Pipeline:**
+1. **Query Generation**: Primary LLM (GPT-5 Nano or GPT-4.1) generates search queries and defines extraction schema
+2. **Web Search**: Search provider (SERP or Exa) retrieves relevant pages
+3. **Structured Extraction**: Schematron extracts JSON data from retrieved pages using the schema
+4. **Answer Synthesis**: Primary LLM produces final answer from structured data
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/6626a246891c75742bd19aaf/mU_01IPsf0FvkXYNYstRZ.png)
+
+**Key findings:**
+- Web search paired with JSON extraction improves factuality: Adding Schematron with web retrieval improves GPT-5 Nano's accuracy from 8.54% to 82.87%—nearly a 10x improvement
+- Search provider matters: Exa (82.9%) significantly outperforms SERP (64.2%) for factual retrieval, while also being more cost-effective
+- Structured extraction beats raw HTML: Processing raw HTML would require 100k+ tokens for 10 searches; Schematron's JSON extraction reduces this by orders of magnitude
+- Small specialized models win: Schematron-8B (82.87%) outperforms the much larger Gemini 2.5 Flash (80.61%) on this task, showing that fine-tuning for well-defined tasks beats general purpose models
+- Performance scales with model quality: When paired with GPT-4.1, Schematron achieves 85.58% accuracy, showing the approach benefits from stronger base models
+
+## Minimal Quickstart
+Use these local snippets to prepare HTML and compose a schema‑guided prompt. The model returns strictly valid JSON; validate it against your schema downstream.
+
+```python
+from lxml.html.clean import Cleaner
+import lxml.html as LH
+
+HTML_CLEANER = Cleaner(
+    scripts=True,
+    javascript=True,
+    style=True,
+    inline_style=True,
+    safe_attrs_only=False,
+)
+
+
+def strip_noise(html: str) -> str:
+    """Remove scripts, styles, and JavaScript from HTML using lxml.
+    """
+    if not html or not html.strip():
+        return ""
+    try:
+        doc = LH.fromstring(html)
+        cleaned = HTML_CLEANER.clean_html(doc)
+        return LH.tostring(cleaned, encoding="unicode")
+    except Exception:
+        return ""
 ```
-You are an image annotation API trained to analyze YouTube video keyframes. You must respond with a valid JSON object matching the exact structure below.
-
-Your job is to extract detailed **factual elements directly visible** in the image. Do not speculate or interpret artistic intent, camera focus, or composition. Do not include phrases like "this appears to be", "this looks like", or anything about the image itself. Describe what **is physically present in the frame**, and nothing more.
-
-Return JSON in this structure:
-
-{
-    "description": "A detailed, factual account of what is visibly happening (4 sentences max). Only mention concrete elements or actions that are clearly shown. Do not include anything about how the image is styled, shot, or composed. Do not lead the description with something like 'This image shows' or 'this keyframe is...', just get right into the details.",
-    "objects": ["object1 with relevant visual details", "object2 with relevant visual details", ...],
-    "actions": ["action1 with participants and context", "action2 with participants and context", ...],
-    "environment": "Detailed factual description of the setting and atmosphere based on visible cues (e.g., interior of a classroom with fluorescent lighting, or outdoor forest path with snow-covered trees).",
-    "content_type": "The type of content it is, e.g. 'real-world footage', 'video game', 'animation', 'cartoon', 'CGI', 'VTuber', etc.",
-    "specific_style": "Specific genre, aesthetic, or platform style (e.g., anime, 3D animation, mobile gameplay, vlog, tutorial, news broadcast, etc.)",
-    "production_quality": "Visible production level: e.g., 'professional studio', 'amateur handheld', 'webcam recording', 'TV broadcast', etc.",
-    "summary": "One clear, comprehensive sentence summarizing the visual content of the frame. Like the description, get right to the point.",
-    "logos": ["logo1 with visual description", "logo2 with visual description", ...]
-}
-
-Rules:
-- Be specific and literal. Focus on what is explicitly visible.
-- Do NOT include interpretations of emotion, mood, or narrative unless it's visually explicit.
-- No artistic or cinematic analysis.
-- Always include the language of any text in the image if present as an object, e.g. "English text", "Japanese text", "Russian text", etc.
-- Maximum 10 objects and 5 actions.
-- Return an empty array for 'logos' if none are present.
-- Always output strictly valid JSON with proper escaping.
-- Output **only the JSON**, no extra text or explanation.
-```
-
-### Inference Parameters
-
-- **Temperature**: 0.1 (recommended for consistency)
-- **Max Tokens**: 2000
-- **Response Format**: `{"type": "json_object"}`
-
-### Output Schema
-
-The model outputs a fixed JSON structure with the following fields:
 
-```json
-{
-  "description": "string - Detailed factual description (max 4 sentences)",
-  "objects": ["array of strings - Up to 10 objects with visual details"],
-  "actions": ["array of strings - Up to 5 actions with context"],
-  "environment": "string - Setting and atmosphere description",
-  "content_type": "string - Type of visual content",
-  "specific_style": "string - Genre or style classification",
-  "production_quality": "string - Production level assessment",
-  "summary": "string - Single sentence summary",
-  "logos": ["array of strings - Detected logos with descriptions"]
-}
+Compose messages with your schema and cleaned HTML:
+
+```python
+def construct_messages(schema: str, html: str):
+    """Construct messages for a schema‑guided extraction request."""
+    response_prompt = {
+        "prompt_part_one": (
+            "You are going to be given a JSON schema following the standardized JSON "
+            "Schema format. You are going to be given a HTML page and you are going "
+            "to apply the schema to the HTML page however you see it as applicable "
+            "and return the results in a JSON object. The schema is as follows:"
+        ),
+        "prompt_part_two": "Here is the HTML page:",
+        "prompt_part_three": "MAKE SURE ITS VALID JSON.",
+    }
+
+    user_prompt = (
+        response_prompt['prompt_part_one']
+        + "\n\n" + schema + "\n\n"
+        + response_prompt['prompt_part_two']
+        + "\n\n" + html + "\n\n"
+        + response_prompt['prompt_part_three']
+    )
+
+    return [
+        {"role": "system", "content": "You are a helpful assistant"},
+        {"role": "user", "content": user_prompt},
+    ]
 ```
 
-## Example Output
-
-Given a nature scene with a wooden boardwalk through grassland:
-
-```json
-{
-  "description": "A wooden boardwalk path extends from the foreground into the distance, cutting through a field of tall, vibrant green grass. The path is flanked on both sides by the dense grass. In the background, a line of trees is visible on the horizon under a blue sky with scattered white clouds.",
-  "objects": [
-    "Wooden boardwalk",
-    "Tall green grass",
-    "Blue sky",
-    "White clouds",
-    "Trees"
-  ],
-  "actions": [],
-  "environment": "An outdoor, natural landscape, likely a marsh or wetland, on a clear day. The scene is characterized by a wooden boardwalk, lush green vegetation, and a bright blue sky with wispy clouds.",
-  "content_type": "real-world footage",
-  "specific_style": "landscape photography",
-  "production_quality": "professional photography",
-  "summary": "A wooden boardwalk path winds through a lush green field under a bright blue sky with scattered clouds.",
-  "logos": []
-}
-```
-
-## Use Cases
+> [!NOTE]
+> In the [serverless API](https://inference.net/models/schematron-3b) there's no need to pass anything but the HTML. We handle the prompt formatting for you.
 
-- **Video Search & Discovery** - Build searchable databases with structured metadata
-- **Content Moderation** - Automated content analysis and categorization
-- **Accessibility** - Generate consistent alt-text and scene descriptions
-- **Ad Verification** - Track product visibility and brand appearances
-- **Video Analytics** - Extract insights from large video collections
-- **Content Management** - Automatic tagging and organization of video libraries
 
-## Interested in training your own model?
+## Recommendations
+- Temperature 0 and JSON mode for deterministic, parseable output
+- Validate responses against your schema (e.g., Pydantic or Zod)
+- Pre‑clean HTML (remove scripts/styles) when possible; avoid over‑aggressive removal
+- Using lxml to clean the HTML is not required, but is recommended as it matches the training data.
 
-Contact us at [partners@inference.net](mailto:partners@inference.net) for a free consultation with our research team.
+## Limitations
+- Static HTML only; render client‑side content upstream
+- Very large pages may require truncation
+- Ambiguous fields depend on schema clarity; be explicit in field descriptions
 
-## Support
-
-- **Documentation**: [docs.inference.net](https://docs.inference.net/use-cases/video-understanding)
-- **API Access**: Get $25 in free credits when you [sign up](https://inference.net/register) for an account
-- **Email**: support@inference.net
+## Safety and Responsible Use
+- Extracted data may include personal or sensitive information present in the page—handle and store responsibly
+- Respect site terms, robots.txt, and applicable laws
+- Use downstream validation and guardrails for compliance
 
 ## License
+See license in the metadata above.
 
-This model is released under the Apache-2.0 license, allowing for commercial use and modification with proper attribution.
+## Support
+- Docs: https://docs.inference.net/use-cases/json-extraction
+- Email: support@inference.net